2026-01-24 17:00:08 [scrapy.utils.log] (PID: 96) INFO: Scrapy 2.12.0 started (bot: catalog_extraction) 2026-01-24 17:00:08 [scrapy.utils.log] (PID: 96) INFO: Versions: lxml 5.3.1.0, libxml2 2.12.9, cssselect 1.3.0, parsel 1.10.0, w3lib 2.3.1, Twisted 24.11.0, Python 3.11.13 (main, Jun 10 2025, 23:54:42) [GCC 12.2.0], pyOpenSSL 25.0.0 (OpenSSL 3.4.1 11 Feb 2025), cryptography 44.0.2, Platform Linux-6.9.12-x86_64-with-glibc2.36 2026-01-24 17:00:08 [global_industrial] (PID: 96) INFO: Starting extraction spider global_industrial... 2026-01-24 17:00:08 [scrapy.addons] (PID: 96) INFO: Enabled addons: [] 2026-01-24 17:00:08 [py.warnings] (PID: 96) WARNING: /usr/local/lib/python3.11/site-packages/scrapy/utils/request.py:120: ScrapyDeprecationWarning: 'REQUEST_FINGERPRINTER_IMPLEMENTATION' is a deprecated setting. It will be removed in a future version of Scrapy. return cls(crawler) 2026-01-24 17:00:08 [scrapy.extensions.telnet] (PID: 96) INFO: Telnet Password: f801ca1cea4505b5 2026-01-24 17:00:08 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-24 17:00:08 [scrapy.middleware] (PID: 96) INFO: Enabled extensions: ['scrapy.extensions.corestats.CoreStats', 'scrapy.extensions.telnet.TelnetConsole', 'scrapy.extensions.memusage.MemoryUsage', 'scrapy.extensions.closespider.CloseSpider', 'scrapy.extensions.feedexport.FeedExporter', 'scrapy.extensions.logstats.LogStats', 'spidermon.contrib.scrapy.extensions.Spidermon'] 2026-01-24 17:00:08 [scrapy.crawler] (PID: 96) INFO: Overridden settings: {'BOT_NAME': 'catalog_extraction', 'CONCURRENT_ITEMS': 250, 'DOWNLOAD_DELAY': 0.5, 'FEED_EXPORT_ENCODING': 'utf-8', 'LOG_FILE': '/var/lib/scrapyd/logs/catalog_extraction/global_industrial/1f5910aef94611f0a5d64200a9fe0102.log', 'LOG_FORMAT': '%(asctime)s [%(name)s] (PID: %(process)d) %(levelname)s: ' '%(message)s', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'catalog_extraction.spiders', 'REQUEST_FINGERPRINTER_CLASS': 'scrapy_poet.ScrapyPoetRequestFingerprinter', 'REQUEST_FINGERPRINTER_IMPLEMENTATION': '2.7', 'SPIDER_MODULES': ['catalog_extraction.spiders'], 'TWISTED_REACTOR': 'twisted.internet.asyncioreactor.AsyncioSelectorReactor', 'USER_AGENT': None} 2026-01-24 17:00:09 [scrapy_poet.injection] (PID: 96) INFO: Loading providers: [, , , , , , ] 2026-01-24 17:00:09 [scrapy.middleware] (PID: 96) INFO: Enabled downloader middlewares: ['scrapy.downloadermiddlewares.offsite.OffsiteMiddleware', 'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware', 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware', 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware', 'scraping_utils.middlewares.downloaders.ProxyManagerDownloaderMiddleware', 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware', 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware', 'scraping_utils.middlewares.downloaders.HeadersSpooferDownloaderMiddleware', 'scrapy_poet.InjectionMiddleware', 'scrapy.downloadermiddlewares.retry.RetryMiddleware', 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware', 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware', 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware', 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware', 'scrapy_poet.DownloaderStatsMiddleware'] 2026-01-24 17:00:09 [NotFoundHandlerSpiderMiddleware] (PID: 96) INFO: NotFoundHandlerSpiderMiddleware running on PRODUCTION environment. 2026-01-24 17:00:09 [scrapy.middleware] (PID: 96) INFO: Enabled spider middlewares: ['catalog_extraction.middlewares.NotFoundHandlerSpiderMiddleware', 'catalog_extraction.middlewares.FixtureSavingMiddleware', 'scrapy_poet.RetryMiddleware', 'scrapy.spidermiddlewares.referer.RefererMiddleware', 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware', 'scrapy.spidermiddlewares.depth.DepthMiddleware'] 2026-01-24 17:00:09 [scrapy.middleware] (PID: 96) INFO: Enabled item pipelines: ['catalog_extraction.pipelines.DuplicatedSKUsFilterPipeline', 'catalog_extraction.pipelines.DiscontinuedProductsAdjustmentPipeline', 'catalog_extraction.pipelines.PriceRoundingPipeline', 'scraping_utils.pipelines.AttachSupplierPipeline', 'spidermon.contrib.scrapy.pipelines.ItemValidationPipeline'] 2026-01-24 17:00:09 [scrapy.core.engine] (PID: 96) INFO: Spider opened 2026-01-24 17:00:09 [scrapy.extensions.closespider] (PID: 96) INFO: Spider will stop when no items are produced after 1800 seconds. 2026-01-24 17:00:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min) 2026-01-24 17:00:09 [scrapy.extensions.telnet] (PID: 96) INFO: Telnet console listening on 127.0.0.1:6025 2026-01-24 17:00:17 [ProxyManagerDownloaderMiddleware] (PID: 96) INFO: Using brd-customer-hl_13cda1e4-zone-main_datacenter_proxy as the default proxy for ProxyManagerDownloaderMiddleware. 2026-01-24 17:01:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:01:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pegboard-strips-green-3-x-32-2-pc>: HTTP status code is not handled or not allowed 2026-01-24 17:01:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flask-clamp-starter-kit-30129-for-use-with-maxq-shaker-platforms>: HTTP status code is not handled or not allowed 2026-01-24 17:01:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/diamond-plate-platform-kit-12-x-18-x-72>: HTTP status code is not handled or not allowed 2026-01-24 17:01:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 83 pages (at 83 pages/min), scraped 80 items (at 80 items/min) 2026-01-24 17:01:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/rack_accessories/bulk_storage_rack_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:01:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-coil-thread-repair-kit-for-metal-1-4-20-x-15d>: HTTP status code is not handled or not allowed 2026-01-24 17:01:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/dry_cleaning_laundry_equipment/laundry_drying_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:01:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks landed on page that is not a product page. 2026-01-24 17:01:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:01:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/soho-modern-home-office-gaming-computer-l-shape-corner-desk-urban-walnut>: HTTP status code is not handled or not allowed 2026-01-24 17:01:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:01:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:01:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/octagonal-dry-erase-sign-stand-16w-x-16h-sign-sig08070101>: HTTP status code is not handled or not allowed 2026-01-24 17:01:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:01:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/legacy-reception-desk-return-shell-ash-grey>: HTTP status code is not handled or not allowed 2026-01-24 17:01:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/rack_accessories/bulk_storage_rack_shelves landed on page that is not a product page. 2026-01-24 17:01:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/dry_cleaning_laundry_equipment already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:01:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/rack_accessories/bulk_storage_rack_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:01:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks>: HTTP status code is not handled or not allowed 2026-01-24 17:01:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:01:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/arts_crafts_supplies/craft_basics already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:01:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:01:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks>: HTTP status code is not handled or not allowed 2026-01-24 17:01:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks landed on page that is not a product page. 2026-01-24 17:01:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-hi-tech-metal-bin-shelving-36-w-x-18-d-x-87-h-725-hallowell-gry-starter-unit-54-bins>: HTTP status code is not handled or not allowed 2026-01-24 17:01:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/arts_crafts_supplies/clay_dough already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:01:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks landed on page that is not a product page. 2026-01-24 17:01:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/dry_cleaning_laundry_equipment landed on page that is not a product page. 2026-01-24 17:01:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-48-w-x-48-d-x-120-h-add-on-3-shelf-levels-wood-deck-sand>: HTTP status code is not handled or not allowed 2026-01-24 17:01:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/arts_crafts_supplies/craft_basics landed on page that is not a product page. 2026-01-24 17:01:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/263184-fry-left-hand-scoop-for-prince-castle>: HTTP status code is not handled or not allowed 2026-01-24 17:01:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011552-pin-for-true-manufacturing>: HTTP status code is not handled or not allowed 2026-01-24 17:01:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/rack_accessories/bulk_storage_rack_shelves landed on page that is not a product page. 2026-01-24 17:01:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/laundry_detergent already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:01:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/envision-25190-c-fold-paper-towel-10-14w-x-13-14h-white-240pack-10carton>: HTTP status code is not handled or not allowed 2026-01-24 17:01:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks landed on page that is not a product page. 2026-01-24 17:01:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/arts_crafts_supplies/clay_dough landed on page that is not a product page. 2026-01-24 17:01:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2721312-motor-brushless-24vdc-for-nieco-corp>: HTTP status code is not handled or not allowed 2026-01-24 17:01:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks landed on page that is not a product page. 2026-01-24 17:01:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arkansas-state-flag-4x6-nylon>: HTTP status code is not handled or not allowed 2026-01-24 17:01:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-48-w-x-36-d-x-96-h-add-on-3-shelf-levels-wood-deck-medium-gray>: HTTP status code is not handled or not allowed 2026-01-24 17:01:45 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/laundry_detergent landed on page that is not a product page. 2026-01-24 17:01:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plug-and-play-22-inch-lcd-cool-blue-display-and-5-remotes>: HTTP status code is not handled or not allowed 2026-01-24 17:01:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dw-portable-forced-air-propane-heater-dxh170favt-125k-to-170k-btu>: HTTP status code is not handled or not allowed 2026-01-24 17:01:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/saddle-extension-kit-for-hw93733-hw93265>: HTTP status code is not handled or not allowed 2026-01-24 17:01:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-inch-x-50-foot-pvc-lay-flat-discharge-hose-coupled-with-cxe-aluminum-cam-and-groove-fittings>: HTTP status code is not handled or not allowed 2026-01-24 17:01:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-ton-air-jack-support-stand-34100>: HTTP status code is not handled or not allowed 2026-01-24 17:01:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1841119-screw-door-handle-10-24thd-for-bakers-pride>: HTTP status code is not handled or not allowed 2026-01-24 17:01:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/electronic_queuing_systems already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:01:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25-ton-axle-jack-with-3-inch-extension-hw93735a>: HTTP status code is not handled or not allowed 2026-01-24 17:01:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-ton-long-chassis-service-jack-with-air-22201>: HTTP status code is not handled or not allowed 2026-01-24 17:01:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-ton-shorty-bottle-jack-b-s12d>: HTTP status code is not handled or not allowed 2026-01-24 17:01:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-3656-9-inch-vegetable-slicer-attachment-for-hobart-22-hubs>: HTTP status code is not handled or not allowed 2026-01-24 17:01:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/electronic_queuing_systems landed on page that is not a product page. 2026-01-24 17:02:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-40-14w-drawer-cabinet6-drawer72-compart-classic-blue-individual-lock>: HTTP status code is not handled or not allowed 2026-01-24 17:02:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/first-aid-only-z6005-cold-compress-4-x-5>: HTTP status code is not handled or not allowed 2026-01-24 17:02:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012231-elbow-oring-boss-90deg2062-6-4s-for-cleveland-range>: HTTP status code is not handled or not allowed 2026-01-24 17:02:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 171 pages (at 88 pages/min), scraped 128 items (at 48 items/min) 2026-01-24 17:02:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uv-black-releasable-cable-tie-502255-100-75-100-pcs>: HTTP status code is not handled or not allowed 2026-01-24 17:02:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/medical_devices_instruments/thermometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:02:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unisex-jade-cotton-polyester-fashion-blend-reversible-set-in-sleeve-scrub-shirt-size-2xl>: HTTP status code is not handled or not allowed 2026-01-24 17:02:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/wiring_devices/wire_connectors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:02:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-powder-exercise-band-100-yd-rolls-5-color-set>: HTTP status code is not handled or not allowed 2026-01-24 17:02:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-free-exercise-band-lime-level-3-6-yard-roll-box>: HTTP status code is not handled or not allowed 2026-01-24 17:02:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/medical_devices_instruments/thermometers>: HTTP status code is not handled or not allowed 2026-01-24 17:02:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/wiring_devices/wire_connectors>: HTTP status code is not handled or not allowed 2026-01-24 17:02:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/70471-high-low-brush-medium-pink>: HTTP status code is not handled or not allowed 2026-01-24 17:02:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hager-full-mortise-spring-single-acting-hinge2>: HTTP status code is not handled or not allowed 2026-01-24 17:02:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_hinges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:02:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_hinges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:02:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-steel-toe-knee-boots-size-13-vinyl-black-1-pair>: HTTP status code is not handled or not allowed 2026-01-24 17:02:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/exit_devices already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:02:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_closers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:02:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/321955-silicone-tubing-foot-for-hoshizaki-of-america>: HTTP status code is not handled or not allowed 2026-01-24 17:02:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cylinder-tube-rack-1-wide-x-1-deep-16w-x-16d-x-30h2-cylinder-cap>: HTTP status code is not handled or not allowed 2026-01-24 17:02:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_hinges landed on page that is not a product page. 2026-01-24 17:02:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_hinges landed on page that is not a product page. 2026-01-24 17:02:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/fasteners/door_window_hardware/exit_devices>: HTTP status code is not handled or not allowed 2026-01-24 17:02:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_closers landed on page that is not a product page. 2026-01-24 17:02:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/69715-1-5-stainless-steel-scraper-white>: HTTP status code is not handled or not allowed 2026-01-24 17:02:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eye-face-wash-32-oz-single-bottle-station>: HTTP status code is not handled or not allowed 2026-01-24 17:02:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/77554-28-foam-blade-squeegee-red>: HTTP status code is not handled or not allowed 2026-01-24 17:02:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-shelf-liner-42w-x-14d-blue>: HTTP status code is not handled or not allowed 2026-01-24 17:02:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12u-wall-mount-vertical-rail-kit-10-32-tapped>: HTTP status code is not handled or not allowed 2026-01-24 17:02:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powered-20-w-x-10-l-belt-conveyor-with-6-h-side-rails>: HTTP status code is not handled or not allowed 2026-01-24 17:02:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/bollards/safety_bollards_posts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:02:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/52-bollard-flex-concrete-installation-white-cover-black-tapes>: HTTP status code is not handled or not allowed 2026-01-24 17:03:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/bollards/safety_bollards_posts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:03:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/52-bollard-flex-with-sign-post-concrete-installation-red-cover-yellow-tapes>: HTTP status code is not handled or not allowed 2026-01-24 17:03:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/bollards/safety_bollards_posts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:03:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/bollards/safety_bollards_posts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:03:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/bollards/safety_bollards_posts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:03:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metal-threaded-handle-0160-1516-x-60>: HTTP status code is not handled or not allowed 2026-01-24 17:03:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/bollards/safety_bollards_posts landed on page that is not a product page. 2026-01-24 17:03:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelate-wire-shelf-truck-60x24x69-1200-pound-capacity-with-brakes>: HTTP status code is not handled or not allowed 2026-01-24 17:03:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 254 pages (at 83 pages/min), scraped 186 items (at 58 items/min) 2026-01-24 17:03:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/bollards/safety_bollards_posts landed on page that is not a product page. 2026-01-24 17:03:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/bollards/safety_bollards_posts>: HTTP status code is not handled or not allowed 2026-01-24 17:03:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/overhead_clearance_bars_door_guards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:03:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/bollards/safety_bollards_posts landed on page that is not a product page. 2026-01-24 17:03:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-1170-chrome-pre-rinse-valve-handle-nut>: HTTP status code is not handled or not allowed 2026-01-24 17:03:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/38-1097-1-4-inch-x-1-1-4-inch-30-amp-fast-acting-ceramic-fuse-250v>: HTTP status code is not handled or not allowed 2026-01-24 17:03:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/bollards/safety_bollards_posts landed on page that is not a product page. 2026-01-24 17:03:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e71t-gs-self-steel-flux-core-welding-wire-0030-dia>: HTTP status code is not handled or not allowed 2026-01-24 17:03:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stronghold-ventilated-divider-cabinet-60-x-24-x-78>: HTTP status code is not handled or not allowed 2026-01-24 17:03:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/duct-access-door-with-no-hinge-18-x-18>: HTTP status code is not handled or not allowed 2026-01-24 17:03:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/overhead_clearance_bars_door_guards landed on page that is not a product page. 2026-01-24 17:03:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stronghold-stainless-steel-double-door-independent-locking-cabinet-60-x-24-x-78>: HTTP status code is not handled or not allowed 2026-01-24 17:03:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-access-door-8-x-8>: HTTP status code is not handled or not allowed 2026-01-24 17:03:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/wardrobe_combination_cabinets/combination_cabients already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:03:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-wardrobe-cabinet-48-x-24-x-78>: HTTP status code is not handled or not allowed 2026-01-24 17:03:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carson-optical-ll-55-lumiloupe-magnifier>: HTTP status code is not handled or not allowed 2026-01-24 17:03:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:03:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/41867-long-handle-scrubbing-brush-stiff-orange>: HTTP status code is not handled or not allowed 2026-01-24 17:03:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/cabinets/wardrobe_combination_cabinets/combination_cabients landed on page that is not a product page. 2026-01-24 17:03:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yellow-black-yellow-belt>: HTTP status code is not handled or not allowed 2026-01-24 17:03:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/security-cage-accessory-shelf-4>: HTTP status code is not handled or not allowed 2026-01-24 17:03:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bolted-teardrop-pallet-rack-add-on-120inw-x-48ind-x-96inh-5630-lbs-shelf-cap>: HTTP status code is not handled or not allowed 2026-01-24 17:03:50 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits landed on page that is not a product page. 2026-01-24 17:03:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storability-31-wire-basket-with-lock-on-brackets>: HTTP status code is not handled or not allowed 2026-01-24 17:03:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dinex-insulated-base-for-insulated-domes-9-1-2-d-12-cs-latte-dx107731>: HTTP status code is not handled or not allowed 2026-01-24 17:03:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-door-wall-mountable-film-box>: HTTP status code is not handled or not allowed 2026-01-24 17:03:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stretch-unlined-bib-overall-2xl-black>: HTTP status code is not handled or not allowed 2026-01-24 17:04:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/soft-pliable-medicine-ball-30-lb-9-diameter-gold>: HTTP status code is not handled or not allowed 2026-01-24 17:04:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ten-quarts-sauce-pan-plain-handle>: HTTP status code is not handled or not allowed 2026-01-24 17:04:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/cookware_bakeware/pots_pans/saute_pots_pans already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:04:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/44-1435-relay-200-240v>: HTTP status code is not handled or not allowed 2026-01-24 17:04:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 350 pages (at 96 pages/min), scraped 254 items (at 68 items/min) 2026-01-24 17:04:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/positioning-half-roll-6w-x-24l-1-1-2-rise-blue>: HTTP status code is not handled or not allowed 2026-01-24 17:04:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/electrochemistry_meters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:04:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/cookware_bakeware/pots_pans/saute_pots_pans>: HTTP status code is not handled or not allowed 2026-01-24 17:04:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/263775-ferrule-breakaway-3-16-inch-15-p>: HTTP status code is not handled or not allowed 2026-01-24 17:04:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/relaxed-fit-twill-safety-cargo-utility-pants-34l-inseam-x-40-waist-navy>: HTTP status code is not handled or not allowed 2026-01-24 17:04:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/electrochemistry_meters landed on page that is not a product page. 2026-01-24 17:04:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insulated-safety-coverall-s-black>: HTTP status code is not handled or not allowed 2026-01-24 17:04:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industries-half-ball-end-cap-for-2-tubing-satin-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 17:04:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/servewell-double-deck-overshelf-3>: HTTP status code is not handled or not allowed 2026-01-24 17:04:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/261161-lock-washer-bx-100-10-18-8-ss-plt>: HTTP status code is not handled or not allowed 2026-01-24 17:04:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1031163-hose-vacuum-inlet-14-1-2-inch-for-ultrafryer>: HTTP status code is not handled or not allowed 2026-01-24 17:04:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-shelving-unit-36-wx18-dx63-h-solid-shelf>: HTTP status code is not handled or not allowed 2026-01-24 17:04:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-metal-cable-seal-1-8-wx12-l-green-50-pack>: HTTP status code is not handled or not allowed 2026-01-24 17:04:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-manila-folders-one-fastener-13-tab-letter-50box>: HTTP status code is not handled or not allowed 2026-01-24 17:04:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clamps-wilton14257>: HTTP status code is not handled or not allowed 2026-01-24 17:04:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/building_construction_tape/antimicrobial_tape already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:04:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/file_folders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:04:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cp100d-4-watt-16-channel-analog-and-digital-uhf-403-470-mhz-non-display>: HTTP status code is not handled or not allowed 2026-01-24 17:04:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1421132-rivet-blind-stainless-steel-42-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 17:04:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/head_face_protection/welding_helmets_lenses already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:04:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/barr8-fl-wh-crowdmaster-1000-barricade-100-by-43-flat-feet-white>: HTTP status code is not handled or not allowed 2026-01-24 17:04:45 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/building_construction_tape/antimicrobial_tape landed on page that is not a product page. 2026-01-24 17:04:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/airhydbottlejacks-jet-456620>: HTTP status code is not handled or not allowed 2026-01-24 17:04:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-poly-oxford-5-in-1-safety-jacket-s-black>: HTTP status code is not handled or not allowed 2026-01-24 17:04:51 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/file_folders landed on page that is not a product page. 2026-01-24 17:04:51 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/head_face_protection/welding_helmets_lenses landed on page that is not a product page. 2026-01-24 17:04:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/screwjacks-jet-441310>: HTTP status code is not handled or not allowed 2026-01-24 17:04:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-50mm-bore-x-600mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 17:04:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/430-14-2-composite-spreader-beam-14-ton-capacity-2-hook-spread>: HTTP status code is not handled or not allowed 2026-01-24 17:04:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/file_folders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:05:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-insulated-quilted-jacket-xs-black>: HTTP status code is not handled or not allowed 2026-01-24 17:05:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/type-r-class-2-heavy-duty-surveyor-safety-vest-zipper-4xl-green-sv59z-2zgd-4x>: HTTP status code is not handled or not allowed 2026-01-24 17:05:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-80mm-bore-x-830mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 17:05:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 442 pages (at 92 pages/min), scraped 319 items (at 65 items/min) 2026-01-24 17:05:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-80mm-bore-x-800mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 17:05:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aristocrat-floor-lectern-medium-oak>: HTTP status code is not handled or not allowed 2026-01-24 17:05:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/file_folders landed on page that is not a product page. 2026-01-24 17:05:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fx-40-balaclava-xl-fluorescent-orange>: HTTP status code is not handled or not allowed 2026-01-24 17:05:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-safety-parka-jacket-4xl-navy>: HTTP status code is not handled or not allowed 2026-01-24 17:05:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/381372-heater-wire-roll-100-ft>: HTTP status code is not handled or not allowed 2026-01-24 17:05:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-17>: HTTP status code is not handled or not allowed 2026-01-24 17:05:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ecos-scented-liquid-laundry-detergent-magnolia-lily-2x-5-gallon-pail>: HTTP status code is not handled or not allowed 2026-01-24 17:05:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dishmate-free-clear-dishwashing-liquid-5-gallon-pail>: HTTP status code is not handled or not allowed 2026-01-24 17:05:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1481008-track-v-true-33-inch-for-true-manufacturing>: HTTP status code is not handled or not allowed 2026-01-24 17:05:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-expandable-starter-rack-48x18x84-gray-with-3-level-wood-deck-1500lb-cap-per-deck>: HTTP status code is not handled or not allowed 2026-01-24 17:05:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/moist-heat-pack-knee-shoulder-10-x-20-12-pk>: HTTP status code is not handled or not allowed 2026-01-24 17:05:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1171304-valve-rebuild-kit-5gpf-urnl-for-zurn-industries-llc>: HTTP status code is not handled or not allowed 2026-01-24 17:05:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lockout-shadow-board-20-padlock-capacity-23w-x-316d-x-22h-red>: HTTP status code is not handled or not allowed 2026-01-24 17:05:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/needle-valve-85973-04-04-screw-adj-14-tube-x-14-swift-fit-composite-body>: HTTP status code is not handled or not allowed 2026-01-24 17:05:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blue-vinyl-reusable-cold-pack-half-size-7-x-11-12-pk>: HTTP status code is not handled or not allowed 2026-01-24 17:05:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hydraulic-raise-n-roll-machinery-dolly-4000-lb-capacity-pair>: HTTP status code is not handled or not allowed 2026-01-24 17:05:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/needle-valve-85978-04-04-knob-adj-14-tube-x-14-swift-fit-composite-body>: HTTP status code is not handled or not allowed 2026-01-24 17:05:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2481018-plate-3-16-inch-for-univex>: HTTP status code is not handled or not allowed 2026-01-24 17:05:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/516-inch-square-mill-stock-undersized-keystock-plain-finish-12-inch-length-6-pcs>: HTTP status code is not handled or not allowed 2026-01-24 17:06:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2mm-square-metric-keystock-plain-finish-1-meter-length>: HTTP status code is not handled or not allowed 2026-01-24 17:06:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cyclone-21052-harness-pass-through-chest-tongue-buckle-legs-back-d-ring-s>: HTTP status code is not handled or not allowed 2026-01-24 17:06:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ex2-deburring-wheel-8-inch-x-1-inch-x-3-inch-fin-9s-grit-silicon-carbide>: HTTP status code is not handled or not allowed 2026-01-24 17:06:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drum-dolly-30-gallon-drum-2>: HTTP status code is not handled or not allowed 2026-01-24 17:06:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 537 pages (at 95 pages/min), scraped 390 items (at 71 items/min) 2026-01-24 17:06:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-additional-shelf-with-laminated-deck-60x24-gray-2>: HTTP status code is not handled or not allowed 2026-01-24 17:06:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/little-giant-3e-34n-series-dual-purpose-small-submersible-pump>: HTTP status code is not handled or not allowed 2026-01-24 17:06:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-record-storage-with-boxes-4>: HTTP status code is not handled or not allowed 2026-01-24 17:06:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50-mm-casters-barrel>: HTTP status code is not handled or not allowed 2026-01-24 17:06:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/union-tee-89230-04-14-tube-metallic-collet>: HTTP status code is not handled or not allowed 2026-01-24 17:06:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1831130-pin-wrap-belt-10-for-roundup-food-equipment>: HTTP status code is not handled or not allowed 2026-01-24 17:06:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48x24x63-stainless-steel-solid-shelving-add-on>: HTTP status code is not handled or not allowed 2026-01-24 17:06:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/presswood-pallet-48x40>: HTTP status code is not handled or not allowed 2026-01-24 17:06:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/happy-feet-grip-mat-black-3x5>: HTTP status code is not handled or not allowed 2026-01-24 17:06:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clean-stride-inserts-60-layers-per-insert-24x30>: HTTP status code is not handled or not allowed 2026-01-24 17:06:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/superscrape-mat-4x6>: HTTP status code is not handled or not allowed 2026-01-24 17:06:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m5-x-0-8-x-25mm-socket-head-cap-screw-304-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 17:06:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qr72h-quick-release-part-ss-hose-clamp-2-inch-5-inch-clamping-diam-10-pcs>: HTTP status code is not handled or not allowed 2026-01-24 17:06:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/small-parts-wire-basket-with-lid-9-l-x-7-w-x-3-h-stainless-steel-price-each-for-qty-5>: HTTP status code is not handled or not allowed 2026-01-24 17:06:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/general_purpose_electric_motors/single_phase_motors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:06:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-tecgen-flame-resistant-work-shirt-2xl-short-light-blue>: HTTP status code is not handled or not allowed 2026-01-24 17:06:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-tecgen-flame-resistant-work-shirt-3xl-tall-light-blue>: HTTP status code is not handled or not allowed 2026-01-24 17:06:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smoking-shelter-dome-roof-four-sided-with-le-front-opening-15-x-5>: HTTP status code is not handled or not allowed 2026-01-24 17:06:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/general_purpose_electric_motors/single_phase_motors landed on page that is not a product page. 2026-01-24 17:06:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/room_dividers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:06:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-waterproof-hard-case-w-p-e-tear-foam-20-l-x-16-3-4-w-x-9-7-16-h-black>: HTTP status code is not handled or not allowed 2026-01-24 17:06:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/foodservice_disposables/disposable_utensils already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:07:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-inch-x-2-inch-x-0003-inch-stainless-steel-slotted-shim-pack-of-20>: HTTP status code is not handled or not allowed 2026-01-24 17:07:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s9616-65w-led-mr16-40-beam-spread-gu53-base-3000k-12v>: HTTP status code is not handled or not allowed 2026-01-24 17:07:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acrylic-transparent-clip-on-sign-holder-with-spring-loaded-4-long>: HTTP status code is not handled or not allowed 2026-01-24 17:07:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/file_cabinets/vertical_file_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:07:09 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/foodservice_disposables/disposable_utensils landed on page that is not a product page. 2026-01-24 17:07:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 635 pages (at 98 pages/min), scraped 465 items (at 75 items/min) 2026-01-24 17:07:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-1834-1-1-4-inch-nps-waste-drain-overflow-head-assembly-for-3-and-3-1-2-inch-sink-openings>: HTTP status code is not handled or not allowed 2026-01-24 17:07:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/partitions_dividers/room_dividers landed on page that is not a product page. 2026-01-24 17:07:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s9151-14w-led-a15-medium-base-frost-2700k>: HTTP status code is not handled or not allowed 2026-01-24 17:07:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/cabinets/file_cabinets/vertical_file_cabinets>: HTTP status code is not handled or not allowed 2026-01-24 17:07:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/roof_pipe_heating_cables already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:07:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-flame-resistant-zip-front-hoodie-l-regular-navy>: HTTP status code is not handled or not allowed 2026-01-24 17:07:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/structural-foam-plastic-platform-truck-with-4-wheels-60-l-x-30-w>: HTTP status code is not handled or not allowed 2026-01-24 17:07:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-assist-truck-axle-jack-w-extension-kit-1-pc-handle-22-ton-cap>: HTTP status code is not handled or not allowed 2026-01-24 17:07:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lightning-lift-heavy-duty-floor-jack-2-pc-handle-3-12-ton-capacity>: HTTP status code is not handled or not allowed 2026-01-24 17:07:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/roof_pipe_heating_cables landed on page that is not a product page. 2026-01-24 17:07:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/business-office-case-pkg-18-b-18l-x-13w-x-6h-black>: HTTP status code is not handled or not allowed 2026-01-24 17:07:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-bead-lift-tire-bar-24l>: HTTP status code is not handled or not allowed 2026-01-24 17:07:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alemite-hydraulicfittings-straight-11-16-in-male-male-1-8-in-ptf-sae-1610-bl>: HTTP status code is not handled or not allowed 2026-01-24 17:07:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-dock-plate-1764-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 17:07:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/13-panel-divider-7-ft-4-inch-high-x-24-ft-1-inch-long-fabric-sand>: HTTP status code is not handled or not allowed 2026-01-24 17:07:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dumpster-pro-odor-neutralizer-3-5-gallons-one-pail>: HTTP status code is not handled or not allowed 2026-01-24 17:07:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-defender-walk-ramp-39w-16l-4000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 17:07:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quintessence-collection-upholstered-sofa-gray-wooden-frame>: HTTP status code is not handled or not allowed 2026-01-24 17:07:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flexline-60-w-x-30-d-dual-sided-workstation-anti-static-top>: HTTP status code is not handled or not allowed 2026-01-24 17:07:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/furniture_parts_accessories/partition_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:07:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/serveware_tableware/dinnerware/condiment_servers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:07:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/serveware_tableware/dinnerware/condiment_servers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:07:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stanley-fatmax-020800r-4-in-1-mobile-workstation>: HTTP status code is not handled or not allowed 2026-01-24 17:07:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/serveware_tableware/dinnerware/condiment_servers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:07:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/262402-swivel-stem-caster5-w-1-1-2-od-tubing>: HTTP status code is not handled or not allowed 2026-01-24 17:07:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-hi-collar-split-lock-washer-385-377-i-d-125-thick-steel-zinc-grade-2-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 17:07:56 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/furniture_parts_accessories/partition_accessories landed on page that is not a product page. 2026-01-24 17:07:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/serveware_tableware/dinnerware/condiment_servers landed on page that is not a product page. 2026-01-24 17:07:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-panel-display-tower-6-ft-5-inch-high-fabric-sand>: HTTP status code is not handled or not allowed 2026-01-24 17:08:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/serveware_tableware/dinnerware/condiment_servers landed on page that is not a product page. 2026-01-24 17:08:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/serveware_tableware/dinnerware/condiment_servers>: HTTP status code is not handled or not allowed 2026-01-24 17:08:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-4-flat-washer-uss-1-3-8-i-d-136-192-thick-steel-galvanized-grade-2-pkg-of-25>: HTTP status code is not handled or not allowed 2026-01-24 17:08:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/appliances/cooking_equipment/deep_fryers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:08:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/push-to-connect-tube-fitting-nylon-plastic-straight-reducer-12mm-tube-od-x-10mm-tube-od>: HTTP status code is not handled or not allowed 2026-01-24 17:08:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 728 pages (at 93 pages/min), scraped 530 items (at 65 items/min) 2026-01-24 17:08:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/diamond-plate-1-2-thick-mat-36-wide-black>: HTTP status code is not handled or not allowed 2026-01-24 17:08:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cayenne-standard-duty-electric-fryers-40705-countertop-fryer-10-lb>: HTTP status code is not handled or not allowed 2026-01-24 17:08:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/261766-baffle-filter-16-x-20-galvanized>: HTTP status code is not handled or not allowed 2026-01-24 17:08:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprayed-glossy-white-ball-6-in-diameter>: HTTP status code is not handled or not allowed 2026-01-24 17:08:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/appliances/cooking_equipment/deep_fryers landed on page that is not a product page. 2026-01-24 17:08:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thank-you-for-not-smoking-sign-4521-3-x-9>: HTTP status code is not handled or not allowed 2026-01-24 17:08:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chemical-resistant-fep-tubing-14id-x-38od-x-10-ft>: HTTP status code is not handled or not allowed 2026-01-24 17:08:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/34-1172-dishwasher-heater-208v-3300w-1-phase>: HTTP status code is not handled or not allowed 2026-01-24 17:08:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-8-drawer-standard-width-cabinet-blue-keyed-alike>: HTTP status code is not handled or not allowed 2026-01-24 17:08:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-11-drawer-standard-width-cabinet-blue-master-keyed>: HTTP status code is not handled or not allowed 2026-01-24 17:08:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-red-pin-heavy-duty-dock-boards-29>: HTTP status code is not handled or not allowed 2026-01-24 17:08:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/servewell-4-caster-set-3809934-ada-ht-w-two-brakes>: HTTP status code is not handled or not allowed 2026-01-24 17:08:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1111310-valve-hose-pre-rinse-w-swivel-for-ts-brass-bronze-works>: HTTP status code is not handled or not allowed 2026-01-24 17:08:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-round-table-with-4-imme-armless-barstools-black-seatmaple-top>: HTTP status code is not handled or not allowed 2026-01-24 17:08:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6t25f34k-nylon-plastic-adjustable-lever-w-steel-components-3-8-16-x-98-stud>: HTTP status code is not handled or not allowed 2026-01-24 17:08:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/68-1209-ice-machine-pump-120v>: HTTP status code is not handled or not allowed 2026-01-24 17:08:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polycarbonate-jar-1414-18-16-oz-capacity-w-almond-top>: HTTP status code is not handled or not allowed 2026-01-24 17:08:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012339-grease-filter-stainless-steel-16-x-20>: HTTP status code is not handled or not allowed 2026-01-24 17:08:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/321930-gh-1640zx-large-spindle-bore-lathe-7-1-2-hp>: HTTP status code is not handled or not allowed 2026-01-24 17:08:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-add-on-rack-72x24x84-with-3-levels-wire-deck-1000-lb-cap-per-level-gray>: HTTP status code is not handled or not allowed 2026-01-24 17:09:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-sleeve-floor-protectors-1-63-diameter-round-8-pack>: HTTP status code is not handled or not allowed 2026-01-24 17:09:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1331571-wheel-kit-2-w-h-ware-shrt-shtl-for-worchester-industrial-products>: HTTP status code is not handled or not allowed 2026-01-24 17:09:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=690177-jvm-836-1-milling-machine-w-acu-rite-200s-dro-1-1-2-hp already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:09:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cable-tie-14-75-lb-8-pk>: HTTP status code is not handled or not allowed 2026-01-24 17:09:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 822 pages (at 94 pages/min), scraped 601 items (at 71 items/min) 2026-01-24 17:09:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-cable-tie-uvb-36-175-lb-10-pk>: HTTP status code is not handled or not allowed 2026-01-24 17:09:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/die-grinder-kit>: HTTP status code is not handled or not allowed 2026-01-24 17:09:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m12-fuel153-cordless-compact-band-saw-kit-12v>: HTTP status code is not handled or not allowed 2026-01-24 17:09:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=690177-jvm-836-1-milling-machine-w-acu-rite-200s-dro-1-1-2-hp landed on page that is not a product page. 2026-01-24 17:09:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slide-card-kit-with-32-assorted-wire-connectors-32-pk>: HTTP status code is not handled or not allowed 2026-01-24 17:09:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/norton-07660789453-metal-small-diameter-cut-off-wheel-4-inch-x-035-inch-x-3-8-inch-60-grit-alum-oxide-type-1>: HTTP status code is not handled or not allowed 2026-01-24 17:09:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/first-aid-only-520-fr-first-responder-kit-large-158-piece-bag>: HTTP status code is not handled or not allowed 2026-01-24 17:09:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/abrasives/sanding_discs/psa_sanding_discs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:09:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ring-terminal-16-14-awg-8-10-stud-blue-15-pk>: HTTP status code is not handled or not allowed 2026-01-24 17:09:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rw30-es1y-general-purpose-rain-bib-pants-hi-viz-lime-m>: HTTP status code is not handled or not allowed 2026-01-24 17:09:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/redco-instacut-3-1-2-corer-dicer-wedger-15026-8-section-core-cut-wall-mount>: HTTP status code is not handled or not allowed 2026-01-24 17:09:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/appliances/refrigerators_freezers/refrigerators/refrigerated_merchandisers_display_cases already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:09:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/abrasives/sanding_discs/psa_sanding_discs landed on page that is not a product page. 2026-01-24 17:09:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/confined-space-umbrella-stand>: HTTP status code is not handled or not allowed 2026-01-24 17:09:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/computers_electronics/audio_video_equipment/cd_players_boomboxes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:09:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/graphic-facility-signs-emergency-eye-wash-plastic-7x7>: HTTP status code is not handled or not allowed 2026-01-24 17:09:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/replacement_shelves_decks/plastic_shelving_additional_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:09:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/task_desk_chairs/task_office_chairs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:09:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1571081-kit-gas-connector-3-4-inch-x36-inch-for-dormont-mfg>: HTTP status code is not handled or not allowed 2026-01-24 17:09:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/appliances/refrigerators_freezers/refrigerators/refrigerated_merchandisers_display_cases landed on page that is not a product page. 2026-01-24 17:09:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-can-type-1-one-gallon-galvanized-steel>: HTTP status code is not handled or not allowed 2026-01-24 17:09:44 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/computers_electronics/audio_video_equipment/cd_players_boomboxes landed on page that is not a product page. 2026-01-24 17:09:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/task_desk_chairs/task_office_chairs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:09:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/266037-shelf-for-glasspro>: HTTP status code is not handled or not allowed 2026-01-24 17:09:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/walls_ceilings/ceiling_tiles already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:09:49 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/task_desk_chairs/task_office_chairs landed on page that is not a product page. 2026-01-24 17:09:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/true-174-ts-49f-hc-reach-freezer-54-13-w-x-29-1-2-d-x-78-38-h>: HTTP status code is not handled or not allowed 2026-01-24 17:09:51 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/replacement_shelves_decks/plastic_shelving_additional_shelves landed on page that is not a product page. 2026-01-24 17:09:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-syle-1-48-3-8-x-24-3-8-pvc-glue-up-tile-in-oil-rubbed-bronze-pg5026>: HTTP status code is not handled or not allowed 2026-01-24 17:09:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011857-service-kit-waveguide-for-turbochef>: HTTP status code is not handled or not allowed 2026-01-24 17:09:56 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/task_desk_chairs/task_office_chairs landed on page that is not a product page. 2026-01-24 17:09:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/walls_ceilings/ceiling_tiles landed on page that is not a product page. 2026-01-24 17:09:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ptp212rd-tough-mark-strip>: HTTP status code is not handled or not allowed 2026-01-24 17:10:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-hotel-luggage-platform-truck>: HTTP status code is not handled or not allowed 2026-01-24 17:10:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kee-klamp-swivel-tee-3>: HTTP status code is not handled or not allowed 2026-01-24 17:10:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kd-12-bushel-canvas-basket-bulk-truck>: HTTP status code is not handled or not allowed 2026-01-24 17:10:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 912 pages (at 90 pages/min), scraped 661 items (at 60 items/min) 2026-01-24 17:10:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tear-down-bench-30x48x29-1-2-black>: HTTP status code is not handled or not allowed 2026-01-24 17:10:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/881081-t2-orifice-cartridgeorifice-3>: HTTP status code is not handled or not allowed 2026-01-24 17:10:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10194945-harness-back-d-ring-quick-connect-leg-straps-standard>: HTTP status code is not handled or not allowed 2026-01-24 17:10:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010608-timer-programmable-for-cleveland-range>: HTTP status code is not handled or not allowed 2026-01-24 17:10:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/wiring_devices/contactors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:10:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/541027-gas-control>: HTTP status code is not handled or not allowed 2026-01-24 17:10:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/impact-xps-153-6-magnetic-insert-bit-holder-matte-nickel-1-4-hex>: HTTP status code is not handled or not allowed 2026-01-24 17:10:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-14-x-42-grey-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 17:10:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lxt-174-4-port-charger-lithium-ion-18v-compact-17-l>: HTTP status code is not handled or not allowed 2026-01-24 17:10:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/wiring_devices/contactors>: HTTP status code is not handled or not allowed 2026-01-24 17:10:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/esd_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:10:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cordless-high-torque-1-2-impact-wrench-kit-5-0ah-18v-lxt-174-li-ion-friction-ring>: HTTP status code is not handled or not allowed 2026-01-24 17:10:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9x12-2-mil-white-block-reclosable-poly-bags>: HTTP status code is not handled or not allowed 2026-01-24 17:10:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergomat-classic-4-x-6>: HTTP status code is not handled or not allowed 2026-01-24 17:10:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-in-bookcase-shelf-34-wide-11-deep-1-high-medium-oak>: HTTP status code is not handled or not allowed 2026-01-24 17:10:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/floor_mats/esd_mats landed on page that is not a product page. 2026-01-24 17:10:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durastripe-supreme-v-6-x-100-orange>: HTTP status code is not handled or not allowed 2026-01-24 17:10:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/boards_easels/enclosed_bulletin_boards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:10:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-boltless-shelving-additional-level-42inw-x-15ind-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 17:10:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1681320-valve-gas-nat-1-2-inch-npt-24v-for-frymaster>: HTTP status code is not handled or not allowed 2026-01-24 17:10:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compact-battery-operated-work-positioner-3>: HTTP status code is not handled or not allowed 2026-01-24 17:10:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/boards_easels/enclosed_bulletin_boards landed on page that is not a product page. 2026-01-24 17:10:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/264621-hinge-kit-for-cres-cor>: HTTP status code is not handled or not allowed 2026-01-24 17:10:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tailgate-strap-combo-pack-slcetsci-with-cam-strap>: HTTP status code is not handled or not allowed 2026-01-24 17:11:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-flame-resistant-coverall-xl-tan-tcgscwn00112xllrg00>: HTTP status code is not handled or not allowed 2026-01-24 17:11:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nonperforated-paper-towel-rolls-8x800ft-white-12-carton>: HTTP status code is not handled or not allowed 2026-01-24 17:11:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/881016-liq-filter-drierdcl163s>: HTTP status code is not handled or not allowed 2026-01-24 17:11:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/shelving_accessories/shelving_enclosure_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:11:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kit2cv11-md10-12-cal-cm2-ultrasoft-arc-flash-kit-with-fr-coverall-md-glove-size-10>: HTTP status code is not handled or not allowed 2026-01-24 17:11:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 1004 pages (at 92 pages/min), scraped 729 items (at 68 items/min) 2026-01-24 17:11:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kit2cv08b2x11-8-cal-cm2-arc-flash-kit-w-fr-coverall-w-balaclava-2xl-glove-size-11>: HTTP status code is not handled or not allowed 2026-01-24 17:11:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mh5006-plus-30-wall-mount-barrier-yellow-banner-closed-for-maintenance>: HTTP status code is not handled or not allowed 2026-01-24 17:11:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/shelving_accessories/shelving_enclosure_kits landed on page that is not a product page. 2026-01-24 17:11:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-add-on-60inw-x-24ind-x-96inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 17:11:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-1-8-npt-double-solenoid-valve-pilot-12v-dc-3w-coil-led-connection>: HTTP status code is not handled or not allowed 2026-01-24 17:11:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/381570-compr-overload-for-ice-o-matic>: HTTP status code is not handled or not allowed 2026-01-24 17:11:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-2-spt-1-white>: HTTP status code is not handled or not allowed 2026-01-24 17:11:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thhn-6-gauge-building-wire-stranded-type-blue>: HTTP status code is not handled or not allowed 2026-01-24 17:11:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-ball-mount-8-inch-tube-2-inch-drop>: HTTP status code is not handled or not allowed 2026-01-24 17:11:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-powered-c-clamp-mount-agitator-2-4-one-half-inch-propeller>: HTTP status code is not handled or not allowed 2026-01-24 17:11:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-add-on-72inw-x-30ind-x-96inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 17:11:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-x-24-kraft-heavy-duty-mailing-tubes-with-caps>: HTTP status code is not handled or not allowed 2026-01-24 17:11:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-x-3-x-60-160-edge-protectors-cased>: HTTP status code is not handled or not allowed 2026-01-24 17:11:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/bar_restaurant_equipment/bartending_supplies/bar_mats_shelf_liners already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:11:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-in-w-x-46-in-h-non-electric-panel-with-raceway-only-gray>: HTTP status code is not handled or not allowed 2026-01-24 17:11:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-add-on-60inw-x-36ind-x-72inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 17:11:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/bar_restaurant_equipment/bartending_supplies/bar_mats_shelf_liners>: HTTP status code is not handled or not allowed 2026-01-24 17:11:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1681557-buzzer-sound-device-w-harness-for-frymaster>: HTTP status code is not handled or not allowed 2026-01-24 17:12:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/881052-dual-pressure-controlauto-auto-36-inch-cap>: HTTP status code is not handled or not allowed 2026-01-24 17:12:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-stacking-bin-16-1-2-x-10-7-8-x-5-beige>: HTTP status code is not handled or not allowed 2026-01-24 17:12:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-flame-resistant-work-shirt-2xl-light-blue-tcgsswn001192xlrg00>: HTTP status code is not handled or not allowed 2026-01-24 17:12:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/outdoor-enclosed-satin-letterboard-96-in-w-x-48-in-h>: HTTP status code is not handled or not allowed 2026-01-24 17:12:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-cal-flame-resistant-coveralls-4x-gray-tcg02150881>: HTTP status code is not handled or not allowed 2026-01-24 17:12:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 1100 pages (at 96 pages/min), scraped 803 items (at 74 items/min) 2026-01-24 17:12:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-hose-double-female-swivel-adapter-379-1521521-1-1-2-nh-x-1-1-2-nh-brass>: HTTP status code is not handled or not allowed 2026-01-24 17:12:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/sign_light_combination_units already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:12:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/contractor-select-green-exit>: HTTP status code is not handled or not allowed 2026-01-24 17:12:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-rolling-laptop-stand-30w-x-16d-x-36h-graphite>: HTTP status code is not handled or not allowed 2026-01-24 17:12:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/264209-door-hingetop-rh-bottom-lh-for-delfield>: HTTP status code is not handled or not allowed 2026-01-24 17:12:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/dunnage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:12:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-heavy-duty-boltless-shelving-starter-36inw-x-18ind-x-72inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 17:12:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stationary-dunnage-rack-48-w-x-24-d-1>: HTTP status code is not handled or not allowed 2026-01-24 17:12:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/sign_light_combination_units landed on page that is not a product page. 2026-01-24 17:12:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-open-g-1-2-single-solenoid-valve-pilot-spr-return-24v-dc-3w-coil-blk-connection>: HTTP status code is not handled or not allowed 2026-01-24 17:12:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silvers-19-oz-45-deluxe-aluminized-coat-2xl-c22nl2xl45>: HTTP status code is not handled or not allowed 2026-01-24 17:12:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/dunnage_racks landed on page that is not a product page. 2026-01-24 17:12:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-58-double-shaft-blower-208-230v-1075rpm-12hp>: HTTP status code is not handled or not allowed 2026-01-24 17:12:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1241427-evaporator-117v-300w-press-sw>: HTTP status code is not handled or not allowed 2026-01-24 17:12:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-18-ga-36-wx18-dx97-h-open-clip-style-6-shelf>: HTTP status code is not handled or not allowed 2026-01-24 17:12:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-starter-36inw-x-12ind-x-84inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 17:12:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-unassembled-deluxe-storage-cabinet-36w-x-24d-x-78-3-4-h-sand>: HTTP status code is not handled or not allowed 2026-01-24 17:12:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cfp1-11um-qualitative-cellulose-filter-paper-4-25cm-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 17:12:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_dishes_plates already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:12:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-work-shirt-4x-tan-tcg01120231>: HTTP status code is not handled or not allowed 2026-01-24 17:12:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plain-magnetic-roll-stock-3-in-50-feet>: HTTP status code is not handled or not allowed 2026-01-24 17:12:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c-thru-deluxe-unassembled-steel-wardrobe-cabinet-36w-x-24d-x-78h-putty>: HTTP status code is not handled or not allowed 2026-01-24 17:12:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pegboard-utility-tool-storage-kit-galvanized-white-32-x-32-x-6>: HTTP status code is not handled or not allowed 2026-01-24 17:13:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pegboard-standard-tool-storage-kit-white-48-x-32-x-9>: HTTP status code is not handled or not allowed 2026-01-24 17:13:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_dishes_plates landed on page that is not a product page. 2026-01-24 17:13:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-starter-72inw-x-48ind-x-72inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 17:13:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-heavy-duty-boltless-shelving-add-on-48inw-x-12ind-x-72inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 17:13:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 1195 pages (at 95 pages/min), scraped 874 items (at 71 items/min) 2026-01-24 17:13:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/264129-bearing-upper-for-dito-dean>: HTTP status code is not handled or not allowed 2026-01-24 17:13:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2561138-cover-conversion-kit-lexan-for-silver-king>: HTTP status code is not handled or not allowed 2026-01-24 17:13:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-pressure-dial-tire-gauge-dual-head-air-chuck-w-12-hose-0-to-120-psi-s-936>: HTTP status code is not handled or not allowed 2026-01-24 17:13:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/card-holder-4-high-2-5-8-weighted-base>: HTTP status code is not handled or not allowed 2026-01-24 17:13:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-371-3-light-pendant-alabaster-bowl-old-bronze-22-5w-x-17-5h>: HTTP status code is not handled or not allowed 2026-01-24 17:13:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alterra-hazelnut-ground-coffee-023-oz-pouch-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 17:13:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-1707-bridgeview-1-light-pendant-mission-dust-bronze-16w-x-15h>: HTTP status code is not handled or not allowed 2026-01-24 17:13:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/decorative_lighting/chandeliers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:13:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-king-sk3000-pallet-rack-108w-x-42d-x-144h-4040-lbs-cap-per-level-add-on-unit>: HTTP status code is not handled or not allowed 2026-01-24 17:13:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/lighted_exit_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:13:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8ab-x-3-8-security-sheet-metal-screw-button-torx-head-302hq-stainless-steel-usa-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 17:13:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nova-eh-20-womens-safety-work-boots-steel-toe-18h-size-105-black>: HTTP status code is not handled or not allowed 2026-01-24 17:13:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/decorative_lighting/chandeliers landed on page that is not a product page. 2026-01-24 17:13:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/badger-neoprene-boots-plain-toe-upper-rubber-sole-steel-shank-12-h-blk-size-12>: HTTP status code is not handled or not allowed 2026-01-24 17:13:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/confined-space-blower-120v-8-inch-dia-canister-15-ft-duct>: HTTP status code is not handled or not allowed 2026-01-24 17:13:42 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/lighted_exit_signs landed on page that is not a product page. 2026-01-24 17:13:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-level-wide-span-rack-72x15-wood-deck-750-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 17:13:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-inch-fnpt-bottom-hole-plated-steel-skimmer-strainer>: HTTP status code is not handled or not allowed 2026-01-24 17:13:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-signs-caution-hard-hat-area-rigid-plastic-10-h-x-14-w>: HTTP status code is not handled or not allowed 2026-01-24 17:13:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-shelving-36x18x96-with-5-shelves-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 17:13:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:13:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/floor_machine_replacement_parts/vacuum_hoses already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:13:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vis-i-rack-vrw85-high-capacity-blueprint-storage-roll-file-wall-rack-8-openings>: HTTP status code is not handled or not allowed 2026-01-24 17:13:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aware-wear-ansi-class-2-economy-mesh-vest-61659-orange>: HTTP status code is not handled or not allowed 2026-01-24 17:14:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-contractor-single-striplight-48-long-4000-lumens>: HTTP status code is not handled or not allowed 2026-01-24 17:14:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-24 17:14:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/floor_machine_replacement_parts/vacuum_hoses landed on page that is not a product page. 2026-01-24 17:14:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-pack-of-letter-legal-file-dividers-black>: HTTP status code is not handled or not allowed 2026-01-24 17:14:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ws10d10a-wall-air-conditioner-9700-btu-cool-115-v>: HTTP status code is not handled or not allowed 2026-01-24 17:14:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 1288 pages (at 93 pages/min), scraped 941 items (at 67 items/min) 2026-01-24 17:14:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-4583-soho-3-light-vanity-fixt-satin-white-glass-polished-chrome-25w-x-7-25h>: HTTP status code is not handled or not allowed 2026-01-24 17:14:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/air_conditioners_chillers/window_air_conditioners already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:14:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-high-legal-starter-empty-black>: HTTP status code is not handled or not allowed 2026-01-24 17:14:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hss-t-shank-4-flute-tmx-core-drill-1-1-16-inch-dia-x-6-5-8-inch-flute-l-x-11-1-4-inch-oal-number-3-mt>: HTTP status code is not handled or not allowed 2026-01-24 17:14:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-8482-12-x-30-super-8-oz-tarp-brown>: HTTP status code is not handled or not allowed 2026-01-24 17:14:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/air_compressors/single_stage_reciprocating_air_compressors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:14:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/air_conditioners_chillers/window_air_conditioners landed on page that is not a product page. 2026-01-24 17:14:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/power_tools/power_saws_blades/power_saw_blades/power_hacksaw_blades already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:14:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-boltless-shelving-add-on-48inw-x-12ind-x-96inh-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 17:14:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-16-w-6-step-steel-rolling-ladder-14-d-top-step>: HTTP status code is not handled or not allowed 2026-01-24 17:14:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-boltless-shelving-add-on-48inw-x-24ind-x-84inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 17:14:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/air_compressors/single_stage_reciprocating_air_compressors landed on page that is not a product page. 2026-01-24 17:14:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/power_tools/power_saws_blades/power_saw_blades/power_hacksaw_blades landed on page that is not a product page. 2026-01-24 17:14:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/266186-leg-36-inch-h-stainless-steel-35-inch-plt>: HTTP status code is not handled or not allowed 2026-01-24 17:14:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lincoln-steel-coupler-1-4-x-1-4-male-npt-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 17:14:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/262842-nut-for-hobart>: HTTP status code is not handled or not allowed 2026-01-24 17:14:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-awg-3-cond-45-ft-15-amp-single-outlet-with-cord-27-lbs>: HTTP status code is not handled or not allowed 2026-01-24 17:14:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n22-32-series-rigid-caster-n32-8-rt-rubber-mold-on-8-dia-500-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 17:14:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-boltless-shelving-starter-48inw-x-12ind-x-72inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 17:14:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/material-handling/casters_wheels/plate_casters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:14:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-1-8-l-x-7-3-8-w-x-3-h-yellow-stacking-hanging-interlocking-polypropylene-bins-6-ct>: HTTP status code is not handled or not allowed 2026-01-24 17:14:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/lockers/cell_phone_box_lockers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:14:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-extinguisher-dry-chemical-10-lb>: HTTP status code is not handled or not allowed 2026-01-24 17:14:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/material-handling/casters_wheels/plate_casters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:14:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-boltless-shelving-add-on-36inw-x-12ind-x-60inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 17:14:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mars-72-in-low-profile-lopro-series-air-curtain-heated-9kw-230-1-60>: HTTP status code is not handled or not allowed 2026-01-24 17:14:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/lockers/cell_phone_box_lockers landed on page that is not a product page. 2026-01-24 17:14:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-style-b-s-hook>: HTTP status code is not handled or not allowed 2026-01-24 17:15:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/material-handling/casters_wheels/plate_casters landed on page that is not a product page. 2026-01-24 17:15:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/material-handling/casters_wheels/plate_casters landed on page that is not a product page. 2026-01-24 17:15:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cyclone-hand-chain-hoist-on-low-headroom-plain-trolley-3-ton-15-ft-lift>: HTTP status code is not handled or not allowed 2026-01-24 17:15:04 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-24 17:15:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lxep4-35hl-dfa-eu-4-led-fixture-5500l-52w-3500k-frosted-acrylic-lens-ip67-dlc>: HTTP status code is not handled or not allowed 2026-01-24 17:15:04 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:1 2026-01-24 17:15:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-uncaged-fixed-access-ladder-yellow-10>: HTTP status code is not handled or not allowed 2026-01-24 17:15:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 1379 pages (at 91 pages/min), scraped 1006 items (at 65 items/min) 2026-01-24 17:15:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/35mr16-gu10-fl-35w-halogen-w-twist-turn-base>: HTTP status code is not handled or not allowed 2026-01-24 17:15:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/plumbing_parts_accessories/shower_parts_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:15:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/bar_restaurant_equipment/restaurant_equipment_replacement_parts_1/steam_table_hot_water_replacement_parts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:15:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010265-water-pump-230v-60hz-for-manitowoc-machines>: HTTP status code is not handled or not allowed 2026-01-24 17:15:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/windsor-nuwave-wave-ivacuum-vacuum-bag>: HTTP status code is not handled or not allowed 2026-01-24 17:15:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25g25-w-25w-incandescent-w-medium-base-130v>: HTTP status code is not handled or not allowed 2026-01-24 17:15:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/outdoor_tools_power_equipment/string_trimmers_edgers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:15:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/plumbing/plumbing_parts_accessories/shower_parts_accessories landed on page that is not a product page. 2026-01-24 17:15:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/bar_restaurant_equipment/restaurant_equipment_replacement_parts_1/steam_table_hot_water_replacement_parts landed on page that is not a product page. 2026-01-24 17:15:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/621148-beverage-and-frothingthermometer>: HTTP status code is not handled or not allowed 2026-01-24 17:15:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-w-diamond-sof-tred-mat-black-yellow-2>: HTTP status code is not handled or not allowed 2026-01-24 17:15:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tuf-r-standard-lldpe-flat-bag-22in-x-36in-1-mil-pkg-qty-500>: HTTP status code is not handled or not allowed 2026-01-24 17:15:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/outdoor_tools_power_equipment/string_trimmers_edgers landed on page that is not a product page. 2026-01-24 17:15:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/d-e-stage-2-pre-filter-for-d-e-defendair-hepa-500>: HTTP status code is not handled or not allowed 2026-01-24 17:15:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:15:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/compact_fluorescent_light_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:15:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ph-hepa-filter-for-ph-390asb>: HTTP status code is not handled or not allowed 2026-01-24 17:15:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/23t2-27-23w-compact-fluorescent-w-medium-base-warm-white>: HTTP status code is not handled or not allowed 2026-01-24 17:15:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/compact_fluorescent_light_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:15:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-height-wobble-stool-swivel-chair-black>: HTTP status code is not handled or not allowed 2026-01-24 17:15:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cft13w-4p-835-13w-compact-fluorescent-w-gx24vq-1-base-neutral-white>: HTTP status code is not handled or not allowed 2026-01-24 17:15:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats landed on page that is not a product page. 2026-01-24 17:15:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/compact_fluorescent_light_bulbs landed on page that is not a product page. 2026-01-24 17:15:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-wall-gaylord-bottom-48l-x-40w-x-48h-kraft-pack-of-5>: HTTP status code is not handled or not allowed 2026-01-24 17:15:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/compact_fluorescent_light_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:15:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/halogen_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:15:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/compact_fluorescent_light_bulbs>: HTTP status code is not handled or not allowed 2026-01-24 17:15:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bung-nut-adapter-for-buttress-threads-2>: HTTP status code is not handled or not allowed 2026-01-24 17:15:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bin-nestable-8-3-8-w-x-17-7-8-d-x-4-h-yellow-2>: HTTP status code is not handled or not allowed 2026-01-24 17:15:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/compact_fluorescent_light_bulbs landed on page that is not a product page. 2026-01-24 17:15:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-hour-task-chair-fabric-black>: HTTP status code is not handled or not allowed 2026-01-24 17:15:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-cardboard-corrugated-boxes-13l-x-11w-x-2h-kraft-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-24 17:15:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/halogen_bulbs landed on page that is not a product page. 2026-01-24 17:15:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/500q-cl-500w-halogen-w-mini-can-base>: HTTP status code is not handled or not allowed 2026-01-24 17:15:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torque-limiting-socket-1-2-drive-19mm-3-4-90-ft-lbs>: HTTP status code is not handled or not allowed 2026-01-24 17:15:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/eye_protection/safety_glasses/half_frame_safety_glasses already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:15:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sb-30-u-6-octagon-sign-base-30-lbs-w-u-channel-adaptor-and-6-u-channel-post>: HTTP status code is not handled or not allowed 2026-01-24 17:16:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/combination_metalworking_machines already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:16:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bin-nestable-13>: HTTP status code is not handled or not allowed 2026-01-24 17:16:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-window-squeegee-18-ss-s-chnl-rbr-blade-10>: HTTP status code is not handled or not allowed 2026-01-24 17:16:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/46-1001-thermostat-type-tb125-temperature-60-250-f-48-inch-capillary>: HTTP status code is not handled or not allowed 2026-01-24 17:16:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/laundry_detergent already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:16:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/power_tools/routers_grinders_sanders/power_grinders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:16:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/eye_protection/safety_glasses/half_frame_safety_glasses landed on page that is not a product page. 2026-01-24 17:16:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/travira-bar-chair-tekwood-natural>: HTTP status code is not handled or not allowed 2026-01-24 17:16:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 1467 pages (at 88 pages/min), scraped 1061 items (at 55 items/min) 2026-01-24 17:16:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/combination_metalworking_machines already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:16:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/combination_metalworking_machines landed on page that is not a product page. 2026-01-24 17:16:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/38-1275-angle-plug-nema-15-50p-3-phase>: HTTP status code is not handled or not allowed 2026-01-24 17:16:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-3-4-thick-x-2-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-24 17:16:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/appliances/ice_machines_equipment/ice_machine_filters_parts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:16:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/laundry_detergent landed on page that is not a product page. 2026-01-24 17:16:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/power_tools/routers_grinders_sanders/power_grinders>: HTTP status code is not handled or not allowed 2026-01-24 17:16:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=dewalt-atomic-20v-max-cordless-2-tool-combo-kit-1-2-brushless-keyless-li-ion-1-5a already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:16:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:16:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/loom-clamps-stainless-steel-2>: HTTP status code is not handled or not allowed 2026-01-24 17:16:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/combination_metalworking_machines landed on page that is not a product page. 2026-01-24 17:16:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:16:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/x5t3-1-4-5w-miniature-sealed-beams-w-mini-wedgebase-24v>: HTTP status code is not handled or not allowed 2026-01-24 17:16:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/cable_management/cable_clamps_clips_hardware already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:16:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/appliances/ice_machines_equipment/ice_machine_filters_parts landed on page that is not a product page. 2026-01-24 17:16:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=dewalt-atomic-20v-max-cordless-2-tool-combo-kit-1-2-brushless-keyless-li-ion-1-5a>: HTTP status code is not handled or not allowed 2026-01-24 17:16:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/cable_management/cable_clamps_clips_hardware already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:16:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40a15-f-40w-incandescent-w-medium-base>: HTTP status code is not handled or not allowed 2026-01-24 17:16:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/katr-recessed-fire-resistant-ceiling-door-stud-22wx30h-katr3022s>: HTTP status code is not handled or not allowed 2026-01-24 17:16:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 17:16:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 17:16:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/cable_management/cable_clamps_clips_hardware already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:16:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-strip-door-curtain-14-w-x-10-h>: HTTP status code is not handled or not allowed 2026-01-24 17:16:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/micro-therm-mini-heat-gun-with-heat-deflector>: HTTP status code is not handled or not allowed 2026-01-24 17:16:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scratch-resistant-strip-door-curtain-6-w-x-8-h>: HTTP status code is not handled or not allowed 2026-01-24 17:16:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/cable_management/cable_clamps_clips_hardware landed on page that is not a product page. 2026-01-24 17:16:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scratch-resistant-strip-door-curtain-8-w-x-8-h>: HTTP status code is not handled or not allowed 2026-01-24 17:16:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/cable_management/cable_clamps_clips_hardware>: HTTP status code is not handled or not allowed 2026-01-24 17:16:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/cable_management/cable_clamps_clips_hardware landed on page that is not a product page. 2026-01-24 17:16:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-pin-compact-fluorescent-light-bulb-gx23-2-base-4100k-13w-cool-white-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 17:16:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stretch-fit-microv-serpentine-drive-belt-gates-k030315sf>: HTTP status code is not handled or not allowed 2026-01-24 17:16:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibco-pneumatic-end-mounted-piston-vibrator-10-5-8s>: HTTP status code is not handled or not allowed 2026-01-24 17:16:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15b9-1-2-f-15w-incandescent-w-candelabra-base>: HTTP status code is not handled or not allowed 2026-01-24 17:16:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11-step-orange-extra-heavy-duty-steel-rolling-ladder-perforated-tread>: HTTP status code is not handled or not allowed 2026-01-24 17:16:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-hybrid-air-service-hose-gates-27584>: HTTP status code is not handled or not allowed 2026-01-24 17:16:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:16:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/13477-water-filter-cartridgecfs9112-for-cuno-inc>: HTTP status code is not handled or not allowed 2026-01-24 17:16:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibco-small-impact-electric-vibrator-sprt-21-230v>: HTTP status code is not handled or not allowed 2026-01-24 17:17:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cross-stack-nest-tote-box-25-1-8x16x8-1-2-gray>: HTTP status code is not handled or not allowed 2026-01-24 17:17:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 17:17:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pld-fluorescent-bulb-gx24q3-base-26w-1800-lumens-3000k-warm-white-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 17:17:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/receptacle-duplex-wall-plate-2-gang-almond-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-24 17:17:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:17:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-sheet-3-4-thick-x-6-wide-x-6-long>: HTTP status code is not handled or not allowed 2026-01-24 17:17:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 1552 pages (at 85 pages/min), scraped 1112 items (at 51 items/min) 2026-01-24 17:17:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/38-1285-locking-plug-nema-l5-20p>: HTTP status code is not handled or not allowed 2026-01-24 17:17:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ushio-esd-jcr120v-150w>: HTTP status code is not handled or not allowed 2026-01-24 17:17:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foam-plates-6-inches-white-round-mediumweight-125pack>: HTTP status code is not handled or not allowed 2026-01-24 17:17:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 17:17:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/840-style-woven-wire-hinge-door-4w-x-7h-10-5-1-4-overall-height>: HTTP status code is not handled or not allowed 2026-01-24 17:17:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/521102-gas-valve1-8-inch-mpt-x-3-8-inch-cc-for-comstock-castle>: HTTP status code is not handled or not allowed 2026-01-24 17:17:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/r134a-brass-fahrenheit-manifold-gauge-set-w-coupler-four-seasons-59198>: HTTP status code is not handled or not allowed 2026-01-24 17:17:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cast-iron-rotary-pump-oil-petroleum-products>: HTTP status code is not handled or not allowed 2026-01-24 17:17:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-g2sc-118-ss>: HTTP status code is not handled or not allowed 2026-01-24 17:17:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:17:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72x36-maple-safety-edge-mobile-power-apron-production-bench-tan-32313651>: HTTP status code is not handled or not allowed 2026-01-24 17:17:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cup-shave-tmb-07dc-circular-7-steel-blade-with-wood-handle-black>: HTTP status code is not handled or not allowed 2026-01-24 17:17:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-a15-light-bulb-e17-base-6w-480-lumens-4000k-cool-white-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-24 17:17:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/clothing_base_layers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:17:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents landed on page that is not a product page. 2026-01-24 17:17:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-ca11-cfc-light-bulb-e12-base-45w-300-lumens-2700k-warm-white-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-24 17:17:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triple-glove-box-holder-stainless-steel-hag16104>: HTTP status code is not handled or not allowed 2026-01-24 17:17:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/led_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:17:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/97-x-24-adjustable-height-5-8-steel-work-platform>: HTTP status code is not handled or not allowed 2026-01-24 17:17:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/clothing_base_layers landed on page that is not a product page. 2026-01-24 17:17:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trapezio-modern-pendant-light-fixture-e26-base-40w-blackbrushed-nickel>: HTTP status code is not handled or not allowed 2026-01-24 17:17:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/180-torsion-spring-0-404-coil-dia-0-048-wire-dia-wound-left-302-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 17:17:49 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/led_bulbs landed on page that is not a product page. 2026-01-24 17:17:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/material-handling/casters_wheels/special_purpose_casters_wheels/furniture_fixture_casters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:17:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-32-32-h3-taper-chamfer-hss-tmx-hand-tap-ground-rh-4-flutes>: HTTP status code is not handled or not allowed 2026-01-24 17:17:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/garage_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:17:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-g1sc-293-b>: HTTP status code is not handled or not allowed 2026-01-24 17:17:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/outdoor_lighting_fixtures/outdoor_sconces already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:17:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/garage_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:17:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ulti-mate-garage-2-0-series-9-piece-cabinet-set-142w-x-21-14d-wall-x-80h-white>: HTTP status code is not handled or not allowed 2026-01-24 17:17:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ses151862-5-hp-oil-less-scroll-compressor-13-gallon-horizontal-116-psi-1-phase-208-230v>: HTTP status code is not handled or not allowed 2026-01-24 17:18:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/material-handling/casters_wheels/special_purpose_casters_wheels/furniture_fixture_casters landed on page that is not a product page. 2026-01-24 17:18:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/styleview-emr-electronic-medical-record-medical-cart-w-lcd-arm-life-powered-sv42-6202-1>: HTTP status code is not handled or not allowed 2026-01-24 17:18:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/cabinets/garage_cabinets landed on page that is not a product page. 2026-01-24 17:18:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/outdoor_lighting_fixtures/outdoor_sconces>: HTTP status code is not handled or not allowed 2026-01-24 17:18:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/cabinets/garage_cabinets landed on page that is not a product page. 2026-01-24 17:18:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-gsc-050-z>: HTTP status code is not handled or not allowed 2026-01-24 17:18:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/fall_protection/portable_frame_fall_protection_systems already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:18:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sts030132-3-hp-oil-less-scroll-compressor-30-gallon-horizontal-116-psi-3-phase-208-230v>: HTTP status code is not handled or not allowed 2026-01-24 17:18:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 1645 pages (at 93 pages/min), scraped 1174 items (at 62 items/min) 2026-01-24 17:18:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-coupling-g2scc-050-050kw>: HTTP status code is not handled or not allowed 2026-01-24 17:18:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-step-steel-stock-picking-ladder-serrated-tread-w-cal-osha-handrail-cal-spl-9-14g>: HTTP status code is not handled or not allowed 2026-01-24 17:18:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-linear-bar-vanity-light-fixture-15w-1200-lumens-18-34l-black>: HTTP status code is not handled or not allowed 2026-01-24 17:18:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-24-w-7-step-steel-rolling-ladder-14-d-top-step-w-handrails-orange-w-cal-osha-handrail-cal-fsh726p-o>: HTTP status code is not handled or not allowed 2026-01-24 17:18:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/fall_protection/portable_frame_fall_protection_systems landed on page that is not a product page. 2026-01-24 17:18:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010173-high-limit-temp-control>: HTTP status code is not handled or not allowed 2026-01-24 17:18:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bca-364213-h-grn-bookcase-all-welded-36-wx13-dx42-h-hunter-green>: HTTP status code is not handled or not allowed 2026-01-24 17:18:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-utility-cart-two-shelves-5-casters-black-500-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-24 17:18:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-valve-5-port-2-pos-man-valve-1-8-nptf-port-black-ext-head-actr-air-spring-ret>: HTTP status code is not handled or not allowed 2026-01-24 17:18:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-step-24-wx90-d-stainless-steel-rolling-safety-ladder-perforated-tread-w-cal-osha-handrail-cal-ss103228p>: HTTP status code is not handled or not allowed 2026-01-24 17:18:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-step-steel-safety-stairway-slope-rolling-ladder-weight-actuated-lock-16-w-expan-step-w-cal-osha-handrail-cal-wa-sw-72414x>: HTTP status code is not handled or not allowed 2026-01-24 17:18:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bussing-tub-20-l-x-15-w-x-5-h-2-compartment-bus-box-polypropylene-gray>: HTTP status code is not handled or not allowed 2026-01-24 17:18:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/educational_materials_books/safety_training_materials already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:18:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zip-a-duct-24in-gray-end-cap>: HTTP status code is not handled or not allowed 2026-01-24 17:18:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/cable_management/cable_ties_wraps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:18:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergotron-neo-flex-lcd-arm>: HTTP status code is not handled or not allowed 2026-01-24 17:18:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rvhl50fr-type-r-class-2-flame-resistant-vest-3xl-hi-vis-lime>: HTTP status code is not handled or not allowed 2026-01-24 17:18:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/educational_materials_books/safety_training_materials>: HTTP status code is not handled or not allowed 2026-01-24 17:18:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/cable_management/cable_ties_wraps>: HTTP status code is not handled or not allowed 2026-01-24 17:18:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trim-bit-shk-wbb-flt-ct-85441m>: HTTP status code is not handled or not allowed 2026-01-24 17:18:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss-hose-braid-w-male-npt-nipple-0-75-x-18>: HTTP status code is not handled or not allowed 2026-01-24 17:18:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-folding-eercise-peddler-with-electronic-display>: HTTP status code is not handled or not allowed 2026-01-24 17:19:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/square-electrodes-for-tens-unit-replacement-electrode-pads>: HTTP status code is not handled or not allowed 2026-01-24 17:19:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fs-42-ts-storage-cabinet-all-welded-42-wx18-dx72-h-tropic-sand>: HTTP status code is not handled or not allowed 2026-01-24 17:19:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/900-mhz-stubby-antenna-for-use-with-dtr600-and-dtr700-portable-radios>: HTTP status code is not handled or not allowed 2026-01-24 17:19:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 1743 pages (at 98 pages/min), scraped 1248 items (at 74 items/min) 2026-01-24 17:19:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-16-w-5-step-steel-rolling-ladder-w-21-d-top-step-w-handrails-orange-w-cal-osha-handrail-cal-fsh51821p-o>: HTTP status code is not handled or not allowed 2026-01-24 17:19:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s201023-20-poly-web-sling-endless-6720-8400-16800-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-24 17:19:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s201001-4-poly-web-sling-endless-2120-2650-5300-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-24 17:19:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forney-tweco-style-mig-welding-nozzle-for-85341-size-12>: HTTP status code is not handled or not allowed 2026-01-24 17:19:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-squeegee-front-for-ice-8118021>: HTTP status code is not handled or not allowed 2026-01-24 17:19:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:19:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/emergency_eyewash_stations_showers/emergency_drench_showers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:19:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-aligning-steel-closed-linear-bearing-wresin-retainer-and-seals-sw6gruu-38id-0875l>: HTTP status code is not handled or not allowed 2026-01-24 17:19:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnetic-label-holder-6-w-x-4-h-50-per-package>: HTTP status code is not handled or not allowed 2026-01-24 17:19:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b151101-8-double-leg-chain-sling-w-grab-hook>: HTTP status code is not handled or not allowed 2026-01-24 17:19:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs>: HTTP status code is not handled or not allowed 2026-01-24 17:19:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/emergency_eyewash_stations_showers/emergency_drench_showers landed on page that is not a product page. 2026-01-24 17:19:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-dunnage-rack-24-inch-w-60-inch-l-12-inch-h>: HTTP status code is not handled or not allowed 2026-01-24 17:19:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-tier-6-door-clear-view-locker-digital-lock12-inch-w-x-12-inch-d-x-78-inch-hgray-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 17:19:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1u-flat-cable-lacing-panel-1903-1-001-01>: HTTP status code is not handled or not allowed 2026-01-24 17:19:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b151042-4-double-leg-chain-sling-w-grab-hook>: HTTP status code is not handled or not allowed 2026-01-24 17:19:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-signs-notice-cylinders-must-be-chained-vinyl-7-h-x-10-w>: HTTP status code is not handled or not allowed 2026-01-24 17:19:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b151098-3-double-leg-chain-sling-w-grab-hook>: HTTP status code is not handled or not allowed 2026-01-24 17:19:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/digital-door-lock-m210-mechanical-keyless-deadbolt-marine-grade>: HTTP status code is not handled or not allowed 2026-01-24 17:19:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leverset-single-step-roses-entry-lock-oil-bronze-clutch>: HTTP status code is not handled or not allowed 2026-01-24 17:19:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-climax-2c-056-s>: HTTP status code is not handled or not allowed 2026-01-24 17:19:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gn63363-hand-knob-26102-8433-m8-x-125-8nx44>: HTTP status code is not handled or not allowed 2026-01-24 17:19:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-skirt-set-apex-for-nilfisk-advance-56510374>: HTTP status code is not handled or not allowed 2026-01-24 17:19:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/three-sided-frame-24x48-nexelon-1>: HTTP status code is not handled or not allowed 2026-01-24 17:20:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dunnage-shelf-24x30-nexelon>: HTTP status code is not handled or not allowed 2026-01-24 17:20:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/letter-size-21-compartment-steel-literature-sorter-putty>: HTTP status code is not handled or not allowed 2026-01-24 17:20:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/desk-accessory-return-4>: HTTP status code is not handled or not allowed 2026-01-24 17:20:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 1838 pages (at 95 pages/min), scraped 1319 items (at 71 items/min) 2026-01-24 17:20:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/157-tn-3-32-hss-general-purpose-tin-118-point-screw-machine-drill>: HTTP status code is not handled or not allowed 2026-01-24 17:20:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-flame-resistant-heavyweight-zip-up-hoodie-l-hi-vis-yellow>: HTTP status code is not handled or not allowed 2026-01-24 17:20:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/19130-3130s-coiled-cable-lanyard-2-lbs-max-lime-polyurethane>: HTTP status code is not handled or not allowed 2026-01-24 17:20:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-x-10-x-9-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 17:20:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=slotted-cap-with-fas-trac-iii-suspension-hi-viz-yellow already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:20:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-climax-1c-100-s>: HTTP status code is not handled or not allowed 2026-01-24 17:20:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pl-57w-cover-for-pl-5w>: HTTP status code is not handled or not allowed 2026-01-24 17:20:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-x-8-x-8-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 17:20:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=slotted-cap-with-fas-trac-iii-suspension-hi-viz-yellow landed on page that is not a product page. 2026-01-24 17:20:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/polymer_bathroom_partitions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:20:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plw-7w-premium-dish-box>: HTTP status code is not handled or not allowed 2026-01-24 17:20:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metal-scrubbing-pads-2-12-x-2-34-copper-coated-metal-3pk-8-pksct>: HTTP status code is not handled or not allowed 2026-01-24 17:20:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cyclone-8482-water-removal-utility-transfer-pump-ac-operation>: HTTP status code is not handled or not allowed 2026-01-24 17:20:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/17-1-4-x-11-1-4-x-7-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 17:20:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/polymer_bathroom_partitions landed on page that is not a product page. 2026-01-24 17:20:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-x-16-x-8-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 17:20:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rfm-2b-fluted-ramekins-2-oz-melamine-bone>: HTTP status code is not handled or not allowed 2026-01-24 17:20:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-small-bottle-holder-holds-3-12-diameter-bottle-5w-x-5-14d-x-5-12h-clear>: HTTP status code is not handled or not allowed 2026-01-24 17:20:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-linen-le-phenolic-sheet-0-015-x-36-000-x-48-000>: HTTP status code is not handled or not allowed 2026-01-24 17:20:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/phenolic-black-core-pilaster-w-shoe-24w-x-82h-almond>: HTTP status code is not handled or not allowed 2026-01-24 17:20:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/phenolic_bathroom_partitions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:20:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-press-fit-fluid-level-sight-glass-w-o-indicator-fits-20mm-79-bore-rd1-a>: HTTP status code is not handled or not allowed 2026-01-24 17:20:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-makrofol-de1-1-g-g-pc-film-0-020-x-24-x-48-50-sheet-case>: HTTP status code is not handled or not allowed 2026-01-24 17:20:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelf-24wx24dx76h-gray-all-welded>: HTTP status code is not handled or not allowed 2026-01-24 17:20:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/phenolic_bathroom_partitions>: HTTP status code is not handled or not allowed 2026-01-24 17:21:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/19146-brow-pad-replacement-blue>: HTTP status code is not handled or not allowed 2026-01-24 17:21:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/patriot-fully-framed-gear-locker-w-1-coat-rod-48wx24dx72h-red-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 17:21:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ice-o-matic-b110ps-ice-cube-storage-bin-48-wx31-dx50-h-854-lbs-storage-cap-top-mount-ice-maker>: HTTP status code is not handled or not allowed 2026-01-24 17:21:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 1936 pages (at 98 pages/min), scraped 1392 items (at 73 items/min) 2026-01-24 17:21:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-1-2-x-30-kra-crimped-end-tubes-case>: HTTP status code is not handled or not allowed 2026-01-24 17:21:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vx-072-0023rp-extended-tank-drain-assembly-300-psi>: HTTP status code is not handled or not allowed 2026-01-24 17:21:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/100-waterproof-window-door-trim-sealant-10-1-oz-brown-7079818418>: HTTP status code is not handled or not allowed 2026-01-24 17:21:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stretch-wrap-film-for-hand-dispenser-blown-70-gauge-12w-x-1500l-clear-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-24 17:21:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-paper-phenolic-sheet-1-000-thick-48-x-96>: HTTP status code is not handled or not allowed 2026-01-24 17:21:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qf-ball-joint-8-dia-galvanized-steel>: HTTP status code is not handled or not allowed 2026-01-24 17:21:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-grate-guard-with-curb-deflector-sediment-high-flow-18l-x-36w-x-4h>: HTTP status code is not handled or not allowed 2026-01-24 17:21:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-nova-3-talk-side-padding-covers>: HTTP status code is not handled or not allowed 2026-01-24 17:21:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v-o-c-free-glass-cleaner-concentrate-neutral-scent-gallon-bottle-2-case-gs006-g2>: HTTP status code is not handled or not allowed 2026-01-24 17:21:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/doral-task-chair-mid-back-articulating-tilt-26-31h-seat-black>: HTTP status code is not handled or not allowed 2026-01-24 17:21:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_tools/cleaning_supply_dispensers_organizers/chemical_dispensers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:21:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/45rd-stainless-steel-scoop-64-oz>: HTTP status code is not handled or not allowed 2026-01-24 17:21:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/lockers/cell_phone_box_lockers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:21:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-couplings-recessed-screw-cc-series-climax-2cc-137-100>: HTTP status code is not handled or not allowed 2026-01-24 17:21:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/gas_engines already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:21:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/132c-ice-logger-boot-regular-tan-7-5>: HTTP status code is not handled or not allowed 2026-01-24 17:21:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-one-piece-clamping-collar-climax-m1c-34>: HTTP status code is not handled or not allowed 2026-01-24 17:21:49 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_tools/cleaning_supply_dispensers_organizers/chemical_dispensers landed on page that is not a product page. 2026-01-24 17:21:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:21:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0359-iron-tuff-arctic-jacket-regular-navy-small>: HTTP status code is not handled or not allowed 2026-01-24 17:21:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/lockers/cell_phone_box_lockers landed on page that is not a product page. 2026-01-24 17:21:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-6-2-inhalation-hazard-adhesive-vinyl-placard>: HTTP status code is not handled or not allowed 2026-01-24 17:21:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/gas_engines landed on page that is not a product page. 2026-01-24 17:21:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-minute-cylinder-threaded-remote-connection-aluminum-less-air>: HTTP status code is not handled or not allowed 2026-01-24 17:21:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coveralls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:21:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats>: HTTP status code is not handled or not allowed 2026-01-24 17:22:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:22:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-radex-co-monitor-outlet-filter>: HTTP status code is not handled or not allowed 2026-01-24 17:22:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96-l-x-30-w-14-gauge-stainless-workbench-w-undershelf-le-sink-2>: HTTP status code is not handled or not allowed 2026-01-24 17:22:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/retail_fixtures_displays_equipment/displays_display_racks/display_trays already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:22:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jet-flux-chipper-lever-lock-r8-series-4600-bpm-93-psi-7-cfm>: HTTP status code is not handled or not allowed 2026-01-24 17:22:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coveralls landed on page that is not a product page. 2026-01-24 17:22:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 2027 pages (at 91 pages/min), scraped 1457 items (at 65 items/min) 2026-01-24 17:22:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats>: HTTP status code is not handled or not allowed 2026-01-24 17:22:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:22:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/tables/folding_tables already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:22:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dxs2153-metric-standard-impact-socket-30mm-drive-black>: HTTP status code is not handled or not allowed 2026-01-24 17:22:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/retail_fixtures_displays_equipment/displays_display_racks/display_trays landed on page that is not a product page. 2026-01-24 17:22:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wheel-axle-size-1-2-9>: HTTP status code is not handled or not allowed 2026-01-24 17:22:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-one-piece-clamping-collar-climax-m1c-70-s>: HTTP status code is not handled or not allowed 2026-01-24 17:22:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-z-link-pivot-assembly>: HTTP status code is not handled or not allowed 2026-01-24 17:22:21 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats landed on page that is not a product page. 2026-01-24 17:22:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lifting-socket-base-for-2500-lb-steel-davit-crane>: HTTP status code is not handled or not allowed 2026-01-24 17:22:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/tables/folding_tables landed on page that is not a product page. 2026-01-24 17:22:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pesc-2-square-storage-container-2-qt-white-polycarbonate>: HTTP status code is not handled or not allowed 2026-01-24 17:22:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cj-7p-condiment-jar-7-oz-plastic>: HTTP status code is not handled or not allowed 2026-01-24 17:22:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-2-shelf-tray-service-utility-cart-40-x-26-5-rubber-casters>: HTTP status code is not handled or not allowed 2026-01-24 17:22:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-climax-c-225-s>: HTTP status code is not handled or not allowed 2026-01-24 17:22:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/maxflow-life-vest-mesh-uscgtc-type-iii-2xl-orange>: HTTP status code is not handled or not allowed 2026-01-24 17:22:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0401-heavyweight-knit-liner-black-s-m>: HTTP status code is not handled or not allowed 2026-01-24 17:22:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-recessed-screw-climax-h2c-093-s>: HTTP status code is not handled or not allowed 2026-01-24 17:22:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-round-picnic-table-polished-gray-limestone-top-tan-river-rock-leg>: HTTP status code is not handled or not allowed 2026-01-24 17:22:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/occumitts-support-gloves-1-pair-x-large-biege>: HTTP status code is not handled or not allowed 2026-01-24 17:22:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plb11ss-universal-bracket-kit>: HTTP status code is not handled or not allowed 2026-01-24 17:22:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fogarty-cameter-bracket-chrome>: HTTP status code is not handled or not allowed 2026-01-24 17:22:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-recessed-screw-climax-h2c-037>: HTTP status code is not handled or not allowed 2026-01-24 17:22:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:22:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-square-platter-led-flush-mount-light-30w-120v-3000k-brush-nickle>: HTTP status code is not handled or not allowed 2026-01-24 17:22:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/power_tools/power_saws_blades/band_saws/band_saw_machines already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:23:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:23:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/lathes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:23:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1457115-swenson-conveyor-chain-04043-075-09>: HTTP status code is not handled or not allowed 2026-01-24 17:23:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/lathes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:23:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-climax-c-062-a>: HTTP status code is not handled or not allowed 2026-01-24 17:23:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:23:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 17:23:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/power_tools/power_saws_blades/band_saws/band_saw_machines>: HTTP status code is not handled or not allowed 2026-01-24 17:23:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 2118 pages (at 91 pages/min), scraped 1523 items (at 66 items/min) 2026-01-24 17:23:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 17:23:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/lathes landed on page that is not a product page. 2026-01-24 17:23:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-spring-rewind-hose-reel-for-air-water-oil-34-i-d-35-hose-1250-psi>: HTTP status code is not handled or not allowed 2026-01-24 17:23:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/lathes landed on page that is not a product page. 2026-01-24 17:23:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyurethane-strip-no-adhesive-60a-116-thick-x-14-wide-x-10-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 17:23:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 17:23:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-steel-drawer-for-rubbermaid-trademaster-carts->: HTTP status code is not handled or not allowed 2026-01-24 17:23:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vinyl-replacement-liner-24-bushel-red-2>: HTTP status code is not handled or not allowed 2026-01-24 17:23:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/bulk_container_trucks_tilt_trucks/box_trucks/plastic_box_trucks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:23:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fiberglass-tote-box-20-1-2x12-7-8x8-white>: HTTP status code is not handled or not allowed 2026-01-24 17:23:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:23:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g-450-galvanized-clip-drop-forged-3-16>: HTTP status code is not handled or not allowed 2026-01-24 17:23:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:23:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:23:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:23:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-roll-no-adhesive-60a-14-thick-x-36-wide-x-4-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 17:23:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/bulk_container_trucks_tilt_trucks/box_trucks/plastic_box_trucks landed on page that is not a product page. 2026-01-24 17:23:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_seating/bleachers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:23:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-row-national-rep-aluminum-bleacher-21-wide-single-footboard>: HTTP status code is not handled or not allowed 2026-01-24 17:23:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 17:23:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets>: HTTP status code is not handled or not allowed 2026-01-24 17:23:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips landed on page that is not a product page. 2026-01-24 17:23:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/bins_totes_containers/baskets_organizers/closet_garment_organizers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:23:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-clamp-for-2-hose-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-24 17:23:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_seating/bleachers landed on page that is not a product page. 2026-01-24 17:23:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-of-boxed-serrated-jaw-inserts-with-4-screws-for-17801780a-vise>: HTTP status code is not handled or not allowed 2026-01-24 17:23:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/l-320cn-s-c-carbon-hook-eye-w-latch-3t-wll>: HTTP status code is not handled or not allowed 2026-01-24 17:23:45 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 17:23:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/riser-for-all-terrain-vise153-10>: HTTP status code is not handled or not allowed 2026-01-24 17:23:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/binders_covers/binders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:23:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/bins_totes_containers/baskets_organizers/closet_garment_organizers>: HTTP status code is not handled or not allowed 2026-01-24 17:23:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:23:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-d-ring-vinyl-view-binder-5-capacity-black>: HTTP status code is not handled or not allowed 2026-01-24 17:23:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:23:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/hand_tools_1/hammers_mallets/ball_peen_cross_peen_hammers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:23:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/din-508-t-slot-nuts-steel-blackened-m12-7-8-w-x-5-8-h>: HTTP status code is not handled or not allowed 2026-01-24 17:23:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s-4055-latch-kit-1-5-2t-carbon-3t-alloy-hook-1-84-l-x-0-82-w>: HTTP status code is not handled or not allowed 2026-01-24 17:23:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/hand_tools_1/hammers_mallets/dead_blow_hammers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:24:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:24:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gd-12251-garage-door-cable-clamps-116-galvanizedpack-of-2>: HTTP status code is not handled or not allowed 2026-01-24 17:24:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 17:24:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/binders_covers/binders landed on page that is not a product page. 2026-01-24 17:24:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 17:24:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n-6936-bi-fold-door-hinge-brass-platedpack-of-2>: HTTP status code is not handled or not allowed 2026-01-24 17:24:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/electric_motor_parts_accessories/motor_brakes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:24:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n-6504-closet-door-roller-14-inch-offset-with-78-inch-nylon-wheelpack-of-2>: HTTP status code is not handled or not allowed 2026-01-24 17:24:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/hand_tools_1/hammers_mallets/ball_peen_cross_peen_hammers landed on page that is not a product page. 2026-01-24 17:24:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-strength-semiclear-fda-silicone-rubber-sheet-no-adhesive-50a-132-thick-x-36-w-x-24-l>: HTTP status code is not handled or not allowed 2026-01-24 17:24:09 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/hand_tools_1/hammers_mallets/dead_blow_hammers landed on page that is not a product page. 2026-01-24 17:24:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 2202 pages (at 84 pages/min), scraped 1572 items (at 49 items/min) 2026-01-24 17:24:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delfin-64-longopac-hepa-vacuum-2-7-hp-mistral-202-ds-lp>: HTTP status code is not handled or not allowed 2026-01-24 17:24:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:24:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/static-conductive-air-hose-assembly-38-id-x-25l-blue>: HTTP status code is not handled or not allowed 2026-01-24 17:24:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips landed on page that is not a product page. 2026-01-24 17:24:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=combination-8-bench-grinder-4-belt-grinder-single-phase-115v-igbb-436 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:24:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/electric_motor_parts_accessories/motor_brakes landed on page that is not a product page. 2026-01-24 17:24:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hydraulic-box-and-pan-brake-5-hp-3-phase-220v-bb-7210h>: HTTP status code is not handled or not allowed 2026-01-24 17:24:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:24:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paramount-locker-double-tier-12x18x36-6-door-blue>: HTTP status code is not handled or not allowed 2026-01-24 17:24:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets>: HTTP status code is not handled or not allowed 2026-01-24 17:24:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/binders_covers/binders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:24:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:24:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=combination-8-bench-grinder-4-belt-grinder-single-phase-115v-igbb-436>: HTTP status code is not handled or not allowed 2026-01-24 17:24:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/side-mount-12-v-aluminum-mount-system-9-1-2-ft-x-8-1-3-ft-5544396>: HTTP status code is not handled or not allowed 2026-01-24 17:24:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 17:24:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-table-cross-base-36w-x-29-1-2h-dark-cherry>: HTTP status code is not handled or not allowed 2026-01-24 17:24:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/packaging/envelopes_mailers/bubble_mailers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:24:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets>: HTTP status code is not handled or not allowed 2026-01-24 17:24:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/packaging/corrugated_boxes_cartons/corrugated_boxes/specialized_moving_boxes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:24:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-strip-no-adhesive-70a-116-thick-x-6-wide-x-5-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 17:24:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/binders_covers/binders landed on page that is not a product page. 2026-01-24 17:24:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/liquid-propane-blue-flame-vent-free-heater-10000-btu-electric>: HTTP status code is not handled or not allowed 2026-01-24 17:24:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=manually-operated-ironworker-10-w-4-stations already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:24:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/walls_ceilings/wall_tiles already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:24:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/heaters/patio_heaters/gas_patio_heaters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:24:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/packaging/envelopes_mailers/bubble_mailers>: HTTP status code is not handled or not allowed 2026-01-24 17:24:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quickpick-double-hopper-bin-9-1-4x14x6-1-2-red>: HTTP status code is not handled or not allowed 2026-01-24 17:24:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11063w-8-24-awg-solid-wire-strippercutter>: HTTP status code is not handled or not allowed 2026-01-24 17:24:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/packaging/corrugated_boxes_cartons/corrugated_boxes/specialized_moving_boxes landed on page that is not a product page. 2026-01-24 17:24:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-roll-no-adhesive-50a-38-thick-x-36-wide-x-10-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 17:24:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=manually-operated-ironworker-10-w-4-stations landed on page that is not a product page. 2026-01-24 17:24:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/walls_ceilings/wall_tiles landed on page that is not a product page. 2026-01-24 17:24:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/heaters/patio_heaters/gas_patio_heaters>: HTTP status code is not handled or not allowed 2026-01-24 17:24:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-high-security-steel-solid-body-padlocks-no-7030kalf>: HTTP status code is not handled or not allowed 2026-01-24 17:24:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:24:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v70529-class-2-highvis-5point-breakaway-vest-fluorescent-orange-2xl3xl>: HTTP status code is not handled or not allowed 2026-01-24 17:24:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:24:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-shelving-closed-shelving-36x12x75-15>: HTTP status code is not handled or not allowed 2026-01-24 17:24:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:24:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-end-platform-truck-16-x-72>: HTTP status code is not handled or not allowed 2026-01-24 17:25:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:25:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hytrol-10-ft-36-inch-w-1-9-inch-dia-galvanized-rollers-33-inch-between-rail-6-inch-rollers-centers>: HTTP status code is not handled or not allowed 2026-01-24 17:25:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 17:25:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-part-carbonless-inventory-tag-3000-3499-8-6-14l-x-3-18w-500pack>: HTTP status code is not handled or not allowed 2026-01-24 17:25:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips landed on page that is not a product page. 2026-01-24 17:25:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paramount-locker-single-tier-12x18x60-1-door-assembled-tan>: HTTP status code is not handled or not allowed 2026-01-24 17:25:09 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-24 17:25:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 2285 pages (at 83 pages/min), scraped 1622 items (at 50 items/min) 2026-01-24 17:25:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips>: HTTP status code is not handled or not allowed 2026-01-24 17:25:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-framed-letter-board-message-center-24w-30h>: HTTP status code is not handled or not allowed 2026-01-24 17:25:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:25:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-af-blend-sheet-0-750-thick-x-12-w-x-24-l>: HTTP status code is not handled or not allowed 2026-01-24 17:25:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-raised-circular-stair-tread-square-nose-204375-x-60-black>: HTTP status code is not handled or not allowed 2026-01-24 17:25:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vented-plastic-shelving-36x18x63-nexelon-finish>: HTTP status code is not handled or not allowed 2026-01-24 17:25:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-af-blend-sheet-2-000-thick-x-24-w-x-24-l>: HTTP status code is not handled or not allowed 2026-01-24 17:25:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 17:25:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merit-ss-316-barstock-hex-head-plug-18-npt-male>: HTTP status code is not handled or not allowed 2026-01-24 17:25:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:25:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silencer-gp-band-saw-blade-1-4-inch-w-035-thick-gauge-10-14-tpi-303-010155>: HTTP status code is not handled or not allowed 2026-01-24 17:25:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:25:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merit-ss-316-barstock-coupling-1-npt-female>: HTTP status code is not handled or not allowed 2026-01-24 17:25:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:25:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0402-m10x1-5-d6-4-flute-bright-plug-chamfer-hand-tap>: HTTP status code is not handled or not allowed 2026-01-24 17:25:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:25:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/abrasives/sanding_discs/flap_sanding_discs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:25:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:25:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls>: HTTP status code is not handled or not allowed 2026-01-24 17:25:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:25:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls>: HTTP status code is not handled or not allowed 2026-01-24 17:25:42 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-24 17:25:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-roll-no-adhesive-60a-316-thick-x-36-wide-x-9-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 17:25:45 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 17:25:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-shelf-bench-rack-with-64-bins-8-wide-2>: HTTP status code is not handled or not allowed 2026-01-24 17:25:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 17:25:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/abrasives/flap_specialty_wheels/flap_wheels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:25:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/abrasives/sanding_discs/flap_sanding_discs landed on page that is not a product page. 2026-01-24 17:25:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gn111-4-spiral-retaining-cables-plastic-2-key-rings-3-94-l-0-71-key-ring-dia>: HTTP status code is not handled or not allowed 2026-01-24 17:25:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 17:25:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1730b07-cross-over-bridge-35-overall-width-7-stairs>: HTTP status code is not handled or not allowed 2026-01-24 17:25:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:25:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:25:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-ketron-peek-rod-q-1-750-dia>: HTTP status code is not handled or not allowed 2026-01-24 17:25:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/power_tools/power_sanders/abrasive_tool_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:25:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tyvek174-shipping-tag-4-34l-x-2-38w-white-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 17:25:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/abrasives/flap_specialty_wheels/flap_wheels landed on page that is not a product page. 2026-01-24 17:25:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/414a-cyan-original-laserjet-toner-cartridge-2100-page-yield>: HTTP status code is not handled or not allowed 2026-01-24 17:26:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3m-s-b-6x1x1-9sfin048011-05132>: HTTP status code is not handled or not allowed 2026-01-24 17:26:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 17:26:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/power_tools/power_sanders/abrasive_tool_accessories>: HTTP status code is not handled or not allowed 2026-01-24 17:26:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 17:26:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trane-compatible-zone-sensors-model-sp155027>: HTTP status code is not handled or not allowed 2026-01-24 17:26:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:26:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7w-12v-warm-white-flood-led-downlight-bronze-texture>: HTTP status code is not handled or not allowed 2026-01-24 17:26:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 2366 pages (at 81 pages/min), scraped 1671 items (at 49 items/min) 2026-01-24 17:26:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tyvek174-shipping-tag-pre-wired-4-14l-x-2-18w-blue-100pack>: HTTP status code is not handled or not allowed 2026-01-24 17:26:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:26:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tyvek174-shipping-tag-5-34l-x-2-78w-yellow-100pack>: HTTP status code is not handled or not allowed 2026-01-24 17:26:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets>: HTTP status code is not handled or not allowed 2026-01-24 17:26:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-strength-neoprene-rubber-strip-with-acrylic-adhesive-60a-116-thick-x-34-w-x-10-ft-l>: HTTP status code is not handled or not allowed 2026-01-24 17:26:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:26:21 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-24 17:26:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-sheet-no-adhesive-50a-14-thick-x-36-wide-x-36-long>: HTTP status code is not handled or not allowed 2026-01-24 17:26:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mb921b-neoprene-steel-toe-snugleg-boots-brick-redbrown-size-15>: HTTP status code is not handled or not allowed 2026-01-24 17:26:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-30-glass-filled-cast-nylon-sheet-1-500-thick-x-24-000-w-x-48-000-l>: HTTP status code is not handled or not allowed 2026-01-24 17:26:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-24 17:26:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-fep-tube-0-170-id-x-250-od-x-50-l>: HTTP status code is not handled or not allowed 2026-01-24 17:26:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/r531007-r-baseline-twinleg-self-retracting-lifeline-7-feet-long-web-with-aluminum-rebar>: HTTP status code is not handled or not allowed 2026-01-24 17:26:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cubby-locker-welded-34-1-2w-x-13-1-2d-x-40h-9-compartments-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 17:26:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gpo-3-u-channel-1166-0-250-thick-x-3-w-x-0-875-leg-x-10-l>: HTTP status code is not handled or not allowed 2026-01-24 17:26:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fs-floor-scrub-food-industry-floor-cleaner-5-gallon-pail-1-pail>: HTTP status code is not handled or not allowed 2026-01-24 17:26:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/906-stm26x1-5-a-steel-threaded-plug-with-m26-x-1-5-tapered-thread>: HTTP status code is not handled or not allowed 2026-01-24 17:26:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/256-flatback-tape-2-x-60-yds-67-mil-white-12pack>: HTTP status code is not handled or not allowed 2026-01-24 17:26:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/walls_ceilings/ceiling_tiles already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:26:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reverse-tuck-folding-cartons-5-58l-x-2-12w-x-5-58h-white-250pack>: HTTP status code is not handled or not allowed 2026-01-24 17:26:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g5-116t-aff-7-5hp-oil-injected-rotary-screw-71-gal-tank-116psi-208-230-460v-w-dryer>: HTTP status code is not handled or not allowed 2026-01-24 17:26:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/outdoor_tools_power_equipment/pressure_washers/electric_pressure_washers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:26:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-z-brite-wire-shelving-36wx14dx54h>: HTTP status code is not handled or not allowed 2026-01-24 17:26:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/libman-commercial-industrial-grade-utility-towels-12-pack>: HTTP status code is not handled or not allowed 2026-01-24 17:26:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/supreme-sliptech-mat-11-16-thick-3-w-cut-length-to-60-black-w-yellow-border>: HTTP status code is not handled or not allowed 2026-01-24 17:27:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/walls_ceilings/ceiling_tiles landed on page that is not a product page. 2026-01-24 17:27:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/libman-commercial-heavy-duty-kitchen-brush>: HTTP status code is not handled or not allowed 2026-01-24 17:27:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/outdoor_tools_power_equipment/pressure_washers/electric_pressure_washers landed on page that is not a product page. 2026-01-24 17:27:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7020m-washer-m16-5507651>: HTTP status code is not handled or not allowed 2026-01-24 17:27:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/wire_shelving/poly_zbrite_wire_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:27:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 2465 pages (at 99 pages/min), scraped 1744 items (at 73 items/min) 2026-01-24 17:27:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/great-lakes-tin-rochester-2-x-2-lay-in-tin-ceiling-tile-in-bronze-burst-y58-06>: HTTP status code is not handled or not allowed 2026-01-24 17:27:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bntl1818-18-x-18-exposed-flange-key-operated-lock>: HTTP status code is not handled or not allowed 2026-01-24 17:27:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stackable-steel-container-solid-48l-x-42w-x-18h-black>: HTTP status code is not handled or not allowed 2026-01-24 17:27:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/wire_shelving/poly_zbrite_wire_shelves landed on page that is not a product page. 2026-01-24 17:27:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bolt-1230r-5235751>: HTTP status code is not handled or not allowed 2026-01-24 17:27:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trayp524s-spillage-tray-for-24-w-flow-series-workstation>: HTTP status code is not handled or not allowed 2026-01-24 17:27:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/laundry-sanitizer-liquid-crisp-linen-90-oz-4-case>: HTTP status code is not handled or not allowed 2026-01-24 17:27:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/welding/arc_welding/mig_welding/mig_wire_feeders_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:27:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/laundry_detergent already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:27:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-weatherly-backless-bench-eco-friendly-synthetic-wood-in-white>: HTTP status code is not handled or not allowed 2026-01-24 17:27:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/embassy-kcc-r-right-end-cap>: HTTP status code is not handled or not allowed 2026-01-24 17:27:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chemical-resistant-trigger-sprayer-with-7-1-4-tube-gray-24-sprayers>: HTTP status code is not handled or not allowed 2026-01-24 17:27:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/laundry_detergent>: HTTP status code is not handled or not allowed 2026-01-24 17:27:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/welding/arc_welding/mig_welding/mig_wire_feeders_accessories landed on page that is not a product page. 2026-01-24 17:27:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-storage-drawer-cabinet-36x24x32-1>: HTTP status code is not handled or not allowed 2026-01-24 17:27:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-sintered-muffler>: HTTP status code is not handled or not allowed 2026-01-24 17:27:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-muffler-1-2-male>: HTTP status code is not handled or not allowed 2026-01-24 17:27:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/laminated-polypropylene-shoe-covers-with-non-skid-aq-sole-water-resistant-white-md-200case>: HTTP status code is not handled or not allowed 2026-01-24 17:27:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-drawer-unit-divider-kit-12-compartment-5>: HTTP status code is not handled or not allowed 2026-01-24 17:27:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/highlighters_markers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:27:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scrubbers-tilegroutbrush-261480>: HTTP status code is not handled or not allowed 2026-01-24 17:27:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8225hl-class-2-standard-vest-orange-2xl-3xl>: HTTP status code is not handled or not allowed 2026-01-24 17:27:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/410-airug-anti-fatigue-mat-38-thick-2w-x-3l-black>: HTTP status code is not handled or not allowed 2026-01-24 17:27:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_tools/cleaning_cloths_towels/wipes_wipers/low_lint_surface_prep_wipes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:27:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8250zhg-class-2-hi-gloss-surveyors-vest-orange-4xl-5xl>: HTTP status code is not handled or not allowed 2026-01-24 17:27:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/highlighters_markers landed on page that is not a product page. 2026-01-24 17:27:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compact-digital-counting-scale-60-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 17:27:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sms-lab-coat-no-pockets-elastic-wrists-snap-front-single-collar-blue-2xl-30case>: HTTP status code is not handled or not allowed 2026-01-24 17:27:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/welding/arc_welding/multi_process_welding_machines already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:27:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8245psv-public-safety-vest-orange-4xl-5xl>: HTTP status code is not handled or not allowed 2026-01-24 17:28:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8229z-class-2-economy-two-tone-vest-orange-s-m>: HTTP status code is not handled or not allowed 2026-01-24 17:28:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_tools/cleaning_cloths_towels/wipes_wipers/low_lint_surface_prep_wipes landed on page that is not a product page. 2026-01-24 17:28:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-gallon-oil-only-spill-kit-spko-10>: HTTP status code is not handled or not allowed 2026-01-24 17:28:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foodsafe-cfswdm3613t-e-ip69-5hp-3500rpm-3ph-60hz-184tc-3634m-tefc-f1>: HTTP status code is not handled or not allowed 2026-01-24 17:28:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/welding/arc_welding/multi_process_welding_machines landed on page that is not a product page. 2026-01-24 17:28:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:28:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/java-wicker-stacking-armchair-bronze-pack-of-16>: HTTP status code is not handled or not allowed 2026-01-24 17:28:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 2558 pages (at 93 pages/min), scraped 1807 items (at 63 items/min) 2026-01-24 17:28:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/electrical_power_testing/conductivity_meters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:28:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jet-jtm-949evs-mill-3-axis-acu-rite-200s-dro-quill-x-y-and-z-axis-powerfeeds-698173>: HTTP status code is not handled or not allowed 2026-01-24 17:28:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n-ferno-6818-knit-cap-black-one-size>: HTTP status code is not handled or not allowed 2026-01-24 17:28:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_bottles_jars already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:28:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents>: HTTP status code is not handled or not allowed 2026-01-24 17:28:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/testing_inspection_instruments/electrical_power_testing/conductivity_meters>: HTTP status code is not handled or not allowed 2026-01-24 17:28:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=jet-jtm-1050evs2-230-mill-3-axis-acu-rite-200s-dro-knee-x-y-axis-powerfeed-air-power-drawbar already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:28:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-adjustable-t-bar-shelf-24-wx36-l>: HTTP status code is not handled or not allowed 2026-01-24 17:28:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_bottles_jars>: HTTP status code is not handled or not allowed 2026-01-24 17:28:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/drilling_hole_making/metalworking_drill_bits/drill_blanks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:28:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-adjustable-solid-brute-shelf-24-wx72-l>: HTTP status code is not handled or not allowed 2026-01-24 17:28:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=jet-jtm-1050evs2-230-mill-3-axis-acu-rite-200s-dro-knee-x-y-axis-powerfeed-air-power-drawbar>: HTTP status code is not handled or not allowed 2026-01-24 17:28:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-3w-x-7l-1-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 17:28:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:28:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8915-class-e-rain-pants-orange-3-x-large>: HTTP status code is not handled or not allowed 2026-01-24 17:28:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:28:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:28:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-9w-x-11l-1-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 17:28:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/drilling_hole_making/metalworking_drill_bits/drill_blanks landed on page that is not a product page. 2026-01-24 17:28:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/long-sleeve-wicking-birdseye-t-shirt-w-pocket-hi-vis-yellow-2xl>: HTTP status code is not handled or not allowed 2026-01-24 17:28:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions landed on page that is not a product page. 2026-01-24 17:28:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-8w-x-19l-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 17:28:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions>: HTTP status code is not handled or not allowed 2026-01-24 17:28:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xd-smooth-aluminum-underbody-truck-box-with-diamond-tread-door-18x18x60-1706515>: HTTP status code is not handled or not allowed 2026-01-24 17:28:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions landed on page that is not a product page. 2026-01-24 17:28:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/folders_filing/wall_files already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:28:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-duty-stainless-steel-wire-rope-thimble-3-4-diameter>: HTTP status code is not handled or not allowed 2026-01-24 17:28:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/binders_covers/dividers_tabs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:28:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-5-dram-natural-pp-graduated-hinged-vet-vial-500pk>: HTTP status code is not handled or not allowed 2026-01-24 17:28:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/binders_covers/dividers_tabs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:28:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/retaining-ring-20mm-8020fw-1328>: HTTP status code is not handled or not allowed 2026-01-24 17:29:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/folders_filing/wall_files landed on page that is not a product page. 2026-01-24 17:29:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/binders_covers/dividers_tabs landed on page that is not a product page. 2026-01-24 17:29:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0-50-dram-amber-compound-vial-w-13-425-green-thermoset-f217-ptfe-caps-144pk>: HTTP status code is not handled or not allowed 2026-01-24 17:29:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/bouffants_hairnets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:29:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/23-x-85mm-6-dram-clear-vial-w-20-400-black-phenolic-polycone-lined-cap-attached-144pk>: HTTP status code is not handled or not allowed 2026-01-24 17:29:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/binders_covers/binders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:29:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/binders_covers/dividers_tabs landed on page that is not a product page. 2026-01-24 17:29:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galvanized-shelf-truck-with-dolly-base-36x24x70-1600-pound-capacity>: HTTP status code is not handled or not allowed 2026-01-24 17:29:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 2643 pages (at 85 pages/min), scraped 1865 items (at 58 items/min) 2026-01-24 17:29:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/food-safe-cfswdm3708t-e-ip69-75hp-1180rpm-3ph-60hz-254tc-3954m-tefc>: HTTP status code is not handled or not allowed 2026-01-24 17:29:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-wire-shelf-platform-36-w-x-21-d>: HTTP status code is not handled or not allowed 2026-01-24 17:29:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/bouffants_hairnets landed on page that is not a product page. 2026-01-24 17:29:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/binders_covers/dividers_tabs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:29:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-16w-x-22l-3-mil-clear-500pack>: HTTP status code is not handled or not allowed 2026-01-24 17:29:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/binders_covers/binders landed on page that is not a product page. 2026-01-24 17:29:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-arm-revers-quick-adj-45-ton-capacity-4-3-4-120-mm-open-3-7-8-100-mm-reach>: HTTP status code is not handled or not allowed 2026-01-24 17:29:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprng-4224-6295822>: HTTP status code is not handled or not allowed 2026-01-24 17:29:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/binders_covers/dividers_tabs landed on page that is not a product page. 2026-01-24 17:29:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cllr-jmd-18vsghd-2020pf-6126-1>: HTTP status code is not handled or not allowed 2026-01-24 17:29:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/notice-safety-glasses-required-7x10-pressure-sensitive-vinyl>: HTTP status code is not handled or not allowed 2026-01-24 17:29:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/cash_register_pos_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:29:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/411kaltblk-safety-padlock-1-12w-x-3h-shackle-black>: HTTP status code is not handled or not allowed 2026-01-24 17:29:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/packaging/envelopes_mailers/shipping_envelopes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:29:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-l-x-30-w-production-bench-plastic-square-edge-blue-2>: HTTP status code is not handled or not allowed 2026-01-24 17:29:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:29:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-wiper-arm-puller-set-w-4-pullers>: HTTP status code is not handled or not allowed 2026-01-24 17:29:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/paper_stationery/cash_register_pos_rolls landed on page that is not a product page. 2026-01-24 17:29:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thermostats-46-1371-fwet-stat-h1>: HTTP status code is not handled or not allowed 2026-01-24 17:29:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/packaging/envelopes_mailers/shipping_envelopes>: HTTP status code is not handled or not allowed 2026-01-24 17:29:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/beds_mattresses/mattresses already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:29:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/springs-26-3076-gar1760401>: HTTP status code is not handled or not allowed 2026-01-24 17:29:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/410kaltylw-safety-padlock-3h-shackle-yellow>: HTTP status code is not handled or not allowed 2026-01-24 17:29:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards landed on page that is not a product page. 2026-01-24 17:29:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s31ltblu-safety-padlock-1-38w-x-3h-shackle-blue>: HTTP status code is not handled or not allowed 2026-01-24 17:29:50 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/beds_mattresses/mattresses landed on page that is not a product page. 2026-01-24 17:29:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-safety-drum-with-engineer-sheeting>: HTTP status code is not handled or not allowed 2026-01-24 17:29:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-10w-x-18l-6-mil-clear-500pack>: HTTP status code is not handled or not allowed 2026-01-24 17:29:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o-ring-194x25-jbj2-11>: HTTP status code is not handled or not allowed 2026-01-24 17:29:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/machinery_parts_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:30:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-unit-5-tier-poly-z-brite-24w-x-36l-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 17:30:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gusseted-poly-bags-20w-x-18d-x-26h-2-mil-clear-200pack>: HTTP status code is not handled or not allowed 2026-01-24 17:30:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gusseted-poly-bags-8w-x-4d-x-15h-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 17:30:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/tool_holders/collets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:30:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cartridges-76-1411-eveev961716>: HTTP status code is not handled or not allowed 2026-01-24 17:30:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 2733 pages (at 90 pages/min), scraped 1926 items (at 61 items/min) 2026-01-24 17:30:09 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/machinery_parts_accessories landed on page that is not a product page. 2026-01-24 17:30:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thermostats-46-1342-kea023897>: HTTP status code is not handled or not allowed 2026-01-24 17:30:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-5w-x-12l-6-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 17:30:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/machining/tool_holders/collets>: HTTP status code is not handled or not allowed 2026-01-24 17:30:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/lathe_chucks_accessories/lathe_chuck_jaws already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:30:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fuse-set-cs275-106>: HTTP status code is not handled or not allowed 2026-01-24 17:30:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/lathe_chucks_accessories/lathe_chucks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:30:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/lathe_chucks_accessories/lathe_centers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:30:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-16w-x-36l-2-mil-clear-250pack>: HTTP status code is not handled or not allowed 2026-01-24 17:30:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/lathe_chucks_accessories/lathe_chucks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:30:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/lathe_chucks_accessories/lathe_chuck_jaws already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:30:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/limit-spring-jsh275-36>: HTTP status code is not handled or not allowed 2026-01-24 17:30:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/lathe_chucks_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:30:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/measurement_layout_tools/micrometers_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:30:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0-1-x-00005-0-001mm-ip65-digimatic-uni-mike-micrometer-w-output-fric-thimble-flat-rod-anvil>: HTTP status code is not handled or not allowed 2026-01-24 17:30:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/lathe_chucks_accessories/lathe_chucks landed on page that is not a product page. 2026-01-24 17:30:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/21-step-steel-caged-walk-through-fixed-access-ladder-safety-yellow-wlfc1121-y>: HTTP status code is not handled or not allowed 2026-01-24 17:30:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/lathe_chucks_accessories/lathe_chucks landed on page that is not a product page. 2026-01-24 17:30:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/lathe_chucks_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:30:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-316-stainless-steel-clipper-carded-hooks>: HTTP status code is not handled or not allowed 2026-01-24 17:30:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/lathe_chucks_accessories/lathe_centers landed on page that is not a product page. 2026-01-24 17:30:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/lathe_chucks_accessories>: HTTP status code is not handled or not allowed 2026-01-24 17:30:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-plated-brass-schedule-40-nipple-12-2-12-npt-male>: HTTP status code is not handled or not allowed 2026-01-24 17:30:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/measurement_layout_tools/micrometers_accessories landed on page that is not a product page. 2026-01-24 17:30:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/lathe_chucks_accessories>: HTTP status code is not handled or not allowed 2026-01-24 17:30:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-ventilated-locker-single-tier-15x18x72-1-door-assembled-4>: HTTP status code is not handled or not allowed 2026-01-24 17:30:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ck09-301-contactor-3-pole-460v>: HTTP status code is not handled or not allowed 2026-01-24 17:30:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-ton-proto-ease-pullers-4234b>: HTTP status code is not handled or not allowed 2026-01-24 17:30:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clkdbm3-insert-and-key-double-bit>: HTTP status code is not handled or not allowed 2026-01-24 17:30:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anti-static-flat-poly-bags-10w-x-20l-6-mil-pink-500pack>: HTTP status code is not handled or not allowed 2026-01-24 17:30:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-gas-infrared-heater-0919r20ngs-straight-configuration-20-l-55000-btu>: HTTP status code is not handled or not allowed 2026-01-24 17:31:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-nexelon-24w-x-24l-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 17:31:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-nexelon-18w-x-72l-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 17:31:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ck12-310-contactor-3-pole-120v>: HTTP status code is not handled or not allowed 2026-01-24 17:31:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 2823 pages (at 90 pages/min), scraped 1990 items (at 64 items/min) 2026-01-24 17:31:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f1212t24hc-wire-trough-hinged-cover-1200x1200x2400-steelgray>: HTTP status code is not handled or not allowed 2026-01-24 17:31:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-10w-x-12l-2-mil-black-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 17:31:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esd-stacking-box-top-overall-dimensions-25-3-4l-x-17-3-4w-x-3h>: HTTP status code is not handled or not allowed 2026-01-24 17:31:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/itm-14-6-14-sds-plus-carbide-hammer-drill-bit>: HTTP status code is not handled or not allowed 2026-01-24 17:31:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proto-174-3-8-drive-ratchet-head-repair-kit-torque-wrench-j6064c-cx-cxcert>: HTTP status code is not handled or not allowed 2026-01-24 17:31:17 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-24 17:31:17 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:2 2026-01-24 17:31:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cotton-terry-towel-12-pack-17-x-14-sw56106>: HTTP status code is not handled or not allowed 2026-01-24 17:31:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/back-end-stops-48-w-x-30-d-black>: HTTP status code is not handled or not allowed 2026-01-24 17:31:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/orange-full-brim-style-4-point-ratchet-suspension-hard-hat>: HTTP status code is not handled or not allowed 2026-01-24 17:31:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gravity-flow-rack-starter-1>: HTTP status code is not handled or not allowed 2026-01-24 17:31:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/index_cards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:31:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/pins_1/dowel_pins already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:31:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/screw-jsm-403-40340>: HTTP status code is not handled or not allowed 2026-01-24 17:31:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/umbrellas_awnings_shades/outdoor_umbrellas_bases/umbrellas already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:31:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-aluminum-bound-pc-face-shield>: HTTP status code is not handled or not allowed 2026-01-24 17:31:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-cabinets-maah-48662rb-b-mobile-storage-cabinet-assembled-36x24x66-black>: HTTP status code is not handled or not allowed 2026-01-24 17:31:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slip-bar-1230r-5233191>: HTTP status code is not handled or not allowed 2026-01-24 17:31:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/paper_stationery/index_cards landed on page that is not a product page. 2026-01-24 17:31:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/pins_1/dowel_pins landed on page that is not a product page. 2026-01-24 17:31:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/outdoor_furniture/umbrellas_awnings_shades/outdoor_umbrellas_bases/umbrellas>: HTTP status code is not handled or not allowed 2026-01-24 17:31:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-cabinets-bca-367218-g-bookcase-assembled-36x18x72-gray>: HTTP status code is not handled or not allowed 2026-01-24 17:31:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-foot-x-40-foot-heavy-duty-white-tarp>: HTTP status code is not handled or not allowed 2026-01-24 17:31:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-foot-x-40-foot-heavy-duty-silver-tarp>: HTTP status code is not handled or not allowed 2026-01-24 17:31:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-chain-3-4-link-100-l-hdpe-brown>: HTTP status code is not handled or not allowed 2026-01-24 17:31:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6802zi-trapper-hat-kit-w--universal-bump-cap-zipper-s-m-black>: HTTP status code is not handled or not allowed 2026-01-24 17:31:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6823-balaclava-face-mask-wind-proof-hinged-navy>: HTTP status code is not handled or not allowed 2026-01-24 17:31:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/resealable-warning-poly-bags-22w-x-24l-15-mil-clear-500pack>: HTTP status code is not handled or not allowed 2026-01-24 17:31:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-precision-ansi-cottered-pin-roller-chain-120-1c-1-1-2-pitch-50ft-reel>: HTTP status code is not handled or not allowed 2026-01-24 17:31:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4947f-double-sided-foam-tape-12-x-5-yds-45-mil-black>: HTTP status code is not handled or not allowed 2026-01-24 17:31:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprng-pin-jtm-4vs-ha-117>: HTTP status code is not handled or not allowed 2026-01-24 17:31:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/airspade-ht114-2000-105-scfm-construction-kit>: HTTP status code is not handled or not allowed 2026-01-24 17:32:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/clips_fasteners/rubber_bands already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:32:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-polypropylene-bags-9l-x-7w-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 17:32:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/clips_fasteners/paper_clips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:32:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/clips_fasteners/binder_clips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:32:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-precision-ansi-roller-chain-160-1r-2-pitch-connecting-link>: HTTP status code is not handled or not allowed 2026-01-24 17:32:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:32:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wood-crate-with-1-polyethylene-foam-lining-four-way-entry-60-l-x-24-w-x-31-12-h>: HTTP status code is not handled or not allowed 2026-01-24 17:32:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-step-50-incline-ladder-24-inch-w-grip-strut>: HTTP status code is not handled or not allowed 2026-01-24 17:32:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 2918 pages (at 95 pages/min), scraped 2055 items (at 65 items/min) 2026-01-24 17:32:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/clips_fasteners/rubber_bands landed on page that is not a product page. 2026-01-24 17:32:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/basic_supplies/clips_fasteners/paper_clips>: HTTP status code is not handled or not allowed 2026-01-24 17:32:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/clips_fasteners/binder_clips landed on page that is not a product page. 2026-01-24 17:32:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40403l-top-and-side-open-blast-cabinet-w-dust-collector-steel>: HTTP status code is not handled or not allowed 2026-01-24 17:32:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits landed on page that is not a product page. 2026-01-24 17:32:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-kit-for-electric-pallet-jack-gwk-epj60r-lw-fits-blue-giant-model-epj60r>: HTTP status code is not handled or not allowed 2026-01-24 17:32:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cantilever-rack-straight-arm-21>: HTTP status code is not handled or not allowed 2026-01-24 17:32:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/skimmer-tin-6-dia>: HTTP status code is not handled or not allowed 2026-01-24 17:32:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/knob-708016-101>: HTTP status code is not handled or not allowed 2026-01-24 17:32:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tube-cutter>: HTTP status code is not handled or not allowed 2026-01-24 17:32:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-block-reclosable-poly-bags-3w-x-6l-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 17:32:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e4pb-push-button-encl-4pb-x-305mm-44mm>: HTTP status code is not handled or not allowed 2026-01-24 17:32:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/plumbing_parts_accessories/pipe_hose_tubing_accessories/pipe_markers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:32:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-conductor-unshielded-plenum-cable-2-conductor-12-awg-19-0185-bc-natural>: HTTP status code is not handled or not allowed 2026-01-24 17:32:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bin-nestable-11-1-8-w-x-17-7-8-d-x-6-h-yellow>: HTTP status code is not handled or not allowed 2026-01-24 17:32:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/plumbing/plumbing_parts_accessories/pipe_hose_tubing_accessories/pipe_markers>: HTTP status code is not handled or not allowed 2026-01-24 17:32:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/bins_totes_containers/storage_totes_tubs/storage_totes_with_lids already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:32:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ahe36x24x6-pull-box-hinged-cover-3600x2400x600-steelgray>: HTTP status code is not handled or not allowed 2026-01-24 17:32:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/minigrip174-reclosable-poly-bags-4w-x-4l-4-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 17:32:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o-ring-jsm-802-80209>: HTTP status code is not handled or not allowed 2026-01-24 17:32:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/bins_totes_containers/storage_totes_tubs/storage_totes_with_lids>: HTTP status code is not handled or not allowed 2026-01-24 17:32:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/csd242012-concept-wall-mount-encl-2400x2000x1200-steelgray>: HTTP status code is not handled or not allowed 2026-01-24 17:32:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-steel-credenza-double-pedestal-24-72-charcoal-mahogany>: HTTP status code is not handled or not allowed 2026-01-24 17:33:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60in-expansion-kit-with-2-win-dows-for-11ft-lifetime-sheds>: HTTP status code is not handled or not allowed 2026-01-24 17:33:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/food_beverages/snacks_candies/crackers_chips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:33:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enclosed-cork-bulletin-board-sliding-door-72-x-48>: HTTP status code is not handled or not allowed 2026-01-24 17:33:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leather-reception-chair-with-arms-black-silver-frame>: HTTP status code is not handled or not allowed 2026-01-24 17:33:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blue-grip-ll301-natural-rubber-gloves-heavy-weight-blue-1-pair-x-large-301429>: HTTP status code is not handled or not allowed 2026-01-24 17:33:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 3014 pages (at 96 pages/min), scraped 2126 items (at 71 items/min) 2026-01-24 17:33:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/work-benches/workbenches_worktables/cabinet_pedestal_workbenches/stationary_cabinet_pedestal_workbenches already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:33:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/125-lf-12-14-hex-bushing-npt-male-female>: HTTP status code is not handled or not allowed 2026-01-24 17:33:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-enzymatic-chute-dumpster-wash-plus-orange-scent-quart-bottle-6-case>: HTTP status code is not handled or not allowed 2026-01-24 17:33:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/food_beverages/snacks_candies/crackers_chips landed on page that is not a product page. 2026-01-24 17:33:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nilotron-hand-held-aerosols-9-5-oz-lemon-scent-12-case>: HTTP status code is not handled or not allowed 2026-01-24 17:33:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:33:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/work-benches/workbenches_worktables/cabinet_pedestal_workbenches/stationary_cabinet_pedestal_workbenches>: HTTP status code is not handled or not allowed 2026-01-24 17:33:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a1412chnf-j-box-hinged-cover-type-4-1400x1200x600-steelgray>: HTTP status code is not handled or not allowed 2026-01-24 17:33:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speedi-boot-90-175-register-vent-boot-w-adj-hangers-for-hvac-duct-work-4-x-10-x-6>: HTTP status code is not handled or not allowed 2026-01-24 17:33:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-solid-stainless-steel-shelving-add-on-unit-5-tier-48w-x-24d-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 17:33:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats landed on page that is not a product page. 2026-01-24 17:33:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-solid-galvanized-steel-shelving-add-on-unit-5-tier-48w-x-18d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 17:33:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/power_tools/drills_drivers_impact_wrenches/power_drill_combo_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:33:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/asg16x12x4-pull-box-screw-cover-kos-1600x1200x400-galvanized>: HTTP status code is not handled or not allowed 2026-01-24 17:33:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hawkeye-1-amber-lens>: HTTP status code is not handled or not allowed 2026-01-24 17:33:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lcd-cart-26-lbs-capacity-two-tone-gray>: HTTP status code is not handled or not allowed 2026-01-24 17:33:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-inch-acrylic-full-dome-safety-mirror-360-viewing-angle-plastic-back>: HTTP status code is not handled or not allowed 2026-01-24 17:33:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/power_tools/drills_drivers_impact_wrenches/power_drill_combo_kits landed on page that is not a product page. 2026-01-24 17:33:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6inx10in-2-mil-recloseable-poly-bag-clear-pkg-qty-1000>: HTTP status code is not handled or not allowed 2026-01-24 17:33:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s-lp-10-thermaflex-flexible-hvac-duct-9-diameter>: HTTP status code is not handled or not allowed 2026-01-24 17:33:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-60v-max-flexvolt-li-ion-dual-pk>: HTTP status code is not handled or not allowed 2026-01-24 17:33:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cordless-impact-driver-bare-tool>: HTTP status code is not handled or not allowed 2026-01-24 17:33:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10l-x-16w-horizontal-bandsaw-variable-speed-2-hp-460v-3-phase>: HTTP status code is not handled or not allowed 2026-01-24 17:33:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/machinery_parts_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:33:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slimline-wall-desk-single-size-manual-close-keyed-lock-beige>: HTTP status code is not handled or not allowed 2026-01-24 17:33:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mdl-60b-switch-box-60b-429>: HTTP status code is not handled or not allowed 2026-01-24 17:33:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/block-rh-29-shaper-6292993>: HTTP status code is not handled or not allowed 2026-01-24 17:34:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/armorclad-2-layer-epoxy-floor-kit-medium-flake-broadcast-600-sq-ft-designer-gray>: HTTP status code is not handled or not allowed 2026-01-24 17:34:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:34:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprng-pin-jsg-0716n-00-3309>: HTTP status code is not handled or not allowed 2026-01-24 17:34:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/locks_lockout_devices/lockout_tagout_devices/lock_lockout_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:34:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/krylon-camouflage-with-fusion-plastic-paint-brown>: HTTP status code is not handled or not allowed 2026-01-24 17:34:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/machinery_parts_accessories landed on page that is not a product page. 2026-01-24 17:34:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/heaters/patio_heaters/electric_patio_heaters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:34:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classical-b-v-belt-b71>: HTTP status code is not handled or not allowed 2026-01-24 17:34:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 3104 pages (at 90 pages/min), scraped 2187 items (at 61 items/min) 2026-01-24 17:34:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cast-acrylic-sheet-48-l-x-48-w-x-1-4-thick-white>: HTTP status code is not handled or not allowed 2026-01-24 17:34:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 17:34:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:34:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/locks_lockout_devices/lockout_tagout_devices/lock_lockout_accessories landed on page that is not a product page. 2026-01-24 17:34:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelate-silver-epoxy-wire-shelving-starter-unit-5-tier-42w-x-21d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 17:34:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelate-silver-epoxy-wire-shelving-starter-unit-5-tier-36w-x-18d-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 17:34:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:34:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/heaters/patio_heaters/electric_patio_heaters landed on page that is not a product page. 2026-01-24 17:34:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelate-silver-epoxy-wire-shelving-starter-unit-5-tier-42w-x-18d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 17:34:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_lockout_identification_tags already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:34:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents>: HTTP status code is not handled or not allowed 2026-01-24 17:34:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:34:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/self_luminous_emergency_exit_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:34:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/long-poly-mailers-18w-x-48l-4-mil-white-25pack>: HTTP status code is not handled or not allowed 2026-01-24 17:34:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:34:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96262-rearview-mirror-hanging-tags-101-200-parking-permits-blue-2-34w-x-4-34h>: HTTP status code is not handled or not allowed 2026-01-24 17:34:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents landed on page that is not a product page. 2026-01-24 17:34:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/foam/foam_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:34:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:34:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_lockout_identification_tags>: HTTP status code is not handled or not allowed 2026-01-24 17:34:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:34:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gusseted-merchandise-bags-17w-x-4d-x-24h-white-500pack>: HTTP status code is not handled or not allowed 2026-01-24 17:34:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/self_luminous_emergency_exit_signs landed on page that is not a product page. 2026-01-24 17:34:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 17:34:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 17:34:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-1-2-x-12-1-2-x-6-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 17:34:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/spill_control_supplies/chemical_sorbents_spill_kits/spill_kits_stations already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:34:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelving-starter-unit-5-tier-24w-x-18d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 17:34:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 17:34:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs landed on page that is not a product page. 2026-01-24 17:34:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rfp300-re-form-plus-universal-pads15w-x-19lmedium-weightcase100>: HTTP status code is not handled or not allowed 2026-01-24 17:34:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/foam/foam_rolls landed on page that is not a product page. 2026-01-24 17:34:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labels-w-do-not-double-stack-print-3l-x-2w-white-textred-back-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-24 17:34:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-duralast-wheel-6x2-1-roller>: HTTP status code is not handled or not allowed 2026-01-24 17:34:45 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/spill_control_supplies/chemical_sorbents_spill_kits/spill_kits_stations landed on page that is not a product page. 2026-01-24 17:34:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-x-13-x-10-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 17:34:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/104312-toughstripe-floor-marking-tape-polyester-2w-x-100l-yellow>: HTTP status code is not handled or not allowed 2026-01-24 17:34:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:34:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:34:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/spill_control_supplies/chemical_sorbents_spill_kits/spill_kits_stations already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:34:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-poly-tech-wheel-3-12x1-38-12-ball>: HTTP status code is not handled or not allowed 2026-01-24 17:34:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelite-vented-plastic-shelving-starter-unit-5-tier-36w-x-21d-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 17:34:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelite-vented-plastic-shelving-starter-unit-5-tier-42w-x-18d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 17:35:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock landed on page that is not a product page. 2026-01-24 17:35:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylok-nut-14-20nc-4000380>: HTTP status code is not handled or not allowed 2026-01-24 17:35:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents landed on page that is not a product page. 2026-01-24 17:35:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/spill_control_supplies/chemical_sorbents_spill_kits/spill_kits_stations landed on page that is not a product page. 2026-01-24 17:35:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-foam-polycarbonate-film-96-l-x-48-w-x-0-03-thick-clear>: HTTP status code is not handled or not allowed 2026-01-24 17:35:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/evaporative_coolers_swamp_coolers/evaporative_cooler_accessories/evaporative_cooler_hardware_parts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:35:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cast-acrylic-bar-12-l-x-2-w-x-1-4-thick-yellow>: HTTP status code is not handled or not allowed 2026-01-24 17:35:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 3189 pages (at 85 pages/min), scraped 2237 items (at 50 items/min) 2026-01-24 17:35:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultem-pei-sheet-6-l-x-6-w-x-1-thick-amber>: HTTP status code is not handled or not allowed 2026-01-24 17:35:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-petg-tube-24-l-x-3-16-id-x-5-16-od-clear>: HTTP status code is not handled or not allowed 2026-01-24 17:35:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/55859-scorch-adf-welding-helmet>: HTTP status code is not handled or not allowed 2026-01-24 17:35:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-x-6-x-9-insulated-container>: HTTP status code is not handled or not allowed 2026-01-24 17:35:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/600088-cchannel-8h-clr-5-pc>: HTTP status code is not handled or not allowed 2026-01-24 17:35:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultem-pei-sheet-24-l-x-24-w-x-3-4-thick-amber>: HTTP status code is not handled or not allowed 2026-01-24 17:35:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-x-24-self-seal-kra-stayflat-mailers>: HTTP status code is not handled or not allowed 2026-01-24 17:35:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-bar-24-l-x-4-w-x-1-8-thick-gray>: HTTP status code is not handled or not allowed 2026-01-24 17:35:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/evaporative_coolers_swamp_coolers/evaporative_cooler_accessories/evaporative_cooler_hardware_parts landed on page that is not a product page. 2026-01-24 17:35:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:35:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paper-labels-with-jun-print-3l-x-2w-l-fluorescent-red-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-24 17:35:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/versa-loop-rehabilitation-band-ultra-heavy-resistance-gray>: HTTP status code is not handled or not allowed 2026-01-24 17:35:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:35:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/satco-s9951-23pll-led-840-bp-led-2g11-cfl-replacement-lamp-ballast-bypass23w-2800-lumens-4000k>: HTTP status code is not handled or not allowed 2026-01-24 17:35:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:35:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/towing_accessories/hitch_covers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:35:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets>: HTTP status code is not handled or not allowed 2026-01-24 17:35:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/foam/foam_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:35:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:35:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electronic-kraft-dispenser-replacement-brush>: HTTP status code is not handled or not allowed 2026-01-24 17:35:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-dia-round-number-paper-labels-with-1-print-fluorescent-green-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-24 17:35:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/composites/composite_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:35:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-24 17:35:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 17:35:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/towing_accessories/hitch_covers>: HTTP status code is not handled or not allowed 2026-01-24 17:35:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/anchors_hangers/screw_anchors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:35:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/automotive_accessories_cleaning_supplies/interior_covers/seat_covers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:35:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/foam/foam_sheets>: HTTP status code is not handled or not allowed 2026-01-24 17:35:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/grill_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:35:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-12-l-x-4-w-x-3-4-thick-black-antistatic>: HTTP status code is not handled or not allowed 2026-01-24 17:35:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:35:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/composites/composite_bar_stock landed on page that is not a product page. 2026-01-24 17:35:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 17:35:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-24-l-x-1-w-x-3-4-thick-blue-glass-filled>: HTTP status code is not handled or not allowed 2026-01-24 17:35:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/anchors_hangers/screw_anchors landed on page that is not a product page. 2026-01-24 17:35:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/daddy-long-legs-side-panel-30l-grey>: HTTP status code is not handled or not allowed 2026-01-24 17:36:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/floor_mats/grill_mats>: HTTP status code is not handled or not allowed 2026-01-24 17:36:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:36:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/econoline-storage-building-20w-x-12h-x-30l-white-pony-wall>: HTTP status code is not handled or not allowed 2026-01-24 17:36:04 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-24 17:36:04 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/vehicle-fleet-maintenance/automotive_accessories_cleaning_supplies/interior_covers/seat_covers landed on page that is not a product page. 2026-01-24 17:36:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/daddy-long-legs-side-panel-20l-green>: HTTP status code is not handled or not allowed 2026-01-24 17:36:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-60-l-x-4-w-x-1-4-thick-white>: HTTP status code is not handled or not allowed 2026-01-24 17:36:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 3272 pages (at 83 pages/min), scraped 2288 items (at 51 items/min) 2026-01-24 17:36:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:36:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rectangular-inventory-paper-labels-4l-x-2w-green-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-24 17:36:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 17:36:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/locking-toilet-tissue-r260xc>: HTTP status code is not handled or not allowed 2026-01-24 17:36:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:36:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mchg102vp-danger-sign-flammable-liquids-14w-x-10h-plastic>: HTTP status code is not handled or not allowed 2026-01-24 17:36:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wrestling-marial-arts-roll-mats-30-x-6-x-1-5-8-navy-112r-ny>: HTTP status code is not handled or not allowed 2026-01-24 17:36:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-dc-1450c-cyclone-style-dust-collector-1450-cfm-2hp-1ph-220v>: HTTP status code is not handled or not allowed 2026-01-24 17:36:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-24-l-x-3-w-x-3-4-thick-black-antistatic>: HTTP status code is not handled or not allowed 2026-01-24 17:36:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 17:36:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/impact-socket-3-8-drive-19mm-6-point-short>: HTTP status code is not handled or not allowed 2026-01-24 17:36:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 17:36:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/impact-socket-3-8-drive-11mm-6-point-deep>: HTTP status code is not handled or not allowed 2026-01-24 17:36:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mdt187ptp-danger-do-not-enter-tag-rp-plastic-25pack>: HTTP status code is not handled or not allowed 2026-01-24 17:36:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:36:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cpvc-bar-24-l-x-1-2-w-x-1-16-thick-gray>: HTTP status code is not handled or not allowed 2026-01-24 17:36:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-rack-guard-back-panel-with-offset-bracket-108l-x-1-14w-x-48h>: HTTP status code is not handled or not allowed 2026-01-24 17:36:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f6-woven-split-wrappable-sleeve-2-dia-10-black>: HTTP status code is not handled or not allowed 2026-01-24 17:36:34 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-24 17:36:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reversible-gel-foam-seat-cushion-with-fleece-cover-16-x-18-x-2-cream-513-7631-9911>: HTTP status code is not handled or not allowed 2026-01-24 17:36:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2flexo-pet-general-purpose-sleeving-dia-500-red>: HTTP status code is not handled or not allowed 2026-01-24 17:36:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kobi-tall-planter-20-l-x-20-w-x-38-h-square-black>: HTTP status code is not handled or not allowed 2026-01-24 17:36:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/socket-1-4-drive-e7-external-torx>: HTTP status code is not handled or not allowed 2026-01-24 17:36:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:36:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:36:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:36:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:36:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-desk-lamp-36w-black>: HTTP status code is not handled or not allowed 2026-01-24 17:36:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:36:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ntep-12x12-bench-scale-with-led-display-100-lb-x-002-lb>: HTTP status code is not handled or not allowed 2026-01-24 17:36:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-24 17:36:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 17:36:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-24 17:36:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-13-16-x-8-5-16-x-2-1-2-lewisbins-divider-box-light-blue>: HTTP status code is not handled or not allowed 2026-01-24 17:36:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-24 17:36:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/z-line-boltless-shelving-w-steel-industrial-shelves-48-w-x-18-d-x-84-h-starter-5-shelves>: HTTP status code is not handled or not allowed 2026-01-24 17:36:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spectrum-medium-duty-pipe-brush-12-long-1-2-d>: HTTP status code is not handled or not allowed 2026-01-24 17:36:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 17:37:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_seating/outdoor_benches/plastic_benches already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:37:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-x-16-open-end-static-shielding-bags>: HTTP status code is not handled or not allowed 2026-01-24 17:37:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g15l-125-ff-20hp-oil-injected-rotary-screw-132-gal-tank-mnt-125psi-3ph-208-230-460v>: HTTP status code is not handled or not allowed 2026-01-24 17:37:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/air_compressors/single_stage_reciprocating_air_compressors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:37:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/telescopic-aluminum-flat-mop-head-handle-70>: HTTP status code is not handled or not allowed 2026-01-24 17:37:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 3362 pages (at 90 pages/min), scraped 2346 items (at 58 items/min) 2026-01-24 17:37:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_seating/outdoor_benches/plastic_benches landed on page that is not a product page. 2026-01-24 17:37:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mfsd931va-emergency-sign-safety-shower-graphic-10w-x-7h-aluminum>: HTTP status code is not handled or not allowed 2026-01-24 17:37:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:37:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:37:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-offset-wall-flange>: HTTP status code is not handled or not allowed 2026-01-24 17:37:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/air_compressors/single_stage_reciprocating_air_compressors landed on page that is not a product page. 2026-01-24 17:37:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rfp-40vizi2-auto-darkening-welding-helmet-9-13-variable-shade-mo2-black>: HTTP status code is not handled or not allowed 2026-01-24 17:37:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/upsable-bubble-wrap-roll-12-x-300-x-316-4-pack>: HTTP status code is not handled or not allowed 2026-01-24 17:37:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 17:37:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 17:37:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hurricane-360-hoist-and-trolley-combo-standard-2-ton-capacity-10-lift>: HTTP status code is not handled or not allowed 2026-01-24 17:37:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9x12-4-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-24 17:37:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:37:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jat-105-3-4-impact-wrench-1500-ft-lbs-r8-series-5-500-rpm-90-psi-9-5-cfm>: HTTP status code is not handled or not allowed 2026-01-24 17:37:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cpvc-sheet-12-l-x-12-w-x-1-8-thick-gray-ps-cpvc-13>: HTTP status code is not handled or not allowed 2026-01-24 17:37:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20mm-e-z-crimper-standard-seal>: HTTP status code is not handled or not allowed 2026-01-24 17:37:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 17:37:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/designjet-heavyweight-coated-ink-jet-paper-35-lb-54-x-100-roll>: HTTP status code is not handled or not allowed 2026-01-24 17:37:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-freestanding-2-panel-corner-room-divider-36-x-61-blue>: HTTP status code is not handled or not allowed 2026-01-24 17:37:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:37:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:37:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gre-2448-storage-chest-assy-25-x-48-x-24>: HTTP status code is not handled or not allowed 2026-01-24 17:37:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-x-18-1-4-triangle-mailing-tubes>: HTTP status code is not handled or not allowed 2026-01-24 17:37:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0920-1080-specific-gravity-shatterproof-plastic-hydrometer>: HTTP status code is not handled or not allowed 2026-01-24 17:37:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 17:37:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers>: HTTP status code is not handled or not allowed 2026-01-24 17:37:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelate-wire-shelving-30605ep-60w-x-30d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 17:37:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-x-26-white-mailing-tubes-with-caps-case>: HTTP status code is not handled or not allowed 2026-01-24 17:37:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:38:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1200-1420-specific-gravity-hydrometer-for-liquids-heavier-than-water>: HTTP status code is not handled or not allowed 2026-01-24 17:38:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dry-zone-shadow-board-yellow-red-68-x-30-acp-aluminum-composite-panel-sb138acp>: HTTP status code is not handled or not allowed 2026-01-24 17:38:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guardair-14-npt-high-flow-coupler>: HTTP status code is not handled or not allowed 2026-01-24 17:38:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ice-hardened-hole-saw-49560147>: HTTP status code is not handled or not allowed 2026-01-24 17:38:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/electrical_power_testing/electrical_power_testing_parts_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:38:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/electrical_power_testing/multimeters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:38:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluke-323-400a-ac-600v-ac-dc-true-rms-clamp-meter-cat-iv-300v-cat-iii-600-v-safety-rating>: HTTP status code is not handled or not allowed 2026-01-24 17:38:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silver-depressed-center-grinding-wheel-4-1-2-x-1-4-x-7-8-t27-ceramic-grain>: HTTP status code is not handled or not allowed 2026-01-24 17:38:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 3452 pages (at 90 pages/min), scraped 2406 items (at 60 items/min) 2026-01-24 17:38:09 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock landed on page that is not a product page. 2026-01-24 17:38:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:38:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:38:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/virgin-kraft-paper-40-lbs-24w-x-900l-1-roll>: HTTP status code is not handled or not allowed 2026-01-24 17:38:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/electrical_power_testing/electrical_power_testing_parts_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:38:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-rod-36-l-x-1-2-dia-white>: HTTP status code is not handled or not allowed 2026-01-24 17:38:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/testing_inspection_instruments/electrical_power_testing/electrical_power_testing_parts_accessories landed on page that is not a product page. 2026-01-24 17:38:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:38:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/testing_inspection_instruments/electrical_power_testing/multimeters landed on page that is not a product page. 2026-01-24 17:38:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluke-tl950-mini-pincer-test-leads>: HTTP status code is not handled or not allowed 2026-01-24 17:38:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commercialine-stack-chair-polypropylene-black-850-cl-series>: HTTP status code is not handled or not allowed 2026-01-24 17:38:18 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock landed on page that is not a product page. 2026-01-24 17:38:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/testing_inspection_instruments/electrical_power_testing/electrical_power_testing_parts_accessories>: HTTP status code is not handled or not allowed 2026-01-24 17:38:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:38:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-hdpe-bar-24-l-x-3-w-x-1-2-thick-black-marine>: HTTP status code is not handled or not allowed 2026-01-24 17:38:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 17:38:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock>: HTTP status code is not handled or not allowed 2026-01-24 17:38:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:38:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-sonic-ptx7l-bs-battery-6-amps-12v-b-terminals>: HTTP status code is not handled or not allowed 2026-01-24 17:38:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:38:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h-b-frio-temp-calibrated-dual-zone-elec-verification-thermometer-50-70c-58-158f-and-10-50c>: HTTP status code is not handled or not allowed 2026-01-24 17:38:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:38:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers landed on page that is not a product page. 2026-01-24 17:38:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-edge-3w-4l-1-ply-eye-eye-poly-web-sling-twisted-eyes>: HTTP status code is not handled or not allowed 2026-01-24 17:38:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-72-x-36-adj-height-workbench-w-drawer-black-plastic-laminate-square-top>: HTTP status code is not handled or not allowed 2026-01-24 17:38:34 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 17:38:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-60-x-30-adj-height-workbench-w-drawer-riser-black-shop-top-square-edge-top>: HTTP status code is not handled or not allowed 2026-01-24 17:38:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock landed on page that is not a product page. 2026-01-24 17:38:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock landed on page that is not a product page. 2026-01-24 17:38:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h-b-frio-temp-freezer-verification-thermometer-25-to-5c>: HTTP status code is not handled or not allowed 2026-01-24 17:38:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-3-4-x-1-3-8-1-manila-pre-strung-tag-1>: HTTP status code is not handled or not allowed 2026-01-24 17:38:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/bins_totes_containers/bins/cubby_bins_trays already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:38:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:38:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interlocking-storage-cabinet-11-3-4w-11-3-8d-x-11h-6-gray-drawers>: HTTP status code is not handled or not allowed 2026-01-24 17:38:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:38:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-hdpe-bar-48-l-x-1-w-x-1-2-thick-white-marine>: HTTP status code is not handled or not allowed 2026-01-24 17:38:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storage-container-with-lid-52-3-4-x-15-x-8-blue>: HTTP status code is not handled or not allowed 2026-01-24 17:38:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-round-outdoor-bar-table-designer-white-phenolic-top-silver-aluminum-frame>: HTTP status code is not handled or not allowed 2026-01-24 17:38:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock landed on page that is not a product page. 2026-01-24 17:38:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock landed on page that is not a product page. 2026-01-24 17:38:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pmp-283ml-urbanti-high-speed-filter-funnels-6pk>: HTTP status code is not handled or not allowed 2026-01-24 17:38:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/bins_totes_containers/bins/cubby_bins_trays landed on page that is not a product page. 2026-01-24 17:38:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-24-l-x-3-w-x-2-1-4-thick-black>: HTTP status code is not handled or not allowed 2026-01-24 17:38:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:39:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tempco-304ss-immersion-1-14-613d-1000w-120v-t-stat>: HTTP status code is not handled or not allowed 2026-01-24 17:39:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:39:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xbu02pt-x2-36v-blower-kit>: HTTP status code is not handled or not allowed 2026-01-24 17:39:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:39:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multifunction-executive-office-chair-mesh-mid-back-black>: HTTP status code is not handled or not allowed 2026-01-24 17:39:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:39:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-rack-upright-frame-48d-x-120h>: HTTP status code is not handled or not allowed 2026-01-24 17:39:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-wall-mount-wire-shelving-30w-x-14d-x-14h-1-shelf-starter-poly-green>: HTTP status code is not handled or not allowed 2026-01-24 17:39:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-24 17:39:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 3539 pages (at 87 pages/min), scraped 2456 items (at 50 items/min) 2026-01-24 17:39:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/foam/foam_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:39:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tool-crib-chain-cable-lube-20-oz-black-6-cans-case-620-1502>: HTTP status code is not handled or not allowed 2026-01-24 17:39:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:39:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 17:39:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 17:39:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers landed on page that is not a product page. 2026-01-24 17:39:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:39:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-wall-mount-wire-shelving-24w-x-24d-x-54h-3-shelf-add-on-poly-green>: HTTP status code is not handled or not allowed 2026-01-24 17:39:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:39:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/itm-2-58-22-sds-max-hollow-hammer-core-bit>: HTTP status code is not handled or not allowed 2026-01-24 17:39:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/foam/foam_sheets landed on page that is not a product page. 2026-01-24 17:39:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-24 17:39:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-x-18-4-mil-reclosable-poly-bags>: HTTP status code is not handled or not allowed 2026-01-24 17:39:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 17:39:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-wall-mount-wire-shelving-30w-x-18d-x-63h-4-shelf-starter-poly-green>: HTTP status code is not handled or not allowed 2026-01-24 17:39:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers landed on page that is not a product page. 2026-01-24 17:39:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-36-l-x-1-w-x-5-8-thick-black>: HTTP status code is not handled or not allowed 2026-01-24 17:39:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:39:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-sheet-12-l-x-12-w-x-3-8-thick-black-bulk-ps-acb-1789>: HTTP status code is not handled or not allowed 2026-01-24 17:39:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-sheet-48-l-x-24-w-x-1-1-2-thick-black>: HTTP status code is not handled or not allowed 2026-01-24 17:39:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kendall-howard-24-performance-work-bench-with-full-bottom-shelf-no-upper-shelving>: HTTP status code is not handled or not allowed 2026-01-24 17:39:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:39:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reach-in-refrigerator-horizon-glass-half-glass-doors-52w-hrp2-1g>: HTTP status code is not handled or not allowed 2026-01-24 17:39:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers landed on page that is not a product page. 2026-01-24 17:39:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:39:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-24-16-drawers-plastic>: HTTP status code is not handled or not allowed 2026-01-24 17:39:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers>: HTTP status code is not handled or not allowed 2026-01-24 17:39:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-x-5ft-nylon-us-flag-set-with-6ft-aluminum-pole>: HTTP status code is not handled or not allowed 2026-01-24 17:39:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90-degree-corner-connector-kit-72-h-panel>: HTTP status code is not handled or not allowed 2026-01-24 17:39:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:39:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers landed on page that is not a product page. 2026-01-24 17:39:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brass-snap-hook>: HTTP status code is not handled or not allowed 2026-01-24 17:39:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gx-80f-spherical-plain-thrust-bearing-metric-high-capacity>: HTTP status code is not handled or not allowed 2026-01-24 17:39:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/food_beverages/emergency_food_drink already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:39:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:39:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leviton-5362-li-20-amp-125v-heavy-duty-grade-duplex-receptacle-grounding-ivory>: HTTP status code is not handled or not allowed 2026-01-24 17:39:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-5-tier-48w-x-30d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 17:39:56 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 17:39:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:39:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/highlighters_markers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:39:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-acetal-bar-24-l-x-2-w-x-1-4-thick-off-white-antistatic>: HTTP status code is not handled or not allowed 2026-01-24 17:40:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/food_beverages/emergency_food_drink landed on page that is not a product page. 2026-01-24 17:40:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chem-crest-14-general-purpose-wash-solution-5-gallon-pail-700014p>: HTTP status code is not handled or not allowed 2026-01-24 17:40:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/snap-cap-styrene-vials-40-dram-148ml-pe-cap-72pk>: HTTP status code is not handled or not allowed 2026-01-24 17:40:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 17:40:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-24 17:40:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_clamps_tube_fittings already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:40:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-sheet-12-l-x-12-w-x-2-1-2-thick-black-bulk-ps-acb-2169>: HTTP status code is not handled or not allowed 2026-01-24 17:40:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/highlighters_markers landed on page that is not a product page. 2026-01-24 17:40:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 3628 pages (at 89 pages/min), scraped 2509 items (at 53 items/min) 2026-01-24 17:40:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-12-l-x-1-1-2-w-x-3-8-thick-black>: HTTP status code is not handled or not allowed 2026-01-24 17:40:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:40:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/space-saver-polycarbonate-polypropylene-desiccator-031-cu-ft>: HTTP status code is not handled or not allowed 2026-01-24 17:40:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_clamps_tube_fittings landed on page that is not a product page. 2026-01-24 17:40:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-ptfe-rod-24-l-x-1-2-dia-off-white-glass-filled>: HTTP status code is not handled or not allowed 2026-01-24 17:40:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:40:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-folding-bulk-shipping-container-bc4844-27-1500-lb-capacity-48-x-44-1-2-x-27>: HTTP status code is not handled or not allowed 2026-01-24 17:40:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-24 17:40:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-48-l-x-2-1-2-w-x-3-8-thick-white>: HTTP status code is not handled or not allowed 2026-01-24 17:40:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_bottles_jars already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:40:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:40:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-24w-x-14d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 17:40:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-tilt-truck-with-fork-li-pockets-1-cubic-yard-1250-lb-capacity-1>: HTTP status code is not handled or not allowed 2026-01-24 17:40:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-16028-coupler-38-body-size-38-npt-female>: HTTP status code is not handled or not allowed 2026-01-24 17:40:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock landed on page that is not a product page. 2026-01-24 17:40:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 17:40:34 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_bottles_jars landed on page that is not a product page. 2026-01-24 17:40:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9548-15-12-inch-axial-explosion-proof-ex-plastic-blower-w-canister-15-ducting>: HTTP status code is not handled or not allowed 2026-01-24 17:40:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:40:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-round-clear-flood-light-6-leds-1492125>: HTTP status code is not handled or not allowed 2026-01-24 17:40:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/tables/training_tables already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:40:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-stacking-bins-16-1-2-x-36-x-17-1-2-red>: HTTP status code is not handled or not allowed 2026-01-24 17:40:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/roof_pipe_heating_cables already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:40:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/competitor-ventilated-three-tier-locker-starter1-wide12w-x-15d-x-24hunassmebled-blue-frost>: HTTP status code is not handled or not allowed 2026-01-24 17:40:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 17:40:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_dippers_scoops_cups already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:40:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:40:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-hasp-red-dipped-1-5-inch-7546>: HTTP status code is not handled or not allowed 2026-01-24 17:40:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/tables/training_tables landed on page that is not a product page. 2026-01-24 17:40:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24w-x-15d-straight-shelf-w-front-lip-semi-gloss-black>: HTTP status code is not handled or not allowed 2026-01-24 17:40:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/roof_pipe_heating_cables landed on page that is not a product page. 2026-01-24 17:40:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-buna-n-roll-720-l-x-36-w-x-1-16-thick-black-high-strength-80a-acrylic-adhesive>: HTTP status code is not handled or not allowed 2026-01-24 17:40:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:40:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_dippers_scoops_cups landed on page that is not a product page. 2026-01-24 17:40:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 17:40:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-teflon-fep-straight-cover-glass-forceps-4-1-2-2pk>: HTTP status code is not handled or not allowed 2026-01-24 17:41:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-double-tier-locker-3-wide-12w-x-15d-x-30h-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-24 17:41:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/190059a-idis-1-tongue-interlock-switch-angled-act-1nc-1no-12npt-pe>: HTTP status code is not handled or not allowed 2026-01-24 17:41:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls>: HTTP status code is not handled or not allowed 2026-01-24 17:41:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-double-tier-locker-3-wide-15w-x-15d-x-36h-assembled-blue-frost>: HTTP status code is not handled or not allowed 2026-01-24 17:41:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 3715 pages (at 87 pages/min), scraped 2566 items (at 57 items/min) 2026-01-24 17:41:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/seq-dual-speed-electric-chain-hoist-w-push-trolley-1-ton-15-lift-17-3-9-ft-min-115v>: HTTP status code is not handled or not allowed 2026-01-24 17:41:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/foam/foam_tubes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:41:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/foam/foam_tubes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:41:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-neoprene-roll-720-l-x-20-w-x-3-8-thick-black-ultra-strength-50a>: HTTP status code is not handled or not allowed 2026-01-24 17:41:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-foam-neoprene-roll-720-l-x-12-w-x-1-8-thick-black-ultra-strength-70a>: HTTP status code is not handled or not allowed 2026-01-24 17:41:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:41:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/material-handling/ladders_personnel_lifts/step_ladders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:41:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-fiberglass-step-ladder-375-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 17:41:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/foam/foam_tubes landed on page that is not a product page. 2026-01-24 17:41:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/foam/foam_tubes landed on page that is not a product page. 2026-01-24 17:41:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/CVT-3024-30-W-x-24-D-16-ga-Stainless-Steel-Workbench-w-Shelf>: HTTP status code is not handled or not allowed 2026-01-24 17:41:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drain-auger-1-4-x-15>: HTTP status code is not handled or not allowed 2026-01-24 17:41:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 17:41:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-four-tier-locker-1-wide-12w-x-18d-x-15h-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-24 17:41:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/material-handling/ladders_personnel_lifts/step_ladders landed on page that is not a product page. 2026-01-24 17:41:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-way-base-for-grid-semi-gloss-white>: HTTP status code is not handled or not allowed 2026-01-24 17:41:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:41:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-side-action-serratable-sealer-hd-for-3-4w-polypropylene>: HTTP status code is not handled or not allowed 2026-01-24 17:41:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:41:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-level-72x18-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 17:41:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:41:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-head-thumb-screw-10-32-78-thread-3-4-head-dia-55-head-h-pkg-of-5>: HTTP status code is not handled or not allowed 2026-01-24 17:41:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-x-40-3-gauge-vinyl-taffeta-w-window-zipper-white-w-white-trim>: HTTP status code is not handled or not allowed 2026-01-24 17:41:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 17:41:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:41:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-868strc-n5-surface-mount-horn-strobe-for-outdoor-use-120v-ac-clear>: HTTP status code is not handled or not allowed 2026-01-24 17:41:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:41:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/die_cut_labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:41:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-wire-laundry-cart-tapered-no-hanger>: HTTP status code is not handled or not allowed 2026-01-24 17:41:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 17:41:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/die_cut_labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:41:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ecp44256t-4-250-hp-1188-rpm>: HTTP status code is not handled or not allowed 2026-01-24 17:41:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:41:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shim-ring-m45-o-d-x-35mm-i-d-x-0-30mm-thick-din-988>: HTTP status code is not handled or not allowed 2026-01-24 17:41:51 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 17:41:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-spreader-beam-2500-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 17:41:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 17:41:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-24 17:41:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/174-150473-prop-65-regulatory-sign-parking-garage-14-x-10-polyester-black-on-white>: HTTP status code is not handled or not allowed 2026-01-24 17:41:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:41:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/locks_lockout_devices/lockout_tagout_devices/lockout_padlocks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:41:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:41:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs>: HTTP status code is not handled or not allowed 2026-01-24 17:42:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:42:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mighty-products-mn-ms90-b1630-90-shade-cloth-shade-tarp-16-x-30>: HTTP status code is not handled or not allowed 2026-01-24 17:42:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mighty-products-mn-ms90-b0810-90-shade-cloth-shade-tarp-8-x-10>: HTTP status code is not handled or not allowed 2026-01-24 17:42:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels landed on page that is not a product page. 2026-01-24 17:42:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/locks_lockout_devices/lockout_tagout_devices/lockout_padlocks landed on page that is not a product page. 2026-01-24 17:42:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90-tooth-dual-material-offset-flex-head-teardrop-ratchet-w-3-8-drive-12-1-4>: HTTP status code is not handled or not allowed 2026-01-24 17:42:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels landed on page that is not a product page. 2026-01-24 17:42:09 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs landed on page that is not a product page. 2026-01-24 17:42:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 3797 pages (at 82 pages/min), scraped 2613 items (at 47 items/min) 2026-01-24 17:42:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ecp2280t-4-5-hp-880-rpm>: HTTP status code is not handled or not allowed 2026-01-24 17:42:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-outdoor-steel-flat-bench-perforated-metal-green>: HTTP status code is not handled or not allowed 2026-01-24 17:42:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-and-3-8-drive-sae-metric-mechanics-tool-set-of-121-pieces>: HTTP status code is not handled or not allowed 2026-01-24 17:42:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-cast-iron-k9-jaw-pipe-wrench>: HTTP status code is not handled or not allowed 2026-01-24 17:42:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/hvac_electric_motors/oem_replacement_fan_blower_motors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:42:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/storage_accessories/label_holders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:42:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cem4103t-5-25-hp-1770-rpm>: HTTP status code is not handled or not allowed 2026-01-24 17:42:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/he-i-li-hydraulic-li-positioner-2>: HTTP status code is not handled or not allowed 2026-01-24 17:42:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extruded-bumper-no-holes-4-25-w-x-120-l-x-4-h>: HTTP status code is not handled or not allowed 2026-01-24 17:42:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/photo_presentation_paper already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:42:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/hvac_electric_motors/oem_replacement_fan_blower_motors landed on page that is not a product page. 2026-01-24 17:42:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/bins_totes_containers/bins_totes_container_accessories/storage_container_label_holders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:42:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-gaylord-pallet-container-with-lid-gray>: HTTP status code is not handled or not allowed 2026-01-24 17:42:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbl-rivet-boltless-shelving-72wx48dx120h-5-levels-add-on-no-decking-500-lbs-shelf-cap-tan>: HTTP status code is not handled or not allowed 2026-01-24 17:42:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s90-series-manual-chain-hoist-2-ton-20-ft-lift>: HTTP status code is not handled or not allowed 2026-01-24 17:42:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/paper_stationery/photo_presentation_paper>: HTTP status code is not handled or not allowed 2026-01-24 17:42:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cem2333t-5-15-hp-1765-rpm>: HTTP status code is not handled or not allowed 2026-01-24 17:42:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/bins_totes_containers/bins_totes_container_accessories/storage_container_label_holders landed on page that is not a product page. 2026-01-24 17:42:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/combination-cable-lock-72l-no-8154dpf>: HTTP status code is not handled or not allowed 2026-01-24 17:42:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/copy_printer_paper already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:42:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/fall_protection/self_retracting_lifelines_lanyards/self_retracting_lifelines already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:42:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-rivet-boltless-shelving-48-inch-w-x-12-inch-d-x-84-inch-h-8-levels-add-on-w-o-decking-tan>: HTTP status code is not handled or not allowed 2026-01-24 17:42:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cecp84104t-5-30-hp-1765-rpm>: HTTP status code is not handled or not allowed 2026-01-24 17:42:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/paper_stationery/copy_printer_paper>: HTTP status code is not handled or not allowed 2026-01-24 17:42:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/fall_protection/self_retracting_lifelines_lanyards/self_retracting_lifelines landed on page that is not a product page. 2026-01-24 17:42:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/toolbox-z300-greenx-series-1-4-fold-natural-90-sheets-12-bu-case-20109>: HTTP status code is not handled or not allowed 2026-01-24 17:42:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-polymer-shelf-with-solid-mat-48-w-x-18-d>: HTTP status code is not handled or not allowed 2026-01-24 17:43:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-three-phase-odp-motor-125-hp-1785-rpm-460v-odp>: HTTP status code is not handled or not allowed 2026-01-24 17:43:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v-grip-wire-shelving-spill-tray-48-l-x-24-w-x-2-1-4-d-9-gallon-capacity-textured-regal-blue>: HTTP status code is not handled or not allowed 2026-01-24 17:43:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v-grip-pre-configured-wire-shelving-spill-containment-system-36-w-x-24-d-x-84-h-starter-putty>: HTTP status code is not handled or not allowed 2026-01-24 17:43:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 3886 pages (at 89 pages/min), scraped 2678 items (at 65 items/min) 2026-01-24 17:43:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-swivel-run-tee-10mm-tube-x-1-4-bspp-1-l>: HTTP status code is not handled or not allowed 2026-01-24 17:43:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/filter-multi-pleat-xl10-hc-15-w-x-20-h-x-2-d-12-ctn>: HTTP status code is not handled or not allowed 2026-01-24 17:43:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/li-off-dry-erase-board-cleaner-wipes>: HTTP status code is not handled or not allowed 2026-01-24 17:43:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jack-plate-with-non-skid-surface-18-18-1-14472>: HTTP status code is not handled or not allowed 2026-01-24 17:43:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/54-metal-handle-w-threads-12-case-97159>: HTTP status code is not handled or not allowed 2026-01-24 17:43:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/donnelly-foot-stool-24-w-x-16-h-x-16-d-w-handrail-mri-safe>: HTTP status code is not handled or not allowed 2026-01-24 17:43:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-em4107t-25-hp-3525-rpm>: HTTP status code is not handled or not allowed 2026-01-24 17:43:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-l3705-3-hp-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-24 17:43:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lx-22767osb9118r-extreme-heavy-metal-cutting-saw-blade-18tpi-9-x-1-x-042-50-pack>: HTTP status code is not handled or not allowed 2026-01-24 17:43:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-fdl3516tm-2-hp-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-24 17:43:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/39191-42-inch-heavy-duty-mobile-filtered-fan-belt-drive-1-2-hp-5702-cfm>: HTTP status code is not handled or not allowed 2026-01-24 17:43:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/colored-chalk-assorted-chalk-12-box-510816>: HTTP status code is not handled or not allowed 2026-01-24 17:43:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-flow-control-flow-knob-adjustment-14mm-tube-x-1-2-universal-thread-1-2-5-l>: HTTP status code is not handled or not allowed 2026-01-24 17:43:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flt-etsi51-forkliftag-inspection-tag-2-sided-100-pkg-plastic-7inches-by-5-inches-blue-yellow>: HTTP status code is not handled or not allowed 2026-01-24 17:43:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/engineer-grade-legend-parking-with-right-arrow-narrowcade-minicade>: HTTP status code is not handled or not allowed 2026-01-24 17:43:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/divider-for-36448-blue-pack-of-12-269926>: HTTP status code is not handled or not allowed 2026-01-24 17:43:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultragrate-lifter-carrying-case>: HTTP status code is not handled or not allowed 2026-01-24 17:43:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-buckle-ice-traction-12-overshoe-mens-sz-6-waterproof-vulcanized-rubber-blk>: HTTP status code is not handled or not allowed 2026-01-24 17:43:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-gas-heater-infrared-positive-pressure-straight-tube-175000-btu-40l>: HTTP status code is not handled or not allowed 2026-01-24 17:43:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-boot-men-s-size-14-15-h-steel-pr-midsole-cleated-sole-black>: HTTP status code is not handled or not allowed 2026-01-24 17:43:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-63in-clear-combination-spot-flood-light-bar-with-15-led-1492182>: HTTP status code is not handled or not allowed 2026-01-24 17:43:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/medical_laboratory_refrigeration/medical_refrigerator_freezer_combos already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:43:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_supplies_dispensers/urinal_blocks_screens already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:43:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anti-static-zipper-bag-clear-5-x-8-100-pack>: HTTP status code is not handled or not allowed 2026-01-24 17:44:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ejmm3311t-7-5-hp-1770-rpm>: HTTP status code is not handled or not allowed 2026-01-24 17:44:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-shoe-covers-large-ankle-height-blue-100-pack>: HTTP status code is not handled or not allowed 2026-01-24 17:44:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_supplies_dispensers/urinal_blocks_screens landed on page that is not a product page. 2026-01-24 17:44:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dot-light-plug-3-pin-straight-male-with-2-blunt-cut-ends-and-number10-ring-on-ground-5620355>: HTTP status code is not handled or not allowed 2026-01-24 17:44:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/medical_laboratory_cold_storage already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:44:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 3984 pages (at 98 pages/min), scraped 2750 items (at 72 items/min) 2026-01-24 17:44:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/suncast-trends-garage-storage-4-metal-double-hook-black>: HTTP status code is not handled or not allowed 2026-01-24 17:44:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shop-crane-2200lb-fold-away-otc1819>: HTTP status code is not handled or not allowed 2026-01-24 17:44:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/intellisense-adult-defibrillation-pads-powerheart-g-5-replacing-xela-e-d001a>: HTTP status code is not handled or not allowed 2026-01-24 17:44:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-front-wheel-drive-bearing-adapter-kit-ast78825>: HTTP status code is not handled or not allowed 2026-01-24 17:44:18 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/medical_laboratory_cold_storage landed on page that is not a product page. 2026-01-24 17:44:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hemco-le-fume-hood-wexplosion-proof-light-only-72-x-32-x-48>: HTTP status code is not handled or not allowed 2026-01-24 17:44:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-3-black-front-t-shirt-pullover-lime-polyester-md>: HTTP status code is not handled or not allowed 2026-01-24 17:44:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precoiled-strap-on-pipe-marker-steam-supply-4>: HTTP status code is not handled or not allowed 2026-01-24 17:44:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-5-9-gauge-galvanized-barbed-staple-960-qty>: HTTP status code is not handled or not allowed 2026-01-24 17:44:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-vbm3538-5s-5-hp-1750-rpm>: HTTP status code is not handled or not allowed 2026-01-24 17:44:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/air_tools_accessories/air_nailers_staplers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:44:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/acoustical_thermal_insulation already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:44:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fellowes-linen-presentation-binding-covers-8-x-11-letter-size-navy-200-pack>: HTTP status code is not handled or not allowed 2026-01-24 17:44:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hardware-46in-in-ground-mount-ada-round-square-tables>: HTTP status code is not handled or not allowed 2026-01-24 17:44:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/two_way_radios_accessories/two_way_radios already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:44:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-vent-cap-sidewall-vent>: HTTP status code is not handled or not allowed 2026-01-24 17:44:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/air_tools_accessories/air_nailers_staplers landed on page that is not a product page. 2026-01-24 17:44:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15500-top-hanger-heavy-weight-17l-plastic-wh-pkg-qty-100>: HTTP status code is not handled or not allowed 2026-01-24 17:44:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/acoustical_thermal_insulation landed on page that is not a product page. 2026-01-24 17:44:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-vem3665t-5-hp-1750-rpm>: HTTP status code is not handled or not allowed 2026-01-24 17:44:45 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/two_way_radios_accessories/two_way_radios landed on page that is not a product page. 2026-01-24 17:44:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-2-x-3-brass-pipe-nipple-schedule-40>: HTTP status code is not handled or not allowed 2026-01-24 17:44:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48l-rectangle-plastic-height-adjustable-activity-table-set-with-6-chairs-green>: HTTP status code is not handled or not allowed 2026-01-24 17:44:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stacking-elegance-chiavari-chair-plastic-crystal-ice-clear>: HTTP status code is not handled or not allowed 2026-01-24 17:44:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-x-close-welded-pipe-nipple-schedule-40-304-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 17:44:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-3-16-insert-bit-x-1-s2-modified-steel-gunmetal-grey>: HTTP status code is not handled or not allowed 2026-01-24 17:44:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/credenzas already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:44:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:44:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/gloves_sleeves/arc_flash_flame_resistant_gloves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:44:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-vl3507-75-hp-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-24 17:45:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:45:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-pop-up-canopy-slant-leg-10-l-x-10-w-x-8-11-h-tan>: HTTP status code is not handled or not allowed 2026-01-24 17:45:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/gloves_sleeves/disposable_gloves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:45:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/outdoor_structures_storage/canopies_carports_utility_tents/utility_work_tents already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:45:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spanish-plastic-sign-peligro-no-entre-personal-autorizado-solamente>: HTTP status code is not handled or not allowed 2026-01-24 17:45:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/credenzas landed on page that is not a product page. 2026-01-24 17:45:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/gloves_sleeves/arc_flash_flame_resistant_gloves>: HTTP status code is not handled or not allowed 2026-01-24 17:45:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks landed on page that is not a product page. 2026-01-24 17:45:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 4073 pages (at 89 pages/min), scraped 2810 items (at 60 items/min) 2026-01-24 17:45:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-2-tier-wire-shelving-add-on-unit-36w-x-18d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 17:45:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks landed on page that is not a product page. 2026-01-24 17:45:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/gloves_sleeves/disposable_gloves landed on page that is not a product page. 2026-01-24 17:45:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/outdoor_structures_storage/canopies_carports_utility_tents/utility_work_tents landed on page that is not a product page. 2026-01-24 17:45:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/female-hex-mag-bit-holder-w-c-ring-x-3-gunmetal-bronze-s2-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 17:45:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/little-giant-wstc2-3060-all-welded-cabinet-workbench-steel-top-center-shelf>: HTTP status code is not handled or not allowed 2026-01-24 17:45:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-3mm-insert-bit-x-1-gunmetal-bronze-s2-modified-steel>: HTTP status code is not handled or not allowed 2026-01-24 17:45:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-zdvscp2334t-20-hp-5400-rpm>: HTTP status code is not handled or not allowed 2026-01-24 17:45:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-16-hex-acorn-nut-3-4-hex-3-4-height-stainless-steel-an-1ss>: HTTP status code is not handled or not allowed 2026-01-24 17:45:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/protecta-1161521-vest-style-climbing-harness-back-and-front-d-rings-m-l>: HTTP status code is not handled or not allowed 2026-01-24 17:45:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:45:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jat-419-1-4-straight-composite-die-grinder>: HTTP status code is not handled or not allowed 2026-01-24 17:45:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-3-tier-wire-shelving-add-on-unit-36w-x-18d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 17:45:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/r-rtr-1-small-retrieveing-tool-w-r-cables>: HTTP status code is not handled or not allowed 2026-01-24 17:45:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_components/pallet_rack_hardware_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:45:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_supplies_dispensers/hand_soap_sanitizer_dispensers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:45:34 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs landed on page that is not a product page. 2026-01-24 17:45:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/halstead-hemostatic-forceps-curved-5-l-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 17:45:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelf-truck-esd-24w-x-30l-x-60h-polyurethane-4-swivel-2-brake-casters>: HTTP status code is not handled or not allowed 2026-01-24 17:45:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_components/pallet_rack_hardware_accessories>: HTTP status code is not handled or not allowed 2026-01-24 17:45:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v9770770-mid-sole-ice-cleat-high-profile-black-strap>: HTTP status code is not handled or not allowed 2026-01-24 17:45:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_supplies_dispensers/hand_soap_sanitizer_dispensers landed on page that is not a product page. 2026-01-24 17:45:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-step-heavy-duty-mobile-work-platform-handrails-24-x-48-platform>: HTTP status code is not handled or not allowed 2026-01-24 17:45:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-4-tier-esd-18w-x-48l-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 17:45:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-4-tier-esd-30w-x-60l-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 17:45:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-shelf-kit-for-greenhouses>: HTTP status code is not handled or not allowed 2026-01-24 17:45:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oty-78034-4-pvc-roof-drain-with-abs-dome-and-dam-collar>: HTTP status code is not handled or not allowed 2026-01-24 17:45:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/39166-bore-polisher-spindle-1-to-1-1-2-aluminum-oxide-medium>: HTTP status code is not handled or not allowed 2026-01-24 17:46:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honeywell-thermostat-tradeline-with-thermometer>: HTTP status code is not handled or not allowed 2026-01-24 17:46:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caution-sign-14x10-rigid-plastic-ear-protection-required>: HTTP status code is not handled or not allowed 2026-01-24 17:46:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ls-hollow-core-pole-kit-12-feet>: HTTP status code is not handled or not allowed 2026-01-24 17:46:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 4170 pages (at 97 pages/min), scraped 2879 items (at 69 items/min) 2026-01-24 17:46:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fg-series-fiberglass-pole-set-with-wire-raiser>: HTTP status code is not handled or not allowed 2026-01-24 17:46:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/outdoor_tools_power_equipment/chain_pole_saws already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:46:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/goffs-welding-screen-6w-x-8h-grey>: HTTP status code is not handled or not allowed 2026-01-24 17:46:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-electric-office-partition-panel-60-14-w-x-77-12-h-tan>: HTTP status code is not handled or not allowed 2026-01-24 17:46:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-plastic-garbage-can-with-lid-32-gal-brown>: HTTP status code is not handled or not allowed 2026-01-24 17:46:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/outdoor_tools_power_equipment/chain_pole_saws landed on page that is not a product page. 2026-01-24 17:46:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rust-oleum-high-performance-v2100-rust-preventive-enamel-aero-gloss-wh>: HTTP status code is not handled or not allowed 2026-01-24 17:46:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-office-partition-panel-60-14-w-x-73-12-h-tan>: HTTP status code is not handled or not allowed 2026-01-24 17:46:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2200w-portable-inverter-generator-red>: HTTP status code is not handled or not allowed 2026-01-24 17:46:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/individual-character-stencil-36-letter-x>: HTTP status code is not handled or not allowed 2026-01-24 17:46:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/index_cards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:46:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-visi-sign-fire-alarm-2>: HTTP status code is not handled or not allowed 2026-01-24 17:46:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m3-5x16-metric-pan-head-machine-screws-MI316MTP188>: HTTP status code is not handled or not allowed 2026-01-24 17:46:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-safety-sign-fire-extinguisher-plastic-4>: HTTP status code is not handled or not allowed 2026-01-24 17:46:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/greener-notes-recycled-notes-654rpyw-3-x-3-canary-yellow-100-sheets-12pack>: HTTP status code is not handled or not allowed 2026-01-24 17:46:45 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/paper_stationery/index_cards landed on page that is not a product page. 2026-01-24 17:46:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rust-oleum-9100-system-340voc-dtm-epoxy-mastic-white>: HTTP status code is not handled or not allowed 2026-01-24 17:46:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/outfeed-roller-table-for-ecb-1422v>: HTTP status code is not handled or not allowed 2026-01-24 17:46:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/comfortable-sleep-king-10-foam-pocket-spring-mattress-2-gel-memory-foam-topper>: HTTP status code is not handled or not allowed 2026-01-24 17:46:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/combination_metalworking_machines already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:46:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10x1-1-2-pan-head-self-tapping-screws-type-a-b-thread-1024ABPP>: HTTP status code is not handled or not allowed 2026-01-24 17:46:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rust-oleum-industrial-choice-1600-sys-gen-purp-enamel-aero-banner-red>: HTTP status code is not handled or not allowed 2026-01-24 17:46:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:47:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bedford-tufted-upholstered-twin-size-headboard-in-navy>: HTTP status code is not handled or not allowed 2026-01-24 17:47:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/combination_metalworking_machines landed on page that is not a product page. 2026-01-24 17:47:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bedford-tufted-upholstered-twin-size-headboard-in-black>: HTTP status code is not handled or not allowed 2026-01-24 17:47:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:47:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-32x1-truss-head-machine-screws-1116MPTB>: HTTP status code is not handled or not allowed 2026-01-24 17:47:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards landed on page that is not a product page. 2026-01-24 17:47:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/beds_mattresses/mattresses already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:47:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 4262 pages (at 92 pages/min), scraped 2947 items (at 68 items/min) 2026-01-24 17:47:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-8482-diamond-plate-anti-fatigue-mat-9-16-thick-2-x-cut-to-75-blk-chevron>: HTTP status code is not handled or not allowed 2026-01-24 17:47:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:47:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buna-n-o-ring-3mm-wide-28mm-id-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-24 17:47:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:47:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buna-n-o-ring-dash-367-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 17:47:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/beds_mattresses/mattresses>: HTTP status code is not handled or not allowed 2026-01-24 17:47:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards landed on page that is not a product page. 2026-01-24 17:47:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/stamp_pads already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:47:21 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards landed on page that is not a product page. 2026-01-24 17:47:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50-lbs-umbrella-base-steel-covered-concrete-black>: HTTP status code is not handled or not allowed 2026-01-24 17:47:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/gaskets_seals_orings/orings/viton_orings already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:47:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:47:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges>: HTTP status code is not handled or not allowed 2026-01-24 17:47:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/professional-numberer-self-inking-type-size-1-1-2-six-digits-black>: HTTP status code is not handled or not allowed 2026-01-24 17:47:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:47:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-shelving-unit-36-wx18-dx86-h-solid-shelf>: HTTP status code is not handled or not allowed 2026-01-24 17:47:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/stamp_pads landed on page that is not a product page. 2026-01-24 17:47:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dividers-6-h-drawer-of-global-8482-modular-drawer-cabinet-36-wx24-d-black>: HTTP status code is not handled or not allowed 2026-01-24 17:47:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/plumbing/gaskets_seals_orings/orings/viton_orings landed on page that is not a product page. 2026-01-24 17:47:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges landed on page that is not a product page. 2026-01-24 17:47:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/titan-telescoping-flagpole-kit-15-ft-bronze-black>: HTTP status code is not handled or not allowed 2026-01-24 17:47:34 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges landed on page that is not a product page. 2026-01-24 17:47:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:47:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-digital-crane-scale-w-remote-3000-lb-x-1-b>: HTTP status code is not handled or not allowed 2026-01-24 17:47:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-2-11112-160-psikgcm2-14-cbm-plastic>: HTTP status code is not handled or not allowed 2026-01-24 17:47:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/p-t4-c-sewerooter-t4-drain-cleaning-machine-w-100-x-9-16-cable-cutter-set>: HTTP status code is not handled or not allowed 2026-01-24 17:47:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/bins_totes_containers/trays/parts_trays already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:47:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-8-heavy-hex-nuts-100A563DH>: HTTP status code is not handled or not allowed 2026-01-24 17:47:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/flashlights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:47:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h19r-core-rechargeable-led-headlamp>: HTTP status code is not handled or not allowed 2026-01-24 17:47:49 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges landed on page that is not a product page. 2026-01-24 17:47:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:47:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:47:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:47:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hirsh-industries-10000-series-lateral-file-42in-wide-4-drawer-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 17:47:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/self_luminous_emergency_exit_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:47:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/bins_totes_containers/trays/parts_trays landed on page that is not a product page. 2026-01-24 17:47:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-liter-desk-style-highlighters-chisel-tip-assorted-colors-dozen>: HTTP status code is not handled or not allowed 2026-01-24 17:47:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/flashlights landed on page that is not a product page. 2026-01-24 17:47:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges>: HTTP status code is not handled or not allowed 2026-01-24 17:48:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges landed on page that is not a product page. 2026-01-24 17:48:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/color_coding_labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:48:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/viper-plus-gt-full-reclining-wheelchair-detachable-full-arms-16-seat>: HTTP status code is not handled or not allowed 2026-01-24 17:48:04 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges landed on page that is not a product page. 2026-01-24 17:48:04 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/self_luminous_emergency_exit_signs landed on page that is not a product page. 2026-01-24 17:48:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/partition-t10-torx-bit-steel-hw0037>: HTTP status code is not handled or not allowed 2026-01-24 17:48:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/address_shipping_labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:48:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6304zz-double-shielded-20mm-bore-52mm-od>: HTTP status code is not handled or not allowed 2026-01-24 17:48:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 4342 pages (at 80 pages/min), scraped 2994 items (at 47 items/min) 2026-01-24 17:48:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/color_coding_labels>: HTTP status code is not handled or not allowed 2026-01-24 17:48:12 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-24 17:48:13 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:3 2026-01-24 17:48:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-recycling-extra-plastic-forkli-able-tilt-truck-2-1-2-cu-yard>: HTTP status code is not handled or not allowed 2026-01-24 17:48:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-narrow-procedure-cart-6-drawers-key-lock-18-w-x-18-l-x-40-3-4-h-hammertone-red>: HTTP status code is not handled or not allowed 2026-01-24 17:48:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compact-refrigerator-2-5-cu-ft-dar259w>: HTTP status code is not handled or not allowed 2026-01-24 17:48:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/address_shipping_labels landed on page that is not a product page. 2026-01-24 17:48:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/special_purpose_carts_trucks/medical_supply_carts/medical_storage_carts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:48:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-x-14-working-length-j-hook-style-b>: HTTP status code is not handled or not allowed 2026-01-24 17:48:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dino-lite-am4113t-microtouch-microscope-w-measurement-1-3mp-10x50x-230x>: HTTP status code is not handled or not allowed 2026-01-24 17:48:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/61331-s762p-non-ansi-vest-tricot-hook-loop-closure-hi-vis-pink-md>: HTTP status code is not handled or not allowed 2026-01-24 17:48:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-dia-x-50-l-roll-lifting-beam-yellow-18-l-x-40-w-x-56-h-115lbs-4000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 17:48:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/special_purpose_carts_trucks/medical_supply_carts/medical_storage_carts landed on page that is not a product page. 2026-01-24 17:48:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s371-ansi-class-3-raincoat-hi-vis-lime-md-61480>: HTTP status code is not handled or not allowed 2026-01-24 17:48:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dot-placard-poison-inhalation-hazard-6-1>: HTTP status code is not handled or not allowed 2026-01-24 17:48:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n-ferno-6471-thermal-overalls-black-2xl-41216>: HTTP status code is not handled or not allowed 2026-01-24 17:48:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-series-lt-weight-emergency-cart-6-drawers-key-lock-36-3-4-wx22-lx40-1-2-h-hammertone-green>: HTTP status code is not handled or not allowed 2026-01-24 17:48:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hugh-52-inch-matte-white-with-led-light-kit>: HTTP status code is not handled or not allowed 2026-01-24 17:48:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-diamond-series-undercounter-flammable-storage-refrigerator-5-cu-ft>: HTTP status code is not handled or not allowed 2026-01-24 17:48:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-72-economy-h-beam-adjustable-length-yellow-102-l-x-40-w-x-36-h-425lbs-4000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 17:48:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/markerboard-activity-table-24in-x-36in-rectangle-standard-adjustable-height>: HTTP status code is not handled or not allowed 2026-01-24 17:48:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tall-savary-dilator-drying-cart-hepa-filter-23-89-100-wx22-lx51-89-100-h-purple>: HTTP status code is not handled or not allowed 2026-01-24 17:48:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/120v-1500w-spst-flush-mounting>: HTTP status code is not handled or not allowed 2026-01-24 17:48:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/empty-label-pvc-free-film-152-x-152-mm-500-roll>: HTTP status code is not handled or not allowed 2026-01-24 17:48:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/electrical_boxes_enclosures_raceways/electrical_boxes/steel_electrical_boxes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:48:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/un3082-orange-panel-permanent-vinyl-25-pack>: HTTP status code is not handled or not allowed 2026-01-24 17:49:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-series-aluminium-medical-cart-short-height-36-3-4-w-x-22-l-x-37-1-4-h-brushed-silver>: HTTP status code is not handled or not allowed 2026-01-24 17:49:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_alarms_signals/audible_alarms already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:49:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-short-isolation-cart-5-drawers-e-lock-30-w-x-22-l-x-37-1-4-h-hammertone-black>: HTTP status code is not handled or not allowed 2026-01-24 17:49:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/electrical_boxes_enclosures_raceways/electrical_boxes/steel_electrical_boxes landed on page that is not a product page. 2026-01-24 17:49:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-ultra-hi-flow-automatic-nozzle-n100dau13g>: HTTP status code is not handled or not allowed 2026-01-24 17:49:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 4439 pages (at 97 pages/min), scraped 3065 items (at 71 items/min) 2026-01-24 17:49:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/boss-combo-lateral-file-31-x-22-cherry>: HTTP status code is not handled or not allowed 2026-01-24 17:49:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ptfe-plastic-bar-18-thick-x-12w-x-24l>: HTTP status code is not handled or not allowed 2026-01-24 17:49:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-direct-flow-circulation-fan-36df-w-mount-1-2-hp-13140-cfm-115-230v>: HTTP status code is not handled or not allowed 2026-01-24 17:49:18 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_alarms_signals/audible_alarms landed on page that is not a product page. 2026-01-24 17:49:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyjohn-pjn3-portable-restroom-tan>: HTTP status code is not handled or not allowed 2026-01-24 17:49:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-16-18-carbon-steel-heavy-wall-insert-for-metal-335-5>: HTTP status code is not handled or not allowed 2026-01-24 17:49:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-tall-emergency-crash-cart-6-drawers-36-3-4-w-x-22-l-x-43-3-4-h-white>: HTTP status code is not handled or not allowed 2026-01-24 17:49:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-tall-isolation-cart-4-drawers-37-1-2-w-x-22-l-x-42-63-100-h-hammertone-red>: HTTP status code is not handled or not allowed 2026-01-24 17:49:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/engraved-occupancy-sign-out-black>: HTTP status code is not handled or not allowed 2026-01-24 17:49:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1161442-comfort-vest-style-positioning-climbing-harness-quick-connect-s>: HTTP status code is not handled or not allowed 2026-01-24 17:49:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/framed-fingerless-work-glove-brown-black-2xl-1-pair-gfd-17kbr2l>: HTTP status code is not handled or not allowed 2026-01-24 17:49:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/500ml-bottle-top-filter-pes-filter-material-0-45-181-m-90mm-sterile-24-pk>: HTTP status code is not handled or not allowed 2026-01-24 17:49:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/emergency_vehicle_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:49:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wedgebit-drill-bit-straight-shank-heavy-duty-size-12-usable-11-l-overall-13-l>: HTTP status code is not handled or not allowed 2026-01-24 17:49:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/emergency_vehicle_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:49:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/emergency_vehicle_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:49:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precut-rheumatoid-arthritis-resting-splint-1-8-mini-perforated-small>: HTTP status code is not handled or not allowed 2026-01-24 17:49:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/patient_support_immobilization/splints_supports already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:49:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/graphic-machine-labels-danger-no-smoking>: HTTP status code is not handled or not allowed 2026-01-24 17:49:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/tables/folding_tables already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:49:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/emergency_vehicle_lights>: HTTP status code is not handled or not allowed 2026-01-24 17:49:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/emergency_vehicle_lights landed on page that is not a product page. 2026-01-24 17:49:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c-5-8-25m-stick-staples>: HTTP status code is not handled or not allowed 2026-01-24 17:49:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/emergency_vehicle_lights landed on page that is not a product page. 2026-01-24 17:49:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/patient_support_immobilization/splints_supports landed on page that is not a product page. 2026-01-24 17:49:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/clear_view_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:49:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a36h3612sslp3pt-continuous-hinge-w3point-ltch-type-4x-3600x3600x1200in-ss-type-304>: HTTP status code is not handled or not allowed 2026-01-24 17:49:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/tables/folding_tables landed on page that is not a product page. 2026-01-24 17:49:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tapered-roller-bearing-cone-lm102949>: HTTP status code is not handled or not allowed 2026-01-24 17:50:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/surface-mounted-corner-guard-90-degrees-2-ft-ft-wings-12-ft-h-with-caps-taupe>: HTTP status code is not handled or not allowed 2026-01-24 17:50:04 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/cabinets/clear_view_cabinets landed on page that is not a product page. 2026-01-24 17:50:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:50:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-cantilever-rack-double-sided-starter-hd-48w-x-59d-x-8h-w-lip>: HTTP status code is not handled or not allowed 2026-01-24 17:50:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/composites/composite_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:50:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vinyl-surface-mounted-corner-guard-90-degrees-corner-3-4-inch-wings-4-ft-h-cappuccino-vinyl-w-tape>: HTTP status code is not handled or not allowed 2026-01-24 17:50:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 4530 pages (at 91 pages/min), scraped 3129 items (at 64 items/min) 2026-01-24 17:50:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/american-lock-solid-aluminum-rectangular-padlock-purple-no-a1305prp>: HTTP status code is not handled or not allowed 2026-01-24 17:50:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hoist-ring-180-pivot-4400-lb-load-capacity>: HTTP status code is not handled or not allowed 2026-01-24 17:50:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powerpoint-hoist-ring-360-pivot-8800-lb-load-capacity>: HTTP status code is not handled or not allowed 2026-01-24 17:50:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs landed on page that is not a product page. 2026-01-24 17:50:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/composites/composite_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-24 17:50:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-grade-100-250-ft-dr-black>: HTTP status code is not handled or not allowed 2026-01-24 17:50:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-5-16in-heavy-duty-a-frame-black-cast-coupler-0091595>: HTTP status code is not handled or not allowed 2026-01-24 17:50:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pipe-tongs-2000-lbs-capacity-yellow-steel-adjustable-diameter-4-8-1-31-opening>: HTTP status code is not handled or not allowed 2026-01-24 17:50:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:50:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mpb68-4-1-2-x-4-1-2-nrp-us26d-6-l-x-6-w-x-2-h-8-1-2-dia>: HTTP status code is not handled or not allowed 2026-01-24 17:50:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:50:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-for-7400-reach-pallet-trucks-ra-632-052-b-xl>: HTTP status code is not handled or not allowed 2026-01-24 17:50:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/cart_truck_accessories/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:50:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unisex-excel-flame-resistant-lab-coat-7-oz-light-blue-cotton-xl>: HTTP status code is not handled or not allowed 2026-01-24 17:50:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24x36x74-chrome-wire-shelving-with-21-24-d-hopper-bins-red>: HTTP status code is not handled or not allowed 2026-01-24 17:50:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-24 17:50:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 17:50:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/wall_mount_shelving already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:50:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/odp-w-rigid-base-1-5hp-1200rpm>: HTTP status code is not handled or not allowed 2026-01-24 17:50:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/cart_truck_accessories/pallet_truck_parts_accessories/pallet_truck_wheel_kits landed on page that is not a product page. 2026-01-24 17:50:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-assembly-for-8hbw23-pallet-trucks-to-00590-53386-71-a>: HTTP status code is not handled or not allowed 2026-01-24 17:50:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/cart_truck_accessories/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:50:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/konnect-5-piece-commercial-kit-5-pack-8-case-white>: HTTP status code is not handled or not allowed 2026-01-24 17:50:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acorn-washfountain-semi-circular-3-stations-foot-operated>: HTTP status code is not handled or not allowed 2026-01-24 17:50:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/wall_mount_shelving landed on page that is not a product page. 2026-01-24 17:50:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-sheet-34-thick-x-18-wide-x-36-long>: HTTP status code is not handled or not allowed 2026-01-24 17:50:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:50:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/assorted-refill-strips-white-16-pack>: HTTP status code is not handled or not allowed 2026-01-24 17:50:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/office-konnect-153-stackable-desk-organizer-4-piece-white-6-case-black>: HTTP status code is not handled or not allowed 2026-01-24 17:50:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fuse-holder-for-wave-pallet-trucks-cr-127037>: HTTP status code is not handled or not allowed 2026-01-24 17:50:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-built-in-combo-lock-for-horizontal-latch-box-locker-rh>: HTTP status code is not handled or not allowed 2026-01-24 17:50:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/cart_truck_accessories/pallet_truck_parts_accessories/pallet_truck_wheel_kits landed on page that is not a product page. 2026-01-24 17:50:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-containment-berm-economy-model-4-x-6-x-1>: HTTP status code is not handled or not allowed 2026-01-24 17:50:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/antimicrobial-130-sheet-heavy-duty-stapler-6-case>: HTTP status code is not handled or not allowed 2026-01-24 17:50:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-24 17:51:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-for-pmx-walkie-pallet-truck>: HTTP status code is not handled or not allowed 2026-01-24 17:51:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-built-in-combination-lock-purple-dial-left-hinged>: HTTP status code is not handled or not allowed 2026-01-24 17:51:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lockable-storage-box-7-gallon-269997>: HTTP status code is not handled or not allowed 2026-01-24 17:51:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/composites/composite_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:51:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/accentra-desktop-stapler-20-sheet-capacity-moss>: HTTP status code is not handled or not allowed 2026-01-24 17:51:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 4623 pages (at 93 pages/min), scraped 3189 items (at 60 items/min) 2026-01-24 17:51:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tumbler-clrw-9-5oz-72qty-slate-blue>: HTTP status code is not handled or not allowed 2026-01-24 17:51:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-ner-electric-chain-hoist-hook-suspension-2-15-28>: HTTP status code is not handled or not allowed 2026-01-24 17:51:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-sheet-w-lse-acrylic-adhesive-132-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 17:51:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/composites/composite_bar_stock landed on page that is not a product page. 2026-01-24 17:51:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-bar-18-thick-x-2-12-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 17:51:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/koch-filter-merv-8-standard-capacity-xl8-pleated-panel-ext-surface-12-inch-w-x-24-inch-h-x-1-inch-d>: HTTP status code is not handled or not allowed 2026-01-24 17:51:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slb-lever-hoists-1650-lb-capacity-5-ft-lift-13-head-room>: HTTP status code is not handled or not allowed 2026-01-24 17:51:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-lb-lever-hoist-2-10-foot-lift>: HTTP status code is not handled or not allowed 2026-01-24 17:51:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hydraulic-pump-for-wp-2000-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-24 17:51:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cup-mug-slate-blue>: HTTP status code is not handled or not allowed 2026-01-24 17:51:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-cf-hand-chain-hoist-1-20-lift>: HTTP status code is not handled or not allowed 2026-01-24 17:51:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/koch-filter-merv-8-high-capacity-xl8-pleated-extended-surface-20-inch-w-x-20-inch-h-x-2-inch-d>: HTTP status code is not handled or not allowed 2026-01-24 17:51:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nps-12ft-mobile-cafeteria-stool-unit-with-mdf-core-top-protect-edge-gray-top-gray-stool>: HTTP status code is not handled or not allowed 2026-01-24 17:51:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gripper-front-short-butcher-coat-white-polyestercotton-2xl>: HTTP status code is not handled or not allowed 2026-01-24 17:51:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pin-for-smx-45-walkie-pallet-truck-pr-312651-001>: HTTP status code is not handled or not allowed 2026-01-24 17:51:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/plumbing_valves/laboratory_gas_valves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:51:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-5202-double-row-angular-contact-bearing-open-15mm-bore-x-35mm-od-x-15-9mm-w>: HTTP status code is not handled or not allowed 2026-01-24 17:51:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pre-filled-container-40ml-1-34-oz-click-close-lid-attached-hazard-label-96-pack>: HTTP status code is not handled or not allowed 2026-01-24 17:51:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kwikool-portable-air-conditioner-kib12023-10-ton-120000-btu-with-two-20-in-condenser-flanges>: HTTP status code is not handled or not allowed 2026-01-24 17:51:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_repair_maintenance/tires_wheels/tire_chains already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:51:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cook-pants-black-white-check-polyestercotton-twill-32-x-32>: HTTP status code is not handled or not allowed 2026-01-24 17:51:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:51:51 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/plumbing/plumbing_valves/laboratory_gas_valves landed on page that is not a product page. 2026-01-24 17:51:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:51:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x36x74-chrome-wire-shelving-with-77-4-h-shelf-bins-yellow>: HTTP status code is not handled or not allowed 2026-01-24 17:51:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-plastic-sheet-14-thick-x-12-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 17:51:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ballymore-9-step-perforated-cantilever-ladder-42-overhang>: HTTP status code is not handled or not allowed 2026-01-24 17:51:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gripper-front-butcher-frock-winside-top-pocket-white-polyestercotton-twill-3xl>: HTTP status code is not handled or not allowed 2026-01-24 17:51:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_repair_maintenance/tires_wheels/tire_chains landed on page that is not a product page. 2026-01-24 17:52:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-24 17:52:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits>: HTTP status code is not handled or not allowed 2026-01-24 17:52:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/utility-table-72-x-24-black>: HTTP status code is not handled or not allowed 2026-01-24 17:52:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/composites/composite_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:52:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:52:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/butcher-wrap-white-polyestercombed-cotton-4xl>: HTTP status code is not handled or not allowed 2026-01-24 17:52:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eggcrate-return-grille-surface-mount-10-x-10-white-pk5>: HTTP status code is not handled or not allowed 2026-01-24 17:52:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 4719 pages (at 96 pages/min), scraped 3255 items (at 66 items/min) 2026-01-24 17:52:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/35-ft-15ft-y-7x7-stranded-steel-static-discharge-reel>: HTTP status code is not handled or not allowed 2026-01-24 17:52:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/composites/composite_bar_stock landed on page that is not a product page. 2026-01-24 17:52:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 17:52:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/collarless-butcher-wrap-wpockets-light-blue-polyestercombed-cotton-s>: HTTP status code is not handled or not allowed 2026-01-24 17:52:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pneumatic-hydraulic-mobile-scissor-lift-table-1000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 17:52:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g-10-garolite-sheet-116-thick-x-12-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 17:52:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-kit-for-electric-pallet-jack-truck-gwk-8hbw30-lw-fits-toyota-model-8hbw30>: HTTP status code is not handled or not allowed 2026-01-24 17:52:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-18-thick-x-5-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 17:52:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/19129-womens-cap-style-hard-hat-hi-viz-pink-6-point-slide-lock-suspension>: HTTP status code is not handled or not allowed 2026-01-24 17:52:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:52:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24x48x74-chrome-wire-shelving-with-66-shelf-bins-blue>: HTTP status code is not handled or not allowed 2026-01-24 17:52:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:52:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transfer-pipet-5ml-fine-tip-153mm-sterile-400-pack>: HTTP status code is not handled or not allowed 2026-01-24 17:52:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slide-mailer-polypropylene-for-1-slide-natural-100-pack>: HTTP status code is not handled or not allowed 2026-01-24 17:52:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5x-detector-mono-lelo2co-h2s-so2-ul-charcoal-instrument-only-10116924>: HTTP status code is not handled or not allowed 2026-01-24 17:52:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 17:52:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:52:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 17:52:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-tuftex-yellow-bib-overall-plain-front-pvc-size-extra-large>: HTTP status code is not handled or not allowed 2026-01-24 17:52:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/automotive_shop_equipment/creeper_seats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:52:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:52:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milwaukee-sds-bit-14-4-6-48-20-7431>: HTTP status code is not handled or not allowed 2026-01-24 17:52:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:52:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-24 17:52:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:52:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/diamond-culture-tube-cap-for-20mm-glass-culture-tubes-polypropylene-yellow-500-pack>: HTTP status code is not handled or not allowed 2026-01-24 17:52:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:52:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/vehicle-fleet-maintenance/automotive_shop_equipment/creeper_seats landed on page that is not a product page. 2026-01-24 17:52:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-men-s-boot-16-flex-3-black-plain-toe-pvc-size-13>: HTTP status code is not handled or not allowed 2026-01-24 17:52:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 17:52:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-head-cap-screw-m4-x-0-7-x-18mm-steel-zinc-clear-class-8-8-din-933-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 17:52:50 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 17:52:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-uhmw-polyethylene-plastic-sheet-1-thick-x-8-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-24 17:52:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 17:52:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 17:52:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/goodyear-coin-pattern-rubber-floor-mat-3-5mm-thick-3-x-6-black>: HTTP status code is not handled or not allowed 2026-01-24 17:52:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-lockout-hasp-aluminum-2w-x-5-78l>: HTTP status code is not handled or not allowed 2026-01-24 17:53:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tach-it-sms-tagging-needles>: HTTP status code is not handled or not allowed 2026-01-24 17:53:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/head_face_protection/head_protection_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:53:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tach-it-e7r-ss-stainless-steel-tape-bag-sealer>: HTTP status code is not handled or not allowed 2026-01-24 17:53:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/composites/composite_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:53:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-safety-padlock-short-zenex-316-body-x-1-12-shackle-teal>: HTTP status code is not handled or not allowed 2026-01-24 17:53:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 4808 pages (at 89 pages/min), scraped 3311 items (at 56 items/min) 2026-01-24 17:53:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:53:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/head_face_protection/head_protection_accessories>: HTTP status code is not handled or not allowed 2026-01-24 17:53:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galvanized-steel-flow-rail-10-oal-dual-galvanized-channel-1-1-2-centers>: HTTP status code is not handled or not allowed 2026-01-24 17:53:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/overboots_overshoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:53:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/composites/composite_rod_stock landed on page that is not a product page. 2026-01-24 17:53:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storage-bin-cabinet-186-yellow-hook-on-bins-48w-24d-72h>: HTTP status code is not handled or not allowed 2026-01-24 17:53:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:53:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:53:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 17:53:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/rainwear already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:53:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:53:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chem-lawn-spray-gun>: HTTP status code is not handled or not allowed 2026-01-24 17:53:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/overboots_overshoes landed on page that is not a product page. 2026-01-24 17:53:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 17:53:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fabric-guest-chair-with-wood-frame>: HTTP status code is not handled or not allowed 2026-01-24 17:53:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 17:53:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/rainwear>: HTTP status code is not handled or not allowed 2026-01-24 17:53:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=e-1236vs-with-taper-attachment-and-collet-closer already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:53:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges landed on page that is not a product page. 2026-01-24 17:53:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-drawer-full-height-48-w-cabinet-black-1>: HTTP status code is not handled or not allowed 2026-01-24 17:53:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wireless-tube-rack-13mm-full-size-72-place-nylon-orange>: HTTP status code is not handled or not allowed 2026-01-24 17:53:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/siesta-air-31-square-dining-table-with-laminated-top-black>: HTTP status code is not handled or not allowed 2026-01-24 17:53:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=e-1236vs-with-taper-attachment-and-collet-closer landed on page that is not a product page. 2026-01-24 17:53:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-camwear-spoon-beige>: HTTP status code is not handled or not allowed 2026-01-24 17:53:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slide-mailer-polypropylene-flip-top-lid-for-5-slides-natural-1000-pack>: HTTP status code is not handled or not allowed 2026-01-24 17:53:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=e-1340vs-with-acu-rite-303-css-dro-with-collet-closer already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:53:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/13-camwear-perforated-spoon-black>: HTTP status code is not handled or not allowed 2026-01-24 17:53:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jtm-1254vs-with-acu-rite-203-3x-k-dro-x-powerfeed-air-power-drawbar>: HTTP status code is not handled or not allowed 2026-01-24 17:53:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microscope-slides-glass-25-x-75mm-45-beveled-edges-clipped-corners-pink-frosted1440-pk>: HTTP status code is not handled or not allowed 2026-01-24 17:53:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/lathes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:53:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-quick-coupling-european-high-flow-profile-with-1-4-mnpt-connection>: HTTP status code is not handled or not allowed 2026-01-24 17:53:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mailing-tubes-with-caps-1-1-2-x-20>: HTTP status code is not handled or not allowed 2026-01-24 17:53:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=e-1340vs-with-acu-rite-303-css-dro-with-collet-closer>: HTTP status code is not handled or not allowed 2026-01-24 17:53:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/lathes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:53:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blockhead-fiberglass-full-brim-hard-hat-non-vented-composite-wrap-4-pt-suspension>: HTTP status code is not handled or not allowed 2026-01-24 17:53:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/lathes landed on page that is not a product page. 2026-01-24 17:53:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glassplate-chiller-24-34w-26-58d-33-38h-t-24-gc>: HTTP status code is not handled or not allowed 2026-01-24 17:53:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microscope-slides-glass-25-x-75mm-45-beveled-edges-clipped-corners-green-frosted1440-pk>: HTTP status code is not handled or not allowed 2026-01-24 17:54:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/lathes landed on page that is not a product page. 2026-01-24 17:54:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-36-gal-concrete-receptacle-alum-lid-tan>: HTTP status code is not handled or not allowed 2026-01-24 17:54:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=etm-949-mill-3-axis-acu-rite-303-quill-dro-x-axis-jet-powerfeed-usa-pwrd-draw-bar already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:54:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microcentrifuge-tube-20ml-polypropylene-orange-stand-up-zipbag-mg-500-pack>: HTTP status code is not handled or not allowed 2026-01-24 17:54:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=etm-949-mill-2-axis-newall-dp700-dro-x-y-axis-jet-powerfeeds-usa-powered-draw-bar already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:54:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-mobile-dunnage-rack-42x20x8>: HTTP status code is not handled or not allowed 2026-01-24 17:54:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 4893 pages (at 85 pages/min), scraped 3365 items (at 54 items/min) 2026-01-24 17:54:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=egh-1740-with-acu-rite-303-dro-with-taper-attachment-and-collet-closer already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:54:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evs-949-mill-with-3-axis-acu-rite-203-quill-dro-and-x-axis-jet-powerfeed>: HTTP status code is not handled or not allowed 2026-01-24 17:54:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15sp15c-1p-1-5-inch-cast-iron-self-priming-cf-pump-130gpm-125psi-epdm-epr-seal-1-1-2hp>: HTTP status code is not handled or not allowed 2026-01-24 17:54:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=etm-949-mill-2-axis-newall-dp700-dro-x-y-axis-jet-powerfeeds-usa-powered-draw-bar landed on page that is not a product page. 2026-01-24 17:54:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wireless-microtube-rack-with-handles-for-15ml-and-20ml-microcentrifuge-tubes-24-place-white>: HTTP status code is not handled or not allowed 2026-01-24 17:54:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=etm-949-mill-3-axis-acu-rite-303-quill-dro-x-axis-jet-powerfeed-usa-pwrd-draw-bar landed on page that is not a product page. 2026-01-24 17:54:18 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=egh-1740-with-acu-rite-303-dro-with-taper-attachment-and-collet-closer landed on page that is not a product page. 2026-01-24 17:54:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camrack-base-rack-7-1-2-inch-inside-stack-height-red-nsf>: HTTP status code is not handled or not allowed 2026-01-24 17:54:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-corrugated-boxes-20-x-15-x-6>: HTTP status code is not handled or not allowed 2026-01-24 17:54:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lt579g01-rendezvous-undercounter-sg-lavatory-cotton-white>: HTTP status code is not handled or not allowed 2026-01-24 17:54:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/580xl100-standard-timing-belt-xl-1-x-58-t290-trapezoidal>: HTTP status code is not handled or not allowed 2026-01-24 17:54:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1846-sb-6-5-8-x-1-4-aluminum-cut-cylindrical-bur-w-end-cut>: HTTP status code is not handled or not allowed 2026-01-24 17:54:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forklift-container-ramp-72in-w-x-48in-l-15-000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 17:54:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-x-60-mat-ultimat-gray-um-3060g>: HTTP status code is not handled or not allowed 2026-01-24 17:54:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carboy-with-handles-autoclavable-polypropylene-10-liter>: HTTP status code is not handled or not allowed 2026-01-24 17:54:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/210xl050-standard-timing-belt-xl-1-2-x-21-t105-trapezoidal>: HTTP status code is not handled or not allowed 2026-01-24 17:54:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-shelf-landscaping-cart>: HTTP status code is not handled or not allowed 2026-01-24 17:54:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/p100-respirator-filter-cartridge-7580p100-2-pk>: HTTP status code is not handled or not allowed 2026-01-24 17:54:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fusion-earplugs-regular-corded-blue-yellow-fus30-hp-1000-pairs>: HTTP status code is not handled or not allowed 2026-01-24 17:54:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/combo-tester-x3-tester-only>: HTTP status code is not handled or not allowed 2026-01-24 17:54:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-joint-3-way-fittings-1dia-furniture-grade-pvc-white>: HTTP status code is not handled or not allowed 2026-01-24 17:54:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-bollard-with-base-removable-plastic-cap-chain-slots-yellow-60-height>: HTTP status code is not handled or not allowed 2026-01-24 17:55:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/structural-plastic-solid-shelving-66-x-36-x-51>: HTTP status code is not handled or not allowed 2026-01-24 17:55:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ma1500nist-true-rms-acdc-clamp-meter-and-ncv-orangegreen-nist-certified>: HTTP status code is not handled or not allowed 2026-01-24 17:55:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foil-insulation-tape-3-inch-x-50-yards>: HTTP status code is not handled or not allowed 2026-01-24 17:55:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fx8-12b-flexelene-tubing12-id-x-34-od-100-length-blue>: HTTP status code is not handled or not allowed 2026-01-24 17:55:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/lockers/bulk_storage_lockers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:55:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 4991 pages (at 98 pages/min), scraped 3438 items (at 73 items/min) 2026-01-24 17:55:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/graduated-cylinder-polypropylene-molded-graduations-250ml>: HTTP status code is not handled or not allowed 2026-01-24 17:55:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-duty-angle-post-1-7-8-w-x-1-7-8-d-x-120-h-14-gauge>: HTTP status code is not handled or not allowed 2026-01-24 17:55:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/electrical_power_testing/multimeters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:55:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/table-wall-roll-stand-24-width-capacity-48-diameter>: HTTP status code is not handled or not allowed 2026-01-24 17:55:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/office_furniture/literature_mail_sorters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:55:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/lockers/bulk_storage_lockers>: HTTP status code is not handled or not allowed 2026-01-24 17:55:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/extra_heavy_boltless_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:55:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sfp22a-cp-8-in-trim-plate-kit-single-hole-chrome-finish>: HTTP status code is not handled or not allowed 2026-01-24 17:55:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/testing_inspection_instruments/electrical_power_testing/multimeters landed on page that is not a product page. 2026-01-24 17:55:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-oneline-guardrail-steel-hdpe-plastic-yellow-96-x-36>: HTTP status code is not handled or not allowed 2026-01-24 17:55:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/office_furniture/literature_mail_sorters landed on page that is not a product page. 2026-01-24 17:55:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/extra_heavy_boltless_shelves>: HTTP status code is not handled or not allowed 2026-01-24 17:55:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securall-galvanized-steel-floor-grating-b3200>: HTTP status code is not handled or not allowed 2026-01-24 17:55:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hydrofarm-ck64001-vented-humidity-dome-2-for-seeding-tray->: HTTP status code is not handled or not allowed 2026-01-24 17:55:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/adhesives_sealants_tape/adhesives_glue/general_purpose_glue already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:55:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chesapeake-86776-14-inch-high-pvc-boot-safety-toe-size-15-black>: HTTP status code is not handled or not allowed 2026-01-24 17:55:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-o-smith-5-5-8-split-capacitor-condenser-fan-motor-825-rpm-460-volts>: HTTP status code is not handled or not allowed 2026-01-24 17:55:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6702-chill-its-174-evaporative-cooling-bandana-embedded-polymers-tie-solid-blue>: HTTP status code is not handled or not allowed 2026-01-24 17:55:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-childs-rectangular-outdoor-steel-picnic-table-expanded-metal-red>: HTTP status code is not handled or not allowed 2026-01-24 17:55:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-pallet-covers-48-x-48-x-84-3-mil-50-pack>: HTTP status code is not handled or not allowed 2026-01-24 17:55:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/adhesives_sealants_tape/adhesives_glue/general_purpose_glue landed on page that is not a product page. 2026-01-24 17:55:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-pallet-covers-48-x-46-x-72-2-mil-50-pack>: HTTP status code is not handled or not allowed 2026-01-24 17:55:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/morrill-solid-state-unit-bearing-fan-motor-35-50-watts-208-230-volts-1>: HTTP status code is not handled or not allowed 2026-01-24 17:55:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drying-rack-72-place-removable-pegs-high-impact-polystyrene>: HTTP status code is not handled or not allowed 2026-01-24 17:55:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-3-3-shaded-pole-self-cooled-motor-230-volts-1500-rpm>: HTTP status code is not handled or not allowed 2026-01-24 17:55:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/platinum-25-ton-air-hydraulic-axle-jack>: HTTP status code is not handled or not allowed 2026-01-24 17:55:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/number-1-adjustable-height-complete-packing-workbench-laminate-square-edge-72-x-30-black>: HTTP status code is not handled or not allowed 2026-01-24 17:56:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-20bs11hx45-metric-1-14-pitch-45mm-finished-bore-11-teeth>: HTTP status code is not handled or not allowed 2026-01-24 17:56:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60a27hx1-a-plate-34-pitch-1-bore-27-teeth>: HTTP status code is not handled or not allowed 2026-01-24 17:56:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sling-positioning-2000lb-for-shop-cranes>: HTTP status code is not handled or not allowed 2026-01-24 17:56:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 5085 pages (at 94 pages/min), scraped 3506 items (at 68 items/min) 2026-01-24 17:56:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/packing-list-enclosed-2-3-white-red>: HTTP status code is not handled or not allowed 2026-01-24 17:56:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-5-split-capacitor-motor-230-volts-925-rpm>: HTTP status code is not handled or not allowed 2026-01-24 17:56:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/utility-grade-high-adhesion-masking-tape-natural-24mm-x-55m-pack-of-36>: HTTP status code is not handled or not allowed 2026-01-24 17:56:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/biohazard-2-2-fluorescent-red-black>: HTTP status code is not handled or not allowed 2026-01-24 17:56:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forkli-container-ramp-60-w-x-96-l-15-000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 17:56:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/packaging/envelopes_mailers/poly_mailers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:56:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/hvac_controls/thermostats_temperature_controls/thermostats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:56:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60bs38x134-34-pitch-1-34-finished-bore-38-teeth>: HTTP status code is not handled or not allowed 2026-01-24 17:56:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-slotted-pallet-rack-add-on-no-deck-120-inches-w-x-42-inches-d-x-144-inches-h>: HTTP status code is not handled or not allowed 2026-01-24 17:56:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_components/pallet_rack_enclosures already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:56:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-100bs16hx1716-1-14-pitch-1-716-finished-bore-16-teeth>: HTTP status code is not handled or not allowed 2026-01-24 17:56:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/touch-free-condiment-dispenser-silver>: HTTP status code is not handled or not allowed 2026-01-24 17:56:34 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/hvac_controls/thermostats_temperature_controls/thermostats landed on page that is not a product page. 2026-01-24 17:56:34 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/packaging/envelopes_mailers/poly_mailers landed on page that is not a product page. 2026-01-24 17:56:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/13-piece-external-torx-socket-set-with-14-38-12-drive-tang>: HTTP status code is not handled or not allowed 2026-01-24 17:56:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extract-flavor-liquid-4-4-3-4-white-red-black>: HTTP status code is not handled or not allowed 2026-01-24 17:56:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_components/pallet_rack_enclosures>: HTTP status code is not handled or not allowed 2026-01-24 17:56:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-dia-green-disc-with-16>: HTTP status code is not handled or not allowed 2026-01-24 17:56:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-100bs12hx1716-1-14-pitch-1-716-finished-bore-12-teeth>: HTTP status code is not handled or not allowed 2026-01-24 17:56:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-dj10p1hm-10-hp-3-phase-3505-rpm>: HTTP status code is not handled or not allowed 2026-01-24 17:56:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-dj7p2hm-7-5-hp-3-phase-1765-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:56:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-ec09b-1-hp-1-phase-3450-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:56:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-8330-1-4-hp-1-phase-1350-rpm>: HTTP status code is not handled or not allowed 2026-01-24 17:56:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/food_beverages/snacks_candies/cookies_pastries already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:56:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-1964-1-8-hp-1-phase-1500-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:56:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-ec1002-1-hp-1-phase-3450-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:56:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20500-l-47-impact-labels-50g-range-red-50-box>: HTTP status code is not handled or not allowed 2026-01-24 17:56:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-t13cm2j-1-3-hp-1-phase-1725-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:56:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/non-dairy-powdered-creamer-original-11-oz-canister-12-carton>: HTTP status code is not handled or not allowed 2026-01-24 17:56:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=us-motor-dj7p2hm-7-5-hp-3-phase-1765-rpm landed on page that is not a product page. 2026-01-24 17:56:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=us-motor-ec09b-1-hp-1-phase-3450-rpm landed on page that is not a product page. 2026-01-24 17:56:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-t2c2jh-2-hp-1-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-24 17:57:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/food_beverages/snacks_candies/cereal_breakfast_snacks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:57:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=us-motor-1964-1-8-hp-1-phase-1500-rpm landed on page that is not a product page. 2026-01-24 17:57:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=us-motor-ec1002-1-hp-1-phase-3450-rpm landed on page that is not a product page. 2026-01-24 17:57:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hershey-s-miniatures-assortment-56-oz-180-pieces-gr>: HTTP status code is not handled or not allowed 2026-01-24 17:57:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=us-motor-t13cm2j-1-3-hp-1-phase-1725-rpm landed on page that is not a product page. 2026-01-24 17:57:04 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/food_beverages/snacks_candies/cookies_pastries landed on page that is not a product page. 2026-01-24 17:57:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-inch-job-site-table-saw-with-scissor-stand-dwe7491x>: HTTP status code is not handled or not allowed 2026-01-24 17:57:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-2204-1-1-2-hp-1-phase-1075-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:57:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-hd-double-panel-traffic-door-8-w-x-8-h-black>: HTTP status code is not handled or not allowed 2026-01-24 17:57:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 5172 pages (at 87 pages/min), scraped 3562 items (at 56 items/min) 2026-01-24 17:57:09 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/food_beverages/snacks_candies/cereal_breakfast_snacks landed on page that is not a product page. 2026-01-24 17:57:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-eh755-3-hp-3-phase-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-24 17:57:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regulator-ng-2-psig-min-inlet-boss-1000>: HTTP status code is not handled or not allowed 2026-01-24 17:57:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-u5v1bc-5-hp-3-phase-3520-rpm>: HTTP status code is not handled or not allowed 2026-01-24 17:57:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/outdoor_lighting_fixtures/outdoor_area_lighting already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:57:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-2910-1-4-hp-1-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-24 17:57:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bar_sheet_racks/bar_pipe_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:57:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=us-motor-2204-1-1-2-hp-1-phase-1075-rpm landed on page that is not a product page. 2026-01-24 17:57:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/telescopic-transmission-jack-0-5-ton-195-80-lbs>: HTTP status code is not handled or not allowed 2026-01-24 17:57:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/closed-shelving-drawer-unit-36x18x75-108-drawers-yellow>: HTTP status code is not handled or not allowed 2026-01-24 17:57:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/outdoor_lighting_fixtures/outdoor_area_lighting landed on page that is not a product page. 2026-01-24 17:57:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/utility_racks/bar_sheet_racks/bar_pipe_racks>: HTTP status code is not handled or not allowed 2026-01-24 17:57:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-ready-built-ii-locker-single-tier-1-wide-12x12x78-blue>: HTTP status code is not handled or not allowed 2026-01-24 17:57:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/181501-rfx-pt300dw-double-wide-wireless-wheel-load-scale-30000-x-50-lb>: HTTP status code is not handled or not allowed 2026-01-24 17:57:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/threaded-rod-suspension-bracket-glass-series-infrared-patio-heaters>: HTTP status code is not handled or not allowed 2026-01-24 17:57:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/outdoor_tools_power_equipment/leaf_blowers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:57:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dandux-rectangular-plastic-box-truck-blue-32-bushel-1120-lb-cap>: HTTP status code is not handled or not allowed 2026-01-24 17:57:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-xs13ca1jcr-1-3-hp-1-phase-3450-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:57:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:57:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-inch-heavy-duty-plastic-chain-25-feet-white>: HTTP status code is not handled or not allowed 2026-01-24 17:57:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:57:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mattress-bags-54x8x90-1-5-mil-100-rl>: HTTP status code is not handled or not allowed 2026-01-24 17:57:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/outdoor_tools_power_equipment/leaf_blowers landed on page that is not a product page. 2026-01-24 17:57:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-7096-3-4-hp-1-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-24 17:57:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-6766-1-2-hp-1-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-24 17:57:51 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=us-motor-xs13ca1jcr-1-3-hp-1-phase-3450-rpm landed on page that is not a product page. 2026-01-24 17:57:51 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves landed on page that is not a product page. 2026-01-24 17:57:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves landed on page that is not a product page. 2026-01-24 17:57:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-5846-3-4-hp-1-phase-1075-rpm>: HTTP status code is not handled or not allowed 2026-01-24 17:57:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/square-food-container-2-quart-7-1-4x7-1-4x3-7-8-clear-green-graduation>: HTTP status code is not handled or not allowed 2026-01-24 17:57:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/versa-food-bars-work-table-cold-food-60-x-29-low-kentucky-green>: HTTP status code is not handled or not allowed 2026-01-24 17:58:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-roll-file-4-compartment>: HTTP status code is not handled or not allowed 2026-01-24 17:58:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-compartment-economy-literature-organizer-black>: HTTP status code is not handled or not allowed 2026-01-24 17:58:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camwear-food-pan-1-2-size-2-1-2-deep-polycarbonate-white-nsf>: HTTP status code is not handled or not allowed 2026-01-24 17:58:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 5263 pages (at 91 pages/min), scraped 3626 items (at 64 items/min) 2026-01-24 17:58:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manual-18-diameter-turntable-7-8-h-1000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 17:58:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/seal-cover-for-camwear-6-and-8-qt-round-storage-container-translucent-sheer-blue>: HTTP status code is not handled or not allowed 2026-01-24 17:58:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camshelving-elements-add-on-unit-24-w-x-36-l-x-64-h-4-shelf-brushed-graphite>: HTTP status code is not handled or not allowed 2026-01-24 17:58:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-run-tee-metal-release-collet-12mm-tube-x-3-8>: HTTP status code is not handled or not allowed 2026-01-24 17:58:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camshelving-elements-mobile-shelf-plate-kit-24-w-x-60-l-venteded-brushed-graphite>: HTTP status code is not handled or not allowed 2026-01-24 17:58:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manifold-8mm-tube-x-1-4-bspp>: HTTP status code is not handled or not allowed 2026-01-24 17:58:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-8-balldriver-screwdriver-long-10707>: HTTP status code is not handled or not allowed 2026-01-24 17:58:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forced-air-circulation-incubator-108-liters-230v>: HTTP status code is not handled or not allowed 2026-01-24 17:58:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/measuring-tool-set-7-7-8-inch-l-2-piece>: HTTP status code is not handled or not allowed 2026-01-24 17:58:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/material-handling/winches_accessories/winches_ratchet_pullers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:58:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shield-curved-direct-mount-w-o-interlock-12-inch-l-x-8-inch-w>: HTTP status code is not handled or not allowed 2026-01-24 17:58:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ecp-heavy-duty-kit-for-5-8-inch-polyester-strap>: HTTP status code is not handled or not allowed 2026-01-24 17:58:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/birdwire-stainless-steel-1000l>: HTTP status code is not handled or not allowed 2026-01-24 17:58:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/faosc-retrofit-grids-for-business-cabinet-w-o-meds-100-persons>: HTTP status code is not handled or not allowed 2026-01-24 17:58:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/snow_ice_removal/snow_blowers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:58:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sj-18-inch-cordless-snow-blower-kit-48v-4-amp-hours>: HTTP status code is not handled or not allowed 2026-01-24 17:58:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbw-poly-bumper-truck-8-bushel-capacity-gray>: HTTP status code is not handled or not allowed 2026-01-24 17:58:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/general_purpose_cleaners already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:58:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_seating/adirondack_lounger_chairs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:58:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/material-handling/winches_accessories/winches_ratchet_pullers landed on page that is not a product page. 2026-01-24 17:58:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bistro-36-dining-height-table-round-whitewash>: HTTP status code is not handled or not allowed 2026-01-24 17:58:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_tables/outdoor_dining_bar_tables already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:58:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_chair_table_sets/patio_bar_dining_sets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:58:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:58:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cape-coral-silver-lines-slab-door-wood-30w-x-80h-whiskey-oak>: HTTP status code is not handled or not allowed 2026-01-24 17:58:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:58:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/snow_ice_removal/snow_blowers landed on page that is not a product page. 2026-01-24 17:58:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:58:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:58:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/key-west-night-lines-slab-door-wood-32w-x-80h-whiskey-oak>: HTTP status code is not handled or not allowed 2026-01-24 17:58:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:58:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/general_purpose_cleaners landed on page that is not a product page. 2026-01-24 17:58:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:58:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:58:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/naples-gold-lines-slab-door-wood-30w-x-80h-ice-maple>: HTTP status code is not handled or not allowed 2026-01-24 17:58:56 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_seating/adirondack_lounger_chairs landed on page that is not a product page. 2026-01-24 17:58:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:58:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_tables/outdoor_dining_bar_tables landed on page that is not a product page. 2026-01-24 17:58:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:58:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/largo-night-lines-slab-door-wood-24w-x-80h-ice-maple>: HTTP status code is not handled or not allowed 2026-01-24 17:58:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_chair_table_sets/patio_bar_dining_sets landed on page that is not a product page. 2026-01-24 17:59:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:59:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:59:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:59:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/butterfly-hinge-chrome>: HTTP status code is not handled or not allowed 2026-01-24 17:59:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:59:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_plates_braces already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:59:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:59:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:59:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hog-heavenanti-fatigue-mat-78-thick-10l-x-5w-black-striped-border>: HTTP status code is not handled or not allowed 2026-01-24 17:59:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:59:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hog-heavenanti-fatigue-mat-58-thick-20l-x-2w-black>: HTTP status code is not handled or not allowed 2026-01-24 17:59:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/doors_windows/doors landed on page that is not a product page. 2026-01-24 17:59:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 5339 pages (at 76 pages/min), scraped 3672 items (at 46 items/min) 2026-01-24 17:59:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/doors_windows/doors landed on page that is not a product page. 2026-01-24 17:59:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/doors_windows/doors>: HTTP status code is not handled or not allowed 2026-01-24 17:59:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/doors_windows/doors landed on page that is not a product page. 2026-01-24 17:59:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/doors_windows/doors landed on page that is not a product page. 2026-01-24 17:59:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/doors_windows/doors>: HTTP status code is not handled or not allowed 2026-01-24 17:59:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_plates_braces landed on page that is not a product page. 2026-01-24 17:59:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/doors_windows/doors landed on page that is not a product page. 2026-01-24 17:59:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chillshield-mens-insulated-pants-4xl-gray>: HTTP status code is not handled or not allowed 2026-01-24 17:59:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/doors_windows/doors landed on page that is not a product page. 2026-01-24 17:59:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:59:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coffee-classic-roast-1-2-oz-pack-of-42>: HTTP status code is not handled or not allowed 2026-01-24 17:59:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-pop-up-aluminum-foil-sheets-10-3-4-l-x-9-w-silver-pack-of-500>: HTTP status code is not handled or not allowed 2026-01-24 17:59:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/foodservice_disposables/disposable_plates_bowls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:59:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/individually-wrapped-moist-towelettes-6-l-x-4-w-lemon-scent-pack-of-1000>: HTTP status code is not handled or not allowed 2026-01-24 17:59:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caterline-casuals-platters-18-dia-black-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-24 17:59:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/placesetter-deluxe-bowl-12-oz-6-dia-white-pack-of-1000>: HTTP status code is not handled or not allowed 2026-01-24 17:59:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats landed on page that is not a product page. 2026-01-24 17:59:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premier-chromatography-refrigerator-36-cuft-capacity-glass-door>: HTTP status code is not handled or not allowed 2026-01-24 17:59:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/floor_machine_accessories_supplies/floor_pads_pad_holders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:59:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/foodservice_disposables/disposable_plates_bowls landed on page that is not a product page. 2026-01-24 17:59:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-drawers-vertical-file-cabinet-letter-size-22d-x-40-316h-black>: HTTP status code is not handled or not allowed 2026-01-24 17:59:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bariatric-knockdown-transfer-bench>: HTTP status code is not handled or not allowed 2026-01-24 17:59:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carbon-fiber-rollator-w-comfort-seat-18-12w-x-24h-black>: HTTP status code is not handled or not allowed 2026-01-24 17:59:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/curad-cushioned-compression-hosiery-w-15-20-mmhg-knee-high-small-size-a-regular-white>: HTTP status code is not handled or not allowed 2026-01-24 17:59:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/floor_machine_accessories_supplies/floor_pads_pad_holders landed on page that is not a product page. 2026-01-24 17:59:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vision-type-e-protection-safety-eyewear-blue-mirror-lens-gray-frame>: HTTP status code is not handled or not allowed 2026-01-24 17:59:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5th-wheel-garden-truck-steel-1000-lb-capacity-50-3-8l-x-24-1-4w-x-38-1-4h>: HTTP status code is not handled or not allowed 2026-01-24 17:59:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deck-mount-pre-rinse-faucet-w-8-spout>: HTTP status code is not handled or not allowed 2026-01-24 17:59:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/advance-sc100-complete-upright-floor-scrubber-12-316-cleaning-width>: HTTP status code is not handled or not allowed 2026-01-24 17:59:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/versa-o-tubing-band-heavy-resistance-12l-blue>: HTTP status code is not handled or not allowed 2026-01-24 17:59:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/seating/reception_sofas_sectionals already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 17:59:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shop-portable-wet-and-dry-vacuum-6-hp-10-gallon>: HTTP status code is not handled or not allowed 2026-01-24 18:00:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/husky-rack-wire-velox-machine-guard-sliding-door-post-8-h-safety-yellow>: HTTP status code is not handled or not allowed 2026-01-24 18:00:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/organizer-cabinet-w-fixed-shelves-assembled-48w-x-18d-x-72h-wine-red>: HTTP status code is not handled or not allowed 2026-01-24 18:00:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cabinet-w-drawer--shelves-assembled-30w-x-18d-x-72h-pastel-green>: HTTP status code is not handled or not allowed 2026-01-24 18:00:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/seating/reception_sofas_sectionals landed on page that is not a product page. 2026-01-24 18:00:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 5431 pages (at 92 pages/min), scraped 3732 items (at 60 items/min) 2026-01-24 18:00:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fr-waterproof-flame-resistant-winter-quilted-jacket-class-1-blue-3xl>: HTTP status code is not handled or not allowed 2026-01-24 18:00:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/diy-4th-gen-e-star-ductless-mini-split-heat-pump-24000-btu-205-seer-208-230v>: HTTP status code is not handled or not allowed 2026-01-24 18:00:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-truck-w-retaining-lip-shelves-2000-lb-cap-41-12l-x-24w-x-35h-gray>: HTTP status code is not handled or not allowed 2026-01-24 18:00:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-class-2-hi-vis-safety-vest-2-reflective-strips-polyester-solid-lime-s-m>: HTTP status code is not handled or not allowed 2026-01-24 18:00:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8381-hi-vis-4-in-1-winter-bomber-jacket-small-orange>: HTTP status code is not handled or not allowed 2026-01-24 18:00:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-7001-abrasion-resistant-gloves-nitrile-coated-xs-black-144-pairs>: HTTP status code is not handled or not allowed 2026-01-24 18:00:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-550-padded-knee-sleeves-w-3-layer-foam-cap-sm-black>: HTTP status code is not handled or not allowed 2026-01-24 18:00:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-7041-cut-resistant-gloves-nitrile-coated-ansi-a4-m-lime-1-pair>: HTTP status code is not handled or not allowed 2026-01-24 18:00:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n-ferno-6436-long-sleeve-lightweight-base-layer-shirt-xl-black>: HTTP status code is not handled or not allowed 2026-01-24 18:00:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-7071-cut-resistant-gloves-polyurethane-coated-ansi-a7-l-gray-1-pair>: HTTP status code is not handled or not allowed 2026-01-24 18:00:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arsenal-5935-web-handle-canvas-hoist-bucket-xl-gray>: HTTP status code is not handled or not allowed 2026-01-24 18:00:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iqv-20-lithium-ion-battery-20v-25-ah-extended-capacity>: HTTP status code is not handled or not allowed 2026-01-24 18:00:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-tier-6-door-digital-locker-12in-w-x-18in-d-x-78in-h-blue-assembled>: HTTP status code is not handled or not allowed 2026-01-24 18:00:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/air_tools_accessories/impact_wrenches already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:00:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trushot-20-multi-surface-cleaner-clean-fresh-scent-10-oz-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-24 18:00:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drive-belt-2-for-hl-m1120ld-frm-1120c-frm-1010-frm-1120ld>: HTTP status code is not handled or not allowed 2026-01-24 18:00:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-convex-mirror-plastic-outdoor-30-dia>: HTTP status code is not handled or not allowed 2026-01-24 18:00:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergonomic-angle-grinder-w-lock-on-paddle-switch-10-amp-11000-rpm-4-12-wheel-dia-blue>: HTTP status code is not handled or not allowed 2026-01-24 18:00:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ball-shaped-bur-double-cut-2l-x-14-shank-dia-316-cut-dia>: HTTP status code is not handled or not allowed 2026-01-24 18:00:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ball-shaped-bur-single-cut-2l-x-18-shank-dia>: HTTP status code is not handled or not allowed 2026-01-24 18:00:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylon-pouch-w-snap-for-bur-sets-blue>: HTTP status code is not handled or not allowed 2026-01-24 18:00:50 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/air_tools_accessories/impact_wrenches landed on page that is not a product page. 2026-01-24 18:00:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-dual-band-ceiling-light-fixture-15w-1050-lumens-120v-12-size-white>: HTTP status code is not handled or not allowed 2026-01-24 18:00:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magic-clamp-universal-platform-for-flask-tubes-16x16-12>: HTTP status code is not handled or not allowed 2026-01-24 18:00:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heated-lid>: HTTP status code is not handled or not allowed 2026-01-24 18:00:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/led_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:00:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precision-balance-500g-capacity-0001g-readability-230v>: HTTP status code is not handled or not allowed 2026-01-24 18:01:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbi-sala174-sealed-blok-self-retracting-lifeline-w-bracket-steel-cable-360l>: HTTP status code is not handled or not allowed 2026-01-24 18:01:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-heavy-duty-cabinet-w-4-shelves-96-blue-bins-14-ga-36w-x-24d-x-76h>: HTTP status code is not handled or not allowed 2026-01-24 18:01:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ball-end-mill-power-a-2-flute-3-12l-x-58-dia-1-14-cut-length>: HTTP status code is not handled or not allowed 2026-01-24 18:01:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/finesse-it-natural-wool-buffing-pad-81470-3-dia-white-pack-of-500>: HTTP status code is not handled or not allowed 2026-01-24 18:01:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/led_bulbs landed on page that is not a product page. 2026-01-24 18:01:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drum-liner-8-mil-thick-55-gal-capacity-pack-of-75>: HTTP status code is not handled or not allowed 2026-01-24 18:01:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 5527 pages (at 96 pages/min), scraped 3797 items (at 65 items/min) 2026-01-24 18:01:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/end-unit-w-t-joiner-support-clip-48l-x-12w-x-72h-black>: HTTP status code is not handled or not allowed 2026-01-24 18:01:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v-gard-h1pro-safety-helmet-non-vented-fas-trac-iii-pivot-ratchet-suspension-red>: HTTP status code is not handled or not allowed 2026-01-24 18:01:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/air_conditioners_chillers/wall_air_conditioners already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:01:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8251hdz-bk-two-tone-hi-vis-safety-vest-ansi-class-2-sm-lime>: HTTP status code is not handled or not allowed 2026-01-24 18:01:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flask-volumetric-wide-mouth-globe-glass-class-a-to-contain-tc-astm-e288-50ml-6-box>: HTTP status code is not handled or not allowed 2026-01-24 18:01:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/air_conditioners_chillers/wall_air_conditioners landed on page that is not a product page. 2026-01-24 18:01:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobilecare-keyed-lock-cart-27w-x-27d-x-39h-white>: HTTP status code is not handled or not allowed 2026-01-24 18:01:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hand-sanitizer-gel-refill-for-ltx-12-dispensers-fragrance-free-1200-ml-capacity-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-24 18:01:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hip-pack-w-guard-logo-ppe-supply-royal-blue-7-pieces>: HTTP status code is not handled or not allowed 2026-01-24 18:01:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ring-buoy-usgc-approved-30-dia-white>: HTTP status code is not handled or not allowed 2026-01-24 18:01:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=protect-liquid-cooled-standby-generator-130kw-120-240v-1-phase-ng already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:01:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60a36x1-a-plate-34-pitch-1-bore-36-teeth>: HTTP status code is not handled or not allowed 2026-01-24 18:01:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/work-premier-casual-chukka-boots-steel-toe-size-9eee-4-inch-h-dark-cognac>: HTTP status code is not handled or not allowed 2026-01-24 18:01:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:01:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/groundbreaker-pull-on-boots-composite-toe-size-15w-11-inch-h-brown>: HTTP status code is not handled or not allowed 2026-01-24 18:01:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:01:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=protect-liquid-cooled-standby-generator-130kw-120-240v-1-phase-ng>: HTTP status code is not handled or not allowed 2026-01-24 18:01:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/groundbreaker-work-boots-w-internal-met-guard-steel-toe-size-6w-6-inch-h-brown>: HTTP status code is not handled or not allowed 2026-01-24 18:01:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:01:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zig-elusion-heritage-work-high-top-sneaker-composite-toe-size-8-5w-black-blue>: HTTP status code is not handled or not allowed 2026-01-24 18:01:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-24 18:01:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steri-drape-mayo-stand-cover-9036-23-inch-x-54-inch-60-bx-4-bx-cs>: HTTP status code is not handled or not allowed 2026-01-24 18:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:01:50 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-24 18:01:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/groundbreaker-work-boots-w-external-met-guard-steel-toe-size-10-5m-8-inch-h-brown>: HTTP status code is not handled or not allowed 2026-01-24 18:01:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/groundbreaker-work-boots-w-internal-met-guard-steel-toe-size-9-5w-6-inch-h-brown>: HTTP status code is not handled or not allowed 2026-01-24 18:01:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-24 18:01:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:01:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:01:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-120e18h-qd-bushed-1-12-pitch-18-teeth>: HTTP status code is not handled or not allowed 2026-01-24 18:01:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 18:02:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zig-elusion-heritage-work-women-s-low-cut-sneaker-composite-toe-size-8w-black>: HTTP status code is not handled or not allowed 2026-01-24 18:02:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vitals-skate-inspired-work-shoes-composite-toe-size-10-5m-wintermoss-whitecap-gray>: HTTP status code is not handled or not allowed 2026-01-24 18:02:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-24 18:02:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 18:02:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-centrifuge-tubes-with-8-bags-of-25-tubes-sterile-25-ml-200-pack>: HTTP status code is not handled or not allowed 2026-01-24 18:02:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vitals-skate-inspired-work-shoes-composite-toe-size-11-5m-navy-aged-indigo>: HTTP status code is not handled or not allowed 2026-01-24 18:02:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 5618 pages (at 91 pages/min), scraped 3859 items (at 62 items/min) 2026-01-24 18:02:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:02:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-reservoir-for-multi-channel-pipettes-50-ml-sterile-5-per-bag-200-pack>: HTTP status code is not handled or not allowed 2026-01-24 18:02:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_vials_tubes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:02:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-aspir-8-reservoir-for-8-channel-sterile-25-ml-200-pack>: HTTP status code is not handled or not allowed 2026-01-24 18:02:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:02:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35bs25hx34-38-pitch-34-finished-bore-25-teeth>: HTTP status code is not handled or not allowed 2026-01-24 18:02:21 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 18:02:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_vials_tubes>: HTTP status code is not handled or not allowed 2026-01-24 18:02:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:02:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50bs13hx118-58-pitch-1-18-finished-bore-13-teeth>: HTTP status code is not handled or not allowed 2026-01-24 18:02:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:02:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35bs34hx34-38-pitch-34-finished-bore-34-teeth>: HTTP status code is not handled or not allowed 2026-01-24 18:02:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 18:02:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guldmann-black-box-medical-replacement-battery-9ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 18:02:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-24 18:02:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:02:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nellcor-npb-3940-medical-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 18:02:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paper-twist-ties-4-x-5-32-orange-2000-pack>: HTTP status code is not handled or not allowed 2026-01-24 18:02:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:02:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lumiscope-x24-medical-replacement-battery-9ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 18:02:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50b20hx58mpb-58-pitch-58-plain-bore-20-teeth>: HTTP status code is not handled or not allowed 2026-01-24 18:02:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 18:02:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 18:02:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premier-work-casual-slip-on-sneaker-steel-toe-size-9-5eee-cognac>: HTTP status code is not handled or not allowed 2026-01-24 18:02:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-24 18:02:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premier-work-casual-oxford-shoes-steel-toe-size-8d-navy>: HTTP status code is not handled or not allowed 2026-01-24 18:02:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ledsm9dnkl-8-dimmable-led-can-disk-light-42w-4000-cct-3500-lumens-82-cri-white>: HTTP status code is not handled or not allowed 2026-01-24 18:02:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nanoflex-tr-athletic-work-shoes-composite-toe-size-11w-black-gray>: HTTP status code is not handled or not allowed 2026-01-24 18:02:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50btl26h-taper-bushed-58-pitch-26-teeth>: HTTP status code is not handled or not allowed 2026-01-24 18:02:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/hand_tools_1/hand_saws_cutting_tools_scrapers/cable_bolt_cutters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:02:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nano-x1-adventure-athletic-work-shoes-composite-toe-size-7m-black>: HTTP status code is not handled or not allowed 2026-01-24 18:03:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/altronix-smp5pmp16-alarm-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 18:03:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/hand_tools_1/hand_saws_cutting_tools_scrapers/cable_bolt_cutters>: HTTP status code is not handled or not allowed 2026-01-24 18:03:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lavante-trail-2-running-work-shoes-composite-toe-size-8w-gray-red-black>: HTTP status code is not handled or not allowed 2026-01-24 18:03:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 5708 pages (at 90 pages/min), scraped 3923 items (at 64 items/min) 2026-01-24 18:03:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-06bs18hx16-metric-38-pitch-16mm-finished-bore-18-teeth>: HTTP status code is not handled or not allowed 2026-01-24 18:03:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nano-x1-adventure-athletic-work-mid-cut-shoes-composite-toe-size-3-5m-black>: HTTP status code is not handled or not allowed 2026-01-24 18:03:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lavante-trail-2-running-work-shoes-composite-toe-size-10m-gray-red-black>: HTTP status code is not handled or not allowed 2026-01-24 18:03:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merits-sp44-wheelchair-replacement-battery-8ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 18:03:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/61390-food-container-anti-allergen-wlid-polypropylene-27-qt-16-size-purple>: HTTP status code is not handled or not allowed 2026-01-24 18:03:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:03:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:03:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evolve-skate-inspired-high-top-work-shoes-composite-toe-size-3-5w-triple-black>: HTTP status code is not handled or not allowed 2026-01-24 18:03:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:03:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:03:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/column-protector-12x-12-square-opening-with-fe-cutout-24od-x-42h-yellow>: HTTP status code is not handled or not allowed 2026-01-24 18:03:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:03:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rotary-screw-air-compressor-230v-125-psi-28-cfm-75hp-3ph>: HTTP status code is not handled or not allowed 2026-01-24 18:03:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:03:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:03:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 18:03:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:03:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-24 18:03:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:03:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers>: HTTP status code is not handled or not allowed 2026-01-24 18:03:34 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 18:03:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ul1618-type-2-tile-stove-board-gray-slate-36-x-36-t2ul3636gt-1c>: HTTP status code is not handled or not allowed 2026-01-24 18:03:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:03:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elan-1mb6b-emergency-light-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 18:03:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-24 18:03:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-24 18:03:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-tapered-roller-bearing-e30209j-45mm-bore-85mm-od>: HTTP status code is not handled or not allowed 2026-01-24 18:03:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exide-q-101-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 18:03:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats landed on page that is not a product page. 2026-01-24 18:03:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 18:03:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ratchet-logistic-strap-641201-with-spring-loaded-fitting-12-x-2-gold>: HTTP status code is not handled or not allowed 2026-01-24 18:03:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-14ah-bs-powersports-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 18:03:44 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats landed on page that is not a product page. 2026-01-24 18:03:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elsar-2318-emergency-light-replacement-battery-7ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 18:03:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honeywell-remote-mount-wall-indoor-sensor-c7189u1005-for-visionpro-8000-thermostat>: HTTP status code is not handled or not allowed 2026-01-24 18:03:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 18:03:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powerstar-pm30l-bs-powersports-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 18:03:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:03:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18u-rack-enclosure-server-cabinet-33-deep-w-doors-and-sides-sr18ub>: HTTP status code is not handled or not allowed 2026-01-24 18:03:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:03:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:03:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cc-pump-motor-wwe75-36-184jp-tefc-rigid-c-3-ph-184jp-75-hp-3600-rpm>: HTTP status code is not handled or not allowed 2026-01-24 18:03:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:03:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:03:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enviro-world-solid-brass-outdoor-grade-spigot-rain-barrels-ewc-11>: HTTP status code is not handled or not allowed 2026-01-24 18:03:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pgo-pmx-naked-50-50cc-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 18:04:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 18:04:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 18:04:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/general_purpose_electric_motors/three_phase_motors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:04:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adly-moto-rt-50-50cc-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 18:04:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 18:04:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/definite_purpose_electric_motors/pool_pump_motors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:04:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 18:04:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-24 18:04:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cc-pump-motor-wwe10-36-215jp-tefc-rigid-c-3-ph-215jp-10-hp-3600-rpm>: HTTP status code is not handled or not allowed 2026-01-24 18:04:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 5794 pages (at 86 pages/min), scraped 3971 items (at 48 items/min) 2026-01-24 18:04:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/general_purpose_electric_motors/three_phase_motors landed on page that is not a product page. 2026-01-24 18:04:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-rated-insulated-metal-access-panel-36l-x-36w>: HTTP status code is not handled or not allowed 2026-01-24 18:04:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/computers_electronics/cables_connectors/cables already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:04:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/definite_purpose_electric_motors/pool_pump_motors landed on page that is not a product page. 2026-01-24 18:04:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/welded-wire-security-partition-swing-door-hrso-4w-x-8h-black>: HTTP status code is not handled or not allowed 2026-01-24 18:04:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leoch-lp12-3-2-sealed-lead-acid-replacement-battery-3-2ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 18:04:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jump-n-carry-jnc105-jump-starter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 18:04:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/computers_electronics/cables_connectors/cables>: HTTP status code is not handled or not allowed 2026-01-24 18:04:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mk-es3-6-h-sealed-lead-acid-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 18:04:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yellow-post-safety-barrier-15-ft-yellow-belt>: HTTP status code is not handled or not allowed 2026-01-24 18:04:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bb-hrl33-12-sealed-lead-acid-replacement-battery-26ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 18:04:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kydz-ladderback-chair-pair-12-height>: HTTP status code is not handled or not allowed 2026-01-24 18:04:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caterpillar-d350c-industrial-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 18:04:34 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-24 18:04:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/calm63-151-145tc-aluminum-worm-gear-reducer-63mm-151-145tc-nema-frame>: HTTP status code is not handled or not allowed 2026-01-24 18:04:35 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:4 2026-01-24 18:04:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/remco-rm1212-sealed-lead-acid-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 18:04:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/970-08-platform-landing-48-x-48-floor-height-96>: HTTP status code is not handled or not allowed 2026-01-24 18:04:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rm43-43-glyphosate-plus-weed-control-tvc-32-oz-bottle-76502>: HTTP status code is not handled or not allowed 2026-01-24 18:04:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:04:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-pack-for-models-411-611>: HTTP status code is not handled or not allowed 2026-01-24 18:04:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elk-06120-f1-sealed-lead-acid-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 18:04:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fiamm-fg21202-fg-21202-12v-12ah-emergency-light>: HTTP status code is not handled or not allowed 2026-01-24 18:04:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats landed on page that is not a product page. 2026-01-24 18:04:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-sonic-ps-12550-ps12550-12v-55ah-emergency-light>: HTTP status code is not handled or not allowed 2026-01-24 18:04:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leoch-djw12-90-t2-djw-12-90-t2-12v-9ah-ups-battery>: HTTP status code is not handled or not allowed 2026-01-24 18:04:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-manual-chain-hoist-5-ton-capacity-20-lift>: HTTP status code is not handled or not allowed 2026-01-24 18:05:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labclean-cleanroom-wipes-10037-polyester-4l-x-4w-pack-of-600>: HTTP status code is not handled or not allowed 2026-01-24 18:05:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-in-3-in-galvanized-steel-pipe-nipple-150-psi-lead-free>: HTTP status code is not handled or not allowed 2026-01-24 18:05:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acme-sdc602-home-alarm-alarm-replacement-battery-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 18:05:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 5891 pages (at 97 pages/min), scraped 4042 items (at 71 items/min) 2026-01-24 18:05:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/air_conditioners_chillers/portable_air_conditioners already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:05:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galvanized-cable-assembly-14dia-x-90-l>: HTTP status code is not handled or not allowed 2026-01-24 18:05:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/plumbing_fittings/pipe_clamps_fittings/black_malleable_pipe_fittings already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:05:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-lambswool-monsoon-strip-window-washer-w-handle-pad-wc18>: HTTP status code is not handled or not allowed 2026-01-24 18:05:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/liebert-gxt2-48vbatt-battery-cabinet-12v-7ah-ups-battery>: HTTP status code is not handled or not allowed 2026-01-24 18:05:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/type-i-steel-safety-can-2-5-gallon-9-5l-self-close-lid-blue-7125300>: HTTP status code is not handled or not allowed 2026-01-24 18:05:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/air_conditioners_chillers/portable_air_conditioners landed on page that is not a product page. 2026-01-24 18:05:21 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/plumbing/plumbing_fittings/pipe_clamps_fittings/black_malleable_pipe_fittings landed on page that is not a product page. 2026-01-24 18:05:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wardrobe-z-locker-w-padlock-hasp-2-tier-assembled-18w-x-18d-x-72h-deep-blue>: HTTP status code is not handled or not allowed 2026-01-24 18:05:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reagent-bottle-narrow-mouth-hdpe-1000ml-capacity-clear-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-24 18:05:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/62092rs-p53-deep-groove-ball-bearing-double-sealed-abec-5-45mm-bore-85mm-od>: HTTP status code is not handled or not allowed 2026-01-24 18:05:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-straight-leg-slotted-shim-20-a-005>: HTTP status code is not handled or not allowed 2026-01-24 18:05:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/combination-shower-floor-mount-closed-abs-bowl-galvanized-steel-pipe-bottom-inlet>: HTTP status code is not handled or not allowed 2026-01-24 18:05:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hsm314-shredder-oil-16-oz-bottle>: HTTP status code is not handled or not allowed 2026-01-24 18:05:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/calibration-weight-astm-class-1-nvlap-certificate-stainless-steel-2kg>: HTTP status code is not handled or not allowed 2026-01-24 18:05:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cyclone-pt-100-100-lb-capacity-portable-pressure-blast-system>: HTTP status code is not handled or not allowed 2026-01-24 18:05:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aerial-shelf-for-bench-22648wh-reflective-white>: HTTP status code is not handled or not allowed 2026-01-24 18:05:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/permobil-c500-aeron-wheelchair-replacement-battery-75ah-12v-it>: HTTP status code is not handled or not allowed 2026-01-24 18:05:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pleasant-hearth-alpine-fireplace-glass-door-medium-black-an-1011>: HTTP status code is not handled or not allowed 2026-01-24 18:05:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:05:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mirado-classic-2-pencil-with-eraser-black-dozen>: HTTP status code is not handled or not allowed 2026-01-24 18:05:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ticonderoga-hb-2-pencil-with-eraser-yellow-barrel-72box>: HTTP status code is not handled or not allowed 2026-01-24 18:05:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/highlighters_markers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:05:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60440201550-cloth-belt-341d-6w-x-48l-60-grit>: HTTP status code is not handled or not allowed 2026-01-24 18:05:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:05:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelf-truck-60l-x-14w-x-69h-polyurethane-brake-casters>: HTTP status code is not handled or not allowed 2026-01-24 18:05:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-24 18:05:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:05:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:05:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelf-truck-36l-x-24w-x-80h-polyurethane-brake-casters>: HTTP status code is not handled or not allowed 2026-01-24 18:05:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/highlighters_markers>: HTTP status code is not handled or not allowed 2026-01-24 18:06:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-speed-hump-middle-section>: HTTP status code is not handled or not allowed 2026-01-24 18:06:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/100-gallon-plastic-storage-tank-square-end-with-flat-bottom-987270>: HTTP status code is not handled or not allowed 2026-01-24 18:06:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-24 18:06:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-24 18:06:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dr-grip-ballpoint-retractable-pen-medium-blue-barrelink>: HTTP status code is not handled or not allowed 2026-01-24 18:06:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bisley-airflow153-womens-long-sleeve-shirt-class-2-m-hi-vis-yellownavy>: HTTP status code is not handled or not allowed 2026-01-24 18:06:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 5981 pages (at 90 pages/min), scraped 4102 items (at 60 items/min) 2026-01-24 18:06:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bisley-airflow153-womens-long-sleeve-shirt-class-2-xl-hi-vis-yellownavy>: HTTP status code is not handled or not allowed 2026-01-24 18:06:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-24 18:06:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/807-0016-external-spindle-series-12>: HTTP status code is not handled or not allowed 2026-01-24 18:06:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yard-machines-46-lawn-and-garden-replacement-battery-10ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 18:06:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f301-ballpoint-retractable-pen-stainless-steel-barrel-07mm-black-ink>: HTTP status code is not handled or not allowed 2026-01-24 18:06:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/highlighters_markers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:06:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/power_tools/power_sanders/cordless_sanders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:06:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/46-square-ada-in-ground-mount-picnic-table-blue>: HTTP status code is not handled or not allowed 2026-01-24 18:06:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/highlighters_markers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:06:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/skate-wheel-conveyor-flexible-extendable-42-to-168l-x-34w-zinc-plated-steel>: HTTP status code is not handled or not allowed 2026-01-24 18:06:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/light-duty-no-bump-dump-hopper-steel-2-cu-yd-2000-lb-capacity-machine-gray>: HTTP status code is not handled or not allowed 2026-01-24 18:06:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/power_tools/power_sanders/cordless_sanders>: HTTP status code is not handled or not allowed 2026-01-24 18:06:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/highlighters_markers landed on page that is not a product page. 2026-01-24 18:06:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ansmike-remote-microphone>: HTTP status code is not handled or not allowed 2026-01-24 18:06:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/highlighters_markers landed on page that is not a product page. 2026-01-24 18:06:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/johnlite-2928-emergency-light-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 18:06:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/material-handling/ladders_personnel_lifts/step_ladders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:06:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lithonia-x-el-emergency-light-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 18:06:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/jobsite_portable_lighting/job_site_work_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:06:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bb-bp46-t1-6v-45ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-24 18:06:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-12xl30-emergency-light-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 18:06:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/material-handling/ladders_personnel_lifts/step_ladders>: HTTP status code is not handled or not allowed 2026-01-24 18:06:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-rack-placard-label-holder-1-x-100-ft-rollwhite-797044>: HTTP status code is not handled or not allowed 2026-01-24 18:06:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/jobsite_portable_lighting/job_site_work_lights landed on page that is not a product page. 2026-01-24 18:06:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/umbrellas_awnings_shades/outdoor_umbrellas_bases/umbrellas already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:06:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/prescolite-rb4-emergency-light-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 18:06:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-rack-placard-label-holder-3-x-100-ft-rollred-797066>: HTTP status code is not handled or not allowed 2026-01-24 18:06:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dyna-ray-707-14s-emergency-light-replacement-battery-10ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 18:06:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nerm010ld-sd-fg-10-ner-food-grade-hoist-with-motorized-trolley-1-ton-capacity>: HTTP status code is not handled or not allowed 2026-01-24 18:06:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/outdoor_furniture/umbrellas_awnings_shades/outdoor_umbrellas_bases/umbrellas landed on page that is not a product page. 2026-01-24 18:07:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forbest-sewer-camera-system-w-10-lcd-monitor-200l-cable>: HTTP status code is not handled or not allowed 2026-01-24 18:07:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-12-879-emergency-light-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 18:07:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5000-watt-power-inverter-pwrinv500012w>: HTTP status code is not handled or not allowed 2026-01-24 18:07:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 6074 pages (at 93 pages/min), scraped 4168 items (at 66 items/min) 2026-01-24 18:07:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/structural_framing already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:07:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-w-x-36-d-production-workbench-with-drawer-upright-shelf-phenolic-resin-safety-edge-blue>: HTTP status code is not handled or not allowed 2026-01-24 18:07:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/measurement_layout_tools/test_indicators_holders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:07:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2015blu-1010-end-cap-1-x-1-x-18-blue>: HTTP status code is not handled or not allowed 2026-01-24 18:07:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-and-decker-cst1100-type-1-9-cordless-trimmer-12v-34ah-battery>: HTTP status code is not handled or not allowed 2026-01-24 18:07:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/structural_framing landed on page that is not a product page. 2026-01-24 18:07:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motorcycle-trail-traffic-safety-sign-egp-aluminum-18-x-12-whitebrown>: HTTP status code is not handled or not allowed 2026-01-24 18:07:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/measurement_layout_tools/test_indicators_holders landed on page that is not a product page. 2026-01-24 18:07:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caution-wide-right-turns-sign-graphic-adhesive-dura-vinyl-14-x-10-yellow>: HTTP status code is not handled or not allowed 2026-01-24 18:07:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caution-move-on-green-only-sign-mirror-image-aluminum-14-x-20-blackyellow>: HTTP status code is not handled or not allowed 2026-01-24 18:07:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ner025s-15-ner-electric-hoist-hook-suspension-2-12-ton-15-lift-22-ftmin-208v>: HTTP status code is not handled or not allowed 2026-01-24 18:07:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/huot-toolscoot-tree-for-hsk-100acnc-holders>: HTTP status code is not handled or not allowed 2026-01-24 18:07:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kawasaki-zx6r-zzr-600cc-motorcycle-replacement-battery-2005-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 18:07:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n-ferno-6495-rechargeable-heated-vest-with-battery-power-bank-l-black>: HTTP status code is not handled or not allowed 2026-01-24 18:07:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honda-c90-m-90cc-motorcycle-replacement-battery-1982-1993-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 18:07:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nerp025s-20-ner-electric-hoist-push-trolley-2-12-ton-20-lift-22-ftmin-208v>: HTTP status code is not handled or not allowed 2026-01-24 18:07:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/al100-hand-chain-hoist-aluminum-1-12-ton-capacity-15-ft-lift>: HTTP status code is not handled or not allowed 2026-01-24 18:07:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electra-voy-electric-scooter-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 18:07:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:07:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polaris-slt-h-700cc-personal-watercraft-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 18:07:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/piaggio-vespa-fr-50cc-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 18:07:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ceramic-round-base-magnet-rb60c-45-lbs-pull>: HTTP status code is not handled or not allowed 2026-01-24 18:07:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/import-1-8-inch-hss-aircraft-extension-drill-12-inch-long>: HTTP status code is not handled or not allowed 2026-01-24 18:07:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs landed on page that is not a product page. 2026-01-24 18:08:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-x-48-slatwall-panel-walnut-laminate-with-aluminum-inserts-pkg-qty-4>: HTTP status code is not handled or not allowed 2026-01-24 18:08:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hook-tape-individual-strips-with-adhesive-75l-x-34w-white>: HTTP status code is not handled or not allowed 2026-01-24 18:08:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnetic-business-card-3-1-2-inch-x-2-inch-black-25-cards-pack>: HTTP status code is not handled or not allowed 2026-01-24 18:08:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/landscaping/planters_pots/planters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:08:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-side-printable-business-card-2-inch-x-3-1-2-inch-white-200-cards-pack>: HTTP status code is not handled or not allowed 2026-01-24 18:08:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 6170 pages (at 96 pages/min), scraped 4239 items (at 71 items/min) 2026-01-24 18:08:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hook-tape-individual-strips-with-adhesive-75l-x-1w-white>: HTTP status code is not handled or not allowed 2026-01-24 18:08:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/binders_covers/dividers_tabs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:08:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durapro-knee-boots-safety-loc-outsole-steel-toe-size-6-14h-blue>: HTTP status code is not handled or not allowed 2026-01-24 18:08:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cyclops-spotlight-thor-s250-sealed-lead-acid-replacement-battery-8ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 18:08:18 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/landscaping/planters_pots/planters landed on page that is not a product page. 2026-01-24 18:08:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ready-index-toc-reference-divider-1-to-12-85x11-12-tabs-white-multi>: HTTP status code is not handled or not allowed 2026-01-24 18:08:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-height-aluminum-gantry-crane-4000-lb-capacity-15w-x-12-1516h>: HTTP status code is not handled or not allowed 2026-01-24 18:08:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/binders_covers/dividers_tabs landed on page that is not a product page. 2026-01-24 18:08:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stackable-shelf-bin-11w-x-18d-x-7h-ivory>: HTTP status code is not handled or not allowed 2026-01-24 18:08:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cap-style-industrial-climbing-helmet-vented-hdpe-suspension-hi-vis-yellow>: HTTP status code is not handled or not allowed 2026-01-24 18:08:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/personal-headset-with-in-line-microphone-and-trrs-plug>: HTTP status code is not handled or not allowed 2026-01-24 18:08:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pipette-storage-box-w-adjustable-compartments-large-25w-x-3d-x-16h>: HTTP status code is not handled or not allowed 2026-01-24 18:08:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stove-bolt-screw-kit-w-hex-nuts-steel-14-20-size-assorted-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 18:08:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quickie-targa-16-inch-22nf-agm-wheelchair-replacement-battery-55ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 18:08:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/miller-electric-aea-200e-aea-200le-industrial-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 18:08:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0009309-1-pressure-reducing-valve>: HTTP status code is not handled or not allowed 2026-01-24 18:08:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conduit-body-gasket-nitrile-12-size>: HTTP status code is not handled or not allowed 2026-01-24 18:08:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ef-ct-fr-dress-uniform-shirt-slu2-light-blue-size-l-long>: HTTP status code is not handled or not allowed 2026-01-24 18:08:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ef-ct-fr-dress-uniform-shirt-slu2-navy-size-m-regular>: HTTP status code is not handled or not allowed 2026-01-24 18:08:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/climate-control-air-curtain-96-inch-w-door-120v-unheated-1-ph-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 18:08:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kpex-9k-98-98-26-us-7-pc-pliers-screwdriver-insulated-tool-set-1000v-nylon-pouch>: HTTP status code is not handled or not allowed 2026-01-24 18:08:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simplicity-trail-cruiser-hydrostatic-lawn-and-garden-replacement-battery-9ah-12v>: HTTP status code is not handled or not allowed 2026-01-24 18:08:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wheelcare-superlight-scooter-patriot-wheelchair-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 18:09:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibration-mount-1-tapped-hole-1-threaded-stud-313-inch-dia-225-inch-h-1-2-13-thread>: HTTP status code is not handled or not allowed 2026-01-24 18:09:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:09:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:09:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sonnenschein-m84001a5060042s-emergency-light-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 18:09:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/line-package-nylon-34-x-450-size>: HTTP status code is not handled or not allowed 2026-01-24 18:09:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 6269 pages (at 99 pages/min), scraped 4313 items (at 74 items/min) 2026-01-24 18:09:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/69000-series-rectangular-desk-shell-72-inchw-x-36-inchd-x-29-1-2-inchh-cherry>: HTTP status code is not handled or not allowed 2026-01-24 18:09:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-24 18:09:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/upholstered-stacking-chair-18-inchw-x-22-inchd-x-34-1-2-inchh-black-4-carton>: HTTP status code is not handled or not allowed 2026-01-24 18:09:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-24 18:09:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reducing-bushing-zinc-alloy-1-x-12-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 18:09:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48h-x-48w-slatwall-panelrecoatable-white-with-aluminum-inserts-968343RI>: HTTP status code is not handled or not allowed 2026-01-24 18:09:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/staplers_staples/staplers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:09:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrf13351r56c-cast-iron-right-angle-worm-gear-reducer-51-ratio-56c-frame>: HTTP status code is not handled or not allowed 2026-01-24 18:09:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrf133101r56c-cast-iron-right-angle-worm-gear-reducer-101-ratio-56c-frame>: HTTP status code is not handled or not allowed 2026-01-24 18:09:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/record-book-record-ruled-7-1-4-x-12-1-4-granite-cover-300-pages-pad>: HTTP status code is not handled or not allowed 2026-01-24 18:09:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/sticky_notes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:09:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/basic_supplies/staplers_staples/staplers>: HTTP status code is not handled or not allowed 2026-01-24 18:09:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-washer-silicon-bronze-0-316-id-0-916-od-10-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 18:09:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/staplers_staples/staples already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:09:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/card-reel-with-ez-clip-30-cord-1-1-4-diameter-black>: HTTP status code is not handled or not allowed 2026-01-24 18:09:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/notebooks_pads already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:09:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrf262301r145tc-cast-iron-right-angle-worm-gear-reducer-301-ratio-145t-frame>: HTTP status code is not handled or not allowed 2026-01-24 18:09:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harley-davidson-night-train-fxstbi-1450cc-motorcycle-battery-2001-2006-12v>: HTTP status code is not handled or not allowed 2026-01-24 18:09:45 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/staplers_staples/staples landed on page that is not a product page. 2026-01-24 18:09:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drywall-screw-phillips-bugle-head-fine-thread-8-x-2-12-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 18:09:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/paper_stationery/sticky_notes landed on page that is not a product page. 2026-01-24 18:09:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bourget-bike-works-fat-daddy-motorcycle-replacement-battery-1997-2011-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 18:09:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/paper_stationery/notebooks_pads landed on page that is not a product page. 2026-01-24 18:09:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/bags_cases/computer_bags_briefcases already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:09:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-e-fr-overall-snap-fly-front-fluorescent-yellowgreen-xl>: HTTP status code is not handled or not allowed 2026-01-24 18:09:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c0001-container-placard-label-holder-cstp5-wplace-label-here-6x8-white>: HTTP status code is not handled or not allowed 2026-01-24 18:09:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-head-tap-bolt-low-carbon-steel-516-18-x-6-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 18:09:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/bags_cases/computer_bags_briefcases>: HTTP status code is not handled or not allowed 2026-01-24 18:10:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polaris-v92c-deluxe-cruiser-1507cc-motorcycle-replacement-battery-1998-2004-12v>: HTTP status code is not handled or not allowed 2026-01-24 18:10:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yamaha-wave-runner-650-lx-650cc-personal-watercraft-battery-1990-1996-12v>: HTTP status code is not handled or not allowed 2026-01-24 18:10:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kawasaki-zx1000-ninja-zx-10r-abs-30th-anniversary-1000cc-motorcycle-battery-2015-12v>: HTTP status code is not handled or not allowed 2026-01-24 18:10:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 6361 pages (at 92 pages/min), scraped 4379 items (at 66 items/min) 2026-01-24 18:10:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yamaha-xv1700atm-road-star-midnight-silverado-1700cc-motorcycle-battery-2004-2007-12v>: HTTP status code is not handled or not allowed 2026-01-24 18:10:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wal-1-8-1-tap-extractor-set>: HTTP status code is not handled or not allowed 2026-01-24 18:10:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yamaha-fx-ho-1812cc-personal-watercraft-replacement-battery-2009-2014-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 18:10:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-hcpk204-12-pillow-block-bearing-low-base-ecc-lock-collar-bore-19-05-mm>: HTTP status code is not handled or not allowed 2026-01-24 18:10:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/amigo-mobility-classicfd-mobility-scooter-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 18:10:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=double-row-spherical-roller-bearing-23220ex1w33c3-100mm-bore-180mm-od already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:10:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/heaters/electric_heaters/portable_electric_heaters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:10:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-row-spherical-roller-bearing-24038ew33c3-190mm-bore-290mm-od>: HTTP status code is not handled or not allowed 2026-01-24 18:10:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quantum-square-partition-wall-divider-10w-x-6h-chrome>: HTTP status code is not handled or not allowed 2026-01-24 18:10:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pride-mobility-jazzyz-chair-mobility-scooter-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 18:10:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=double-row-spherical-roller-bearing-23220ex1w33c3-100mm-bore-180mm-od landed on page that is not a product page. 2026-01-24 18:10:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/checklist-caddy-with-book-for-loading-dock>: HTTP status code is not handled or not allowed 2026-01-24 18:10:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/heaters/electric_heaters/portable_electric_heaters landed on page that is not a product page. 2026-01-24 18:10:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/golden-technologies-alero-mobility-scooter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 18:10:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/everest-jennings-153302004-mobility-scooter-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 18:10:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-ball-transfer-conveyor-table-btit220403-with-3-center-75-lb-cap>: HTTP status code is not handled or not allowed 2026-01-24 18:10:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-in-t316-stainless-steel-full-port-ball-valve-3-piece-1000-psi>: HTTP status code is not handled or not allowed 2026-01-24 18:10:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dalton-tacahe-rehab-pc1465rhb-mobility-scooter-replacement-battery-55ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 18:10:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pillar-technology-blazerplus-mobility-scooter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 18:10:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-chrome-esd-wire-shelving-unit-starter-60w-x-21d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 18:10:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-in-x-4-in-lead-free-seamless-red-brass-pipe-nipple-140-psi-sch-40-import>: HTTP status code is not handled or not allowed 2026-01-24 18:10:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-in-304-stainless-steel-cap-fnpt-125-psi-import>: HTTP status code is not handled or not allowed 2026-01-24 18:10:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-chrome-wire-shelving-unit-starter-60w-x-12d-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 18:10:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dalton-primechair-mid-wheel-pc-mp3cm20-ar-bl-mobility-scooter-battery-35ah-12v>: HTTP status code is not handled or not allowed 2026-01-24 18:10:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hotpod-8-in-duct-mounted-heater-hardwired-hp8-1440120-2t-1440-720w-120v>: HTTP status code is not handled or not allowed 2026-01-24 18:11:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pre-inked-message-stamp-past-due-1-5-8-x-1-2-red>: HTTP status code is not handled or not allowed 2026-01-24 18:11:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-mx-z-tnt-ace-900-899cc-snowmobile-replacement-battery-2014-2015-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 18:11:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-corrugated-boxes-14-x-12-x-3-200lb-test-ect-32-25-pack>: HTTP status code is not handled or not allowed 2026-01-24 18:11:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ez-correct-correction-tape-1-6-in-x-400-in-white-10-box>: HTTP status code is not handled or not allowed 2026-01-24 18:11:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/forms already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:11:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 6455 pages (at 94 pages/min), scraped 4446 items (at 67 items/min) 2026-01-24 18:11:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pre-inked-message-stamp-void-1-2-x-1-5-8-blue>: HTTP status code is not handled or not allowed 2026-01-24 18:11:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/petty-cash-book-2-part-carbonless-11-x-5-1-2-white-canary-200-sets-book>: HTTP status code is not handled or not allowed 2026-01-24 18:11:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sales-order-book-2-part-carbonless-3-11-32-x-7-3-16-50-sets-pad>: HTTP status code is not handled or not allowed 2026-01-24 18:11:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/paper_stationery/forms>: HTTP status code is not handled or not allowed 2026-01-24 18:11:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-renegade-backcountry-x-800r-800cc-snowmobile-battery-2014-2016-12v>: HTTP status code is not handled or not allowed 2026-01-24 18:11:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-mx-z-600-ho-renegade-renegade-x-600cc-snowmobile-battery-2004-2013-12v>: HTTP status code is not handled or not allowed 2026-01-24 18:11:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/money-receipt-book-3-part-carbonless-2-3-4-x-6-7-8-200-sets-book>: HTTP status code is not handled or not allowed 2026-01-24 18:11:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-corrugated-boxes-11-1-4-x-8-3-4-x-4-200lb-test-ect-32-25-pack>: HTTP status code is not handled or not allowed 2026-01-24 18:11:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-ski-doo-mx-z-adrenaline-800-ho-ptek-800cc-snowmobile-battery-2007-12v>: HTTP status code is not handled or not allowed 2026-01-24 18:11:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36w-square-pedestal-table-with-grey-nebula-top-round-silver-base-bistro-height>: HTTP status code is not handled or not allowed 2026-01-24 18:11:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/door-frame-with-drywall-afterset-curries-hinge-location-left-hand-48w-x-84h>: HTTP status code is not handled or not allowed 2026-01-24 18:11:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/queuepro-250-retractable-belt-barrier-40-silver-post-11-black-belt-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-24 18:11:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electrically-conductive-floor-mat-916-thick-3-x-12-black>: HTTP status code is not handled or not allowed 2026-01-24 18:11:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:11:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-plus-phenolic-locker-11-8z1212721-z-style-12-x-12-x-72-graphite-grafix>: HTTP status code is not handled or not allowed 2026-01-24 18:11:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/non-stick-sheet-protector-8-1-2w-x-11h-clear-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 18:11:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sky-3-piece-outdoor-lounge-sets-23-5-8-w-x-15-11-16-h-table-black>: HTTP status code is not handled or not allowed 2026-01-24 18:11:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sky-3-piece-outdoor-lounge-sets-23-5-8-w-x-15-11-16-h-table-red>: HTTP status code is not handled or not allowed 2026-01-24 18:11:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions landed on page that is not a product page. 2026-01-24 18:11:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4991-double-sided-vhb-acrylic-foam-tape-1-x-5-yds-90-mil-gray>: HTTP status code is not handled or not allowed 2026-01-24 18:11:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/x032-1-2hp-115-230v-1625rpm-1ph-teao-48yz-fr>: HTTP status code is not handled or not allowed 2026-01-24 18:12:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/explosion-proof-motor-g855-056c17g5321-1-2hp-115-208-230v-1800rpm-1ph-epfc>: HTTP status code is not handled or not allowed 2026-01-24 18:12:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-531nl-lf-brass-hose-bibb-12-mnpt>: HTTP status code is not handled or not allowed 2026-01-24 18:12:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cell-phone-locker-with-access-panel-19158-25arc-recessed-mounted-combo-locks-25-a-doors-aluminum>: HTTP status code is not handled or not allowed 2026-01-24 18:12:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 6552 pages (at 97 pages/min), scraped 4520 items (at 74 items/min) 2026-01-24 18:12:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gruber-power-gpsu1-34-sealed-lead-acid-replacement-battery-3-2ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 18:12:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/severe-duty-motor-e854-449ttfs6540-200hp-460v-1800rpm-3ph-447-9t-fr-tefc>: HTTP status code is not handled or not allowed 2026-01-24 18:12:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cr1-series-triton-corrosion-resistant-washdown-unit-heater-25000w-600v-3-phase>: HTTP status code is not handled or not allowed 2026-01-24 18:12:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zeus-pc1-3-12f1-sealed-lead-acid-replacement-battery-1-3ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 18:12:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cxf-series-screw-plug-immersion-heater-2-12npt-6000w-600v-30l>: HTTP status code is not handled or not allowed 2026-01-24 18:12:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dk-2053a-push-pull-rod-set-waccessories-10-meters>: HTTP status code is not handled or not allowed 2026-01-24 18:12:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-3-classic-wicking-birdseye-t-shirt-with-pocket-yellow-4xl>: HTTP status code is not handled or not allowed 2026-01-24 18:12:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-gal-sat-blk-ash-trash-recept-sat-alum-lid-w-sand-cover-192-bk-sa>: HTTP status code is not handled or not allowed 2026-01-24 18:12:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fan-blower-motor-c1160-5kcr49sn0150x-15hp-1725rpm-115-208-230v-1ph-56h-dp>: HTTP status code is not handled or not allowed 2026-01-24 18:12:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nm20-c5-plugmold-120v-15a-60l-5-outlets>: HTTP status code is not handled or not allowed 2026-01-24 18:12:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-cal-coin-grip-anti-slip-rubber-mat-2mm-x-4-x-5-rolled-rubber-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 18:12:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v20gb506-plugmold-125v-15a-5l-10-outlets>: HTTP status code is not handled or not allowed 2026-01-24 18:12:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-nexelon-blue-wire-shelving-unit-starter-36w-x-21d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 18:12:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/designer-pvc-ceiling-tile-75307-waterproof-washable-icon-coffer-2l-x-2w-satin-blk>: HTTP status code is not handled or not allowed 2026-01-24 18:12:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-shelf-nexelon-blue-wire-shelving-unit-starter-24w-x-24d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 18:12:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gogreen-power-gg-16103msbk-6-outlet-surge-protector-3ft-cord-black>: HTTP status code is not handled or not allowed 2026-01-24 18:12:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unison-union-mx-12120-sealed-lead-acid-replacement-battery-9ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 18:12:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/general_purpose_sealed_lead_acid_batteries already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:12:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ironguard-laminated-dock-bumper-24-3494-26-w-x-9-d-x-10-h>: HTTP status code is not handled or not allowed 2026-01-24 18:13:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/19-gallon-2-door-self-close-fume-hood-acid-cabinet-30x21-5-8x35-3-4-lgt-neutral>: HTTP status code is not handled or not allowed 2026-01-24 18:13:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/orthofab-lifestyles-vip-wheelchair-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 18:13:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/general_purpose_sealed_lead_acid_batteries landed on page that is not a product page. 2026-01-24 18:13:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90-gallon-1-sliding-door-self-close-acid-corrosive-cabinet-43w-x-34d-x-65h-blue>: HTTP status code is not handled or not allowed 2026-01-24 18:13:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 6648 pages (at 96 pages/min), scraped 4594 items (at 74 items/min) 2026-01-24 18:13:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-two-wheel-trash-cart-32-gallon-greenstone-ana32-gns>: HTTP status code is not handled or not allowed 2026-01-24 18:13:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-nexelate-silver-epoxy-wire-shelving-unit-starter-60w-x-30d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 18:13:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/retractable-locking-panel-26w-x-3-4d-x-11-1-4h-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 18:13:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electric-mobility-sparky-wheelchair-replacement-battery-75ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 18:13:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/hvac_system_parts_components/replacement_fan_blades_blower_wheels/plastic_fan_blades already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:13:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-shelf-poly-z-brite-wire-shelving-unit-add-on-24w-x-21d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 18:13:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/bags_cases/computer_bags_briefcases already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:13:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/bags_cases/computer_bags_briefcases already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:13:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/backpack-pvc-plastic-12-1-2-x-17-1-2-x-5-1-2-clear>: HTTP status code is not handled or not allowed 2026-01-24 18:13:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:13:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-inlet-blower-wheel-9-1-8-inch-dia-ccw-1750-rpm-1-2-inch-bore-3-3-4-inch-w-galvanized>: HTTP status code is not handled or not allowed 2026-01-24 18:13:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/hvac_system_parts_components/replacement_fan_blades_blower_wheels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:13:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thurst-collars-5-8-inch-shaft-diameter-2pk>: HTTP status code is not handled or not allowed 2026-01-24 18:13:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:13:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/bags_cases/computer_bags_briefcases>: HTTP status code is not handled or not allowed 2026-01-24 18:13:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-crank-handle-for-psz-slm-portable-safety-zone>: HTTP status code is not handled or not allowed 2026-01-24 18:13:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-24 18:13:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-poly-z-brite-wire-shelving-unit-starter-24w-x-18d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 18:13:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-poly-z-brite-wire-shelving-unit-starter-36w-x-24d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 18:13:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/hvac_system_parts_components/replacement_fan_blades_blower_wheels landed on page that is not a product page. 2026-01-24 18:13:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergi-lite-m-9-2-p-emergency-light-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 18:13:42 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-24 18:13:42 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/bags_cases/computer_bags_briefcases landed on page that is not a product page. 2026-01-24 18:13:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:13:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-inlet-blower-wheel-4-3-4-inch-dia-cw-3450-rpm-1-2-inch-bore-2-15-16-inch-w-galvanized>: HTTP status code is not handled or not allowed 2026-01-24 18:13:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/small-hubless-fan-blade-7-inch-dia-21-pitch-cw-1-1-8-inch-blade-depth-5-blade>: HTTP status code is not handled or not allowed 2026-01-24 18:13:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-stainless-steel-solid-shelving-unit-add-on-48w-x-18d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 18:13:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portalac-lac-pe9f2-emergency-light-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 18:13:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/designer-suites-monitor-riser>: HTTP status code is not handled or not allowed 2026-01-24 18:13:56 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-24 18:13:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bolted-pallet-rack-upright-frame-42ind-x-120inh>: HTTP status code is not handled or not allowed 2026-01-24 18:13:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-wide-boltless-shelving-add-on-72-in-w-x-48-in-d-x-84-in-h-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 18:14:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pk-15307ei-electronics-tool-kit-50-pcs>: HTTP status code is not handled or not allowed 2026-01-24 18:14:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vertical-stationary-compressor-gt5b80v230v-1ph-5hp-230v-1ph-175-psi-80-gal>: HTTP status code is not handled or not allowed 2026-01-24 18:14:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-fg424288bla-stock-tank-100-gallon-black-270079>: HTTP status code is not handled or not allowed 2026-01-24 18:14:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/label_tapes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:14:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/p-touch-tze-labeling-tape-3-4w-blue-on-white>: HTTP status code is not handled or not allowed 2026-01-24 18:14:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 6739 pages (at 91 pages/min), scraped 4656 items (at 62 items/min) 2026-01-24 18:14:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-workbench-72-w-x-30-d-with-2-backsplash-undershelf>: HTTP status code is not handled or not allowed 2026-01-24 18:14:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-extra-hd-boltless-shelving-add-on-36-in-w-x-12-in-d-x-60-in-h-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 18:14:18 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/label_tapes landed on page that is not a product page. 2026-01-24 18:14:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-high-cap-boltless-shelving-steel-deck-add-on-72-in-w-x-24-in-d-x-96-in-h>: HTTP status code is not handled or not allowed 2026-01-24 18:14:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/york-wide-light-m2e12-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 18:14:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ff61bisshhada-ada-compliant-built-in-undercounter-all-refrigerator-white-23-58w>: HTTP status code is not handled or not allowed 2026-01-24 18:14:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wirecraftersrackback-wire-mesh-pallet-rack-enclosure-hinged-door-120x120-797137>: HTTP status code is not handled or not allowed 2026-01-24 18:14:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ps-70-f-red-privacy-screen>: HTTP status code is not handled or not allowed 2026-01-24 18:14:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/room_dividers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:14:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sonnenschein-a512-550a-emergency-light-replacement-battery-55ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 18:14:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/acoustical_partition_panels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:14:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regenerative-blower-ab-902-3-phase-2-stage-85-hp>: HTTP status code is not handled or not allowed 2026-01-24 18:14:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ps-88-f-ocean-privacy-screen>: HTTP status code is not handled or not allowed 2026-01-24 18:14:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/partitions_dividers/room_dividers landed on page that is not a product page. 2026-01-24 18:14:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wildgameinnovations-th-350dfeeder-decoy-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 18:14:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/partitions_dividers/acoustical_partition_panels>: HTTP status code is not handled or not allowed 2026-01-24 18:14:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/acoustical_partition_panels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:14:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/awrd-partition-88x84-f-beige>: HTTP status code is not handled or not allowed 2026-01-24 18:14:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/five-shelf-heavy-duty-boltless-shelving-add-on-48in-w-x-24in-d-x-60in-h>: HTTP status code is not handled or not allowed 2026-01-24 18:14:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wildgameinnovations-blue-iris-scb3-trailcamera-decoy-battery-5ah-6v>: HTTP status code is not handled or not allowed 2026-01-24 18:14:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/partitions_dividers/acoustical_partition_panels landed on page that is not a product page. 2026-01-24 18:14:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qvtrk-2-168est-single-unit-track-assembly-kit-2-tracks-168>: HTTP status code is not handled or not allowed 2026-01-24 18:14:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/big-dog-vintage-classic-1570cc-motorcycle-replacement-battery-1998-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 18:15:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/big-dog-chopper-dt-1750cc-motorcycle-replacement-battery-2003-2004-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 18:15:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/k32005-6-carbon-eyebolt-plain-pattern-1-2-13-6-shank>: HTTP status code is not handled or not allowed 2026-01-24 18:15:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-153-non-para-extra-strength-urinal-screen-evergreen-12-screens-case>: HTTP status code is not handled or not allowed 2026-01-24 18:15:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/odor_control already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:15:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbi-sala-nano-lok-arc-flash-self-retracting-lifeline-web--aluminum-rebar-lock-hook-8l>: HTTP status code is not handled or not allowed 2026-01-24 18:15:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 6830 pages (at 91 pages/min), scraped 4723 items (at 67 items/min) 2026-01-24 18:15:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-vis-cooling-safety-t-shirt-long-sleeve-ansi-class-3-s-yellow>: HTTP status code is not handled or not allowed 2026-01-24 18:15:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sterile-burn-sheet-21-620>: HTTP status code is not handled or not allowed 2026-01-24 18:15:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/heaters/electric_heaters/electric_baseboard_heaters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:15:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/heaters/electric_heaters/electric_baseboard_heaters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:15:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/odor_control>: HTTP status code is not handled or not allowed 2026-01-24 18:15:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-welded-workbench-72-x-30-pegboard-panel-butcher-block-square-edge>: HTTP status code is not handled or not allowed 2026-01-24 18:15:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd-welded-workbench-48-x-24-adj-height-pegboard-panel-steel-square-edge>: HTTP status code is not handled or not allowed 2026-01-24 18:15:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-ski-doo-gtx-4-tec-1500cc-personal-watercraft-battery-2003-07-12v>: HTTP status code is not handled or not allowed 2026-01-24 18:15:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/heaters/electric_heaters/electric_baseboard_heaters landed on page that is not a product page. 2026-01-24 18:15:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-cabinet-flush-door-with-264-blue-bins-16-ga-all-welded-cabinet-72-w-x-24-d-x-84-h-gray>: HTTP status code is not handled or not allowed 2026-01-24 18:15:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/bin_cabinets/stationary_bin_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:15:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/heaters/electric_heaters/electric_baseboard_heaters landed on page that is not a product page. 2026-01-24 18:15:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/badsey-hot-scoot-mobility-scooter-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 18:15:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/summit-ff1532bss-all-refrigerator-for-built-in-or-freestanding-use-3-cu-ft-15-wide>: HTTP status code is not handled or not allowed 2026-01-24 18:15:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/permobil-lowrider-mobility-scooter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 18:15:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/theradyne-investigator-mobility-scooter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 18:15:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/cabinets/bin_cabinets/stationary_bin_cabinets landed on page that is not a product page. 2026-01-24 18:15:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/invacare-p9000xdt-mobility-scooter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 18:15:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merits-pioneer-4-s141-s1412-s1413-mobility-scooter-battery-75ah-12v>: HTTP status code is not handled or not allowed 2026-01-24 18:15:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bollard-cover-cl1385u-4-1-2dia-x-52h-brown-w-white-tape>: HTTP status code is not handled or not allowed 2026-01-24 18:15:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bollard-cover-cl1385vv-4-1-2dia-x-52h-beige-w-no-tape>: HTTP status code is not handled or not allowed 2026-01-24 18:15:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelate-42w-x-21d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 18:15:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-oal-straight-steel-roller-conveyor-19-roller-dia-22-bf-3-axle-center>: HTTP status code is not handled or not allowed 2026-01-24 18:15:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hook-loop-fastener-78-buttons-78-hook-white>: HTTP status code is not handled or not allowed 2026-01-24 18:15:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u1549-cantilever-rack-single-sided-upright-49d-x-15h>: HTTP status code is not handled or not allowed 2026-01-24 18:15:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-strip-hanger-plastic>: HTTP status code is not handled or not allowed 2026-01-24 18:15:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelate-48w-x-30d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 18:16:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xu1876-cantilever-rack-single-sided-upright-76d-x-18h>: HTTP status code is not handled or not allowed 2026-01-24 18:16:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-drum-spill-containment-platform-1620st-51-1-2-w-x-27-d-x-13-h-steel>: HTTP status code is not handled or not allowed 2026-01-24 18:16:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-gsx-se-e-tec-800r-snowmobile-replacement-battery-2012-2015-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 18:16:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 6930 pages (at 100 pages/min), scraped 4796 items (at 73 items/min) 2026-01-24 18:16:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polaris-wide-trak-iq-snowmobile-replacement-battery-2009-2010-12v-i>: HTTP status code is not handled or not allowed 2026-01-24 18:16:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cubby-cabinet-cc-66-03-welded-34-1-2w-x-13-1-2d-x-66h-black>: HTTP status code is not handled or not allowed 2026-01-24 18:16:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ventilated-locker-vdl-121836-c-02-double-tier-no-legs-3-wide-12x18x36-welded-med-gray>: HTTP status code is not handled or not allowed 2026-01-24 18:16:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/ink_toner/inkjet_ink_cartridges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:16:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-2-sided-pegboard-lean-tool-rack-48w-x-24d>: HTTP status code is not handled or not allowed 2026-01-24 18:16:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nursery-platform-truck-perforated-48x24-lip-deck-rubber-casters-1000-lbs>: HTTP status code is not handled or not allowed 2026-01-24 18:16:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-drawer-steel-flat-file-for-30-42-d-black>: HTTP status code is not handled or not allowed 2026-01-24 18:16:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-drawer-steel-flat-file-for-24-36-documents-black>: HTTP status code is not handled or not allowed 2026-01-24 18:16:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/ink_toner/inkjet_ink_cartridges landed on page that is not a product page. 2026-01-24 18:16:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-a95-4l970-v-belt-1-2-x-97>: HTTP status code is not handled or not allowed 2026-01-24 18:16:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-b186-v-belt-5-8-x-189>: HTTP status code is not handled or not allowed 2026-01-24 18:16:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ww3060-8phfl-heavy-duty-7-gauge-mobile-workbench-8-phenolic-casters-floor-lock>: HTTP status code is not handled or not allowed 2026-01-24 18:16:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/freestanding-office-partitions-gray-60-14w-x-72h-238640FGY>: HTTP status code is not handled or not allowed 2026-01-24 18:16:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-b257-v-belt-5-8-x-260>: HTTP status code is not handled or not allowed 2026-01-24 18:16:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelite-36w-x-21d-x-80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 18:16:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/severe-duty-motor-ecp83770t-5-3-ph-75-hp-575-v-1765-rpm-tefc-213t-frame>: HTTP status code is not handled or not allowed 2026-01-24 18:16:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30l-x-60d-electric-height-adjustable-28-45h-table-black>: HTTP status code is not handled or not allowed 2026-01-24 18:16:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelite-48w-x-21d-x-80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 18:16:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brake-motor-ebm3116t-3-phase-208-230-460-volts-1-hp-1765-rpm-open-143t-frame>: HTTP status code is not handled or not allowed 2026-01-24 18:16:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-black-epoxy-48w-x-21d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 18:16:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g-tek-polykor-seamless-knit-blended-glove-polyurethane-coated-flat-grip-large-salt-and-pepper-12pk>: HTTP status code is not handled or not allowed 2026-01-24 18:17:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-flat-free-hand-truck-wheel-354dcswpu242-410-350-4-2-1-4-x-5-8-offset-hub>: HTTP status code is not handled or not allowed 2026-01-24 18:17:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelon-72w-x-21d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 18:17:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/prefilled-homogenizer-tubes-2-0ml-zirconium-beads-1-0mm-triple-pure-50-pk>: HTTP status code is not handled or not allowed 2026-01-24 18:17:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/abus-combination-dial-padlock-78-50-green-78815>: HTTP status code is not handled or not allowed 2026-01-24 18:17:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 7028 pages (at 98 pages/min), scraped 4870 items (at 74 items/min) 2026-01-24 18:17:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/severe-duty-motor-vecp3667t-3-ph-15-hp-208-230-460-v-1170-rpm-tefc-182tc-frame>: HTTP status code is not handled or not allowed 2026-01-24 18:17:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/74hb40-75-keyed-alike-lockout-padlock-non-conductive-3-inch-shackle-green-06775>: HTTP status code is not handled or not allowed 2026-01-24 18:17:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/combi-rotor-quick-lock-lid-for-z216-series-24-x-1-5-2-0ml-4-x-pcr-strips>: HTTP status code is not handled or not allowed 2026-01-24 18:17:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2e-mach-rod-end-machined-black-oxided-c1030-1035-5-16-18-x-6>: HTTP status code is not handled or not allowed 2026-01-24 18:17:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/severe-duty-motor-cecp84103t-4-3-ph-25-hp-460-v-1780-rpm-tefc-284tc-frame>: HTTP status code is not handled or not allowed 2026-01-24 18:17:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machined-chrome-steel-needle-roller-bearing-w-inner-ring-80-x-110-x-35mm>: HTTP status code is not handled or not allowed 2026-01-24 18:17:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machined-chrome-steel-needle-roller-bearing-w-inner-ring-95-x-125-x-36mm>: HTTP status code is not handled or not allowed 2026-01-24 18:17:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-poly-green-24wx18dx80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 18:17:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drt-l163-lifebuoy-light-uscg-solas-med-atex-orange-20-30-inch-liferings>: HTTP status code is not handled or not allowed 2026-01-24 18:17:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-poly-green-30wx24dx60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 18:17:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/import-3-8-16-transfer-screw-set>: HTTP status code is not handled or not allowed 2026-01-24 18:17:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ninja-bnf-gloves-15-gauge-nylon-nitrile-coated-palm-fingertips-gray-black-m>: HTTP status code is not handled or not allowed 2026-01-24 18:17:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rpm-iii-dc-motor-tachometer-kit-417708-34-dpg-dpfv-frames-dc180-c180-dc2112>: HTTP status code is not handled or not allowed 2026-01-24 18:17:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-nexelon-30w-x-24d-x-80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 18:17:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/awn-h23-5ptr-seam-slope-awning-5-8-w-x-3d-x-2h-metal>: HTTP status code is not handled or not allowed 2026-01-24 18:17:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-ac-motor-c-face-kit-field-conversion-10-1305gld-tefc-10-284-6uc-tc-frame>: HTTP status code is not handled or not allowed 2026-01-24 18:17:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-nexelon-30w-x-24d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 18:17:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-spot-cooler-system-3825-female-thread-one-cold-outlet>: HTTP status code is not handled or not allowed 2026-01-24 18:18:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelon-72w-x-18d-x-69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 18:18:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waring-cts1000-commercial-conveyor-toaster-450-slices-per-hour-208v>: HTTP status code is not handled or not allowed 2026-01-24 18:18:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-poly-sealer-12-material-width-for-use-with-off-set-serrated-seals-polyethylene-strapping>: HTTP status code is not handled or not allowed 2026-01-24 18:18:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/celltreat-scientific-products-800l-pipette-mla-precision-silver>: HTTP status code is not handled or not allowed 2026-01-24 18:18:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 7125 pages (at 97 pages/min), scraped 4945 items (at 75 items/min) 2026-01-24 18:18:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cutlery-combo-forks-knives-spoons-napkins-250-carton-white>: HTTP status code is not handled or not allowed 2026-01-24 18:18:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/celltreat-scientific-products-60l-pipette-mla-dtipper-silver>: HTTP status code is not handled or not allowed 2026-01-24 18:18:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rc-121-relay-socket-300v-10-amps-din-rail-mountable>: HTTP status code is not handled or not allowed 2026-01-24 18:18:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/topoge-series-180-manway-gasket-12-x-15-x-1-black-rubber-elliptical-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 18:18:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cl-saltix-10-hepa-canister-vacuum-107410362>: HTTP status code is not handled or not allowed 2026-01-24 18:18:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jurni-round-bistro-table-36dia-columbian-walnut-topsilver-base>: HTTP status code is not handled or not allowed 2026-01-24 18:18:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-thread-o-ring-connector-h5315x12-3-4-inch-tube-od-3-4-inch-port-size>: HTTP status code is not handled or not allowed 2026-01-24 18:18:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/heaters/gas_heaters_furnaces/gas_wall_heaters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:18:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/worldwide-fm13-18-56-farm-duty-motor-13hp-1800rpm-56-115230v-tefc>: HTTP status code is not handled or not allowed 2026-01-24 18:18:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/filter-head-fh325-1-1-2-inch-npt-25-psi-bypass>: HTTP status code is not handled or not allowed 2026-01-24 18:18:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bg9-belt-guard-fits-reducer-styles-smr9-or-wsmr9>: HTTP status code is not handled or not allowed 2026-01-24 18:18:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/heaters/gas_heaters_furnaces already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:18:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-series-clamp-for-hose-ssch100-1-inch-id-1531-od>: HTTP status code is not handled or not allowed 2026-01-24 18:18:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hangcha-a-series-electric-lithiumion-high-level-order-picker-3000-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-24 18:18:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/salamander-heater-electric-wall-mount-fan-forced-240v-15-kw-3-phase-36-1-amps>: HTTP status code is not handled or not allowed 2026-01-24 18:18:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/heaters/gas_heaters_furnaces>: HTTP status code is not handled or not allowed 2026-01-24 18:18:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sa-whiffle-typical-14-double-48-gray>: HTTP status code is not handled or not allowed 2026-01-24 18:18:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/bins_totes_containers/bulk_containers/plastic_bulk_containers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:18:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forged-welding-flange-fdf025-1-4-inch-forged-steel-1312-inch-od-1312-inch-pilot>: HTTP status code is not handled or not allowed 2026-01-24 18:18:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90-street-elbow-h3409x4-1-4-inch-x-1-4-inch-male-to-female>: HTTP status code is not handled or not allowed 2026-01-24 18:18:56 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-24 18:18:56 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:5 2026-01-24 18:18:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pu-erh-tea-emperors-pu-erh-single-cup-bags-0125-oz-16-box>: HTTP status code is not handled or not allowed 2026-01-24 18:18:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sbl2-lp840-2-led-square-basket-wraparound-mvolt-20l-4000-cct>: HTTP status code is not handled or not allowed 2026-01-24 18:18:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/bins_totes_containers/bulk_containers/plastic_bulk_containers landed on page that is not a product page. 2026-01-24 18:19:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-long-sleeve-button-down-work-shirt-xl-brown>: HTTP status code is not handled or not allowed 2026-01-24 18:19:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:19:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/worldwide-electric-winl67-3-5-1-182tc-helical-inline-speed-reducer-182tc-input-flange-3-5-1-ratio>: HTTP status code is not handled or not allowed 2026-01-24 18:19:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:19:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:19:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-vis-long-sleeve-shirt-tear-and-rip-resistant-xl-orange>: HTTP status code is not handled or not allowed 2026-01-24 18:19:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 7216 pages (at 91 pages/min), scraped 5013 items (at 68 items/min) 2026-01-24 18:19:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-44w-x-34l-gray>: HTTP status code is not handled or not allowed 2026-01-24 18:19:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts landed on page that is not a product page. 2026-01-24 18:19:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clark-foam-products-1001356y-kitting-sheet-polyethylene-yellow-1>: HTTP status code is not handled or not allowed 2026-01-24 18:19:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats landed on page that is not a product page. 2026-01-24 18:19:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ds-407200w-shrink-wrap-40w-x-200l-7mil-white>: HTTP status code is not handled or not allowed 2026-01-24 18:19:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:19:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:19:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5409800-36x18-shelf>: HTTP status code is not handled or not allowed 2026-01-24 18:19:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts landed on page that is not a product page. 2026-01-24 18:19:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:19:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_drawers_keyboard_trays already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:19:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-189524>: HTTP status code is not handled or not allowed 2026-01-24 18:19:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:19:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:19:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72706tb1-6-mini-srd-twin-leg-carabiner-steel-snap-hooks>: HTTP status code is not handled or not allowed 2026-01-24 18:19:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/work-benches/bench_tops_accessories/workbench_legs_frames_stringers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:19:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/automotive_accessories_cleaning_supplies/cab_protectors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:19:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats landed on page that is not a product page. 2026-01-24 18:19:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats landed on page that is not a product page. 2026-01-24 18:19:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-33w-x-34l-green>: HTTP status code is not handled or not allowed 2026-01-24 18:19:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:19:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:19:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_drawers_keyboard_trays landed on page that is not a product page. 2026-01-24 18:19:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/marino-wool-watch-cap-black-one-size-0043rblkosa>: HTTP status code is not handled or not allowed 2026-01-24 18:19:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:19:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 18:19:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/md-vertical-unit-heater-v279s01-279000-btu-5460-cfm-115v>: HTTP status code is not handled or not allowed 2026-01-24 18:19:34 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/work-benches/bench_tops_accessories/workbench_legs_frames_stringers landed on page that is not a product page. 2026-01-24 18:19:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-34w-x-33l-tan>: HTTP status code is not handled or not allowed 2026-01-24 18:19:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/decor_linens/artificial_plants already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:19:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 18:19:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 18:19:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 18:19:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts landed on page that is not a product page. 2026-01-24 18:19:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/automotive_accessories_cleaning_supplies already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:19:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-2-economical-digital-force-gauge-50-lbf-capacity>: HTTP status code is not handled or not allowed 2026-01-24 18:19:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:19:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 18:19:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-42w-x-34l-brown>: HTTP status code is not handled or not allowed 2026-01-24 18:19:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:19:44 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/decor_linens/artificial_plants landed on page that is not a product page. 2026-01-24 18:19:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/automatic-1000-ml-bulk-foam-soap-dispenserwhite-sf2150-17-640430>: HTTP status code is not handled or not allowed 2026-01-24 18:19:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:19:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:19:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/vehicle-fleet-maintenance/automotive_accessories_cleaning_supplies landed on page that is not a product page. 2026-01-24 18:19:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 18:19:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:19:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/450hs-34-inch-top-chest-6-drawer-black-34-inch-l-x-27-inch-h-x-25-inchd>: HTTP status code is not handled or not allowed 2026-01-24 18:19:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 18:19:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:19:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-housekeeping-cart-beige-2104-be>: HTTP status code is not handled or not allowed 2026-01-24 18:19:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:19:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:19:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 18:19:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 18:19:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-38w-x-30l-black>: HTTP status code is not handled or not allowed 2026-01-24 18:20:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 18:20:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-50w-x-34l-navy>: HTTP status code is not handled or not allowed 2026-01-24 18:20:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 18:20:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:20:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 18:20:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 18:20:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:20:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spectra-glitter-assortment-3-4-oz-6-colors-12-jars-pack>: HTTP status code is not handled or not allowed 2026-01-24 18:20:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 7289 pages (at 73 pages/min), scraped 5045 items (at 32 items/min) 2026-01-24 18:20:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/side-loading-mesh-letter-trays-black-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 18:20:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 18:20:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:20:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/water-tank-brass-float-valve>: HTTP status code is not handled or not allowed 2026-01-24 18:20:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 18:20:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-12-gauge-combination-cabinet-w-drawers-shelves-36w-x-24d-x-78h>: HTTP status code is not handled or not allowed 2026-01-24 18:20:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:20:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:20:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wedge-locking-washer-carbon-steel-zinc-coated-3-8-large-o-d-pkg-of-200>: HTTP status code is not handled or not allowed 2026-01-24 18:20:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:20:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:20:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 18:20:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:20:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:20:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-jeans-indura-denim-14-oz-34w-x-unhemmed-flame-resistant-blue>: HTTP status code is not handled or not allowed 2026-01-24 18:20:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:20:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 18:20:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 18:20:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cap-mount-headgear-used-with-faceshields-10-each>: HTTP status code is not handled or not allowed 2026-01-24 18:20:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 18:20:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:20:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 18:20:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:20:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-work-coveralls-56-regular-navy>: HTTP status code is not handled or not allowed 2026-01-24 18:20:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:20:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 18:20:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-38w-x-30l-dark-navy>: HTTP status code is not handled or not allowed 2026-01-24 18:20:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 18:20:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 18:20:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:20:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-work-coveralls-44-tall-navy>: HTTP status code is not handled or not allowed 2026-01-24 18:20:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 18:20:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:20:44 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 18:20:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96-performance-300-series-lan-station-african-mahogany>: HTTP status code is not handled or not allowed 2026-01-24 18:20:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 18:20:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-compartment-hanging-garment-dispenser-locker-gray-w-turn-knob-locks>: HTTP status code is not handled or not allowed 2026-01-24 18:20:50 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 18:20:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:20:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a88k-v-belt-kevlar-1-2-x-90>: HTTP status code is not handled or not allowed 2026-01-24 18:20:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:20:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 18:20:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-jeans-indura-denim-14-oz-52w-x-unhemmed-flame-resistant-blue>: HTTP status code is not handled or not allowed 2026-01-24 18:20:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/attic-armour-43251-white-48-inch-x-48-inch>: HTTP status code is not handled or not allowed 2026-01-24 18:20:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:20:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:20:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-work-pants-reflective-material-flame-resistant-34w-x-unhemmed-red>: HTTP status code is not handled or not allowed 2026-01-24 18:21:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 18:21:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:21:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 18:21:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:21:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:21:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wallbase-dry-back-75259-48-inchl-x-4-inchw-beige>: HTTP status code is not handled or not allowed 2026-01-24 18:21:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 18:21:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 18:21:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:21:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 7364 pages (at 75 pages/min), scraped 5084 items (at 39 items/min) 2026-01-24 18:21:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 18:21:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 18:21:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 18:21:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:21:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:21:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-westex-52w-x-30l-tan>: HTTP status code is not handled or not allowed 2026-01-24 18:21:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:21:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cargo-pants-with-double-reinforced-knees-flame-resistant-34w-x-32l-navy>: HTTP status code is not handled or not allowed 2026-01-24 18:21:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/painting_supplies/spray_paint_primer already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:21:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-decor-decorative-metallic-spray-12-oz-aerosol-can-silver-metallic-793304>: HTTP status code is not handled or not allowed 2026-01-24 18:21:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 18:21:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:21:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-72w-x-14d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 18:21:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 18:21:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:21:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 18:21:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:21:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 18:21:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:21:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/painting_supplies/spray_paint_primer>: HTTP status code is not handled or not allowed 2026-01-24 18:21:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 18:21:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/x-o-rust-anti-rust-enamel-galvanized-aluminum-primer-white-quart-738856>: HTTP status code is not handled or not allowed 2026-01-24 18:21:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 18:21:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-48w-x-14d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 18:21:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 18:21:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-9a-1ph-remote-start-start-stop-100-250v-1-7-2-3a>: HTTP status code is not handled or not allowed 2026-01-24 18:21:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 18:21:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smooth-bollard-post-sleeve-4-yellow-436851>: HTTP status code is not handled or not allowed 2026-01-24 18:21:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manitowoc-ice-qm-45a-ice-maker-with-bin-cube-style-air-cooled-self-contained-condenser>: HTTP status code is not handled or not allowed 2026-01-24 18:21:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cab-command-center-7-1-2-ford-transit-ram-9010-3-01>: HTTP status code is not handled or not allowed 2026-01-24 18:21:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_storage_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:21:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_roof_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:21:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-fastener-hfef150-stalguardcoated-steel-anticorrosive-150pk>: HTTP status code is not handled or not allowed 2026-01-24 18:21:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qwiklug-3-terminal-repair-lugs-qt2810-10-awg-2-leads-with-nut>: HTTP status code is not handled or not allowed 2026-01-24 18:21:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-16a-1ph-remote-start-start-stop-24-60v-10-13a>: HTTP status code is not handled or not allowed 2026-01-24 18:21:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:21:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/494003-hex-bolt-1-4-20-x-3-4-grade-a-steel-zinc-unc-ft-pkg-of-150>: HTTP status code is not handled or not allowed 2026-01-24 18:21:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:22:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/011207-socket-cap-screw-5-16-18-x-1-1-2-steel-black-oxide-ft-unc-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 18:22:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rhino-34-green-vinyl-tape-white-print>: HTTP status code is not handled or not allowed 2026-01-24 18:22:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_cabinets_drawers?token=null already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:22:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/349006-split-lock-washer-1-4-steel-zinc-pkg-of-1000>: HTTP status code is not handled or not allowed 2026-01-24 18:22:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 7447 pages (at 83 pages/min), scraped 5139 items (at 55 items/min) 2026-01-24 18:22:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_cabinets_drawers?token=null already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:22:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e40c-extension-spring-0375-od-x-00475-wire-dia-x-25-oal-559-lbs-in-mbhd>: HTTP status code is not handled or not allowed 2026-01-24 18:22:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m459rb-sign-cpr-emergency-response-10-x-14-whiteredblack>: HTTP status code is not handled or not allowed 2026-01-24 18:22:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/travira-48-round-dining-table-tekwood-natural>: HTTP status code is not handled or not allowed 2026-01-24 18:22:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_cabinets_drawers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:22:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/d239rb-osha-sign-danger-chemical-storage-area-10-x-14-whiteredblack>: HTTP status code is not handled or not allowed 2026-01-24 18:22:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_cabinets_drawers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:22:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/32-chromebook-8482-laptop-tablet-charging-cart>: HTTP status code is not handled or not allowed 2026-01-24 18:22:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-80a-3ph-remote-start-terminals-reset-button-250-500v-57-68a>: HTTP status code is not handled or not allowed 2026-01-24 18:22:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vsm-abrasive-belt-205925-zirconia-alumina-1-4-x-18-40-grit>: HTTP status code is not handled or not allowed 2026-01-24 18:22:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_cabinets_drawers landed on page that is not a product page. 2026-01-24 18:22:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-antique-book-safe-with-key-lock-8-3-4w-x-2-3-4d-x-13h>: HTTP status code is not handled or not allowed 2026-01-24 18:22:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_cabinets_drawers landed on page that is not a product page. 2026-01-24 18:22:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vsm-abrasive-belt-285782-ceramic-2-x-132-40-grit>: HTTP status code is not handled or not allowed 2026-01-24 18:22:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/voyager-lightweight-headset-kenwood-hyt-relm-series-radios>: HTTP status code is not handled or not allowed 2026-01-24 18:22:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ac-inpass-1-inch-npt-2-in-1-coalescing-filter-178-cfm-8-000-hr-0-01-micron-232-psi>: HTTP status code is not handled or not allowed 2026-01-24 18:22:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/369509-female-connector-6063a-5p-230250vac-9-hr-ip67>: HTTP status code is not handled or not allowed 2026-01-24 18:22:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-impact-wrench-with-6-extension-anvil-sx556-6-1600-ft-lbs>: HTTP status code is not handled or not allowed 2026-01-24 18:22:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acrylic-draft-shiled-small-13-x-14-x-13>: HTTP status code is not handled or not allowed 2026-01-24 18:22:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-set-screw-coupling-mrc15-black-oxide-15mm-5pk>: HTTP status code is not handled or not allowed 2026-01-24 18:22:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/130-ibc-stairway-stair36i-19-130>: HTTP status code is not handled or not allowed 2026-01-24 18:22:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keyless-rigid-coupling-c600e275-steel-275d-x-5826d-234l-shaft>: HTTP status code is not handled or not allowed 2026-01-24 18:22:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-keyless-rigid-coupling-c600m13-c600-series-steel-13mm-shaft>: HTTP status code is not handled or not allowed 2026-01-24 18:22:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-bolt-flange-mount-bronze-bearing-f2psbr100-press-steel-housing-selfalign-1id>: HTTP status code is not handled or not allowed 2026-01-24 18:23:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-set-screw-coupling-mrc15s4h90-steel-15mm-5pk>: HTTP status code is not handled or not allowed 2026-01-24 18:23:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-sided-self-supporting-tank-hs-1200-116-dia-x-44h-1200-gallon-cap-blue>: HTTP status code is not handled or not allowed 2026-01-24 18:23:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-60-inch-w-x-21-inch-d-1000-lbs-cap-removable-open-grid-mat-polymer-gray>: HTTP status code is not handled or not allowed 2026-01-24 18:23:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sz101001-knife-sterilizer-box-no-hole-5-x-8-x-8>: HTTP status code is not handled or not allowed 2026-01-24 18:23:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 7543 pages (at 96 pages/min), scraped 5210 items (at 71 items/min) 2026-01-24 18:23:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-36-inch-w-x-18-inch-d-1000-lbs-cap-removable-open-grid-mat-polymer-gray>: HTTP status code is not handled or not allowed 2026-01-24 18:23:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-24-inch-w-x-21-inch-d-x-74-inch-h-2000-lbs-cap-4-solid-shelves>: HTTP status code is not handled or not allowed 2026-01-24 18:23:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kni-electronics-super-knips-cutter-w-multi-component-handle>: HTTP status code is not handled or not allowed 2026-01-24 18:23:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/air_compressors/portable_electric_air_compressors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:23:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40x40-hd-drainage-3-sided-o-b>: HTTP status code is not handled or not allowed 2026-01-24 18:23:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/38-frl-n33353w-gauge-metal-bowl-wsight-manual-fog-lubricator>: HTTP status code is not handled or not allowed 2026-01-24 18:23:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ag-silver-calcium-alginate-dressing-4l-x-5w-10-pcs>: HTTP status code is not handled or not allowed 2026-01-24 18:23:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-receiver-surge-tank-ar8035-horizontal-30-gal-top-plate>: HTTP status code is not handled or not allowed 2026-01-24 18:23:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/air_compressors/portable_electric_air_compressors landed on page that is not a product page. 2026-01-24 18:23:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triple-antibiotic-ointment-09-gpacket>: HTTP status code is not handled or not allowed 2026-01-24 18:23:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zinc-oxide-ointment-1-oz-tube-pack-of-72>: HTTP status code is not handled or not allowed 2026-01-24 18:23:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-torx-power-bit-16040x-1-4-hexagon-t6-drive-1-15-16-long>: HTTP status code is not handled or not allowed 2026-01-24 18:23:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/two_way_radios_accessories/two_way_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:23:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/long-hex-tip-socket-set-54416l-1-2-drive-6-pieces-w-clips-rail>: HTTP status code is not handled or not allowed 2026-01-24 18:23:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tms313h-traffic-sign-reserved-parking-florida-18-x-12-white>: HTTP status code is not handled or not allowed 2026-01-24 18:23:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transparent-surgical-tape-3w-x-10-yards-pack-of-48>: HTTP status code is not handled or not allowed 2026-01-24 18:23:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tm195k-traffic-sign-detour-inside-arrow-right-sign-12-x-36-orange>: HTTP status code is not handled or not allowed 2026-01-24 18:23:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cubbie-lid-13-1-2l-x-8-5-8w-x-1-4h-red>: HTTP status code is not handled or not allowed 2026-01-24 18:23:42 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/two_way_radios_accessories/two_way_accessories landed on page that is not a product page. 2026-01-24 18:23:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tm015k-traffic-sign-no-parking-any-time-with-double-arrow-18-x-12-white>: HTTP status code is not handled or not allowed 2026-01-24 18:23:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tms316g-traffic-sign-reserved-parking-illinois-24-x-12-white->: HTTP status code is not handled or not allowed 2026-01-24 18:23:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-stic-grip-xtra-comfort-ballpoint-pen-black-12mm-medium-36-pack>: HTTP status code is not handled or not allowed 2026-01-24 18:23:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jumbo-refillable-permanent-marker-chisel-tip-red>: HTTP status code is not handled or not allowed 2026-01-24 18:23:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-3d-printer-filament-abs-1-kg-175-mm-black>: HTTP status code is not handled or not allowed 2026-01-24 18:23:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mini-half-round-bolster-13-l-x-6-dia-burgundy>: HTTP status code is not handled or not allowed 2026-01-24 18:23:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/amscope-fluorescent-ring-light-bulb-8w-2>: HTTP status code is not handled or not allowed 2026-01-24 18:23:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-square-table-on-casters-with-4-chairs-steel-frame-white-table-w-navy-poly-chairs>: HTTP status code is not handled or not allowed 2026-01-24 18:23:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/hvac_service_tools_parts/installation_maintenance_tools_parts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:23:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-gel-wrist-rest-mmmwr420le-196-x-106-black>: HTTP status code is not handled or not allowed 2026-01-24 18:23:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/applique-drafting-film-chadaf8-85h-x-11w-clear-100-sheets>: HTTP status code is not handled or not allowed 2026-01-24 18:24:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paris-london-dual-book-lock-box-with-keyed-lock-cb12470-10-1-2-x-7-1-2-x-3-1-2>: HTTP status code is not handled or not allowed 2026-01-24 18:24:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/data_storage_media already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:24:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24038-black-oxide-drill-bit-29-piece-set-135-3-8-rs-1-16-to-1-1-2>: HTTP status code is not handled or not allowed 2026-01-24 18:24:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/hvac_service_tools_parts/installation_maintenance_tools_parts landed on page that is not a product page. 2026-01-24 18:24:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fl-slip-on-boat-shoe-smooth-leather-cognac-14eee>: HTTP status code is not handled or not allowed 2026-01-24 18:24:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:24:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rp-works-ruched-bootie-full-grain-leather-black-10m>: HTTP status code is not handled or not allowed 2026-01-24 18:24:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 7638 pages (at 95 pages/min), scraped 5273 items (at 63 items/min) 2026-01-24 18:24:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/data_storage_media>: HTTP status code is not handled or not allowed 2026-01-24 18:24:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/77150-backing-pad-for-sanding-discs-polishing-bonnets-1-4-arbor-5>: HTTP status code is not handled or not allowed 2026-01-24 18:24:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-24 18:24:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rp-works-ruched-bootie-full-grain-leather-black-10w>: HTTP status code is not handled or not allowed 2026-01-24 18:24:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:24:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wooden-treatment-table-with-shelf-blue-upholstery-natural-frame-78l-x-24w-x-30h>: HTTP status code is not handled or not allowed 2026-01-24 18:24:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/physical_therapy/glassless_mirrors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:24:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rp-works-two-eye-tie-work-sneaker-breathable-knit-textile-leather-brown-tan-11w>: HTTP status code is not handled or not allowed 2026-01-24 18:24:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-linear-guided-slide-mtcl40x60st-ball-bearing-14-npt-40mm-bore-60mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 18:24:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-24 18:24:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f37915-0510-borosilicate-glass-10ml-automatic-self-zeroing-burette-500ml-reservoir>: HTTP status code is not handled or not allowed 2026-01-24 18:24:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:24:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/physical_therapy/glassless_mirrors>: HTTP status code is not handled or not allowed 2026-01-24 18:24:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/press_brakes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:24:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-varitemp-heat-gun-kit-1200-degrees-f>: HTTP status code is not handled or not allowed 2026-01-24 18:24:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commercial-products-microfiber-economy-cloth-16-x-16-red-polyester-nylon-red>: HTTP status code is not handled or not allowed 2026-01-24 18:24:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u1282-1a-mp-black-tie-locker-single-tier-12x18x72-1-door-assembled-black-light-grey>: HTTP status code is not handled or not allowed 2026-01-24 18:24:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/press_brakes landed on page that is not a product page. 2026-01-24 18:24:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/awa282-2-gym-pe-locker-double-tier-12x18x36-assembled>: HTTP status code is not handled or not allowed 2026-01-24 18:24:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 18:24:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-chrome-2-tier-wire-shelf-basket-starter-unit-36w-x-18d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 18:24:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/indexing-plunger-multi-pin-w-spindle-lock-nut-black-11-5-40-0n-pressure-m20-1-5-thread-12-15mm-pin>: HTTP status code is not handled or not allowed 2026-01-24 18:24:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-lithium-battery-for-auto-floor-scrubber-713170>: HTTP status code is not handled or not allowed 2026-01-24 18:24:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ac-power-supply-psh75a-enclosed-single-75va-120-208-240-277-480-24vac>: HTTP status code is not handled or not allowed 2026-01-24 18:24:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enclosed-relay-rib2401d-n4-10a-nema-4-4x-dpdt-24vac-dc-120vac>: HTTP status code is not handled or not allowed 2026-01-24 18:24:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/electric_motor_controls/ac_drives already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:24:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compact-single-spring-sign-stand-for-roll-up-signs-36-inch-and-48-inch-powder-coated-orange-steel>: HTTP status code is not handled or not allowed 2026-01-24 18:24:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pistol-case-qf540sgr-watertight-5-gun-cap-15-x12-1-8-x12-1-8-gray>: HTTP status code is not handled or not allowed 2026-01-24 18:25:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/building_construction_tape/marking_warning_tape already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:25:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/electric_motor_controls/ac_drives landed on page that is not a product page. 2026-01-24 18:25:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbwp-16-bushel-vinyl-basket-truck-2-rigid-2-swivel-casters-navy>: HTTP status code is not handled or not allowed 2026-01-24 18:25:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/horizon-mfg-economy-sds-center-1-1-2w-binder-wire-rack-3032-3-language>: HTTP status code is not handled or not allowed 2026-01-24 18:25:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/bins_totes_containers/protective_equipment_cases/waterproof_equipment_cases already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:25:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/bulk_container_trucks_tilt_trucks/basket_trucks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:25:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ll-wl-4-feet-led-stairwell-wall-mount-fixture-3000-lumens-4000k-white>: HTTP status code is not handled or not allowed 2026-01-24 18:25:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/fans_blowers/exhaust_fans/duct_fans already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:25:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 7728 pages (at 90 pages/min), scraped 5336 items (at 63 items/min) 2026-01-24 18:25:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/building_construction_tape/marking_warning_tape>: HTTP status code is not handled or not allowed 2026-01-24 18:25:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/toggle-switch-125-277v-10-20a-black-silver-for-merco-000716sp>: HTTP status code is not handled or not allowed 2026-01-24 18:25:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ajc-eagle-picher-cf6v5-5-6v-7ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-24 18:25:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/guest_chairs/conference_training_chairs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:25:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/fans_blowers/exhaust_fans/duct_fans>: HTTP status code is not handled or not allowed 2026-01-24 18:25:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/special_purpose_chairs/gaming_chairs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:25:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ajc-interstate-bsl1075-12v-7ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-24 18:25:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/bins_totes_containers/protective_equipment_cases/waterproof_equipment_cases landed on page that is not a product page. 2026-01-24 18:25:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ws099001av-70-amp-stick-welder>: HTTP status code is not handled or not allowed 2026-01-24 18:25:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/bulk_container_trucks_tilt_trucks/basket_trucks landed on page that is not a product page. 2026-01-24 18:25:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-x-2-performance-tpr-wheel-with-roller-bearing-for-1-2-axle-600-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-24 18:25:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/weathermaker-rooftop-gas-heat-electric-cool-unit-7-5-ton-3-ph>: HTTP status code is not handled or not allowed 2026-01-24 18:25:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/special_purpose_chairs/gaming_chairs landed on page that is not a product page. 2026-01-24 18:25:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vive-700-series-5plus1plus1-large-screen-thermostat-programmable-1h-1c>: HTTP status code is not handled or not allowed 2026-01-24 18:25:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speed-change-mop-handle-plastic-yellow-gjo80160>: HTTP status code is not handled or not allowed 2026-01-24 18:25:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/122032-vertical-wall-mount-sign-holder-w-adhesive-tape-11-x-17-10-pack>: HTTP status code is not handled or not allowed 2026-01-24 18:25:34 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/guest_chairs/conference_training_chairs landed on page that is not a product page. 2026-01-24 18:25:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/199604-adhesive-back-c-channel-nameplate-6-x-1-5-acrylic>: HTTP status code is not handled or not allowed 2026-01-24 18:25:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wern-proform-f3-construction-fall-protection-harness-w-quick-connect-leg-s>: HTTP status code is not handled or not allowed 2026-01-24 18:25:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700017-metal-prong-hook-for-slatwall-pegboard-1-25-high-20-pack>: HTTP status code is not handled or not allowed 2026-01-24 18:25:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wern-bull-ring-anchorage-connector-steel>: HTTP status code is not handled or not allowed 2026-01-24 18:25:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/21-115l-rigid-to-swing-adapter-top-of-riser-low-lead>: HTTP status code is not handled or not allowed 2026-01-24 18:25:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-extension-cord-for-model-jf-dcs-yellow>: HTTP status code is not handled or not allowed 2026-01-24 18:25:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22-616-replacement-face-strainer-for-3-1-2-waste-drains>: HTTP status code is not handled or not allowed 2026-01-24 18:25:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700509-org-two-sided-non-revolving-pegboard-countertop-display-8-x-20-orange>: HTTP status code is not handled or not allowed 2026-01-24 18:26:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vs-jlt4-flocculator-100-240v-50-60hz>: HTTP status code is not handled or not allowed 2026-01-24 18:26:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/375-piece-cotter-pin-assortment-1-16-to-5-32-zinc>: HTTP status code is not handled or not allowed 2026-01-24 18:26:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lyn-1-tier-1-door-locker-15-inch-w-x-18-inch-d-x-78-inch-h-gray-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 18:26:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inplant-office-vinyl-cov-hdbd-8x8-2-wall-1-light-2-dplx-1-door-white>: HTTP status code is not handled or not allowed 2026-01-24 18:26:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 7822 pages (at 94 pages/min), scraped 5402 items (at 66 items/min) 2026-01-24 18:26:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-electric-kit-12x16-inplant-office>: HTTP status code is not handled or not allowed 2026-01-24 18:26:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anflex-ultralight-cut-resistant-gloves-a4-cut-protection-size-10>: HTTP status code is not handled or not allowed 2026-01-24 18:26:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zp-morado-super-cleaner-5-gallon-pail-mild-butyl-scent>: HTTP status code is not handled or not allowed 2026-01-24 18:26:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cat-2010sp-ultra-quiet-oil-free-1-hp-2-gal-tank-air-compressor>: HTTP status code is not handled or not allowed 2026-01-24 18:26:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inplant-office-vinyl-cov-drywall-20x20-2-wall-8-lgt-6-dplx-1-dr-gray>: HTTP status code is not handled or not allowed 2026-01-24 18:26:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/air_compressors/rotary_screw_air_compressors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:26:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-cantilever-rack-starter-heavy-duty-2-in-lip-48-in-w-x-45-in-d-72-in-h>: HTTP status code is not handled or not allowed 2026-01-24 18:26:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-plastic-barrier-chain-1-1-2-inch-x-50-feet-l-slate-gray>: HTTP status code is not handled or not allowed 2026-01-24 18:26:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esd-open-bin-box-12w-x-18d-x-4-1-2h>: HTTP status code is not handled or not allowed 2026-01-24 18:26:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-2-1-2-inch-medium-duty-stanchion-kit-w-2-inch-x-10-feet-l-chain-black-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-24 18:26:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/air_compressors/rotary_screw_air_compressors landed on page that is not a product page. 2026-01-24 18:26:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-spigot-for-autoclavable-carboys-911846005-polypropylene-1-pk-blue-white>: HTTP status code is not handled or not allowed 2026-01-24 18:26:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-putty-peach-x-soft-1-pound>: HTTP status code is not handled or not allowed 2026-01-24 18:26:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emery-panel-moulding-corner-pml15x15em-1534w-x-1534h-x-118d>: HTTP status code is not handled or not allowed 2026-01-24 18:26:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-putty-lime-medium-2-ounce>: HTTP status code is not handled or not allowed 2026-01-24 18:26:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/designer-wood-locker-32365-double-tier-3-wide-12w-x-15d-x-36h-cherry-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 18:26:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gel-hand-exercise-ball-small-circular-tan-xx-soft>: HTTP status code is not handled or not allowed 2026-01-24 18:26:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hand-finger-exerciser-yellow-x-light>: HTTP status code is not handled or not allowed 2026-01-24 18:26:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/finger-exerciser-10-finger-set-white>: HTTP status code is not handled or not allowed 2026-01-24 18:26:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hand-finger-exerciser-8-piece-set>: HTTP status code is not handled or not allowed 2026-01-24 18:26:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/lockers/locker_accessories/locker_panels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:26:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hand-exerciser-medium-black-x-heavy>: HTTP status code is not handled or not allowed 2026-01-24 18:26:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/antimicrobial-beige-pe-foam-roller-round-6-dia-x-12l>: HTTP status code is not handled or not allowed 2026-01-24 18:26:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/boards_easels/chalkboards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:26:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disc-361f-2-inch-diameter-tr-aluminum-oxide-50-grit>: HTTP status code is not handled or not allowed 2026-01-24 18:26:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/lockers/locker_accessories/locker_panels>: HTTP status code is not handled or not allowed 2026-01-24 18:26:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/work-benches/workbenches_worktables/open_base_workbenches/stationary_open_base_workbenches/standard_duty_stationary_workbenches already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:26:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/checkers-industrial-delineator-with-whips-airlines-delfs3obwuni>: HTTP status code is not handled or not allowed 2026-01-24 18:26:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-ton-cap-aluminum-gantry-crane-8-ft-span-adj-height-8-11-min-11-5-max>: HTTP status code is not handled or not allowed 2026-01-24 18:26:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lug-all-winch-hoist-height-adjustment-kit-for-adjustable-height-gantry-cranes-03-015>: HTTP status code is not handled or not allowed 2026-01-24 18:26:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fs-943-pvc-floor-sink-full-grate-dome-strainer-3-solvent-weld-outlet-for-sch-40-pvc-pipe>: HTTP status code is not handled or not allowed 2026-01-24 18:26:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/21504-z-large-sump-roof-drain-poly-dome-and-4-no-hub-outlet>: HTTP status code is not handled or not allowed 2026-01-24 18:26:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/boards_easels/chalkboards landed on page that is not a product page. 2026-01-24 18:27:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6in-x-3-8in-acrylic-mini-roller-covers-12-pack-8-case-6cr038q-12>: HTTP status code is not handled or not allowed 2026-01-24 18:27:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/work-benches/workbenches_worktables/open_base_workbenches/stationary_open_base_workbenches/standard_duty_stationary_workbenches landed on page that is not a product page. 2026-01-24 18:27:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9in-frame-for-a-4in-mini-roller-24-case-4rl-09>: HTTP status code is not handled or not allowed 2026-01-24 18:27:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7351d-dura-polyurethane-chair-aluminum-base-mushroom-glides-black>: HTTP status code is not handled or not allowed 2026-01-24 18:27:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 7914 pages (at 92 pages/min), scraped 5462 items (at 60 items/min) 2026-01-24 18:27:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25-ml-flask-clip-for-precision-shaking-water-baths>: HTTP status code is not handled or not allowed 2026-01-24 18:27:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-24-x-3-4-partition-size-sex-bolt-set-one-way-truss-head-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 18:27:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-16-x-3-4-security-cap-screw-button-socket-head-18-8-stainless-steel-pkg-of-25>: HTTP status code is not handled or not allowed 2026-01-24 18:27:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-32-x-1-security-machine-screw-button-torx-head-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 18:27:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-office-partition-panel-with-partial-window-and-pass-thru-cable-60-1-4w-x-77-1-2h-gray>: HTTP status code is not handled or not allowed 2026-01-24 18:27:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-office-partition-panel-with-partial-window-and-raceway-36-1-4w-x-47-1-2h-blue>: HTTP status code is not handled or not allowed 2026-01-24 18:27:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roof-cable-de-icer-160-ft-120v-7-watts-per-foot>: HTTP status code is not handled or not allowed 2026-01-24 18:27:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/mouse_pads_wrist_rests already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:27:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-tamperproof-spanner-bit-1-4-hex-shank>: HTTP status code is not handled or not allowed 2026-01-24 18:27:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flap-wheel-741a-6-inch-x-1-inch-x-1-inch-50-grit-ceramic>: HTTP status code is not handled or not allowed 2026-01-24 18:27:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelving-42w-x-24d-x-54h-3-shelf-add-on-chrome>: HTTP status code is not handled or not allowed 2026-01-24 18:27:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/mouse_pads_wrist_rests landed on page that is not a product page. 2026-01-24 18:27:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ball-valve-1-8-female-x-1-8-male-bspt>: HTTP status code is not handled or not allowed 2026-01-24 18:27:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/28-400-14-in-1-hp-steel-frame-band-saw>: HTTP status code is not handled or not allowed 2026-01-24 18:27:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-epoxy-wire-shelving-add-on-189370gz-60w-x-24d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 18:27:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-epoxy-wire-shelving-add-on-189353gz-54w-x-18d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 18:27:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-epoxy-wire-shelving-189218gz-36w-x-24d-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 18:27:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_storage_bench_protection/splash_guards_shields already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:27:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:27:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-48-w-x-48-d-x-120-h-add-on-3-shelf-levels-steel-deck-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 18:27:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:27:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:27:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:27:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/block-heater-for-compact-dry-bath-block-heater-3-x-50-ml-tubes-1-2-block>: HTTP status code is not handled or not allowed 2026-01-24 18:27:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/bins_totes_containers/baskets_organizers/closet_garment_organizers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:27:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1l-erlenmeyer-flask-clamp-236015-for-use-with-maxq-8000-shaker>: HTTP status code is not handled or not allowed 2026-01-24 18:27:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_storage_bench_protection/splash_guards_shields landed on page that is not a product page. 2026-01-24 18:27:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks>: HTTP status code is not handled or not allowed 2026-01-24 18:28:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks>: HTTP status code is not handled or not allowed 2026-01-24 18:28:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks landed on page that is not a product page. 2026-01-24 18:28:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks landed on page that is not a product page. 2026-01-24 18:28:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-hi-tech-metal-bin-shelving-36-w-x-24-d-x-87-h-725-hallowell-gray-add-on-unit-14-bins>: HTTP status code is not handled or not allowed 2026-01-24 18:28:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/bins_totes_containers/baskets_organizers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:28:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:28:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-yellow-kit-includes-a-carton-of-three-yellow-panels-and-one-set-of-98-black-reflective-locking-straps>: HTTP status code is not handled or not allowed 2026-01-24 18:28:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/arts_crafts_supplies/craft_basics already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:28:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011983-dial-for-rational-cooking-systems>: HTTP status code is not handled or not allowed 2026-01-24 18:28:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 8002 pages (at 88 pages/min), scraped 5523 items (at 61 items/min) 2026-01-24 18:28:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-72-w-x-36-d-x-72-h-starter-3-shelf-levels-wood-deck-sand>: HTTP status code is not handled or not allowed 2026-01-24 18:28:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:28:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/bins_totes_containers/baskets_organizers>: HTTP status code is not handled or not allowed 2026-01-24 18:28:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/locator-jr-cryogenic-rack-and-box-system-with-level-monitor-60-liters>: HTTP status code is not handled or not allowed 2026-01-24 18:28:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks landed on page that is not a product page. 2026-01-24 18:28:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/locator-4-cryogenic-rack-and-box-system-111-liters>: HTTP status code is not handled or not allowed 2026-01-24 18:28:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/cutting_tools/scissors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:28:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ese2020101-gourmet-elite-by-sink-20l-x-20w-x-10-1-8d>: HTTP status code is not handled or not allowed 2026-01-24 18:28:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:28:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:28:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks>: HTTP status code is not handled or not allowed 2026-01-24 18:28:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/arts_crafts_supplies/craft_basics landed on page that is not a product page. 2026-01-24 18:28:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:28:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kimberly-clark-41600-wypall-x70-wipersjumbo-rollperf12-12-x-13-25white>: HTTP status code is not handled or not allowed 2026-01-24 18:28:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_alarms_signals/audible_alarms already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:28:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_alarms_signals/audible_alarms already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:28:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/cutting_tools/scissors landed on page that is not a product page. 2026-01-24 18:28:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-48-w-x-36-d-x-96-h-starter-3-shelf-levels-steel-deck-sand>: HTTP status code is not handled or not allowed 2026-01-24 18:28:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks landed on page that is not a product page. 2026-01-24 18:28:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:28:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/rack_accessories/bulk_storage_rack_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:28:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lockable-gate-latch>: HTTP status code is not handled or not allowed 2026-01-24 18:28:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks landed on page that is not a product page. 2026-01-24 18:28:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks>: HTTP status code is not handled or not allowed 2026-01-24 18:28:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_alarms_signals/audible_alarms landed on page that is not a product page. 2026-01-24 18:28:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_alarms_signals/audible_alarms landed on page that is not a product page. 2026-01-24 18:28:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alera-aleva638232my-valencia-series-bookcase-6-shelves-31-34w-x-14d-x-80-38h-mahogany>: HTTP status code is not handled or not allowed 2026-01-24 18:28:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks landed on page that is not a product page. 2026-01-24 18:28:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-1042-21-3-4-inch-x-3-inch-cast-iron-top-broiler-grate>: HTTP status code is not handled or not allowed 2026-01-24 18:28:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/rack_accessories/bulk_storage_rack_shelves landed on page that is not a product page. 2026-01-24 18:28:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alera-aleva327236my-valencia-series-reception-desk-wcounter-71w-x-35-12d-x-42-12h-mahogany>: HTTP status code is not handled or not allowed 2026-01-24 18:28:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:28:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-inch-transfer-pump-hose-kit-w-aluminum-and-polypropylene-fittings>: HTTP status code is not handled or not allowed 2026-01-24 18:28:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-72-w-x-36-d-x-72-h-add-on-3-shelf-levels-wood-deck-medium-gray>: HTTP status code is not handled or not allowed 2026-01-24 18:28:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/walnut-laminate-office-suite-desking-473x236x295-bridge>: HTTP status code is not handled or not allowed 2026-01-24 18:28:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/portable_restrooms already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:28:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks landed on page that is not a product page. 2026-01-24 18:28:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tylenol-97477-extra-strength-caplets-30-two-packsbox>: HTTP status code is not handled or not allowed 2026-01-24 18:28:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2271159-lid-gasket-screw-hp-4req-for-henny-penny>: HTTP status code is not handled or not allowed 2026-01-24 18:29:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/portable_restrooms>: HTTP status code is not handled or not allowed 2026-01-24 18:29:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tubing-exerciser-red-22>: HTTP status code is not handled or not allowed 2026-01-24 18:29:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/wire_cable/electrical_wire_cable already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:29:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-ah21b-black-epoxy-utility-cart-handle-21w>: HTTP status code is not handled or not allowed 2026-01-24 18:29:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 8089 pages (at 87 pages/min), scraped 5579 items (at 56 items/min) 2026-01-24 18:29:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-inch-exhaust-shutter>: HTTP status code is not handled or not allowed 2026-01-24 18:29:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/681531-motor-convection-oven-208-240v-for-southbend-range>: HTTP status code is not handled or not allowed 2026-01-24 18:29:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fm-pro-series-steel-industrial-mobile-sanitzation-cart-41w-x-20d-x-43h-red>: HTTP status code is not handled or not allowed 2026-01-24 18:29:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unisex-white-80-polyester-20-cotton-poplin-snap-front-lab-coat-size-xl>: HTTP status code is not handled or not allowed 2026-01-24 18:29:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interion-174-restaurant-lunchroom-square-counter-height-table-42-x-42-x-36-h-mahogany>: HTTP status code is not handled or not allowed 2026-01-24 18:29:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/wire_cable/electrical_wire_cable landed on page that is not a product page. 2026-01-24 18:29:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/physical_therapy/exercise_fitness_equipment/stretch_bands_tubing already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:29:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unisex-navy-cotton-polyester-fashion-blend-reversible-scrub-pants-size-xl>: HTTP status code is not handled or not allowed 2026-01-24 18:29:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2801657-muddler-10-inch>: HTTP status code is not handled or not allowed 2026-01-24 18:29:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/battery-jumper-cable-7781-001-12-long-1-pc>: HTTP status code is not handled or not allowed 2026-01-24 18:29:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-foxx-2hp-ufo-101h2-vertical-bag-dust-collector>: HTTP status code is not handled or not allowed 2026-01-24 18:29:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/81023-8-mil-die-cut-apron-50-length-blue>: HTTP status code is not handled or not allowed 2026-01-24 18:29:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/physical_therapy/exercise_fitness_equipment/stretch_bands_tubing landed on page that is not a product page. 2026-01-24 18:29:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/70476-high-low-brush-medium-yellow>: HTTP status code is not handled or not allowed 2026-01-24 18:29:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_hinges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:29:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rolling-service-stock-cart-2-shelves-1200-lbs-capacity-60-x-30>: HTTP status code is not handled or not allowed 2026-01-24 18:29:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tx4000-esd-cleanroom-apparel-frock-3xl-white>: HTTP status code is not handled or not allowed 2026-01-24 18:29:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-free-exercise-band-plum-50-yard-roll-box>: HTTP status code is not handled or not allowed 2026-01-24 18:29:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010631-fuse-15a-for-cleveland-range>: HTTP status code is not handled or not allowed 2026-01-24 18:29:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/locksets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:29:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_hinges landed on page that is not a product page. 2026-01-24 18:29:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/321897-gas-connector-kit-1-inch-x48-inch-w-qd-and-elbow-for-dormont-mfg>: HTTP status code is not handled or not allowed 2026-01-24 18:29:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-surface-mounted-towel-shelf-w-towel-bar-24-w-bright-polished>: HTTP status code is not handled or not allowed 2026-01-24 18:29:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/70374-tank-brush-stiff-red>: HTTP status code is not handled or not allowed 2026-01-24 18:29:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_tools/cleaning_tool_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:29:51 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/door_window_hardware/locksets landed on page that is not a product page. 2026-01-24 18:29:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cylinder-tube-rack-4-wide-x-4-deep-58w-x-50d-x-30h16-cylinder-cap>: HTTP status code is not handled or not allowed 2026-01-24 18:29:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-classicseries-25-cent-recessed-semi-recessed-sanitary-vendor>: HTTP status code is not handled or not allowed 2026-01-24 18:29:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-battery-jumper-cable-4-awg-25l-blackred>: HTTP status code is not handled or not allowed 2026-01-24 18:29:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_tools/cleaning_tool_accessories landed on page that is not a product page. 2026-01-24 18:29:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-battery-jumper-cable-4t-awg-25l-blackred>: HTTP status code is not handled or not allowed 2026-01-24 18:30:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/bollards/safety_bollards_posts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:30:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gn852-latch-type-toggle-clamp-852-2800-t2s-welding-w-pull-latch-forged-steel>: HTTP status code is not handled or not allowed 2026-01-24 18:30:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/bollards/safety_bollards_posts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:30:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-classicseries-surface-mounted-waste-receptacle>: HTTP status code is not handled or not allowed 2026-01-24 18:30:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/bollards/safety_bollards_posts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:30:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/38-1430-13-32-inch-x-1-1-2-inch-10a-fast-acting-ktk-10-glass-fuse-600v>: HTTP status code is not handled or not allowed 2026-01-24 18:30:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 8179 pages (at 90 pages/min), scraped 5639 items (at 60 items/min) 2026-01-24 18:30:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-w-x-36-d-production-workbench-esd-laminate-square-edge-with-drawers-shelf-blue>: HTTP status code is not handled or not allowed 2026-01-24 18:30:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/bollards/safety_bollards_posts landed on page that is not a product page. 2026-01-24 18:30:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-polyester-filter-20-w-x-30-h-x-1-d>: HTTP status code is not handled or not allowed 2026-01-24 18:30:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/bollards/safety_bollards_posts landed on page that is not a product page. 2026-01-24 18:30:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-w-x-30-d-production-workbench-maple-butcher-block-safety-complete-bench-blue>: HTTP status code is not handled or not allowed 2026-01-24 18:30:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/panel-moulding-pml06x02pn1-618h-x-2d-x-9412l>: HTTP status code is not handled or not allowed 2026-01-24 18:30:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/bollards/safety_bollards_posts landed on page that is not a product page. 2026-01-24 18:30:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-office-partition-panel-with-pass-thru-cable-24-1-4w-x-65-1-2h-blue>: HTTP status code is not handled or not allowed 2026-01-24 18:30:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kenda-super-turf-tire-16x650-8-4pr>: HTTP status code is not handled or not allowed 2026-01-24 18:30:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/shop_presses already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:30:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/riser-straight-with-carpet-96l-x-18w-x-32h-black>: HTTP status code is not handled or not allowed 2026-01-24 18:30:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-32-tamperproof-hex-screwdriver>: HTTP status code is not handled or not allowed 2026-01-24 18:30:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sewn-liquid-bag-filter-nylon-mesh-5-1-2-dia-x-21-l-1000-micron-steel-ring-pkg-qty-50>: HTTP status code is not handled or not allowed 2026-01-24 18:30:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/shop_presses landed on page that is not a product page. 2026-01-24 18:30:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-32-x-3-4-tri-wing-machine-screw-pan-head-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 18:30:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sewn-liquid-bag-filter-polyester-multifilament-4-1-8-x-8-800-micron-steel-ring-pkg-50>: HTTP status code is not handled or not allowed 2026-01-24 18:30:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-office-partition-panel-with-partial-window-and-pass-thru-cable-60-1-4w-x-65-1-2h-blue>: HTTP status code is not handled or not allowed 2026-01-24 18:30:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-8-tamperproof-hex-key-double-ended>: HTTP status code is not handled or not allowed 2026-01-24 18:30:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/seating/toddler_chairs_seating/high_chairs_boosters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:30:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:30:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-27-tamperproof-torx-power-bit>: HTTP status code is not handled or not allowed 2026-01-24 18:30:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-tamperproof-spanner-bit-1-4-hex-shank>: HTTP status code is not handled or not allowed 2026-01-24 18:30:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/hvac_system_parts_components/heating_parts_components/thermocouples_thermopiles already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:30:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelving-60w-x-14d-x-14h-1-shelf-starter-chrome>: HTTP status code is not handled or not allowed 2026-01-24 18:30:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/seating/toddler_chairs_seating/high_chairs_boosters landed on page that is not a product page. 2026-01-24 18:30:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks landed on page that is not a product page. 2026-01-24 18:30:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/etx-15-tank-non-potable-water-expansion-tank>: HTTP status code is not handled or not allowed 2026-01-24 18:30:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelving-24w-x-14d-x-54h-3-shelf-add-on-chrome>: HTTP status code is not handled or not allowed 2026-01-24 18:31:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/hvac_system_parts_components/heating_parts_components/thermocouples_thermopiles>: HTTP status code is not handled or not allowed 2026-01-24 18:31:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-control-6mm-tube-x-1-8-metal-release-collet-flow-in-screw-adjustment>: HTTP status code is not handled or not allowed 2026-01-24 18:31:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/biosystem-stirrer-without-controller-4-position-5l-per-position>: HTTP status code is not handled or not allowed 2026-01-24 18:31:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-48-x-36-adj-height-workbench-w-drawer-riser-gray-maple-safety-edge-top>: HTTP status code is not handled or not allowed 2026-01-24 18:31:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 8275 pages (at 96 pages/min), scraped 5706 items (at 67 items/min) 2026-01-24 18:31:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dirty-equipment-tag-red-2-5-16w-4-3-4l>: HTTP status code is not handled or not allowed 2026-01-24 18:31:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/outdoor_tools_power_equipment/pressure_washers/gas_pressure_washers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:31:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-equipment-cover-clear-on-roll-38w-48l>: HTTP status code is not handled or not allowed 2026-01-24 18:31:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/rack_accessories/bulk_storage_rack_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:31:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:31:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metal-shelves-galvanized-3-x-48-2-pc>: HTTP status code is not handled or not allowed 2026-01-24 18:31:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:31:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-leg-ironing-board-w-iron-rest-pad-white-blue-stripes-black-base-pp-plastic-steel-foam-cotton>: HTTP status code is not handled or not allowed 2026-01-24 18:31:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/outdoor_tools_power_equipment/pressure_washers/gas_pressure_washers landed on page that is not a product page. 2026-01-24 18:31:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smooth-platform-kit-8-x-18-x-18>: HTTP status code is not handled or not allowed 2026-01-24 18:31:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/rack_accessories/bulk_storage_rack_shelves landed on page that is not a product page. 2026-01-24 18:31:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks>: HTTP status code is not handled or not allowed 2026-01-24 18:31:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks>: HTTP status code is not handled or not allowed 2026-01-24 18:31:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:31:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-hi-tech-metal-bin-shelving-36-w-x-18-d-x-87-h-725-hallowell-gray-starter-unit-29-bins>: HTTP status code is not handled or not allowed 2026-01-24 18:31:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:31:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:31:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1901298-screw-handle-stainless-steel-slotted-for-bunn-o-matic>: HTTP status code is not handled or not allowed 2026-01-24 18:31:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/furniture_parts_accessories/partition_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:31:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:31:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/furniture_parts_accessories/partition_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:31:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fort-knox-pegboard-2-pieces-22-w-x-075-d-x-4425-h-red-textured-1>: HTTP status code is not handled or not allowed 2026-01-24 18:31:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks landed on page that is not a product page. 2026-01-24 18:31:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2041228-pin-door-hinge-10-32-thd-for-hatco-corp>: HTTP status code is not handled or not allowed 2026-01-24 18:31:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks landed on page that is not a product page. 2026-01-24 18:31:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_alarms_signals/audible_alarms already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:31:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks landed on page that is not a product page. 2026-01-24 18:31:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/timber-jack-tmb-55-37-wood-handle-with-14-1-4-steel-hook>: HTTP status code is not handled or not allowed 2026-01-24 18:31:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/furniture_parts_accessories/partition_accessories landed on page that is not a product page. 2026-01-24 18:31:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/furniture_parts_accessories/partition_accessories>: HTTP status code is not handled or not allowed 2026-01-24 18:31:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks landed on page that is not a product page. 2026-01-24 18:31:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-72-w-x-36-d-x-96-h-add-on-3-shelf-levels-wood-deck-medium-gray>: HTTP status code is not handled or not allowed 2026-01-24 18:31:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_alarms_signals/audible_alarms landed on page that is not a product page. 2026-01-24 18:31:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/imh60-s-advanced-protocol-security-incubator-2-3-cu-ft-120v>: HTTP status code is not handled or not allowed 2026-01-24 18:31:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1331493-strip-test-fryer-oil-40-vial>: HTTP status code is not handled or not allowed 2026-01-24 18:32:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-inch-x-25-foot-pvc-lay-flat-discharge-hose-coupled-with-cxe-aluminum-cam-and-groove-fittings>: HTTP status code is not handled or not allowed 2026-01-24 18:32:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/office_furniture/literature_mail_sorters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:32:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-ton-service-jack-hw93652>: HTTP status code is not handled or not allowed 2026-01-24 18:32:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1171023-nut-lock-1-2-inch-nps-brass>: HTTP status code is not handled or not allowed 2026-01-24 18:32:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 8359 pages (at 84 pages/min), scraped 5764 items (at 58 items/min) 2026-01-24 18:32:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hein-werner-2-1-2-ton-air-operated-end-lift-with-remote-operation-hw93696u>: HTTP status code is not handled or not allowed 2026-01-24 18:32:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-40-14w-cabinet-15-drawer-243-compart-light-gray-individual-lock>: HTTP status code is not handled or not allowed 2026-01-24 18:32:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/office_furniture/literature_mail_sorters landed on page that is not a product page. 2026-01-24 18:32:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-band-handles-with-d-ring-50-pairs>: HTTP status code is not handled or not allowed 2026-01-24 18:32:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-whitegreen-multi-conductor-flat-jacketed-wire-233203-1000-12-3-gauge-1000-ft>: HTTP status code is not handled or not allowed 2026-01-24 18:32:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stud-to-post-conversion-connector-5514-2002>: HTTP status code is not handled or not allowed 2026-01-24 18:32:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/wire_cable/electrical_wire_cable already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:32:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/wiring_devices/wire_connectors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:32:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-free-exercise-tubing-blue-100-roll-box-10-5724>: HTTP status code is not handled or not allowed 2026-01-24 18:32:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/wiring_devices/wire_connectors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:32:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/wiring_devices/wire_connectors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:32:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/starter-ground-magnalugs-6404-005f-3-8-5-pcs>: HTTP status code is not handled or not allowed 2026-01-24 18:32:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/electrochemistry_meters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:32:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/cable_management/cable_ties_wraps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:32:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/cable_management/corrugated_tubing_fittings already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:32:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-inplant-office-w-2-windows-class-a-12-w-x-12-d-gray>: HTTP status code is not handled or not allowed 2026-01-24 18:32:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/wire_cable/electrical_wire_cable landed on page that is not a product page. 2026-01-24 18:32:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/wiring_devices/wire_connectors landed on page that is not a product page. 2026-01-24 18:32:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/red-black-dual-wall-heat-shrink-kit-506391-001-6-long>: HTTP status code is not handled or not allowed 2026-01-24 18:32:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/wiring_devices/wire_connectors landed on page that is not a product page. 2026-01-24 18:32:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/wiring_devices/wire_connectors landed on page that is not a product page. 2026-01-24 18:32:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/electrochemistry_meters>: HTTP status code is not handled or not allowed 2026-01-24 18:32:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/cable_management/cable_ties_wraps landed on page that is not a product page. 2026-01-24 18:32:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leviton-23654-twp-miniature-base-t5-bi-pin-fluorescent-lampholder>: HTTP status code is not handled or not allowed 2026-01-24 18:32:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/cable_management/corrugated_tubing_fittings landed on page that is not a product page. 2026-01-24 18:32:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/77155-28-double-blade-ultra-hygiene-squeegee-white>: HTTP status code is not handled or not allowed 2026-01-24 18:32:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leviton-13654-snp-miniature-base-t5-bi-pin-standard-fluorescent-lampholder>: HTTP status code is not handled or not allowed 2026-01-24 18:32:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/locksets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:32:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/71404-16-single-blade-ultra-hygiene-squeegee-red>: HTTP status code is not handled or not allowed 2026-01-24 18:32:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-exercise-band-yellow-6-yard-roll-box>: HTTP status code is not handled or not allowed 2026-01-24 18:32:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_hinges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:32:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_closers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:32:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mlw-m4-20ah-battery>: HTTP status code is not handled or not allowed 2026-01-24 18:32:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/fasteners/door_window_hardware/locksets>: HTTP status code is not handled or not allowed 2026-01-24 18:32:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-channel-frame-mirror-shelf-combination-18-x-24>: HTTP status code is not handled or not allowed 2026-01-24 18:33:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_closers>: HTTP status code is not handled or not allowed 2026-01-24 18:33:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_hinges landed on page that is not a product page. 2026-01-24 18:33:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:33:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-cylinder-tube-rack-1-wide-x-1-deep-16w-x-16d-x-30h2-cylinder-cap>: HTTP status code is not handled or not allowed 2026-01-24 18:33:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eatrth5s1pextreme-duty-industrial-air-ratchet12-driveextended-handle18-cfm14-npt-inlet>: HTTP status code is not handled or not allowed 2026-01-24 18:33:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:33:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 8443 pages (at 84 pages/min), scraped 5819 items (at 55 items/min) 2026-01-24 18:33:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-w-x-36-d-production-workbench-maple-butcher-block-safety-edge-with-drawers-shelf-blue>: HTTP status code is not handled or not allowed 2026-01-24 18:33:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots>: HTTP status code is not handled or not allowed 2026-01-24 18:33:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/screws/machine_screws/flat_head_machine_screws already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:33:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots>: HTTP status code is not handled or not allowed 2026-01-24 18:33:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/synthetic-winch-line-with-stainless-steel-thimble-58-dia-x-75l>: HTTP status code is not handled or not allowed 2026-01-24 18:33:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/needle-roller-cage-assembly-18mm-bore-22mm-od-10mm-w-steel-cage-material>: HTTP status code is not handled or not allowed 2026-01-24 18:33:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-classicseries-recessed-towel-dispenser-receptacle>: HTTP status code is not handled or not allowed 2026-01-24 18:33:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/screws/machine_screws/flat_head_machine_screws landed on page that is not a product page. 2026-01-24 18:33:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-truck-4-shelves-60x30-3000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 18:33:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/toilet-plunger-tbp100>: HTTP status code is not handled or not allowed 2026-01-24 18:33:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelate-wire-shelf-utility-cart-with-brakes-36x18-2-shelves-800-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 18:33:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-3-soow-black-250-ft>: HTTP status code is not handled or not allowed 2026-01-24 18:33:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/ventilated_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:33:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/line-set-38-suction-insulation-38-od-x-78-od-x-50l-black>: HTTP status code is not handled or not allowed 2026-01-24 18:33:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galvanized-roof-hatch-30-x-36>: HTTP status code is not handled or not allowed 2026-01-24 18:33:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dinex-rectangular-soup-bowl-lid-1000-cs-clear-dxhh30>: HTTP status code is not handled or not allowed 2026-01-24 18:33:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/261370-nut>: HTTP status code is not handled or not allowed 2026-01-24 18:33:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/681391-pump-motor-assembly>: HTTP status code is not handled or not allowed 2026-01-24 18:33:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-access-door-14-x-14>: HTTP status code is not handled or not allowed 2026-01-24 18:33:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wardrobe-cabinet-with-full-width-rod-48-x-24-x-66>: HTTP status code is not handled or not allowed 2026-01-24 18:33:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-30-steel-top-workbench-adjustable-height>: HTTP status code is not handled or not allowed 2026-01-24 18:33:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/wardrobe_combination_cabinets/combination_cabients already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:33:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/29606-51-ultra-hygiene-handle-yellow>: HTTP status code is not handled or not allowed 2026-01-24 18:33:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:33:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/29752-62-113-telescopic-aluminum-handle-green>: HTTP status code is not handled or not allowed 2026-01-24 18:33:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-unit-blk-yellow-30-caution-do-not-enter-yellow-w-clamp-belt-end>: HTTP status code is not handled or not allowed 2026-01-24 18:33:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/cabinets/ventilated_cabinets landed on page that is not a product page. 2026-01-24 18:33:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pure-air-gel-8-ounce-cup>: HTTP status code is not handled or not allowed 2026-01-24 18:34:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medium-cube-ice-machine-356lb>: HTTP status code is not handled or not allowed 2026-01-24 18:34:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits>: HTTP status code is not handled or not allowed 2026-01-24 18:34:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/cabinets/wardrobe_combination_cabinets/combination_cabients landed on page that is not a product page. 2026-01-24 18:34:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forkli-able-shipping-container-with-lid-44-x-23-5>: HTTP status code is not handled or not allowed 2026-01-24 18:34:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/abt-hc-sls-26-standard-solid-door-laboratory-refrigerator-26-cu-ft>: HTTP status code is not handled or not allowed 2026-01-24 18:34:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 8541 pages (at 98 pages/min), scraped 5888 items (at 69 items/min) 2026-01-24 18:34:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pack-of-2-rolls-of-duct-tape>: HTTP status code is not handled or not allowed 2026-01-24 18:34:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cgc08025-machine-screw>: HTTP status code is not handled or not allowed 2026-01-24 18:34:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/cookware_bakeware/pots_pans/frying_pans_skillets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:34:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50-compartment-sectional-unit-37-w-x-13-d-x-60-h-gray>: HTTP status code is not handled or not allowed 2026-01-24 18:34:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/63-h-nexel-chrome-wire-shelving-48-w-x-14-d>: HTTP status code is not handled or not allowed 2026-01-24 18:34:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-force-rug-4-x-6-46-inch-x-72-inch>: HTTP status code is not handled or not allowed 2026-01-24 18:34:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hand-basket-stand-and-sign-for-28-liter-basket>: HTTP status code is not handled or not allowed 2026-01-24 18:34:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/cookware_bakeware/pots_pans/frying_pans_skillets landed on page that is not a product page. 2026-01-24 18:34:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tbsp-long-handle-measuring-spoon>: HTTP status code is not handled or not allowed 2026-01-24 18:34:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poly-oxford-insulated-safety-bib-overall-xxs-black>: HTTP status code is not handled or not allowed 2026-01-24 18:34:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-2082-control-solution-low-4-ml>: HTTP status code is not handled or not allowed 2026-01-24 18:34:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelite-vented-plastic-mat-shelf-truck-36l-x-18w-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 18:34:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industries-flush-center-post-for-1-tubing-satin-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 18:34:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poly-oxford-insulated-safety-bib-overall-m-black>: HTTP status code is not handled or not allowed 2026-01-24 18:34:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cjc04005-machine-screw>: HTTP status code is not handled or not allowed 2026-01-24 18:34:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelite-vented-plastic-mat-shelf-truck-36l-x-18w-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 18:34:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/360-stretch-waist-flex-twill-cargo-pants-32l-inseam-x-29-30-waist-navy>: HTTP status code is not handled or not allowed 2026-01-24 18:34:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/switches_receptacles_sensors/wall_switches already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:34:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/tool_storage/chests_roller_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:34:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-file-folders-15-cut-assorted-one-ply-top-tab-legal-manila-100box>: HTTP status code is not handled or not allowed 2026-01-24 18:34:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-colored-file-folders-13-cut-one-ply-top-tab-letter-violet-100box>: HTTP status code is not handled or not allowed 2026-01-24 18:34:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/wire_shelving/poly_zbrite_wire_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:34:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/54-h-nexel-chrome-wire-shelving-24-w-x-14-d>: HTTP status code is not handled or not allowed 2026-01-24 18:34:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-hanging-file-folders-15-tab-11-point-stock-letter-standard-green-25box>: HTTP status code is not handled or not allowed 2026-01-24 18:34:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/tool_storage/chests_roller_cabinets landed on page that is not a product page. 2026-01-24 18:35:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/performance-gravity-hvlp-atomizing-head-refill-kit-26716-1-6mm-purple-pack-of-5>: HTTP status code is not handled or not allowed 2026-01-24 18:35:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1331391-bar-adapter-21-inch-l-steam-table-for-vollrath-idea-company>: HTTP status code is not handled or not allowed 2026-01-24 18:35:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/wire_shelving/poly_zbrite_wire_shelves landed on page that is not a product page. 2026-01-24 18:35:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cp100d-5-watt-160-channel-analog-and-digital-vhf-136-174-mhz-display-with-limited-keypad>: HTTP status code is not handled or not allowed 2026-01-24 18:35:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/switches_receptacles_sensors/wall_switches landed on page that is not a product page. 2026-01-24 18:35:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smead-2-capacity-closed-side-flexible-hanging-file-pockets-letter-sky-blue-25box>: HTTP status code is not handled or not allowed 2026-01-24 18:35:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/expanding_file_folders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:35:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cct-32-cruiser-bench-counting-scale-70-lb-x-0-002-lb>: HTTP status code is not handled or not allowed 2026-01-24 18:35:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 8634 pages (at 93 pages/min), scraped 5952 items (at 64 items/min) 2026-01-24 18:35:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hubbel-commercial-14-4-20>: HTTP status code is not handled or not allowed 2026-01-24 18:35:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fox-40-whistle-orange-10-421-org>: HTTP status code is not handled or not allowed 2026-01-24 18:35:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/expanding_file_folders landed on page that is not a product page. 2026-01-24 18:35:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acrylic-coated-fiberglass-welding-blanket-8w-x-8h-yellow>: HTTP status code is not handled or not allowed 2026-01-24 18:35:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/85-1036-high-temperature-small-porcelain-wire-connectors-100-pack>: HTTP status code is not handled or not allowed 2026-01-24 18:35:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/281311-wheel5-inch>: HTTP status code is not handled or not allowed 2026-01-24 18:35:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/material-handling/cranes_hoists/crane_parts_accessories/lifting_spreader_beams already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:35:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8ft-long-shirred-pleat-skirting-8h-stage-blue>: HTTP status code is not handled or not allowed 2026-01-24 18:35:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/file_folders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:35:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chicago-pneumatic-mini-angle-die-grinder-cp875-22500-rpm>: HTTP status code is not handled or not allowed 2026-01-24 18:35:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/hi_vis_jackets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:35:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1751154-caster-plate-2-inch-rigid-for-magikitchen-products>: HTTP status code is not handled or not allowed 2026-01-24 18:35:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/material-handling/cranes_hoists/crane_parts_accessories/lifting_spreader_beams>: HTTP status code is not handled or not allowed 2026-01-24 18:35:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/file_folders>: HTTP status code is not handled or not allowed 2026-01-24 18:35:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1331317-handle-top-kit-short-shuttle-for-worchester-industrial-products>: HTTP status code is not handled or not allowed 2026-01-24 18:35:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paraffin-bath-refill-6-lb-blocks-rose-blossom-fragrance>: HTTP status code is not handled or not allowed 2026-01-24 18:35:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/hi_vis_jackets landed on page that is not a product page. 2026-01-24 18:35:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/square-edge-work-bench-top-12-gauge-steel-2>: HTTP status code is not handled or not allowed 2026-01-24 18:35:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/moist-heat-pack-cover-all-terry-microfiber-half-size-12-pk>: HTTP status code is not handled or not allowed 2026-01-24 18:35:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-wide-span-rack-60x36x96-with-3-shelves-wood-deck-1200-lb-capacity-per-level>: HTTP status code is not handled or not allowed 2026-01-24 18:35:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-expandable-add-on-rack-96x36x84-gray-with-3-level-wood-deck-800lb-cap-per-level>: HTTP status code is not handled or not allowed 2026-01-24 18:35:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/741259-door-gasket-23-inch-x-59-inch-for-delfield>: HTTP status code is not handled or not allowed 2026-01-24 18:35:50 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-24 18:35:51 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:6 2026-01-24 18:35:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2051112-handle-product-grip-for-hobart>: HTTP status code is not handled or not allowed 2026-01-24 18:35:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quart-size-standup-pouch-reclosable-bag-with-write-on-strips-8-x-7-175-mil>: HTTP status code is not handled or not allowed 2026-01-24 18:36:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-32mm-bore-x-25mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 18:36:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-6432-single-acting-magnetic-cylinder-10mm-bore-x-10mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 18:36:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/85-1085-high-temperature-large-porcelain-wire-connectors-25-pack>: HTTP status code is not handled or not allowed 2026-01-24 18:36:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-40mm-bore-x-730mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 18:36:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/small-insulated-ice-bag-with-hook-loop-band-5-x-13>: HTTP status code is not handled or not allowed 2026-01-24 18:36:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/moto-vest-4-pockets-cottonpolyester-lt-black>: HTTP status code is not handled or not allowed 2026-01-24 18:36:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 8728 pages (at 94 pages/min), scraped 6020 items (at 68 items/min) 2026-01-24 18:36:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eagle-acid-corrosive-cabinet-self-close-60-gallon>: HTTP status code is not handled or not allowed 2026-01-24 18:36:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-inch-id-x-34-inch-od-x-0093-inch-steel-arbor-shim-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 18:36:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15022-12-rapid-deployment-rescue-ladder-clear-zinc-osha-compliant>: HTTP status code is not handled or not allowed 2026-01-24 18:36:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/finish-thompson-wall-mounting-bracket-pf-tt-pumps>: HTTP status code is not handled or not allowed 2026-01-24 18:36:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-12-inch-square-keystock-plain-finish-36-inch-length>: HTTP status code is not handled or not allowed 2026-01-24 18:36:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011546-hinge-kit-bottom-for-true-manufacturing>: HTTP status code is not handled or not allowed 2026-01-24 18:36:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-welded-slim-line-cabinet-1>: HTTP status code is not handled or not allowed 2026-01-24 18:36:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0003-inch-stainless-steel-shim-stock-12-inch-x-50-inch-roll>: HTTP status code is not handled or not allowed 2026-01-24 18:36:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-branch-tee-89210-02-02-18-tube-x-18-swift-fit-metallic-collet>: HTTP status code is not handled or not allowed 2026-01-24 18:36:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/025mm-brass-shim-stock-150mm-x-25m-roll>: HTTP status code is not handled or not allowed 2026-01-24 18:36:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/union-60040-8-8mm-tube-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 18:36:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8013605-door-seal-402s-e4-e4s-for-merry-chef>: HTTP status code is not handled or not allowed 2026-01-24 18:36:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0005-inch-brass-shim-stock-6-inch-x-60-inch-roll>: HTTP status code is not handled or not allowed 2026-01-24 18:36:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30066-500-plastic-barrier-chain-hdpe-1-5-x500-6-38mm-blue>: HTTP status code is not handled or not allowed 2026-01-24 18:36:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/262418-threaded-stem-caster4-w-1-2-13-x-1>: HTTP status code is not handled or not allowed 2026-01-24 18:36:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/definite_purpose_electric_motors/agricultural_motors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:36:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/51009-100-heavy-duty-plastic-barrier-chain-hdpe-2-x100-54mm-gold>: HTTP status code is not handled or not allowed 2026-01-24 18:36:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/definite_purpose_electric_motors/agricultural_motors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:36:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-aquatic-exercise-kit-jogger-belt-hand-bars-large-blue>: HTTP status code is not handled or not allowed 2026-01-24 18:36:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50324-10-s-hook-acetal-copolymer-2-blue-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 18:36:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8014367-bearing-kit-pk-4-sets-for-roundup-food-equipment>: HTTP status code is not handled or not allowed 2026-01-24 18:36:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/definite_purpose_electric_motors/agricultural_motors landed on page that is not a product page. 2026-01-24 18:36:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/definite_purpose_electric_motors/agricultural_motors landed on page that is not a product page. 2026-01-24 18:36:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ms48ss-all-ss-military-worm-gear-hose-clamp-2-916-inch-3-12-inch-clmpg-diam-10-pcs>: HTTP status code is not handled or not allowed 2026-01-24 18:36:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/262388-swivel-stem-caster4-w-1-5-8-od-tubing>: HTTP status code is not handled or not allowed 2026-01-24 18:36:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dixie-ultra-pathways-heavyweight-paper-bowls-20-oz-greenburgundy-500ct>: HTTP status code is not handled or not allowed 2026-01-24 18:36:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cs32h-collared-screw-worm-gear-hose-clamp-1-916-inch-2-12-inch-clamping-diam-10-pcs>: HTTP status code is not handled or not allowed 2026-01-24 18:36:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tex-gold-fp-kinesiology-tape-2-x-5-5-yds-beige-6-rolls>: HTTP status code is not handled or not allowed 2026-01-24 18:37:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-round-basket-12-3-4-dia-x-12-3-8-h-plain-steel-price-each-for-qty-4>: HTTP status code is not handled or not allowed 2026-01-24 18:37:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eco-products-renewable-psm-cutlery-teaspoon-cream-1000carton>: HTTP status code is not handled or not allowed 2026-01-24 18:37:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1901253-hopper-assembly-for-bunn-o-matic>: HTTP status code is not handled or not allowed 2026-01-24 18:37:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8014275-contactor-3pole-240v-40a>: HTTP status code is not handled or not allowed 2026-01-24 18:37:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 8828 pages (at 100 pages/min), scraped 6089 items (at 69 items/min) 2026-01-24 18:37:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-kit-for-electric-pallet-truck-gwk-8hbw23-lw-fits-toyota-model-8hbw24>: HTTP status code is not handled or not allowed 2026-01-24 18:37:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50mm-x-50mm-x-2mm-stainless-steel-metric-slotted-shim-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 18:37:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hammer-drill-bits-irwin-322017>: HTTP status code is not handled or not allowed 2026-01-24 18:37:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10500-series-laminate-bookcase-three-shelf-36-w-x-13-1-8-d-x-43-3-8-h-harvest>: HTTP status code is not handled or not allowed 2026-01-24 18:37:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s9491-65w-mr16-led-25-beam-spread-gu53-base-3000k-12v>: HTTP status code is not handled or not allowed 2026-01-24 18:37:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/184516-rfx-cs1500-ntep-crane-scale-with-led-display-2000-lb-x-1-lb>: HTTP status code is not handled or not allowed 2026-01-24 18:37:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-stacking-bin-clear-8>: HTTP status code is not handled or not allowed 2026-01-24 18:37:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/heaters/electric_heaters/electric_unit_heaters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:37:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vci-co-extruded-gusseted-bags-49-x-46-x-75-3-mil-50-bags-per-roll>: HTTP status code is not handled or not allowed 2026-01-24 18:37:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-tear-drop-pallet-rack-add-on-144w-x-48d-x-96h>: HTTP status code is not handled or not allowed 2026-01-24 18:37:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-inch-plastic-classroom-chair-set-of-five-green>: HTTP status code is not handled or not allowed 2026-01-24 18:37:34 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/heaters/electric_heaters/electric_unit_heaters landed on page that is not a product page. 2026-01-24 18:37:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-inch-plastic-classroom-chair-set-of-five-gray>: HTTP status code is not handled or not allowed 2026-01-24 18:37:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/positioning-adjustable-lanyard-6l-2-snaphooks>: HTTP status code is not handled or not allowed 2026-01-24 18:37:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insecticide-fruit-fly-vapor-strip-10-strips-per-case>: HTTP status code is not handled or not allowed 2026-01-24 18:37:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lab-pack-open-head-poly-drum-metal-lever-lock-14-gallon-capacity-black>: HTTP status code is not handled or not allowed 2026-01-24 18:37:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/collapsible-bulk-container-1800-lb-capacity-32l-x-30w-x-25h-black>: HTTP status code is not handled or not allowed 2026-01-24 18:37:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/computer-lan-workstation-mobile-2>: HTTP status code is not handled or not allowed 2026-01-24 18:37:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/guest_chairs/guest_reception_chairs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:37:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/264602-grease-filter-aluminum-16-x-16-x-2>: HTTP status code is not handled or not allowed 2026-01-24 18:37:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mixer-dough-hook-xmix0704-for-7-quart-mixer>: HTTP status code is not handled or not allowed 2026-01-24 18:37:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/door_guards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:37:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-door-track-protector-48-pair-black>: HTTP status code is not handled or not allowed 2026-01-24 18:37:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/guest_chairs/guest_reception_chairs>: HTTP status code is not handled or not allowed 2026-01-24 18:38:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wear-ever-fry-pan-with-ceramiguard-ii-interior-67954-8-gauge-10-7-8-bottom-diameter>: HTTP status code is not handled or not allowed 2026-01-24 18:38:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/door_guards>: HTTP status code is not handled or not allowed 2026-01-24 18:38:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-36-esd-square-edge-work-bench-fixed-height-1-1-4-top>: HTTP status code is not handled or not allowed 2026-01-24 18:38:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 8926 pages (at 98 pages/min), scraped 6164 items (at 75 items/min) 2026-01-24 18:38:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/new-haven-6-ash-wood-flat-bench-without-back-brown>: HTTP status code is not handled or not allowed 2026-01-24 18:38:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wear-ever-ever-smooth-fry-pan-es4012-8-gauge-9-3-4-bottom-diameter>: HTTP status code is not handled or not allowed 2026-01-24 18:38:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/264592-grease-filter-stainless-steel-20-x-16-x-2>: HTTP status code is not handled or not allowed 2026-01-24 18:38:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fowler-digi-pro-electronic-protractor>: HTTP status code is not handled or not allowed 2026-01-24 18:38:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1ul-inoculating-loop-bulk-pack-sterile-white-218mm-2000-case>: HTTP status code is not handled or not allowed 2026-01-24 18:38:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slicing-plate-msg2003-1-8-fits-40785>: HTTP status code is not handled or not allowed 2026-01-24 18:38:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-round-table-with-4-imme-armless-caster-chairs-black-seatdesigner-white-top>: HTTP status code is not handled or not allowed 2026-01-24 18:38:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iec-metric-motor-112m-ip55-3ph-c112t17fz1cc>: HTTP status code is not handled or not allowed 2026-01-24 18:38:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2000a-adjustable-case-sealer-w-accuglide-nph-taping-head-bottom-belt-drive-2w-tape>: HTTP status code is not handled or not allowed 2026-01-24 18:38:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/professional-3-commercial-hepa-air-purifier-stainless>: HTTP status code is not handled or not allowed 2026-01-24 18:38:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grinder-knife-40750-no-22-fits-40744>: HTTP status code is not handled or not allowed 2026-01-24 18:38:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-9-drawer-standard-width-cabinet-blue-keyed-alike>: HTTP status code is not handled or not allowed 2026-01-24 18:38:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-round-table-with-4-imme-armless-chairs-navy-seatdesigner-white-top>: HTTP status code is not handled or not allowed 2026-01-24 18:38:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/vbelts_timing_belts/wrapped_vbelts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:38:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pack-of-1-ptfe-o-ring-dash-350>: HTTP status code is not handled or not allowed 2026-01-24 18:38:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traex-squeeze-dispensers-5224-13-standard-24-oz>: HTTP status code is not handled or not allowed 2026-01-24 18:38:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/bins_totes_containers/organizers_caddies/drawer_organizers_dividers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:38:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/100-amp-solar-charge-controller-12-24-36-48-vdc-mppt>: HTTP status code is not handled or not allowed 2026-01-24 18:38:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/vbelts_timing_belts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:38:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-round-table-with-4-imme-armless-chairs-coral-seatriver-cherry-top>: HTTP status code is not handled or not allowed 2026-01-24 18:38:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/lathes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:38:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/antimicrobial-film-tape-for-touch-screens-60-h-x-7-w-clear>: HTTP status code is not handled or not allowed 2026-01-24 18:38:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/lathes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:38:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2691021-shelf-left-21-inch-x-15-3-4-inch-for-perlick-refrigeration>: HTTP status code is not handled or not allowed 2026-01-24 18:38:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-round-table-with-4-imme-armless-chairs-black-seatdesigner-white-top>: HTTP status code is not handled or not allowed 2026-01-24 18:38:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/vbelts_timing_belts landed on page that is not a product page. 2026-01-24 18:38:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/machining/machinery/lathes>: HTTP status code is not handled or not allowed 2026-01-24 18:38:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/bins_totes_containers/organizers_caddies/drawer_organizers_dividers landed on page that is not a product page. 2026-01-24 18:38:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cutlery-bins-99700-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 18:38:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/lathes landed on page that is not a product page. 2026-01-24 18:38:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-starter-rack-60x48x96-with-3-levels-wire-deck-1300lb-cap-per-shelf-gray>: HTTP status code is not handled or not allowed 2026-01-24 18:39:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-square-table-with-4-imme-armless-caster-chairs-navy-seatdesigner-white-top>: HTTP status code is not handled or not allowed 2026-01-24 18:39:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=jtm-1050-mill-200s-3-q-xandytpfa-airdrawbr already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:39:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-square-table-with-4-imme-armless-caster-chairs-sky-blue-seatmaple-top>: HTTP status code is not handled or not allowed 2026-01-24 18:39:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vinyl-back-seat-oversized-stack-chairs-black>: HTTP status code is not handled or not allowed 2026-01-24 18:39:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 9015 pages (at 89 pages/min), scraped 6223 items (at 59 items/min) 2026-01-24 18:39:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hessaire-36-direct-drive-box-fan-36d370-n-1-2hp-1ph-11070-cfm>: HTTP status code is not handled or not allowed 2026-01-24 18:39:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=jtm-1050-mill-200s-3-q-xandytpfa-airdrawbr landed on page that is not a product page. 2026-01-24 18:39:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/easy-adjust-boltless-shelf-truck-with-laminate-shelves-7>: HTTP status code is not handled or not allowed 2026-01-24 18:39:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/centurion-sauce-pan-3707-7-quart-6-depth>: HTTP status code is not handled or not allowed 2026-01-24 18:39:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-level-96x48-high-capacity-rack-wire-deck-gray>: HTTP status code is not handled or not allowed 2026-01-24 18:39:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/norton-66261126280-metalite-cloth-roll-1-1-2-inch-w-x-50-yds-aluminum-oxide-p320-grit>: HTTP status code is not handled or not allowed 2026-01-24 18:39:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-x-4-shelf-retainer-divider>: HTTP status code is not handled or not allowed 2026-01-24 18:39:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dividable-grid-container-16-1-2x10-7-8x8-blue>: HTTP status code is not handled or not allowed 2026-01-24 18:39:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/air_tools_accessories/air_grinders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:39:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/redco-king-kutter-6015-replacement-cone-includes-5-3-16-cone>: HTTP status code is not handled or not allowed 2026-01-24 18:39:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n-ferno-6492-2-piece-wind-resistant-multi-band-lime>: HTTP status code is not handled or not allowed 2026-01-24 18:39:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/no-pets-shoes-shirt-required-sign-4523-3-x-9>: HTTP status code is not handled or not allowed 2026-01-24 18:39:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/half-mask-low-pressure-system-3-workers-100-hose>: HTTP status code is not handled or not allowed 2026-01-24 18:39:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/air_tools_accessories/air_grinders landed on page that is not a product page. 2026-01-24 18:39:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-viz-heavy-duty-single-scba-wall-case-green>: HTTP status code is not handled or not allowed 2026-01-24 18:39:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-x-72-table-with-6-imme-armless-barstools-black-seatstudio-teak-top>: HTTP status code is not handled or not allowed 2026-01-24 18:39:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/two_way_radios_accessories/two_way_radio_headsets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:39:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/graphic-facility-signs-fire-extinguisher-plastic-7x7>: HTTP status code is not handled or not allowed 2026-01-24 18:39:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/cookware_bakeware/chafing_dishes_fuel already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:39:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mounted-aluminum-arm-w-swivel-ext-sensor-and-red-strobe-6-power-cord-12v>: HTTP status code is not handled or not allowed 2026-01-24 18:39:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/cookware_bakeware/chafing_dishes_fuel already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:39:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_alarms_signals/forklift_alert_sensors_systems already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:39:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-x-72-table-with-6-imme-armless-barstools-coral-seatmaple-top>: HTTP status code is not handled or not allowed 2026-01-24 18:39:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/two_way_radios_accessories/two_way_radio_headsets>: HTTP status code is not handled or not allowed 2026-01-24 18:39:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/cookware_bakeware/chafing_dishes_fuel>: HTTP status code is not handled or not allowed 2026-01-24 18:39:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/border-fill-48-3-8-x-24-3-8-pvc-glue-up-tile-in-gloss-white-pg5600>: HTTP status code is not handled or not allowed 2026-01-24 18:39:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/cookware_bakeware/chafing_dishes_fuel landed on page that is not a product page. 2026-01-24 18:40:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_alarms_signals/forklift_alert_sensors_systems landed on page that is not a product page. 2026-01-24 18:40:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/locksets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:40:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/266356-shelf-white-epoxy-for-true-manufacturing>: HTTP status code is not handled or not allowed 2026-01-24 18:40:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pin-nails-p23-58-23-gauge-5-8-2000-bx>: HTTP status code is not handled or not allowed 2026-01-24 18:40:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-syle-1-48-3-8-x-24-3-8-pvc-glue-up-tile-in-vintage-metal-pg5034>: HTTP status code is not handled or not allowed 2026-01-24 18:40:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kee-klamp-side-outlet-elbow-1>: HTTP status code is not handled or not allowed 2026-01-24 18:40:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 9107 pages (at 92 pages/min), scraped 6289 items (at 66 items/min) 2026-01-24 18:40:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/door_window_hardware/locksets landed on page that is not a product page. 2026-01-24 18:40:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3x4-4-mil-parts-bags-w-hang-holes>: HTTP status code is not handled or not allowed 2026-01-24 18:40:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pizza-dough-box-dolly>: HTTP status code is not handled or not allowed 2026-01-24 18:40:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-gravity-flow-rack-27539s-3-flat-steel-shelves-52w-x-28d-x-72h>: HTTP status code is not handled or not allowed 2026-01-24 18:40:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eye-face-wash-deck-mounted-autoflow-90-degree-right-hand-mounting-g1805>: HTTP status code is not handled or not allowed 2026-01-24 18:40:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-x725-6-mil-poly-tubing>: HTTP status code is not handled or not allowed 2026-01-24 18:40:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swing-glass-door-merchandiser-73-high>: HTTP status code is not handled or not allowed 2026-01-24 18:40:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/shelving_accessories/shelving_enclosure_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:40:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cantilever-rack-straight-arm-5>: HTTP status code is not handled or not allowed 2026-01-24 18:40:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-18-x-24-dark-yellow-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 18:40:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36x18x18-double-wall-boxes>: HTTP status code is not handled or not allowed 2026-01-24 18:40:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/drainage_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:40:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-task-light-pivoting-head-10-degree-spot-3x3-watt-led-24v-ac-dc-w-out-transformer>: HTTP status code is not handled or not allowed 2026-01-24 18:40:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/weblok-assembly-867010-2-synthetic-to-attachment-3-8-6250-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-24 18:40:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/shelving_accessories/shelving_enclosure_kits landed on page that is not a product page. 2026-01-24 18:40:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tutco-crankcase-54-240-39-58>: HTTP status code is not handled or not allowed 2026-01-24 18:40:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/floor_mats/drainage_mats landed on page that is not a product page. 2026-01-24 18:40:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-plastic-chain-evergreen>: HTTP status code is not handled or not allowed 2026-01-24 18:40:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tutco-crankcase-70-240-20-1316>: HTTP status code is not handled or not allowed 2026-01-24 18:40:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-add-on-36in-w-x-24in-d-x-96in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 18:40:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40-gallon-square-rubbermaid-brute-waste-receptacles-gray>: HTTP status code is not handled or not allowed 2026-01-24 18:40:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lxt-174-power-tool-battery-charger-starter-pk-5-0ah-li-ion-18v-45-min-chrg-time>: HTTP status code is not handled or not allowed 2026-01-24 18:40:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-18-x-24-red-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 18:40:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/trash_recycling/trash_recycling_containers/indoor_trash_cans/plastic_indoor_trash_cans already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:40:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/641005-slicing-knife-set-for-hobart>: HTTP status code is not handled or not allowed 2026-01-24 18:40:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/locking-storage-cabinet-36-w-x-18-d-x-42-h-with-removable-bins-2>: HTTP status code is not handled or not allowed 2026-01-24 18:40:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/circulation_reception_desks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:40:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/desk-top-organizer-low-profile-58-wide-12-deep-12-high-mahogany>: HTTP status code is not handled or not allowed 2026-01-24 18:40:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/esd_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:40:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-21-x-72-grey-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 18:41:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/trash_recycling/trash_recycling_containers/indoor_trash_cans/plastic_indoor_trash_cans landed on page that is not a product page. 2026-01-24 18:41:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/outdoor-enclosed-bulletin-board-cabinet-1-door-3-feet-h-x-2-feet-w-silver-trim-pacific-blue>: HTTP status code is not handled or not allowed 2026-01-24 18:41:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/461426-control-temperature-temp-for-ranco>: HTTP status code is not handled or not allowed 2026-01-24 18:41:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-1-2x10-documents-enclosed-transportation-envelopes-pl496>: HTTP status code is not handled or not allowed 2026-01-24 18:41:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/floor_mats/esd_mats landed on page that is not a product page. 2026-01-24 18:41:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergomat-classic-3-x-4>: HTTP status code is not handled or not allowed 2026-01-24 18:41:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-heavy-duty-boltless-shelving-add-on-60in-w-x-48in-d-x-72in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 18:41:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 9203 pages (at 96 pages/min), scraped 6352 items (at 63 items/min) 2026-01-24 18:41:09 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/circulation_reception_desks landed on page that is not a product page. 2026-01-24 18:41:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spatula-8-str-flex-blk-nylon>: HTTP status code is not handled or not allowed 2026-01-24 18:41:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2271109-hp-pump-fitting-for-henny-penny>: HTTP status code is not handled or not allowed 2026-01-24 18:41:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1681454-valve-gas-lp-for-frymaster>: HTTP status code is not handled or not allowed 2026-01-24 18:41:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-24-x-72-light-grey-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 18:41:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2531311-module-ext-f-occupancy-sensor>: HTTP status code is not handled or not allowed 2026-01-24 18:41:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010269-motor-fan-16w-115v-for-manitowoc-machines>: HTTP status code is not handled or not allowed 2026-01-24 18:41:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durastripe-supreme-v-3-x-100-black-yellow>: HTTP status code is not handled or not allowed 2026-01-24 18:41:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-extra-heavy-duty-boltless-shelving-starter-96inwx-36ind-x-72inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 18:41:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-12-x-36-light-blue-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 18:41:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plus-banner-head-15-banner-yellow-caution-do-not-enter>: HTTP status code is not handled or not allowed 2026-01-24 18:41:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/shelving_accessories/shelving_enclosure_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:41:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-pres-cntr-dbl-solenoid-vlv-pilot-spr-cntred-1-8-nptf-220v-ac-5va-coil-black-conn>: HTTP status code is not handled or not allowed 2026-01-24 18:41:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fr-hi-vis-work-shirt-type-r-class-3-2xl-fluorescent-yellow-shrtv3c32xlrg>: HTTP status code is not handled or not allowed 2026-01-24 18:41:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-add-on-48inw-x-36ind-x-72inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 18:41:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-12-x-18-blue-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 18:41:45 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/shelving_accessories/shelving_enclosure_kits landed on page that is not a product page. 2026-01-24 18:41:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-t3-steam-scrub-floor-mop>: HTTP status code is not handled or not allowed 2026-01-24 18:41:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-long-sleeve-t-shirt-l-gray-c541ngelslg>: HTTP status code is not handled or not allowed 2026-01-24 18:41:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/roof_pipe_heating_cables already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:41:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-heavy-duty-boltless-shelving-add-on-96inw-x-48ind-x-72inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 18:41:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adam-platform-floor-scale-660lb>: HTTP status code is not handled or not allowed 2026-01-24 18:41:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11-1-8-x-8-3-4-x-2-kraft-literature-mailers>: HTTP status code is not handled or not allowed 2026-01-24 18:41:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kit4sc40ng-md-40-cal-compliance-arc-flash-kit-short-coat-bib-overall-md-no-gloves>: HTTP status code is not handled or not allowed 2026-01-24 18:41:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/roof_pipe_heating_cables landed on page that is not a product page. 2026-01-24 18:42:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-x-42-cryogenic-apron-a02crc24x42>: HTTP status code is not handled or not allowed 2026-01-24 18:42:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-x-1-phillips-pan-head-tapping-screw-pkg-of-100-fra21032>: HTTP status code is not handled or not allowed 2026-01-24 18:42:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-in-w-x-76-in-h-panel-with-pass-thru-cable-gray>: HTTP status code is not handled or not allowed 2026-01-24 18:42:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/481140-hi-limit-lcha>: HTTP status code is not handled or not allowed 2026-01-24 18:42:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 9301 pages (at 98 pages/min), scraped 6424 items (at 72 items/min) 2026-01-24 18:42:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kit4sc100-lg12-100-cal-cm2-arc-flash-kit-lg-glove-size-12>: HTTP status code is not handled or not allowed 2026-01-24 18:42:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-dbl-solenoid-vlv-ext-pilot-closed-cntr-spr-cntred-1-4-nptf-220v-ac-5va-coil-led>: HTTP status code is not handled or not allowed 2026-01-24 18:42:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-flame-resistant-work-shirt-xl-gray-tcgsswn00115xlrg00>: HTTP status code is not handled or not allowed 2026-01-24 18:42:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sleeve-anchor-3-8-x-3-3-4-round-head-steel-zinc-pkg-of-50-426030>: HTTP status code is not handled or not allowed 2026-01-24 18:42:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-nacho-peanut-popcorn-warmer>: HTTP status code is not handled or not allowed 2026-01-24 18:42:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/264557-extension-shaft-thermostat-for-frymaster>: HTTP status code is not handled or not allowed 2026-01-24 18:42:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/divider-10-7-8-d-x-5-h-premium-stacking-bin-pack-of-6-2>: HTTP status code is not handled or not allowed 2026-01-24 18:42:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/led_signs_message_displays already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:42:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-24w-12-step-steel-rolling-ladder-14d-top-step-wcal-osha-handrail-ca>: HTTP status code is not handled or not allowed 2026-01-24 18:42:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/die-rack-shelf-60-x-18>: HTTP status code is not handled or not allowed 2026-01-24 18:42:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/led_signs_message_displays>: HTTP status code is not handled or not allowed 2026-01-24 18:42:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-18-ga-36-wx30-dx85-h-open-clip-style-5-shelf>: HTTP status code is not handled or not allowed 2026-01-24 18:42:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/hvac_electric_motors/general_purpose_hvac_motors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:42:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-work-shirt-xl-ln-navy-tcg01160223>: HTTP status code is not handled or not allowed 2026-01-24 18:42:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stanchion-kit-yellow-6pk-50-of-2-chain-w-c-hooks-incl-2-pole-14-base-41-height>: HTTP status code is not handled or not allowed 2026-01-24 18:42:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-add-on-60inw-x-24ind-x-60inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 18:42:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010969-bushing-gauge-for-hobart>: HTTP status code is not handled or not allowed 2026-01-24 18:42:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-cal-flame-resistant-coveralls-xl-ln-navy-tcg02160870>: HTTP status code is not handled or not allowed 2026-01-24 18:42:56 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/hvac_electric_motors/general_purpose_hvac_motors landed on page that is not a product page. 2026-01-24 18:42:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-unassembled-storage-cabinet-recessed-handle36w-x-24d-x-72h-medium-grey>: HTTP status code is not handled or not allowed 2026-01-24 18:43:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/264278-p-trap-1-1-8-inch>: HTTP status code is not handled or not allowed 2026-01-24 18:43:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/clear_view_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:43:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-starter-72inw-x-48ind-x-84inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 18:43:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/441020-electrode-for-dynamic-cooking-systems>: HTTP status code is not handled or not allowed 2026-01-24 18:43:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 9394 pages (at 93 pages/min), scraped 6496 items (at 72 items/min) 2026-01-24 18:43:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pegboard-utility-tool-storage-kit-galvanized-blue-32-x-32-x-6>: HTTP status code is not handled or not allowed 2026-01-24 18:43:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012706-sifter-brush-assembly-black-for-ayrking-corporation>: HTTP status code is not handled or not allowed 2026-01-24 18:43:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quick-adjust-shelf-24x48-poly-z-brite>: HTTP status code is not handled or not allowed 2026-01-24 18:43:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/cabinets/clear_view_cabinets landed on page that is not a product page. 2026-01-24 18:43:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-2-single-solenoid-valve-ext-pilot-spring-return-g-1-4-220v-ac-5va-coil-led-conn>: HTTP status code is not handled or not allowed 2026-01-24 18:43:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-32-x-1-1-2-security-machine-screw-flat-torx-plus-head-18-8-stainless-steel-ft-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 18:43:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-double-solenoid-valve-ext-pilot-g-1-4-110v-ac-5va-coil-led-connection>: HTTP status code is not handled or not allowed 2026-01-24 18:43:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pegboard-garden-tool-board-organizer-gray-black-48-x-32-x-9>: HTTP status code is not handled or not allowed 2026-01-24 18:43:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mixing-paddle-4-75x9-75-paddle-42x1-tubular-handle>: HTTP status code is not handled or not allowed 2026-01-24 18:43:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-wall-mounted-misting-fan-outdoor-rated-oscillating-7435-cfm-1-7-hp>: HTTP status code is not handled or not allowed 2026-01-24 18:43:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dry-erase-marker-fine-tip-black-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-24 18:43:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carson-optical-cf-618-closeup-6x18mm-close-focus-monocular>: HTTP status code is not handled or not allowed 2026-01-24 18:43:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/76-690-1-light-pendant-frosted-prismatic-old-bronze-15w-x-10-5h>: HTTP status code is not handled or not allowed 2026-01-24 18:43:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/decorative_lighting/chandeliers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:43:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sign-reserved-2x6-heavy-weight-tent-style-plastic>: HTTP status code is not handled or not allowed 2026-01-24 18:43:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/sign_light_combination_units already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:43:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-swing-panel-pegboard-48w-x-1-12d-x-48h-black>: HTTP status code is not handled or not allowed 2026-01-24 18:43:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:43:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gravity-womens-safety-work-boots-steel-toe-4h-size-95w-cognac-brown>: HTTP status code is not handled or not allowed 2026-01-24 18:43:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010669-thumb-screw-for-globe-food-equipment>: HTTP status code is not handled or not allowed 2026-01-24 18:43:49 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/decorative_lighting/chandeliers landed on page that is not a product page. 2026-01-24 18:43:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-32-x-1-2-security-machine-screw-button-torx-head-302hq-stainless-steel-ft-unc-100-pk-91120>: HTTP status code is not handled or not allowed 2026-01-24 18:43:51 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/sign_light_combination_units landed on page that is not a product page. 2026-01-24 18:43:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-24 18:43:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-x-30-plastic-square-edge-work-bench-fixed-height-1-5-8-top>: HTTP status code is not handled or not allowed 2026-01-24 18:43:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-2-inch-f-polypropylene-cam-and-groove-adapter-x-male-npt>: HTTP status code is not handled or not allowed 2026-01-24 18:43:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/badger-neoprene-boots-plain-toe-upper-rubber-sole-steel-shank-15-h-blk-size-11>: HTTP status code is not handled or not allowed 2026-01-24 18:44:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:44:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iron-eagle-174-overall-green-knee-patch-pockets-loto-straps-xs>: HTTP status code is not handled or not allowed 2026-01-24 18:44:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-signs-caution-hot-rigid-plastic-7-h-x-10-w>: HTTP status code is not handled or not allowed 2026-01-24 18:44:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-t-shirt-short-sleeve-1-pocket-fl-lime-3xl>: HTTP status code is not handled or not allowed 2026-01-24 18:44:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 9493 pages (at 99 pages/min), scraped 6567 items (at 71 items/min) 2026-01-24 18:44:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009895-spring-guide-vct2010-gen-3-for-roundup-food-equipment>: HTTP status code is not handled or not allowed 2026-01-24 18:44:09 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-24 18:44:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/floor_machine_replacement_parts/vacuum_hoses already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:44:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-525-1-light-die-cast-bulk-head-frosted-diffuser-architectural-bronze-6-125w-x-10h>: HTTP status code is not handled or not allowed 2026-01-24 18:44:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/41-main-roller-brush-for-b-300-sweeper-scrubber-hard-6-680-369-0>: HTTP status code is not handled or not allowed 2026-01-24 18:44:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009908-slide-rail-kit-hct-for-roundup-food-equipment>: HTTP status code is not handled or not allowed 2026-01-24 18:44:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-5141-patton-3-light-ceiling-flush-fixt-frosted-prairie-bronze-15-75w-x-7-75h>: HTTP status code is not handled or not allowed 2026-01-24 18:44:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gravity-womens-safety-work-boots-steel-toe-4h-size-8w-cognac-brown>: HTTP status code is not handled or not allowed 2026-01-24 18:44:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/floor_machine_replacement_parts/vacuum_hoses landed on page that is not a product page. 2026-01-24 18:44:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deep-cleaning-ribbed-mat-3x4-blue-1>: HTTP status code is not handled or not allowed 2026-01-24 18:44:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cobalt-tmx-silver-and-deming-drill-1-2-inch-shank-135-splt-pt-11-16-inch-dia-x-3-inch-flute-l-x-6-inch-oal>: HTTP status code is not handled or not allowed 2026-01-24 18:44:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/decorative_lighting/flush_mount_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:44:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-high-letter-adder-empty-black>: HTTP status code is not handled or not allowed 2026-01-24 18:44:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/881306-refrigeration-hard-startkit>: HTTP status code is not handled or not allowed 2026-01-24 18:44:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/bar_restaurant_equipment/bartending_supplies/bar_mats_shelf_liners already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:44:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/76-269-2-light-ceiling-fixt-round-sunburst-white-12w-x-4-75h>: HTTP status code is not handled or not allowed 2026-01-24 18:44:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/bar_restaurant_equipment/bartending_supplies/bar_mats_shelf_liners already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:44:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/decorative_lighting/flush_mount_lights landed on page that is not a product page. 2026-01-24 18:44:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adder-unit-legal-31-inch-38-3-471-1-2-inch-6-opening-bone-white>: HTTP status code is not handled or not allowed 2026-01-24 18:44:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-4921-banyan-1-light-outdoor-wall-clear-water-white-6-125w-x-12-25h>: HTTP status code is not handled or not allowed 2026-01-24 18:44:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/air_compressors/single_stage_reciprocating_air_compressors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:44:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reelcraft-l-3030-123-7q-compact-steel-power-cord-reel-20a-12-3-30-cord-w-quad-box>: HTTP status code is not handled or not allowed 2026-01-24 18:44:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/bar_restaurant_equipment/bartending_supplies/bar_mats_shelf_liners landed on page that is not a product page. 2026-01-24 18:44:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009541-thermometer-led-for-true-manufacturing>: HTTP status code is not handled or not allowed 2026-01-24 18:44:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/snow_ice_removal/snow_blowers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:44:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/77-924-2-light-ceiling-flush-mount-medium-white-mushroom-antique-brass-9-5w-x-6h>: HTTP status code is not handled or not allowed 2026-01-24 18:44:49 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/bar_restaurant_equipment/bartending_supplies/bar_mats_shelf_liners landed on page that is not a product page. 2026-01-24 18:44:51 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/air_compressors/single_stage_reciprocating_air_compressors landed on page that is not a product page. 2026-01-24 18:44:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-3168-empire-4-light-vanity-wall-frosted-white-mahogany-bronze-28-75w-x-6-125h>: HTTP status code is not handled or not allowed 2026-01-24 18:44:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/spill_control_supplies/containment_berms_pools already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:44:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/pneumatic_hoses_connectors/pneumatic_fittings already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:44:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/neoprene-floor-mount-vibration-isolator-3-long-2-wide-red>: HTTP status code is not handled or not allowed 2026-01-24 18:44:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/snow_ice_removal/snow_blowers landed on page that is not a product page. 2026-01-24 18:44:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qp-pro-two-stage-air-compressor-54-hp-60-gallon-vertical-230v-1-phase>: HTTP status code is not handled or not allowed 2026-01-24 18:45:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-flammable-cabinet-30-gallon-self-close-door-43-w-x-22-d-x-44-h>: HTTP status code is not handled or not allowed 2026-01-24 18:45:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/spill_control_supplies already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:45:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/pneumatic_hoses_connectors/pneumatic_fittings landed on page that is not a product page. 2026-01-24 18:45:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pot-n-pan-cleaner-1-5-oz-packs-2307544>: HTTP status code is not handled or not allowed 2026-01-24 18:45:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/sneeze_guards_distancing_barriers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:45:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alliance-hose-spiral-wrap-hose-guard-for-1-2-to-11-16-od-25>: HTTP status code is not handled or not allowed 2026-01-24 18:45:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 9580 pages (at 87 pages/min), scraped 6624 items (at 57 items/min) 2026-01-24 18:45:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-1-8-l-x-5-3-8-w-x-3-h-red-stacking-hanging-interlocking-polypropylene-bins-6-ct>: HTTP status code is not handled or not allowed 2026-01-24 18:45:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/spill_control_supplies landed on page that is not a product page. 2026-01-24 18:45:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/material-handling/casters_wheels/plate_casters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:45:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-boltless-shelving-add-on-48inw-x-18ind-x-48inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 18:45:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/desktop_dividers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:45:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rotating-deck-dome-oven-flat-top-47-inside-deck-gas-92500btu-208-240v-digital-control>: HTTP status code is not handled or not allowed 2026-01-24 18:45:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/personal_care_hygiene/hand_sanitizers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:45:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mars-combination-roller-plunger-door-limit-switch-single-phase-20-amps-6>: HTTP status code is not handled or not allowed 2026-01-24 18:45:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/material-handling/casters_wheels/plate_casters>: HTTP status code is not handled or not allowed 2026-01-24 18:45:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-20-spread-beam-telescoping-adjustable-10-000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 18:45:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/personal_care_hygiene/hand_sanitizers>: HTTP status code is not handled or not allowed 2026-01-24 18:45:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interion-174-restaurant-lunchroom-table-60-x-30-charcoal>: HTTP status code is not handled or not allowed 2026-01-24 18:45:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/partitions_dividers/desktop_dividers landed on page that is not a product page. 2026-01-24 18:45:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-vinyl-gloves-powder-free-medium>: HTTP status code is not handled or not allowed 2026-01-24 18:45:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-uncaged-fixed-access-ladder-yellow-8>: HTTP status code is not handled or not allowed 2026-01-24 18:45:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/35mr16-fl-med-35w-halogen-w-medium-base>: HTTP status code is not handled or not allowed 2026-01-24 18:45:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5011007-probe-meat-for-food-warming-equipment-co-inc>: HTTP status code is not handled or not allowed 2026-01-24 18:45:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/parcel-drop-box-front-access-10-1-2-w-x-15-2-5-d-x-41-1-2-h-black>: HTTP status code is not handled or not allowed 2026-01-24 18:45:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-34-inch-high-post-silver-epoxy-4-pack>: HTTP status code is not handled or not allowed 2026-01-24 18:45:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011806-capillary-tubing-for-victory>: HTTP status code is not handled or not allowed 2026-01-24 18:45:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:45:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/underhoist-component-stand-w-foot-pedal-1-650-capacity-lbs>: HTTP status code is not handled or not allowed 2026-01-24 18:45:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torque-limiting-extension-1-2-drive-150-ft-lbs>: HTTP status code is not handled or not allowed 2026-01-24 18:45:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:45:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8898-strapping-tape-3-4-x-60-yds-46-mil-blue>: HTTP status code is not handled or not allowed 2026-01-24 18:45:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits landed on page that is not a product page. 2026-01-24 18:45:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transmission-jack-1-100-capacity-lbs>: HTTP status code is not handled or not allowed 2026-01-24 18:46:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/266258-padlock-1-3-4-inch-w-w-3-inch-shackle>: HTTP status code is not handled or not allowed 2026-01-24 18:46:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-cal-wide-rib-corrugated-rubber-floor-mat-1-8-thick-x-4-x-4-utility-runner>: HTTP status code is not handled or not allowed 2026-01-24 18:46:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ez-mount-backer-plate-a100-bpev>: HTTP status code is not handled or not allowed 2026-01-24 18:46:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats landed on page that is not a product page. 2026-01-24 18:46:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/floor-jack-2-ton-low-profile>: HTTP status code is not handled or not allowed 2026-01-24 18:46:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 9674 pages (at 94 pages/min), scraped 6692 items (at 68 items/min) 2026-01-24 18:46:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/compact_fluorescent_light_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:46:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tall-cardboard-corrugated-boxes-9l-x-9w-x-48h-kraft-pack-of-20>: HTTP status code is not handled or not allowed 2026-01-24 18:46:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s4041-anti-fog-safety-glasses-gloss-black-frame-gray-lens-scratch-resistant>: HTTP status code is not handled or not allowed 2026-01-24 18:46:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/compact_fluorescent_light_bulbs>: HTTP status code is not handled or not allowed 2026-01-24 18:46:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roller-follower-full-comp-inch-cry56vuu-double-sealed-3-12-od>: HTTP status code is not handled or not allowed 2026-01-24 18:46:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/personal_care_hygiene/personal_wipes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:46:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bin-nestable-6>: HTTP status code is not handled or not allowed 2026-01-24 18:46:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tire-inflation-safety-cage-2-bar-heavy-duty-steel>: HTTP status code is not handled or not allowed 2026-01-24 18:46:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/combination_metalworking_machines already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:46:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dspbl-dust-head-36x5-cttnsyn-12>: HTTP status code is not handled or not allowed 2026-01-24 18:46:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/personal_care_hygiene/personal_wipes>: HTTP status code is not handled or not allowed 2026-01-24 18:46:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20v-max-cordless-cable-stapler-kit-1-l-staple-li-ion-d-style>: HTTP status code is not handled or not allowed 2026-01-24 18:46:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/combination_metalworking_machines>: HTTP status code is not handled or not allowed 2026-01-24 18:46:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manual-chain-hoist-stainless-steel-2-ton-capacity-20-lift>: HTTP status code is not handled or not allowed 2026-01-24 18:46:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dsb-214sm-surface-mounted-access-door-for-all-surf-lock-16wx16h-smp1616l>: HTTP status code is not handled or not allowed 2026-01-24 18:46:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/cable_management/cable_clamps_clips_hardware already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:46:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:46:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-strip-door-curtain-8-w-x-7-h>: HTTP status code is not handled or not allowed 2026-01-24 18:46:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-step-stairway-slope-work-platform-w-handrails-24in-x-72in-platform>: HTTP status code is not handled or not allowed 2026-01-24 18:46:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:46:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/390-premium-replacement-face-shield-window-8l-x-12w-x-116-thick-dark-green>: HTTP status code is not handled or not allowed 2026-01-24 18:46:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/cable_management/cable_clamps_clips_hardware>: HTTP status code is not handled or not allowed 2026-01-24 18:46:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 18:46:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drip-dam-leak-diverter-10-x-12-x-1-4-461012-ye>: HTTP status code is not handled or not allowed 2026-01-24 18:47:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e44-1-58w-miniature-sealed-beams-w-minature-bayonet-base>: HTTP status code is not handled or not allowed 2026-01-24 18:47:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 18:47:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/fluorescent_linear_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:47:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/step-on-ppe-waste-can-25-1-gallon-white>: HTTP status code is not handled or not allowed 2026-01-24 18:47:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/fluorescent_linear_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:47:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plt-u-bend-fluorescent-bulb-gx24q3-base-32w-2400-lumens-6500k-daylight-pk-of-10>: HTTP status code is not handled or not allowed 2026-01-24 18:47:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/csg15-5in-concrete-surfacing-grinder>: HTTP status code is not handled or not allowed 2026-01-24 18:47:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 9762 pages (at 88 pages/min), scraped 6757 items (at 65 items/min) 2026-01-24 18:47:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-75-digital-atomic-clock-with-calendar-and-indoor-temperature-display-silver>: HTTP status code is not handled or not allowed 2026-01-24 18:47:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_tools/cleaning_cloths_towels/shop_towels_cloth_rags/microfiber_cloths_towels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:47:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/fluorescent_linear_bulbs landed on page that is not a product page. 2026-01-24 18:47:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ushio-jcr12v-100w-h10>: HTTP status code is not handled or not allowed 2026-01-24 18:47:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/fluorescent_linear_bulbs landed on page that is not a product page. 2026-01-24 18:47:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/verishield-153-behind-the-neck-ear-muffs-22-db-black>: HTTP status code is not handled or not allowed 2026-01-24 18:47:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/specialty_light_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:47:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wrapped-cutlery-6-14-knife-white>: HTTP status code is not handled or not allowed 2026-01-24 18:47:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/specialty_light_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:47:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_tools/cleaning_cloths_towels/shop_towels_cloth_rags/microfiber_cloths_towels>: HTTP status code is not handled or not allowed 2026-01-24 18:47:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milton-s-746w-amt-style-universal-coupler-1-4-mnpt>: HTTP status code is not handled or not allowed 2026-01-24 18:47:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flush-gun-tool-kit-four-seasons-59172>: HTTP status code is not handled or not allowed 2026-01-24 18:47:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milton-777-a-style-aro-plug-1-4-mnpt-10-pack>: HTTP status code is not handled or not allowed 2026-01-24 18:47:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/specialty_light_bulbs landed on page that is not a product page. 2026-01-24 18:47:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/specialty_light_bulbs landed on page that is not a product page. 2026-01-24 18:47:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/p-208-chemical-sorbent-mini-boom-3-diameter-x-8-6-boomscase>: HTTP status code is not handled or not allowed 2026-01-24 18:47:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72x30-stainless-square-edge-mobile-power-apron-production-bench-tan-32313648>: HTTP status code is not handled or not allowed 2026-01-24 18:47:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/self_luminous_emergency_exit_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:47:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:47:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-16-w-2-step-steel-rolling-ladder-10-d-top-step-2>: HTTP status code is not handled or not allowed 2026-01-24 18:47:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-24-w-6-step-steel-rolling-ladder-20-d-top-step>: HTTP status code is not handled or not allowed 2026-01-24 18:47:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fiberglass-handle-with-compression-lock-60-l-x-1-d>: HTTP status code is not handled or not allowed 2026-01-24 18:47:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/self_luminous_emergency_exit_signs landed on page that is not a product page. 2026-01-24 18:47:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/removable-model-a-corner-bumper-guard-white>: HTTP status code is not handled or not allowed 2026-01-24 18:47:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/761416-claris-system-l-for-everpure-filter>: HTTP status code is not handled or not allowed 2026-01-24 18:47:45 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents landed on page that is not a product page. 2026-01-24 18:47:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drainage-mat-accessory-ramp-1>: HTTP status code is not handled or not allowed 2026-01-24 18:47:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/garage_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:47:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-two-piece-clamping-collar-gm2c-05-b>: HTTP status code is not handled or not allowed 2026-01-24 18:47:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/garage_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:47:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fold-store-service-cart-350-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 18:47:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/garage_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:47:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-t8-tube-light-bulb-med-bi-pin-g13-base-12w-1700-lumens-50k-white-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-24 18:47:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/garage_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:47:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/23-75-in-round-tempered-glass-metal-table>: HTTP status code is not handled or not allowed 2026-01-24 18:48:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-coupling-g2scc-087-087>: HTTP status code is not handled or not allowed 2026-01-24 18:48:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/cabinets/garage_cabinets landed on page that is not a product page. 2026-01-24 18:48:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/cabinets/garage_cabinets landed on page that is not a product page. 2026-01-24 18:48:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/cabinets/garage_cabinets>: HTTP status code is not handled or not allowed 2026-01-24 18:48:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/cabinets/garage_cabinets landed on page that is not a product page. 2026-01-24 18:48:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-vertical-file-legal-size-4-drawer-light-gray-18w-x-25d-x-52h>: HTTP status code is not handled or not allowed 2026-01-24 18:48:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 9855 pages (at 93 pages/min), scraped 6818 items (at 61 items/min) 2026-01-24 18:48:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010582-burner-valve-for-garland-range>: HTTP status code is not handled or not allowed 2026-01-24 18:48:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bca-368418-n-bookcase-all-welded-36-wx18-dx84-h-navy>: HTTP status code is not handled or not allowed 2026-01-24 18:48:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-step-steel-safety-stairway-slope-rolling-ladder-weight-actuated-lock-16-w-expan-step-w-cal-osha-handrail-cal-wa-sw-82414x>: HTTP status code is not handled or not allowed 2026-01-24 18:48:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modern-cylinder-vanity-light-fixture-60w-22l-brushed-nickel>: HTTP status code is not handled or not allowed 2026-01-24 18:48:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drum-dolly-5-30-55-gallon-drums-1>: HTTP status code is not handled or not allowed 2026-01-24 18:48:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hvy-dty-cable-ties-mtg-tab-natural-14-100-pieces>: HTTP status code is not handled or not allowed 2026-01-24 18:48:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rolling-privacy-screen-for-partition-panels-frosted-polycarbonate>: HTTP status code is not handled or not allowed 2026-01-24 18:48:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-vis-fr-control-2-0-short-sleeve-tee-type-r-class-2-3xt>: HTTP status code is not handled or not allowed 2026-01-24 18:48:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-w-x-20-h-x-12-d-carbon-v-bank-air-filter-carbon-potassium-global-industrial>: HTTP status code is not handled or not allowed 2026-01-24 18:48:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/cable_management/cable_ties_wraps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:48:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/retail_fixtures_displays_equipment/displays_display_racks/gondola_shelving already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:48:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-w-x-24-h-x-1-d-pleated-merv-8-high-capacity-air-filter-global-industrial>: HTTP status code is not handled or not allowed 2026-01-24 18:48:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/retail_fixtures_displays_equipment/displays_display_racks/gondola_shelving already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:48:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-step-16-wx71-d-stainless-steel-rolling-safety-ladder-serrated-grating-w-cal-osha-handrail-cal-ss082421g>: HTTP status code is not handled or not allowed 2026-01-24 18:48:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/maah-36782rb-h-grn-mobile-cabinet-all-welded-36-wx24-dx78-h-hunter-green>: HTTP status code is not handled or not allowed 2026-01-24 18:48:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/cable_management/cable_ties_wraps landed on page that is not a product page. 2026-01-24 18:48:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/retail_fixtures_displays_equipment/displays_display_racks/gondola_shelving landed on page that is not a product page. 2026-01-24 18:48:42 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/retail_fixtures_displays_equipment/displays_display_racks/gondola_shelving landed on page that is not a product page. 2026-01-24 18:48:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/printer-fax-stand-mobile-20x19-7-8x14-1-8-mahogany>: HTTP status code is not handled or not allowed 2026-01-24 18:48:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spill-containment-sump-two-drums-without-lid>: HTTP status code is not handled or not allowed 2026-01-24 18:48:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1341101-pro-elastic-shock-absorbing-lanyard-6l-130-310-cap-lbs>: HTTP status code is not handled or not allowed 2026-01-24 18:48:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1341125-pro-elastic-shock-absorbing-lanyard-6l-130-310-cap-lbs>: HTTP status code is not handled or not allowed 2026-01-24 18:48:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perdiem174-general-purpose-cleaner-with-hydrogen-peroxide-845-oz-bottle-2case>: HTTP status code is not handled or not allowed 2026-01-24 18:48:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fs-30-l-grn-storage-cabinet-all-welded-30-wx18-dx72-h-leaf-green>: HTTP status code is not handled or not allowed 2026-01-24 18:48:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/residential-air-conditioner-condensing-unit-16-seer2-2-ton-208230v>: HTTP status code is not handled or not allowed 2026-01-24 18:49:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fs-30-wal-storage-cabinet-all-welded-30-wx18-dx72-h-walnut>: HTTP status code is not handled or not allowed 2026-01-24 18:49:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forney-e7014-steel-electrode-110-140a-18-dia>: HTTP status code is not handled or not allowed 2026-01-24 18:49:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dmr-digital-handheld-radio-4-watts-uhf-400-470-mhz-32-channels-ip65-approved>: HTTP status code is not handled or not allowed 2026-01-24 18:49:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 9951 pages (at 96 pages/min), scraped 6889 items (at 71 items/min) 2026-01-24 18:49:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/q-series-vertical-terminal-air-conditioner-17600-btu-cool-5-kw-208230v>: HTTP status code is not handled or not allowed 2026-01-24 18:49:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/material-handling/cranes_hoists/crane_parts_accessories/lifting_slings already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:49:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacment-squeegee-set-for-factory-cat-28-770u-factory-cat-28-754u-factory-cat-28-755u>: HTTP status code is not handled or not allowed 2026-01-24 18:49:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/foodservice_disposables/takeout_containers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:49:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-closed-linear-bearing-wresin-retainer-sw10g-58id-15l>: HTTP status code is not handled or not allowed 2026-01-24 18:49:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bradley-eye-face-washpedestal-mountss-s19-210fw>: HTTP status code is not handled or not allowed 2026-01-24 18:49:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forney-universal-top-terminals-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 18:49:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/material-handling/cranes_hoists/crane_parts_accessories/lifting_slings landed on page that is not a product page. 2026-01-24 18:49:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/foodservice_disposables/takeout_containers landed on page that is not a product page. 2026-01-24 18:49:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-w-x-36-d-x-1-3-4-thick-steel-square-edge-workbench-top-12-gauge-steel-black>: HTTP status code is not handled or not allowed 2026-01-24 18:49:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-elevated-plastic-box-truck-4-bushel-capacity>: HTTP status code is not handled or not allowed 2026-01-24 18:49:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/warehouse-safety-employee-booklet>: HTTP status code is not handled or not allowed 2026-01-24 18:49:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1818-1-4-6in-oal-hss-heavy-duty-sand-blasted-118-point-carbide-tipped-masonry-drill>: HTTP status code is not handled or not allowed 2026-01-24 18:49:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/krsk-knurled-rim-clamping-knob-75-191-6316-3ttt1>: HTTP status code is not handled or not allowed 2026-01-24 18:49:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylon-set-screw-collar-nc-series-climax-nc-031>: HTTP status code is not handled or not allowed 2026-01-24 18:49:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-climax-1c-075-a>: HTTP status code is not handled or not allowed 2026-01-24 18:49:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1892-17-32-hss-steam-oxide-118-point-1-2-reduced-shank-silver-deming-drill>: HTTP status code is not handled or not allowed 2026-01-24 18:49:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-safe-step-on-plastic-container-8-gallon-beige-fg614300beig>: HTTP status code is not handled or not allowed 2026-01-24 18:49:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mcr-safety-9178t-cut-pro-dupont-kevlar-fiber-sleeve-18-thumb-slot>: HTTP status code is not handled or not allowed 2026-01-24 18:49:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polymer-pilaster-w-shoe-4w-x-82h-moss>: HTTP status code is not handled or not allowed 2026-01-24 18:49:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1898-8-00mm-hss-general-purpose-bright-118-point-jobber-length-drill>: HTTP status code is not handled or not allowed 2026-01-24 18:49:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lateral-file-accessory-two-edp-adapters-2>: HTTP status code is not handled or not allowed 2026-01-24 18:49:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arrow-shed-steel-woodridge-10-x-8>: HTTP status code is not handled or not allowed 2026-01-24 18:50:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/science-table-with-book-boxes-casters-phenolic-top-adjustable-height-30-x-60-black-oak>: HTTP status code is not handled or not allowed 2026-01-24 18:50:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/test-tube-racks-orange-for-30mm-tubes-case-of-8>: HTTP status code is not handled or not allowed 2026-01-24 18:50:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12660-6602mf-evaporative-microfiber-cooling-towel-blue>: HTTP status code is not handled or not allowed 2026-01-24 18:50:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 10048 pages (at 97 pages/min), scraped 6962 items (at 73 items/min) 2026-01-24 18:50:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-g-10-fr-4-garolite-rod-1-diameter-x-1-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 18:50:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polymer-pilaster-w-shoe-3w-x-82h-olive>: HTTP status code is not handled or not allowed 2026-01-24 18:50:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-recessed-screw-climax-h1c-193>: HTTP status code is not handled or not allowed 2026-01-24 18:50:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/led_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:50:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plb-g-oval-platter-baskets>: HTTP status code is not handled or not allowed 2026-01-24 18:50:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/abc-industries-enviroduct-15wk-portable-heater-duct-14-dia-x-25l-x-6-pitch-white>: HTTP status code is not handled or not allowed 2026-01-24 18:50:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-toilet-scrubber-refill-bluewhite-10pack>: HTTP status code is not handled or not allowed 2026-01-24 18:50:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-vis-heated-safety-nano-bomber-jacket-ansi-class-3-hi-vis-yellow-s>: HTTP status code is not handled or not allowed 2026-01-24 18:50:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/shelf_trucks/aluminum_shelf_trucks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:50:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-laminate-pilaster-w-shoe-7w-x-82h-khaki-brown>: HTTP status code is not handled or not allowed 2026-01-24 18:50:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/led_bulbs landed on page that is not a product page. 2026-01-24 18:50:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mw-series-2-hp-solids-handling-sewage-pump>: HTTP status code is not handled or not allowed 2026-01-24 18:50:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deep-cut-band-saw-bare-tool-0-490-sfpm-44-78l-blade-24v>: HTTP status code is not handled or not allowed 2026-01-24 18:50:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-x-8-x-8-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 18:50:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/packaging/stretch_wrap_shrink_wrap/shrink_wrap already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:50:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/skullerz-8955-lightweight-bump-cap-short-brim-navy>: HTTP status code is not handled or not allowed 2026-01-24 18:50:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/shelf_trucks/aluminum_shelf_trucks landed on page that is not a product page. 2026-01-24 18:50:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/office_packing_tape/office_tape already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:50:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-climax-2c-131-s>: HTTP status code is not handled or not allowed 2026-01-24 18:50:44 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-24 18:50:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-x-20-x-16-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 18:50:44 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:7 2026-01-24 18:50:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-canvas-ce-phenolic-sheet-0-250-x-48-000-x-96-000>: HTTP status code is not handled or not allowed 2026-01-24 18:50:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/packaging/stretch_wrap_shrink_wrap/shrink_wrap landed on page that is not a product page. 2026-01-24 18:50:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/office_packing_tape/office_tape>: HTTP status code is not handled or not allowed 2026-01-24 18:50:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/plastic_laminate_bathroom_partitions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:50:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/patriot-fully-framed-ta-50-locker-wperforated-doors-36wx24dx78h-redall-welded>: HTTP status code is not handled or not allowed 2026-01-24 18:50:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5sh-a-2448-72-heavy-duty-adjustable-steel-shelving-24-x-48-5-shelves>: HTTP status code is not handled or not allowed 2026-01-24 18:50:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/air_conditioners_chillers/ductless_air_conditioners/ductless_full_systems already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:50:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pmcd-9k-double-fold-menu-cover-9-1-2w-12h-plastic-w-vinyl-binding-double-page-black>: HTTP status code is not handled or not allowed 2026-01-24 18:51:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/plastic_laminate_bathroom_partitions landed on page that is not a product page. 2026-01-24 18:51:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/plastic_laminate_bathroom_partitions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:51:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-trash-can-lid-20-gal-cap-black>: HTTP status code is not handled or not allowed 2026-01-24 18:51:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/plastic_laminate_bathroom_partitions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:51:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-x-18-kra-mailing-tubes-with-caps-case>: HTTP status code is not handled or not allowed 2026-01-24 18:51:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/air_conditioners_chillers/ductless_air_conditioners/ductless_full_systems landed on page that is not a product page. 2026-01-24 18:51:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/upright-controlled-room-temperature-cabinet-glass-door-23-cu-ft>: HTTP status code is not handled or not allowed 2026-01-24 18:51:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 10139 pages (at 91 pages/min), scraped 7026 items (at 64 items/min) 2026-01-24 18:51:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ptp-20c-pebbled-tumblers-20-oz-1375h>: HTTP status code is not handled or not allowed 2026-01-24 18:51:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/plastic_laminate_bathroom_partitions landed on page that is not a product page. 2026-01-24 18:51:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vinyl-adhesive-caulk-10-oz-metal-gray-5113604102>: HTTP status code is not handled or not allowed 2026-01-24 18:51:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/plastic_laminate_bathroom_partitions landed on page that is not a product page. 2026-01-24 18:51:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-paper-phenolic-sheet-0-125-x-48-000-x-96-000>: HTTP status code is not handled or not allowed 2026-01-24 18:51:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rigid-coupling-4-set-screws-2-90-rc4h-series-climax-rc-062-kw4h90>: HTTP status code is not handled or not allowed 2026-01-24 18:51:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stretch-wrap-film-for-hand-dispenser-cast-80-gauge-18w-x-1500l-black-opaque-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-24 18:51:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-one-piece-industry-standard-clamping-couplings-climax-miscc-06-06-s>: HTTP status code is not handled or not allowed 2026-01-24 18:51:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-grate-guard-with-curb-deflector-sediment-high-flow-24l-x-48w-x-4h>: HTTP status code is not handled or not allowed 2026-01-24 18:51:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commercial-turf-series-gas-engine-vertical-shaft-electric-start-20-hp>: HTTP status code is not handled or not allowed 2026-01-24 18:51:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poly-z-brite-wire-shelf-14x36>: HTTP status code is not handled or not allowed 2026-01-24 18:51:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/afp-10s-fry-pan>: HTTP status code is not handled or not allowed 2026-01-24 18:51:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/floor_machine_accessories_supplies/floor_brushes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:51:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slimline-rectangular-trash-can-lid-with-open-top-1623-gal-cap-black>: HTTP status code is not handled or not allowed 2026-01-24 18:51:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40-door-cell-phone-locker-cp10-091572-dbk4-wide-whasp36w-x-15d-x-82-34h-black>: HTTP status code is not handled or not allowed 2026-01-24 18:51:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:51:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-day-capture-cartridges-for-fly-patrol-replacement-carts-6pk-72319>: HTTP status code is not handled or not allowed 2026-01-24 18:51:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/azar-700778-pnk-two-sided-pegboard-floor-display-w-studio-base-17-w-x-17-d-x-60-h>: HTTP status code is not handled or not allowed 2026-01-24 18:51:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/floor_machine_accessories_supplies/floor_brushes landed on page that is not a product page. 2026-01-24 18:51:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-gx4-heavy-duty-field-case>: HTTP status code is not handled or not allowed 2026-01-24 18:51:51 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats landed on page that is not a product page. 2026-01-24 18:51:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0344r-iron-tuff-minus-50-suit-tall-navy-4xl>: HTTP status code is not handled or not allowed 2026-01-24 18:51:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0344r-iron-tuff-minus-50-suit-tall-navy-xl>: HTTP status code is not handled or not allowed 2026-01-24 18:51:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tebyas-open-carry-handgun-law-prohibited-m461r>: HTTP status code is not handled or not allowed 2026-01-24 18:51:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-x-30-maple-top-pedestal-workbench-w-4-drawers-cabinet>: HTTP status code is not handled or not allowed 2026-01-24 18:51:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coveralls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:52:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0034-04-stanford-bouillon-spoon>: HTTP status code is not handled or not allowed 2026-01-24 18:52:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/floor_cleaners already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:52:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powerquad153-pulseair-vacuum-kit-with-2-inlet-30-gallon-capacity>: HTTP status code is not handled or not allowed 2026-01-24 18:52:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sstc-60-cover-for-sst-24>: HTTP status code is not handled or not allowed 2026-01-24 18:52:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:52:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 10233 pages (at 94 pages/min), scraped 7093 items (at 67 items/min) 2026-01-24 18:52:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wheel-axle-size-1-2-12>: HTTP status code is not handled or not allowed 2026-01-24 18:52:09 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coveralls landed on page that is not a product page. 2026-01-24 18:52:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0344l2-hivis-minus-50-suit-regular-hivis-lime-yellow-3xl>: HTTP status code is not handled or not allowed 2026-01-24 18:52:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-locker-single-tier-12x12x72-3-door-assembled-parchment>: HTTP status code is not handled or not allowed 2026-01-24 18:52:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-c40-top-case>: HTTP status code is not handled or not allowed 2026-01-24 18:52:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats landed on page that is not a product page. 2026-01-24 18:52:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/floor_cleaners landed on page that is not a product page. 2026-01-24 18:52:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-rubbermaid-brute-20-gallon-trash-container-4>: HTTP status code is not handled or not allowed 2026-01-24 18:52:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:52:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-label-holder-wire-shelf-1-1-4-h-x-48-w>: HTTP status code is not handled or not allowed 2026-01-24 18:52:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0409-herringbone-grip-glove-black-large>: HTTP status code is not handled or not allowed 2026-01-24 18:52:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-30-esd-top-pedestal-workbench-w-3-drawers-cabinet>: HTTP status code is not handled or not allowed 2026-01-24 18:52:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/respiratory_protection/powered_air_purifying_respirator_systems already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:52:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/235339-disposable-polystyrene-beakers-50ml-translucent-case-of-500>: HTTP status code is not handled or not allowed 2026-01-24 18:52:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-series-outdoor-enclosure-air-conditioner-cool-only-2000-btu-115v>: HTTP status code is not handled or not allowed 2026-01-24 18:52:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ribbed-bollard-post-sleeve-6-lime-green>: HTTP status code is not handled or not allowed 2026-01-24 18:52:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats landed on page that is not a product page. 2026-01-24 18:52:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/respiratory_protection/powered_air_purifying_respirator_systems>: HTTP status code is not handled or not allowed 2026-01-24 18:52:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-recessed-screw-climax-h1c-100-s>: HTTP status code is not handled or not allowed 2026-01-24 18:52:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jtm-2-milling-machine-with-acu-rite-203-dro-x-axis-powerfeed-2-hp-115230v>: HTTP status code is not handled or not allowed 2026-01-24 18:52:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/air_compressors/compressed_air_treatment_dryers_frls/compressed_air_parts_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:52:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-lp-48-7-ton-louver-press-with-3-louver-48-throat-depth-2-hp-110v>: HTTP status code is not handled or not allowed 2026-01-24 18:52:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1114-8-high-pressure-regulator-38-npt-250-psi-gauge-not-included>: HTTP status code is not handled or not allowed 2026-01-24 18:52:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:52:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-roll-no-adhesive-50a-12-thick-x-36-wide-x-8-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 18:52:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/air_compressors/compressed_air_treatment_dryers_frls/compressed_air_parts_accessories landed on page that is not a product page. 2026-01-24 18:52:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42006-spiral-band-1-2-inch-x-1-inch-60-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-24 18:53:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls>: HTTP status code is not handled or not allowed 2026-01-24 18:53:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:53:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-tier-shelving-starter-unit-solid-galvanized-steel-72w-x-18d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 18:53:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/hose_cord_reels/power_cord_reels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:53:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrosion-resistant-set-screw-collar-cr-climax-crc-043>: HTTP status code is not handled or not allowed 2026-01-24 18:53:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 10324 pages (at 91 pages/min), scraped 7159 items (at 66 items/min) 2026-01-24 18:53:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:53:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-climax-c-118-s>: HTTP status code is not handled or not allowed 2026-01-24 18:53:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 18:53:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dandux-vinyl-basket-truck-bulk-truck-24-bushel-gray>: HTTP status code is not handled or not allowed 2026-01-24 18:53:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/hose_cord_reels/power_cord_reels landed on page that is not a product page. 2026-01-24 18:53:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips>: HTTP status code is not handled or not allowed 2026-01-24 18:53:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fiber-lids-for-fiber-containers-7-8-x-10-1-x-0-5-natural-400-carton>: HTTP status code is not handled or not allowed 2026-01-24 18:53:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/70099-2a-flap-wheel-3-inch-x-1-inch-x-1-4-inch-40-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-24 18:53:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:53:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-frame-for-rubbermaid-tilt-truck>: HTTP status code is not handled or not allowed 2026-01-24 18:53:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fiberglass-tote-lid-20-1-2x12-7-8-gray>: HTTP status code is not handled or not allowed 2026-01-24 18:53:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-avant-caster-with-swivel-plate-with-total-lock-2-1-2-x-3-5-8-plate-220-lb-cap>: HTTP status code is not handled or not allowed 2026-01-24 18:53:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tn-800-end-mill-style-tube-pipe-notcher-eccentric-cut-1-3-od-1-hp-110v>: HTTP status code is not handled or not allowed 2026-01-24 18:53:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-24 18:53:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/air_curtains already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:53:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-sheet-with-acrylic-adhesive-50a-38-thick-x-36-wide-x-36-long>: HTTP status code is not handled or not allowed 2026-01-24 18:53:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/bins_totes_containers/baskets_organizers/closet_garment_organizers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:53:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:53:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g-450-galvanized-clip-drop-forged-7-8>: HTTP status code is not handled or not allowed 2026-01-24 18:53:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:53:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s-4320-latch-kit-new-style-1-5t-carbon-2t-alloy-hook-1-66-l-x-0-63-w>: HTTP status code is not handled or not allowed 2026-01-24 18:53:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/air_curtains>: HTTP status code is not handled or not allowed 2026-01-24 18:53:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22481-cut-off-wheel-type-1-ceramic-5-inch-x-045-inch-x-7-8-inch-ceramic-aluminum>: HTTP status code is not handled or not allowed 2026-01-24 18:53:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/bins_totes_containers/baskets_organizers/closet_garment_organizers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:53:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/23325-cut-off-wheel-type-1-z-tech-5-inch-x-045-inch-x-7-8-inch-zirconium>: HTTP status code is not handled or not allowed 2026-01-24 18:53:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/binders_covers/binders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:53:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets>: HTTP status code is not handled or not allowed 2026-01-24 18:53:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/bins_totes_containers/baskets_organizers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:53:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-strip-with-acrylic-adhesive-60a-14-thick-x-1-w-x-10-ft-l>: HTTP status code is not handled or not allowed 2026-01-24 18:53:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/binders_covers/binders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:53:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-roll-no-adhesive-60a-38-thick-x-36-wide-x-5-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 18:53:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 18:53:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/bins_totes_containers/baskets_organizers/closet_garment_organizers>: HTTP status code is not handled or not allowed 2026-01-24 18:54:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/hand_tools_1/hammers_mallets/ball_peen_cross_peen_hammers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:54:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/binders_covers/binders landed on page that is not a product page. 2026-01-24 18:54:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:54:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n-6817-mirror-door-roller-with-1-12-inch-concave-nylon-wheel>: HTTP status code is not handled or not allowed 2026-01-24 18:54:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/bins_totes_containers/baskets_organizers landed on page that is not a product page. 2026-01-24 18:54:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/236-quad-ring-xring-314id-x-312od-70-duro-round-black-pkg-100>: HTTP status code is not handled or not allowed 2026-01-24 18:54:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/electric_motor_controls/dc_drives already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:54:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bash-4-lb-steel-dead-blow-hammer-16l-handle>: HTTP status code is not handled or not allowed 2026-01-24 18:54:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/binders_covers/binders landed on page that is not a product page. 2026-01-24 18:54:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 10408 pages (at 84 pages/min), scraped 7213 items (at 54 items/min) 2026-01-24 18:54:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls>: HTTP status code is not handled or not allowed 2026-01-24 18:54:09 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/hand_tools_1/hammers_mallets/ball_peen_cross_peen_hammers landed on page that is not a product page. 2026-01-24 18:54:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/locker-six-tier-12x15x12-6-door-tan>: HTTP status code is not handled or not allowed 2026-01-24 18:54:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=heavy-duty-hydraulic-sheet-metal-shear-7-5-hp-3-phase-220v-sh-5210-hd already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:54:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/electric_motor_controls/dc_drives landed on page that is not a product page. 2026-01-24 18:54:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/l100-150wo-20-hand-chain-hoist-with-overload-protection-1-12-ton-capacity-20-lift>: HTTP status code is not handled or not allowed 2026-01-24 18:54:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:54:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/17-in-nylon-scrub-brush-for-lvt-soft-white>: HTTP status code is not handled or not allowed 2026-01-24 18:54:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=manually-operated-box-and-pan-hardened-finger-brake-40-l-12-gauge already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:54:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=heavy-duty-hydraulic-sheet-metal-shear-7-5-hp-3-phase-220v-sh-5210-hd>: HTTP status code is not handled or not allowed 2026-01-24 18:54:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:54:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/address_shipping_labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:54:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-extinguisher-bracket-vehicle-mount-use-for-models-b456-322>: HTTP status code is not handled or not allowed 2026-01-24 18:54:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips landed on page that is not a product page. 2026-01-24 18:54:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leeson-single-phase-explosion-proof-motor-1-2hp-1140rpm-56-epfc-60hz-automatic-1-0sf-rigid>: HTTP status code is not handled or not allowed 2026-01-24 18:54:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=manually-operated-box-and-pan-hardened-finger-brake-40-l-12-gauge landed on page that is not a product page. 2026-01-24 18:54:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/shrinker_stretchers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:54:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tackboard-black-fabric-for-66-hutch-black-fabric>: HTTP status code is not handled or not allowed 2026-01-24 18:54:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=3-in-1-combination-shear-brake-and-roll-12-bed-width-20-gauge already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:54:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls>: HTTP status code is not handled or not allowed 2026-01-24 18:54:34 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/address_shipping_labels landed on page that is not a product page. 2026-01-24 18:54:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:54:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrislynn-helical-thread-single-size-repair-kit-m20x15-rh>: HTTP status code is not handled or not allowed 2026-01-24 18:54:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=belt-grinder-3-w-x-79-l-3-phase-220v-bg-379 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:54:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-arm-air-powered-tapping-machine-1-8-1-tapping-capacity-220-rpm>: HTTP status code is not handled or not allowed 2026-01-24 18:54:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/shrinker_stretchers landed on page that is not a product page. 2026-01-24 18:54:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=manually-operated-8-open-ended-letter-brake-20-gauge-mild-steel-capacity already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:54:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smo28-2-large-capacity-forced-air-oven-27-5-cu-ft-781-l-230v>: HTTP status code is not handled or not allowed 2026-01-24 18:54:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=3-in-1-combination-shear-brake-and-roll-12-bed-width-20-gauge landed on page that is not a product page. 2026-01-24 18:54:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flush-handle-instrument-cart-lip-shelves-24-x-36>: HTTP status code is not handled or not allowed 2026-01-24 18:54:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips landed on page that is not a product page. 2026-01-24 18:54:44 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=belt-grinder-3-w-x-79-l-3-phase-220v-bg-379 landed on page that is not a product page. 2026-01-24 18:54:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-part-carbon-style-inventory-tag-1000-1499-8-6-14l-x-3-18w-500pack>: HTTP status code is not handled or not allowed 2026-01-24 18:54:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=manually-operated-8-open-ended-letter-brake-20-gauge-mild-steel-capacity landed on page that is not a product page. 2026-01-24 18:54:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-shelf-truck-3-flush-shelves-24-x-36>: HTTP status code is not handled or not allowed 2026-01-24 18:54:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carton-sealing-tape-made-in-usa-2-x-110-yds-22-mil-redwhiteblue-18pack>: HTTP status code is not handled or not allowed 2026-01-24 18:54:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:54:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-shelving-open-shelving-36x18x75-8>: HTTP status code is not handled or not allowed 2026-01-24 18:54:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-shelving-open-shelving-36x18x39-14>: HTTP status code is not handled or not allowed 2026-01-24 18:54:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-shelving-closed-shelving-36x18x39-8>: HTTP status code is not handled or not allowed 2026-01-24 18:55:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-shelving-open-shelving-36x12x39-23>: HTTP status code is not handled or not allowed 2026-01-24 18:55:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:55:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluidity-02f-direct-acting-solenoid-valve-2-2-nc-fkm-seal-3-8-nptf-3-mm-24v-dc>: HTTP status code is not handled or not allowed 2026-01-24 18:55:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips landed on page that is not a product page. 2026-01-24 18:55:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mid-size-brass-spray-nozzle-black>: HTTP status code is not handled or not allowed 2026-01-24 18:55:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:55:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hytrol-10-ft-18-inch-w-1-9-inch-dia-galvanized-rollers-15-inch-between-rail-4-5-inch-rollers-centers>: HTTP status code is not handled or not allowed 2026-01-24 18:55:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 10496 pages (at 88 pages/min), scraped 7268 items (at 55 items/min) 2026-01-24 18:55:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o22001-iron-eagle-plain-front-overall-blue-snaplock-buckles-sm>: HTTP status code is not handled or not allowed 2026-01-24 18:55:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 18:55:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-delrin-homopolymer-sheet-0-750-thick-x-12-w-x-24-l>: HTTP status code is not handled or not allowed 2026-01-24 18:55:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips landed on page that is not a product page. 2026-01-24 18:55:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1300-rubber-hitop-overshoes-black-cleated-outsole-md>: HTTP status code is not handled or not allowed 2026-01-24 18:55:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wireway-husky-tear-drop-pallet-rack-add-on-108x42x120>: HTTP status code is not handled or not allowed 2026-01-24 18:55:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:55:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-roll-no-adhesive-60a-116-thick-x-36-wide-x-9-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 18:55:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vinyl-no-toe-wall-base-coil-4-in-x-125-in-x-120-ft-white>: HTTP status code is not handled or not allowed 2026-01-24 18:55:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/full_height_storage_cabinets/stationary_storage_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:55:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-laminating-tag-5-4-34l-x-2-38w-yellow-100pack>: HTTP status code is not handled or not allowed 2026-01-24 18:55:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:55:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-roll-no-adhesive-60a-132-thick-x-36-wide-x-4-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 18:55:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/j53129-comfortbrite-jacket-fluorescent-orange-5xl>: HTTP status code is not handled or not allowed 2026-01-24 18:55:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 18:55:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-circle-tag-3-dia-black-100pack>: HTTP status code is not handled or not allowed 2026-01-24 18:55:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/cabinets/full_height_storage_cabinets/stationary_storage_cabinets landed on page that is not a product page. 2026-01-24 18:55:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 18:55:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/j56107-durascrim-storm-fly-front-hooded-jacket-yellow-3xl>: HTTP status code is not handled or not allowed 2026-01-24 18:55:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/abrasives/sanding_discs/flap_sanding_discs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:55:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-roll-no-adhesive-50a-316-thick-x-36-wide-x-8-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 18:55:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v73859-job-sight-class-2-surveyor-style-vest-fluorescent-orange-sm>: HTTP status code is not handled or not allowed 2026-01-24 18:55:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-tag-12-8l-x-4w-manilla-500pack>: HTTP status code is not handled or not allowed 2026-01-24 18:55:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:55:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/abrasives/sanding_discs/flap_sanding_discs landed on page that is not a product page. 2026-01-24 18:55:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-1-2x7-8-36-grit-alumox-resin-fibre-disc>: HTTP status code is not handled or not allowed 2026-01-24 18:55:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/l-shaped-electric-height-adjustable-desk-60-w-x-24-d-cherry-w-gray-base>: HTTP status code is not handled or not allowed 2026-01-24 18:55:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/expandable-starter-rack-36x18x84-gray-with-3-level-wood-deck-1500lb-cap-per-deck>: HTTP status code is not handled or not allowed 2026-01-24 18:55:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 18:55:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-sheet-no-adhesive-50a-18-thick-x-36-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 18:56:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-tag-5-4-34l-x-2-38w--fluorescent-green-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 18:56:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:56:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-tag-1-2-34l-x-1-38w--fluorescent-yellow-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 18:56:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:56:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mb922b-neoprene-steel-toe-snugleg-boots-brick-redbrown-size-9>: HTTP status code is not handled or not allowed 2026-01-24 18:56:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-cabinet-1-door-4-drawer-36-inch-w-x-18-inch-d-x-35-inch-h>: HTTP status code is not handled or not allowed 2026-01-24 18:56:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/abrasives/grinding_cutoff_wheels/surface_grinding_wheels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:56:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 10586 pages (at 90 pages/min), scraped 7329 items (at 61 items/min) 2026-01-24 18:56:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3m-s-b-6x2x1-2scrs048011-13176>: HTTP status code is not handled or not allowed 2026-01-24 18:56:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-24 18:56:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h31107-webdri-detachable-hood-yellow-large>: HTTP status code is not handled or not allowed 2026-01-24 18:56:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-24 18:56:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/softback-sanding-sponge>: HTTP status code is not handled or not allowed 2026-01-24 18:56:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/abrasives/grinding_cutoff_wheels/surface_grinding_wheels>: HTTP status code is not handled or not allowed 2026-01-24 18:56:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/57-natural-rubber-tape-3-x-110-yds-17-mil-clear-6pack>: HTTP status code is not handled or not allowed 2026-01-24 18:56:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-height-welded-workbench-w-open-base-hardboard-over-steel-24-x-48>: HTTP status code is not handled or not allowed 2026-01-24 18:56:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/weefhb-workez-executive-laptop-cooling-stand-w-2-fans-usb-ports-black>: HTTP status code is not handled or not allowed 2026-01-24 18:56:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wa-2860-e-60w-x-28d-wa-2860-e-welded-workbench-w-back-stop-end-stops>: HTTP status code is not handled or not allowed 2026-01-24 18:56:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nomex-410-sheet-0-003-thick-x-24-000-w-x-36-000-l>: HTTP status code is not handled or not allowed 2026-01-24 18:56:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/red-gpo-3-sheet-0-750-thick-x-24-w-x-48-l>: HTTP status code is not handled or not allowed 2026-01-24 18:56:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_shelving already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:56:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oxy-stain-remover-32-oz-trigger-spray-bottle-6-bottles-case>: HTTP status code is not handled or not allowed 2026-01-24 18:56:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-top-machine-table-24-x-48-x-24>: HTTP status code is not handled or not allowed 2026-01-24 18:56:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/walls_ceilings/ceiling_tiles already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:56:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/office_furniture/literature_mail_sorters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:56:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-7-gauge-steel-workbench-fixed-height-42-x-84>: HTTP status code is not handled or not allowed 2026-01-24 18:56:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:56:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/417766-9-tool-cuff-adapter-38mm-for-1-hose>: HTTP status code is not handled or not allowed 2026-01-24 18:56:51 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/walls_ceilings/ceiling_tiles landed on page that is not a product page. 2026-01-24 18:56:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/office_furniture/literature_mail_sorters>: HTTP status code is not handled or not allowed 2026-01-24 18:56:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/outdoor_tools_power_equipment/pressure_washers/electric_pressure_washers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:56:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-z-brite-wire-add-on-72wx14dx54h>: HTTP status code is not handled or not allowed 2026-01-24 18:56:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_cabinets_drawers?token=null already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:56:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-oz-o-ring-silicone-grease-tube>: HTTP status code is not handled or not allowed 2026-01-24 18:57:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gem-urinal-screen-lasts-30-days-red-spiced-apple-fragrance-12-box>: HTTP status code is not handled or not allowed 2026-01-24 18:57:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/outdoor_tools_power_equipment/pressure_washers/electric_pressure_washers landed on page that is not a product page. 2026-01-24 18:57:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aquaba-38-round-resin-pedestal-table-w-umbrella-hole-zinc-on-charcoal-frame>: HTTP status code is not handled or not allowed 2026-01-24 18:57:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_cabinets_drawers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:57:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/libman-commercial-large-angled-toilet-bowl-brush-caddy>: HTTP status code is not handled or not allowed 2026-01-24 18:57:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 10675 pages (at 89 pages/min), scraped 7394 items (at 65 items/min) 2026-01-24 18:57:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-dry-chemical-fire-extinguisher-wmetal-strap-bracket-wall-mount-type-a-b-c-25-lb-cap>: HTTP status code is not handled or not allowed 2026-01-24 18:57:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/odor_control already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:57:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/great-lakes-tin-rochester-2-x-2-nail-up-tin-ceiling-tile-in-copper-t58-08>: HTTP status code is not handled or not allowed 2026-01-24 18:57:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spin-grit-brush-w-quick-connect-for-716-58-id-tubes-pack-of-15>: HTTP status code is not handled or not allowed 2026-01-24 18:57:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_cabinets_drawers>: HTTP status code is not handled or not allowed 2026-01-24 18:57:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/great-lakes-tin-rochester-2-x-2-nail-up-tin-ceiling-tile-in-argento-t58-07>: HTTP status code is not handled or not allowed 2026-01-24 18:57:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/walls_ceilings/ceiling_tiles already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:57:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-in-one-package-opener-multi-function>: HTTP status code is not handled or not allowed 2026-01-24 18:57:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ozone-generator-10000-cubic-2>: HTTP status code is not handled or not allowed 2026-01-24 18:57:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/odor_control landed on page that is not a product page. 2026-01-24 18:57:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4210-drive-pulley-shaft-5513025>: HTTP status code is not handled or not allowed 2026-01-24 18:57:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hose-coupler-1>: HTTP status code is not handled or not allowed 2026-01-24 18:57:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/walls_ceilings/ceiling_tiles>: HTTP status code is not handled or not allowed 2026-01-24 18:57:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/toilet-seat-cleaner-dispenser-white-gray-500-ml-6-case-sc500dis>: HTTP status code is not handled or not allowed 2026-01-24 18:57:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/pneumatic_hoses_connectors/pneumatic_quick_connect_couplings already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:57:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reechcraft-pl-x4-powerlift-44-38-platform-height-350-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 18:57:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-suction-hose-641410-641411-floor-scrubbers>: HTTP status code is not handled or not allowed 2026-01-24 18:57:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/appliances/cooking_equipment/ranges/range_hoods_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:57:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a5816-gear-table-raiser-5507514>: HTTP status code is not handled or not allowed 2026-01-24 18:57:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-lab-coat-no-pockets-elastic-wrists-snap-front-single-collar-blue-2xl-30cs>: HTTP status code is not handled or not allowed 2026-01-24 18:57:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/pneumatic_hoses_connectors/pneumatic_quick_connect_couplings landed on page that is not a product page. 2026-01-24 18:57:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chill-its-6202s-vest-only-khaki-s-m>: HTTP status code is not handled or not allowed 2026-01-24 18:57:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/appliances/cooking_equipment/ranges/range_hoods_accessories>: HTTP status code is not handled or not allowed 2026-01-24 18:57:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eye-bolt-5630411>: HTTP status code is not handled or not allowed 2026-01-24 18:57:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7060-screw-pan-hd-m6x16-5512212>: HTTP status code is not handled or not allowed 2026-01-24 18:57:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7020-washer-12-5712941>: HTTP status code is not handled or not allowed 2026-01-24 18:58:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/lockers/bulk_storage_lockers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:58:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/allen-wrench-8mm-5628791>: HTTP status code is not handled or not allowed 2026-01-24 18:58:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/474-ergomat153-anti-fatigue-mat-12-thick-2w-x-75l-black>: HTTP status code is not handled or not allowed 2026-01-24 18:58:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/lockers/bulk_storage_lockers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:58:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylon-hairnet-honeycomb-116-hole-100-latex-free-white-24-100bag>: HTTP status code is not handled or not allowed 2026-01-24 18:58:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-lab-coat-no-pockets-elastic-wrists-velcro-front-single-collar-xl-30cs>: HTTP status code is not handled or not allowed 2026-01-24 18:58:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/lockers/bulk_storage_lockers landed on page that is not a product page. 2026-01-24 18:58:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 10769 pages (at 94 pages/min), scraped 7461 items (at 67 items/min) 2026-01-24 18:58:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-x-3-drywall-screw-phillips-bugle-head-black-phosphate-tek-self-drilling-point-pkg-of-200>: HTTP status code is not handled or not allowed 2026-01-24 18:58:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/lockers/bulk_storage_lockers landed on page that is not a product page. 2026-01-24 18:58:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8210z-class-2-economy-vest-lime-4xl-5xl>: HTTP status code is not handled or not allowed 2026-01-24 18:58:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2221vs-washer-5510262>: HTTP status code is not handled or not allowed 2026-01-24 18:58:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-gravity-flow-rack-1>: HTTP status code is not handled or not allowed 2026-01-24 18:58:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sunset-armless-lounge-chair-solid-gray-sling-on-platinum-gray-frame-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-24 18:58:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cbo8-single-flue-chimney-cap-bolt-on-8-round-black-galvanized-steel>: HTTP status code is not handled or not allowed 2026-01-24 18:58:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/driving-sleeve-5625671>: HTTP status code is not handled or not allowed 2026-01-24 18:58:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/drinking_fountains_bottle_fillers_water_coolers/water_coolers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:58:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stud-4224-6295778>: HTTP status code is not handled or not allowed 2026-01-24 18:58:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/work-benches/workbenches_worktables/preconfigured_workbenches/standard_duty_preconfigured_workbenches already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:58:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0-5oz-amber-wide-mouth-packer-with-28-400-black-polypropylene-cap-ptfe-disc-48pk>: HTTP status code is not handled or not allowed 2026-01-24 18:58:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-heavy-duty-3-shelf-rack-20-wx60-hx60-l>: HTTP status code is not handled or not allowed 2026-01-24 18:58:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=jet-jtm-1050evs2-230-mill-3-axis-newall-dp700-dro-quill-x-axis-powerfeed-690628 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:58:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jet-jtm-1050evs2-230-mill-newall-dp700-dro-x-and-y-axis-powerfeeds-698174>: HTTP status code is not handled or not allowed 2026-01-24 18:58:42 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/plumbing/drinking_fountains_bottle_fillers_water_coolers/water_coolers landed on page that is not a product page. 2026-01-24 18:58:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_seating/patio_dining_chairs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:58:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triumph-style-t1b-hss-jobbers-drill-black-oxide-316-12>: HTTP status code is not handled or not allowed 2026-01-24 18:58:44 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/work-benches/workbenches_worktables/preconfigured_workbenches/standard_duty_preconfigured_workbenches landed on page that is not a product page. 2026-01-24 18:58:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wicking-birdseye-t-shirt-w-pocket-class-3-hi-vis-yellow-small>: HTTP status code is not handled or not allowed 2026-01-24 18:58:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/tarps_sheeting/surface_protectors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:58:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-4w-x-14l-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 18:58:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_seating/patio_dining_chairs>: HTTP status code is not handled or not allowed 2026-01-24 18:58:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=jet-jtm-1050evs2-230-mill-3-axis-newall-dp700-dro-quill-x-axis-powerfeed-690628 landed on page that is not a product page. 2026-01-24 18:58:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-22w-x-22l-2-mil-clear-500pack>: HTTP status code is not handled or not allowed 2026-01-24 18:58:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8915-class-e-rain-pants-lime-medium>: HTTP status code is not handled or not allowed 2026-01-24 18:58:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/tarps_sheeting/surface_protectors landed on page that is not a product page. 2026-01-24 18:58:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:58:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cart-cover-10-gauge-vinyl-for-24-x-72-x-63-unit-zipper-closure-clear>: HTTP status code is not handled or not allowed 2026-01-24 18:59:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/135oz-amber-jug-w-neck-finish-38-430-black-phenolic-cap-f217-ptfe-cap-vacuum-6pk>: HTTP status code is not handled or not allowed 2026-01-24 18:59:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/stamps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:59:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-4000-single-strap-wrist-support-black-x-large-left>: HTTP status code is not handled or not allowed 2026-01-24 18:59:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts landed on page that is not a product page. 2026-01-24 18:59:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-5-in-1-parka-hi-viz-orange-2xl>: HTTP status code is not handled or not allowed 2026-01-24 18:59:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 10860 pages (at 91 pages/min), scraped 7525 items (at 64 items/min) 2026-01-24 18:59:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1724b04-cross-over-bridge-36-1-2-overall-width-4-stairs>: HTTP status code is not handled or not allowed 2026-01-24 18:59:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/binders_covers/dividers_tabs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:59:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/binders_covers/dividers_tabs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:59:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dewalt-dwe4222n-11-amp-corded-4-5-small-angle-paddle-switch-angle-grinder-w-brake-no-lock-on>: HTTP status code is not handled or not allowed 2026-01-24 18:59:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/stamps landed on page that is not a product page. 2026-01-24 18:59:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/material-handling/lift_tables_work_positioners/lift_table_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:59:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/320-contact-adhesive-1-quart>: HTTP status code is not handled or not allowed 2026-01-24 18:59:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/notebooks_pads already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:59:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/toilet-seat-cover-14-1-2-x-17-white-20-carton-tc0020>: HTTP status code is not handled or not allowed 2026-01-24 18:59:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/binders_covers/dividers_tabs landed on page that is not a product page. 2026-01-24 18:59:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/411kas12prp-safety-padlock-1-12wx1-12h-purple-12set>: HTTP status code is not handled or not allowed 2026-01-24 18:59:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/binders_covers/dividers_tabs landed on page that is not a product page. 2026-01-24 18:59:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slipper-mdl-ghbbdb-1340a-32a5306>: HTTP status code is not handled or not allowed 2026-01-24 18:59:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/narrow-2-arm-revers-quick-adj-35-ton-capacity-4-3-4-120-mm-open-3-7-8-100-mm-reach>: HTTP status code is not handled or not allowed 2026-01-24 18:59:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-l-x-36-w-production-bench-maple-square-edge-tan>: HTTP status code is not handled or not allowed 2026-01-24 18:59:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/paper_stationery/notebooks_pads landed on page that is not a product page. 2026-01-24 18:59:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/material-handling/lift_tables_work_positioners/lift_table_accessories landed on page that is not a product page. 2026-01-24 18:59:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classic-plus-35mm-pvc-poly-ridercoat-collar-blk-267cx2>: HTTP status code is not handled or not allowed 2026-01-24 18:59:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/411kaylw-safety-padlock-1-12w-x-1-12h-shackle-ylw>: HTTP status code is not handled or not allowed 2026-01-24 18:59:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s33ltblk-snap-lock-safety-padlock-1-38w-x-3h-black>: HTTP status code is not handled or not allowed 2026-01-24 18:59:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pull-seal-catalog-envelope-9-x-12-light-brown-100box>: HTTP status code is not handled or not allowed 2026-01-24 18:59:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 18:59:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-piece-slide-ha-mmer-set-sae-5-male-adapters-3-female-adapters>: HTTP status code is not handled or not allowed 2026-01-24 18:59:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washout-construction-washout-w-lid-polypropylene-120-gallon-48-l-x-48-w-x-12-h>: HTTP status code is not handled or not allowed 2026-01-24 18:59:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards>: HTTP status code is not handled or not allowed 2026-01-24 18:59:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anchorage-connsctors-tie-off-adapter-1003000>: HTTP status code is not handled or not allowed 2026-01-24 18:59:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-40w-x-42l-2-mil-clear-200pack>: HTTP status code is not handled or not allowed 2026-01-24 18:59:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nut-spices-bar-maple-glazed-pecan-sea-salt-1-4-oz>: HTTP status code is not handled or not allowed 2026-01-24 19:00:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buckhorn-lid-48x45-black>: HTTP status code is not handled or not allowed 2026-01-24 19:00:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unprinted-reclosable-static-shielding-bags-9w-x-12l-28-mil-clear-100pack>: HTTP status code is not handled or not allowed 2026-01-24 19:00:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11-step-24in-w-steel-safety-angle-rolling-ladder-grip-strut-gray-kdec111242>: HTTP status code is not handled or not allowed 2026-01-24 19:00:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5c-collet-5-16-inch-round-tmx>: HTTP status code is not handled or not allowed 2026-01-24 19:00:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 10952 pages (at 92 pages/min), scraped 7589 items (at 64 items/min) 2026-01-24 19:00:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5c-collet-3-8-inch-round-tmx>: HTTP status code is not handled or not allowed 2026-01-24 19:00:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/lathe_chucks_accessories/lathe_chuck_parts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:00:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s1-safety-quick-coupling-3-8-industrial-interchange-with-1-2-fnpt-connection>: HTTP status code is not handled or not allowed 2026-01-24 19:00:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thermostats-46-1209-jad4615300100>: HTTP status code is not handled or not allowed 2026-01-24 19:00:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/lathe_chucks_accessories/lathe_chuck_parts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:00:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-unit-5-tier-nexelon-21w-x-60l-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 19:00:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/machinery_parts_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:00:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bison-live-centers-mt-5-super-precision-t-i-r-0002>: HTTP status code is not handled or not allowed 2026-01-24 19:00:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a16n12mpp-panel-1300x1050-fits-16x12-med-steelgray>: HTTP status code is not handled or not allowed 2026-01-24 19:00:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/lathe_chucks_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:00:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/lathe_chucks_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:00:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-step-16in-w-10in-d-top-step-steel-rolling-ladder-grip-strut-tread-36in-handrail-kdsr107162>: HTTP status code is not handled or not allowed 2026-01-24 19:00:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cleaning-station-shadow-board-blue-white-72-x-36-pro-series-acrylic>: HTTP status code is not handled or not allowed 2026-01-24 19:00:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/machinery_parts_accessories landed on page that is not a product page. 2026-01-24 19:00:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/construction-inner-outer-reamers-29993>: HTTP status code is not handled or not allowed 2026-01-24 19:00:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/lathe_chucks_accessories landed on page that is not a product page. 2026-01-24 19:00:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/lathe_chucks_accessories landed on page that is not a product page. 2026-01-24 19:00:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-x12-melamine-wood-shelves-with-brackets-package-of-2>: HTTP status code is not handled or not allowed 2026-01-24 19:00:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pure-flow-cartridge-refills-set-of-2>: HTTP status code is not handled or not allowed 2026-01-24 19:00:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oil-cap-jsm4340-26>: HTTP status code is not handled or not allowed 2026-01-24 19:00:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ck28-311-contactor-3-pole-120v>: HTTP status code is not handled or not allowed 2026-01-24 19:00:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/open-end-static-shielding-bags-15w-x-18l-28-mil-transparent-100pack>: HTTP status code is not handled or not allowed 2026-01-24 19:00:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hanging-chain-kit-1800csu204-for-u-configuration-4-0-infrared-heaters-20-l>: HTTP status code is not handled or not allowed 2026-01-24 19:00:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gusseted-poly-bags-12w-x-6d-x-24h-2-mil-clear-500pack>: HTTP status code is not handled or not allowed 2026-01-24 19:00:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-polypropylene-coverall-elastic-wrists-ankles-hood-boots-white-medium-25-case>: HTTP status code is not handled or not allowed 2026-01-24 19:00:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-1-2-gallon-step-on-trash-can-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 19:01:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ztek-eyewear-clear-anti-fog-lens-with-clear-frame>: HTTP status code is not handled or not allowed 2026-01-24 19:01:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/airsol-asflat-anti-fatigue-insoles-h-13-14-flat-style-open-cell-foam>: HTTP status code is not handled or not allowed 2026-01-24 19:01:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sheave-countershaft-3717053>: HTTP status code is not handled or not allowed 2026-01-24 19:01:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lens-cleaning-solution>: HTTP status code is not handled or not allowed 2026-01-24 19:01:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 11045 pages (at 93 pages/min), scraped 7658 items (at 69 items/min) 2026-01-24 19:01:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tinted-shopping-bags-8w-x-4-12d-x-10-14h-brown-250pack>: HTTP status code is not handled or not allowed 2026-01-24 19:01:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/import-precision-ground-dowel-pin-38-x-1-34-40pk>: HTTP status code is not handled or not allowed 2026-01-24 19:01:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-giant-stackable-storage-bin-18-1-4x23-7-8x12-red>: HTTP status code is not handled or not allowed 2026-01-24 19:01:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/release-block-ghd-20pfjmd-18pfn-2450020>: HTTP status code is not handled or not allowed 2026-01-24 19:01:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60in-invader-side-gate-wood-mop-handle-yellow-fgh116000000>: HTTP status code is not handled or not allowed 2026-01-24 19:01:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/index_cards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:01:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/intruder-eyewear-io-mirror-lens-with-io-mirror-frame>: HTTP status code is not handled or not allowed 2026-01-24 19:01:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4004-double-sided-foam-tape-1-x-18-yds-250-mil-natural-9pack>: HTTP status code is not handled or not allowed 2026-01-24 19:01:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/key-ghb-1340-4x4x20-text-7534>: HTTP status code is not handled or not allowed 2026-01-24 19:01:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/highlighters_markers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:01:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-cabinets-maah-3624rb-b-mobile-storage-cabinet-assembled-36x24x66-black>: HTTP status code is not handled or not allowed 2026-01-24 19:01:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/paper_stationery/index_cards landed on page that is not a product page. 2026-01-24 19:01:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-foot-x-100-foot-heavy-duty-silver-tarp>: HTTP status code is not handled or not allowed 2026-01-24 19:01:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/highlighters_markers>: HTTP status code is not handled or not allowed 2026-01-24 19:01:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-174-chrome-wire-shelf-truck-48x24x72-1200-pound-capacity-with-brakes>: HTTP status code is not handled or not allowed 2026-01-24 19:01:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:01:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4951-double-sided-foam-tape-34-x-5-yds-45-mil-white>: HTTP status code is not handled or not allowed 2026-01-24 19:01:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/complete-electronic-packaging-workbench-plastic-safety-edge-60-x-30>: HTTP status code is not handled or not allowed 2026-01-24 19:01:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-foot-x-50-foot-forest-green-tarp>: HTTP status code is not handled or not allowed 2026-01-24 19:01:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/243-1952-30h-hdmi-male-to-male-w-internet-1080p-28-awg-30-ft-gold-plated-black>: HTTP status code is not handled or not allowed 2026-01-24 19:01:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fibre-disc-787c-36-5-x-7-8-in-die-500p-25-per-inner-100-per-case>: HTTP status code is not handled or not allowed 2026-01-24 19:01:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks>: HTTP status code is not handled or not allowed 2026-01-24 19:01:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spacer-jtm-124vs-la-122>: HTTP status code is not handled or not allowed 2026-01-24 19:01:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/clips_fasteners/rubber_bands already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:01:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/clips_fasteners/paper_clips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:01:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ez-stainless-steel-truck-dump-insert-for-8-foot-bed>: HTTP status code is not handled or not allowed 2026-01-24 19:01:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-precision-iso-metric-double-roller-chain-20b-2-1-1-4-pitch-10ft-box>: HTTP status code is not handled or not allowed 2026-01-24 19:01:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enterprise-sink-base-cabinet-grouping-108w-x-30d-x-36h-silver-beige>: HTTP status code is not handled or not allowed 2026-01-24 19:01:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/clips_fasteners/rubber_bands landed on page that is not a product page. 2026-01-24 19:01:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-tube-light-rl70le-84-n-led-22-26v-dc-42w-glass-42>: HTTP status code is not handled or not allowed 2026-01-24 19:01:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/clips_fasteners/paper_clips landed on page that is not a product page. 2026-01-24 19:02:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plunger-jtm-1055-jtm1055-h101>: HTTP status code is not handled or not allowed 2026-01-24 19:02:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poly-tubing-14w-x-725l-6-mil-black-1-roll>: HTTP status code is not handled or not allowed 2026-01-24 19:02:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ring-led-rllq63-6w-100-240v-50-60hz-6-diopter-with-clamp>: HTTP status code is not handled or not allowed 2026-01-24 19:02:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40086-spot-blaster-adapter-kit-rubber>: HTTP status code is not handled or not allowed 2026-01-24 19:02:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 11143 pages (at 98 pages/min), scraped 7726 items (at 68 items/min) 2026-01-24 19:02:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polypropylene-bags-14l-x-11w-15-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 19:02:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/complete-wheel-kit-for-manual-pallet-jack-gwk-am2200-ck-fits-jungheinrich-model-am2200>: HTTP status code is not handled or not allowed 2026-01-24 19:02:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gondola-free-standing-slider-system-with-28-variety-bins-white>: HTTP status code is not handled or not allowed 2026-01-24 19:02:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/automatic-paper-folding-machine-with-adj-stacker-wheels-252457>: HTTP status code is not handled or not allowed 2026-01-24 19:02:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-qt-alum-liquid-measures>: HTTP status code is not handled or not allowed 2026-01-24 19:02:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-tornado-fiberglass-exhaust-poly-blades-shutters>: HTTP status code is not handled or not allowed 2026-01-24 19:02:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:02:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-18w-x-36l-4-mil-clear-250pack>: HTTP status code is not handled or not allowed 2026-01-24 19:02:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pla-precision-material-white>: HTTP status code is not handled or not allowed 2026-01-24 19:02:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/true-temper-19-poly-combo-snow-shovel-with-versagrip>: HTTP status code is not handled or not allowed 2026-01-24 19:02:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/resealable-polypropylene-bags-4l-x-3w-15-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 19:02:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits landed on page that is not a product page. 2026-01-24 19:02:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/resealable-polypropylene-bags-18l-x-12w-15-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 19:02:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/packing-workbench-maple-butcher-block-safety-edge-60-x-36-with-riser-kit>: HTTP status code is not handled or not allowed 2026-01-24 19:02:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adapter-bar-6>: HTTP status code is not handled or not allowed 2026-01-24 19:02:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/complete-mobile-packing-workbench-plastic-square-edge-with-electric-72-x-36>: HTTP status code is not handled or not allowed 2026-01-24 19:02:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprng-jtm-1050-b-21>: HTTP status code is not handled or not allowed 2026-01-24 19:02:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18vx2-36v-lxt-174-li-ion-brushless-couple-shaft-power-head-w-string-trimmer-attachment>: HTTP status code is not handled or not allowed 2026-01-24 19:02:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergotec-soft-rubber-36-12-pack>: HTTP status code is not handled or not allowed 2026-01-24 19:02:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-36-3-drawer-with-maple-top-workbench>: HTTP status code is not handled or not allowed 2026-01-24 19:02:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-credenza-with-box-and-file-drawers-36w-charcoal>: HTTP status code is not handled or not allowed 2026-01-24 19:02:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spring-jjp8bt-115>: HTTP status code is not handled or not allowed 2026-01-24 19:03:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coffee-portion-packs-1-1-2-oz-packs-columbian-decaf>: HTTP status code is not handled or not allowed 2026-01-24 19:03:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c00-series-grip-ring-stem-caster-1>: HTTP status code is not handled or not allowed 2026-01-24 19:03:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c00-series-grip-ring-stem-caster-10>: HTTP status code is not handled or not allowed 2026-01-24 19:03:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 11241 pages (at 98 pages/min), scraped 7799 items (at 73 items/min) 2026-01-24 19:03:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nut-9180-201>: HTTP status code is not handled or not allowed 2026-01-24 19:03:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ingersoll-rand-236-1-2-impact>: HTTP status code is not handled or not allowed 2026-01-24 19:03:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:03:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ingersoll-rand-231ha-impactool>: HTTP status code is not handled or not allowed 2026-01-24 19:03:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/indicating-lights>: HTTP status code is not handled or not allowed 2026-01-24 19:03:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exq-pro-triple-bank-top-chest-72w-x-30d-x-26-14h-green-with-black-drawer-pulls>: HTTP status code is not handled or not allowed 2026-01-24 19:03:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats landed on page that is not a product page. 2026-01-24 19:03:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90-176-ss-corner-guard-16-ga-3m-two-sided-tape-sq-corners-48-lx1-wx1-h>: HTTP status code is not handled or not allowed 2026-01-24 19:03:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brass-breather-valve-with-g-1-2-pipe-thread-881-g1-2-200-ms-m>: HTTP status code is not handled or not allowed 2026-01-24 19:03:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/animal-house>: HTTP status code is not handled or not allowed 2026-01-24 19:03:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_alarms_signals/audible_alarms already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:03:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/open-sided-wire-exchange-truck-five-wire-shelves-800-lb-cap-4>: HTTP status code is not handled or not allowed 2026-01-24 19:03:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/filter-oil-assembly-part-replacement-for-kerosene-heater>: HTTP status code is not handled or not allowed 2026-01-24 19:03:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s-tl-thermaflex-flexible-hvac-duct-5-diameter>: HTTP status code is not handled or not allowed 2026-01-24 19:03:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/armorultra-job-on-pallet-2-layer-epoxy-system-1000-sq-ft-white>: HTTP status code is not handled or not allowed 2026-01-24 19:03:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_alarms_signals/audible_alarms landed on page that is not a product page. 2026-01-24 19:03:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ng-to-lp-conversion-kit-for-modine-high-efficiency-300000-btu-gas-fired-unit-heater>: HTTP status code is not handled or not allowed 2026-01-24 19:03:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/king-pin-lock-tractor-trailers-not-keyed-alike>: HTTP status code is not handled or not allowed 2026-01-24 19:03:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelate-silver-epoxy-wire-shelving-add-on-unit-5-tier-72w-x-21d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 19:03:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/power_tools/power_saws_blades/band_saws/band_saw_machines already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:03:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/foam/foam_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:03:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bollard-safety-net-4-foot-l-by-16-foot-w-fits-7-3-4-inches-to-8-5-8-inches-dia-brsn-416-9>: HTTP status code is not handled or not allowed 2026-01-24 19:03:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-epdm-tube-60-l-x-5-8-id-x-7-8-od-black>: HTTP status code is not handled or not allowed 2026-01-24 19:03:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classical-b-v-belt-b30>: HTTP status code is not handled or not allowed 2026-01-24 19:03:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/foam/foam_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:03:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/power_tools/power_saws_blades/band_saws/band_saw_machines>: HTTP status code is not handled or not allowed 2026-01-24 19:03:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8090250010-two-stage-horizontal-air-compressor-75hp-80-gallon>: HTTP status code is not handled or not allowed 2026-01-24 19:03:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/foam/foam_sheets landed on page that is not a product page. 2026-01-24 19:04:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-foam-buna-n-pvc-sheet-36-l-x-36-w-x-3-8-thick-black>: HTTP status code is not handled or not allowed 2026-01-24 19:04:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/locks_lockout_devices/lockout_tagout_devices/lockout_padlocks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:04:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:04:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/86555-ladder-inspecton-tag-2-sided-10pkg-polyester-3w-x-5-3-34h>: HTTP status code is not handled or not allowed 2026-01-24 19:04:04 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/foam/foam_rolls landed on page that is not a product page. 2026-01-24 19:04:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8090250037-honda-gasoline-driven-horizontal-air-compressor-13hp-30-gallon>: HTTP status code is not handled or not allowed 2026-01-24 19:04:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/red-rack-mobile-gear-storage-rack-locker-double-sided-six-24-sections-red>: HTTP status code is not handled or not allowed 2026-01-24 19:04:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 11333 pages (at 92 pages/min), scraped 7864 items (at 65 items/min) 2026-01-24 19:04:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/locks_lockout_devices/lockout_tagout_devices/lockout_padlocks landed on page that is not a product page. 2026-01-24 19:04:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 19:04:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:04:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/250-1-8-diameter-7x19-stainless-steel-aircraft-cable>: HTTP status code is not handled or not allowed 2026-01-24 19:04:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/ink_toner/laser_toner_cartridges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:04:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/locks_lockout_devices/lockout_tagout_devices/lockout_devices already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:04:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cast-acrylic-sheet-96-l-x-48-w-x-1-4-thick-yellow-ps-cact-80>: HTTP status code is not handled or not allowed 2026-01-24 19:04:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:04:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grade-70-transport-binder-chain-1-2-dia-11300-lb-cap-20-box-bag>: HTTP status code is not handled or not allowed 2026-01-24 19:04:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sparta-paddle-scraper-w-handle-60-long>: HTTP status code is not handled or not allowed 2026-01-24 19:04:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/ink_toner/laser_toner_cartridges landed on page that is not a product page. 2026-01-24 19:04:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/locks_lockout_devices/lockout_tagout_devices/lockout_devices>: HTTP status code is not handled or not allowed 2026-01-24 19:04:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-24 19:04:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelate-silver-epoxy-wire-shelving-starter-unit-5-tier-54w-x-24d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 19:04:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-24 19:04:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:04:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26x12x48-solar-star-greenhouse-w-poly-ends-and-drop-down-sides>: HTTP status code is not handled or not allowed 2026-01-24 19:04:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/wire_shelving/poly_zbrite_wire_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:04:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pointer-jts-10jfjcs-10-10105057>: HTTP status code is not handled or not allowed 2026-01-24 19:04:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/103780-handicapped-parking-sign-bluewhite-hip-reflective-sign-aluminum-12w-x-18h>: HTTP status code is not handled or not allowed 2026-01-24 19:04:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-versa-tech-wheel-8x2-58-roller>: HTTP status code is not handled or not allowed 2026-01-24 19:04:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelving-starter-unit-5-tier-54w-x-21d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 19:04:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-24 19:04:42 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/wire_shelving/poly_zbrite_wire_shelves landed on page that is not a product page. 2026-01-24 19:04:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:04:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/225504-3-compartment-tray-wpushers-and-tracks-9125w-x-15h-x-58125d-clr-1-pc>: HTTP status code is not handled or not allowed 2026-01-24 19:04:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/45a-1-motor-115v60hz1ph-text-6289098>: HTTP status code is not handled or not allowed 2026-01-24 19:04:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/locks_lockout_devices/lockout_tagout_devices/lockout_padlocks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:04:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/material-handling/casters_wheels/wheels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:04:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-x-12-x-7-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 19:04:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_lockout_identification_tags already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:04:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sokt-hd-cap-screw-m10-x-15p-x-50lg-201-6012133>: HTTP status code is not handled or not allowed 2026-01-24 19:04:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents landed on page that is not a product page. 2026-01-24 19:04:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_film_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:04:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polystyrene-bar-24-l-x-6-w-x-1-32-thick-white>: HTTP status code is not handled or not allowed 2026-01-24 19:04:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/locks_lockout_devices/lockout_tagout_devices/lockout_padlocks landed on page that is not a product page. 2026-01-24 19:04:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_film_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:04:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700277pnk-pegboard-floor-stand-135w-x-60h-pk-1-pc>: HTTP status code is not handled or not allowed 2026-01-24 19:05:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_lockout_identification_tags>: HTTP status code is not handled or not allowed 2026-01-24 19:05:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/material-handling/casters_wheels/wheels landed on page that is not a product page. 2026-01-24 19:05:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_film_stock>: HTTP status code is not handled or not allowed 2026-01-24 19:05:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-abs-tube-72-l-x-1-id-x-1-1-4-od-beige>: HTTP status code is not handled or not allowed 2026-01-24 19:05:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_film_stock landed on page that is not a product page. 2026-01-24 19:05:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 11419 pages (at 86 pages/min), scraped 7919 items (at 55 items/min) 2026-01-24 19:05:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-specialty-labels-caution--lithium-battery-4-58l-x-5w-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-24 19:05:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/food_beverages/snacks_candies/cookies_pastries already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:05:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/610112clr-flat-revolving-display-base-075h-x-12dia-clr-10pack>: HTTP status code is not handled or not allowed 2026-01-24 19:05:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-sheet-24-l-x-24-w-x-1-8-thick-gray>: HTTP status code is not handled or not allowed 2026-01-24 19:05:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-x-2-spring-pin-high-carbon-steel-plain-asme-b18-8-2-made-in-usa>: HTTP status code is not handled or not allowed 2026-01-24 19:05:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-wall-mounted-shelf-12-inch-72-inch-shelf-with-hooks>: HTTP status code is not handled or not allowed 2026-01-24 19:05:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:05:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-wall-mounted-shelf-15-inch-36-inch-shelf-with-hooks>: HTTP status code is not handled or not allowed 2026-01-24 19:05:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-wall-mount-magnetic-retractable-barrier-authorized-access-only-belt-30-l>: HTTP status code is not handled or not allowed 2026-01-24 19:05:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/food_beverages/snacks_candies/cookies_pastries landed on page that is not a product page. 2026-01-24 19:05:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rectangular-inventory-paper-labels-4l-x-4w-green-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-24 19:05:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 19:05:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:05:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/600031-spring-pusher-06875w-x-275h-clr-50-pc>: HTTP status code is not handled or not allowed 2026-01-24 19:05:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guard-dog-5-ch-45-degree-right-turn-orange-lidblack-base>: HTTP status code is not handled or not allowed 2026-01-24 19:05:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/foam/foam_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:05:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ada-locker-bench-wall-mount-bracket-each-for-20-deep-top-almond-269863tn>: HTTP status code is not handled or not allowed 2026-01-24 19:05:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/versa-loop-rehabilitation-band-heavy-resistance-light-blue>: HTTP status code is not handled or not allowed 2026-01-24 19:05:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips landed on page that is not a product page. 2026-01-24 19:05:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adj-height-pallet-stand-pdse48-6ph2fllr-48-x-48-solid-deck-and-load-retainers>: HTTP status code is not handled or not allowed 2026-01-24 19:05:42 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/foam/foam_sheets landed on page that is not a product page. 2026-01-24 19:05:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-spill-economy-containment-pallet-p2-2-drum-with-drain>: HTTP status code is not handled or not allowed 2026-01-24 19:05:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/grill_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:05:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-dia-round-paper-labels-with-sept-print-white-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-24 19:05:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:05:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mfsd987vp-eye-wash-station-sign-10w-x-7h-plastic>: HTTP status code is not handled or not allowed 2026-01-24 19:05:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tall-20w-zippered-end-panel-white>: HTTP status code is not handled or not allowed 2026-01-24 19:05:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:05:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:05:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/non-electric-office-partition-panel-with-raceway-24-1-4w-x-100h-tan-gl>: HTTP status code is not handled or not allowed 2026-01-24 19:05:56 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 19:05:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pictogram-corrosion-labels-2l-x-2w-redwhiteblack-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-24 19:05:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:06:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-x-100-gauge-pvc-shrink-tubing>: HTTP status code is not handled or not allowed 2026-01-24 19:06:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 19:06:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fixed-height-welded-steel-workbench-lower-shelf-72-w-x-36-d-x-34-1-16-h>: HTTP status code is not handled or not allowed 2026-01-24 19:06:04 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/floor_mats landed on page that is not a product page. 2026-01-24 19:06:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9x18-2-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-24 19:06:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-24 19:06:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 11508 pages (at 89 pages/min), scraped 7978 items (at 59 items/min) 2026-01-24 19:06:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/msmk050vs-danger-sign-no-smoking-or-open-flame-14w-x-10h-adhesive-vinyl>: HTTP status code is not handled or not allowed 2026-01-24 19:06:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-24-l-x-1-2-w-x-3-16-thick-black-antistatic>: HTTP status code is not handled or not allowed 2026-01-24 19:06:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fixed-height-welded-steel-workbench-locking-drawer-lower-shelf-72-w-x-36-d-x-34-1-16-h>: HTTP status code is not handled or not allowed 2026-01-24 19:06:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:06:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-24-l-x-3-w-x-1-thick-black-antistatic>: HTTP status code is not handled or not allowed 2026-01-24 19:06:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-x-2150-2-mil-anti-static-poly-tubing>: HTTP status code is not handled or not allowed 2026-01-24 19:06:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-48-l-x-1-w-x-3-4-thick-black-antistatic>: HTTP status code is not handled or not allowed 2026-01-24 19:06:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-24-l-x-3-w-x-1-8-thick-black-antistatic>: HTTP status code is not handled or not allowed 2026-01-24 19:06:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 19:06:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:06:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:06:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/d-con-disposable-bait-station-3w-x-3d-x-1-1-4h-6-carton-rac99427>: HTTP status code is not handled or not allowed 2026-01-24 19:06:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-8-inches-open-snap-on-seal-for-steel-5000-per-case>: HTTP status code is not handled or not allowed 2026-01-24 19:06:29 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-24 19:06:30 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:8 2026-01-24 19:06:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/impact-socket-1-2-drive-30mm-6-point-deep>: HTTP status code is not handled or not allowed 2026-01-24 19:06:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 19:06:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 19:06:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/msmk243vs-danger-sign-flammable-material-no-smoking-or-open-14w-x-10h-adhesive-vinyl>: HTTP status code is not handled or not allowed 2026-01-24 19:06:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-bar-12-l-x-1-w-x-3-8-thick-clear-antistatic>: HTTP status code is not handled or not allowed 2026-01-24 19:06:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:06:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:06:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gold-film-disc-roll-255l-5-x-nh-p150-125-discsroll-4-rollscase>: HTTP status code is not handled or not allowed 2026-01-24 19:06:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-sheet-24-l-x-12-w-x-3-32-thick-clear>: HTTP status code is not handled or not allowed 2026-01-24 19:06:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fairfield-horizontal-towel-valet-black>: HTTP status code is not handled or not allowed 2026-01-24 19:06:45 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 19:06:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 19:06:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8x10-4-mil-anti-static-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-24 19:06:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/female-single-swivel-socket-member-4>: HTTP status code is not handled or not allowed 2026-01-24 19:06:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-led-crane-scale-with-remote-2000-lb-x-1-lb>: HTTP status code is not handled or not allowed 2026-01-24 19:06:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/z-line-boltless-shelving-w-steel-industrial-shelves-48-w-x-24-d-x-84-h-starter-5-shelves>: HTTP status code is not handled or not allowed 2026-01-24 19:06:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trs307ptp-inspection-record-tag-pf-cardstock-25pack>: HTTP status code is not handled or not allowed 2026-01-24 19:06:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/duo-sweep-spectrum-1-dia-fiberglass-handle-with-self-locking-flex-tip-48-blue>: HTTP status code is not handled or not allowed 2026-01-24 19:07:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3x18-2-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-24 19:07:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:07:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dust-mop-36-natural>: HTTP status code is not handled or not allowed 2026-01-24 19:07:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durawrap-1050-ballistic-nylon-wrap-w-velcro-closure-6-dia-150-black>: HTTP status code is not handled or not allowed 2026-01-24 19:07:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:07:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:07:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machined-type-needle-roller-bearing-metric-taf121912-12mm-bore-19mm-od>: HTTP status code is not handled or not allowed 2026-01-24 19:07:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-corner-swivel-socket-member-2>: HTTP status code is not handled or not allowed 2026-01-24 19:07:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 11601 pages (at 93 pages/min), scraped 8040 items (at 62 items/min) 2026-01-24 19:07:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-24 19:07:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-bar-24-l-x-1-w-x-1-8-thick-clear-antistatic>: HTTP status code is not handled or not allowed 2026-01-24 19:07:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 19:07:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/add-on-single-handrail-socket-1>: HTTP status code is not handled or not allowed 2026-01-24 19:07:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-24 19:07:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:07:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/efp-i-auto-darkening-welding-helmet-9-13-variable-shade-skeletor-graphic>: HTTP status code is not handled or not allowed 2026-01-24 19:07:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hurricane-360-hoist-and-trolley-combo-geared-3-ton-capacity-20-lift>: HTTP status code is not handled or not allowed 2026-01-24 19:07:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tea-concentrate-dispenser-67-galhr>: HTTP status code is not handled or not allowed 2026-01-24 19:07:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 19:07:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-bar-24-l-x-2-w-x-1-8-thick-clear-antistatic>: HTTP status code is not handled or not allowed 2026-01-24 19:07:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-b-cylinders-with-pour-spout-500ml-case-of-4>: HTTP status code is not handled or not allowed 2026-01-24 19:07:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/Sunex-hd-ext-die-grinder>: HTTP status code is not handled or not allowed 2026-01-24 19:07:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-pump-kit-p7500>: HTTP status code is not handled or not allowed 2026-01-24 19:07:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dax-carbon-fiber-full-brim-50-50-yellow-black>: HTTP status code is not handled or not allowed 2026-01-24 19:07:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/air_conditioners_chillers/window_air_conditioners already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:07:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guardair-ultra-venturi-12-alum-ext-alloy-nozzle-short-trigger>: HTTP status code is not handled or not allowed 2026-01-24 19:07:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kraft-paper-40-lbs-72w-x-900l-1-roll>: HTTP status code is not handled or not allowed 2026-01-24 19:07:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:07:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/air_conditioners_chillers/window_air_conditioners>: HTTP status code is not handled or not allowed 2026-01-24 19:07:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:07:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1400-1625-specific-gravity-and-41-54-degree-baume-dual-scale-hydrometer>: HTTP status code is not handled or not allowed 2026-01-24 19:07:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-24 19:07:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:07:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:07:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 19:08:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wet-zone-shadow-board-blue-black-68-x-30-aluminum-sb116al>: HTTP status code is not handled or not allowed 2026-01-24 19:08:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-sided-bulk-truck-1-1-2-retaining-lip-3600-lbs-mesh-sides-24-x-48-deck>: HTTP status code is not handled or not allowed 2026-01-24 19:08:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock landed on page that is not a product page. 2026-01-24 19:08:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-shelf-truck-6-wheeled-3600-lbs-2-flush-shelves-24-x-48>: HTTP status code is not handled or not allowed 2026-01-24 19:08:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 11693 pages (at 92 pages/min), scraped 8108 items (at 68 items/min) 2026-01-24 19:08:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:08:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_storage_bench_protection/laboratory_bench_racks_stands already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:08:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/heaters/gas_heaters_furnaces/portable_gas_oil_kerosene_heaters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:08:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluke-116-hvac-multimeter-w-temperature-microamps>: HTTP status code is not handled or not allowed 2026-01-24 19:08:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers landed on page that is not a product page. 2026-01-24 19:08:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silver-depressed-center-grinding-wheel-9-x-1-4-x-5-8-11-t27-ceramic-grain-36-grit>: HTTP status code is not handled or not allowed 2026-01-24 19:08:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/laboratory_heating_cooling/cryogenic_storage already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:08:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bridges_returns already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:08:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/work-benches/workbenches_worktables/automotive_garage_workbenches/mobile_automotive_workbenches already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:08:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:08:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lube-adapter-kit-58000>: HTTP status code is not handled or not allowed 2026-01-24 19:08:18 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers landed on page that is not a product page. 2026-01-24 19:08:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_storage_bench_protection/laboratory_bench_racks_stands landed on page that is not a product page. 2026-01-24 19:08:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clamping-ring-for-8-glove-ports>: HTTP status code is not handled or not allowed 2026-01-24 19:08:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:08:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-shade-cloth-8-x-8>: HTTP status code is not handled or not allowed 2026-01-24 19:08:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/heaters/gas_heaters_furnaces/portable_gas_oil_kerosene_heaters landed on page that is not a product page. 2026-01-24 19:08:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/laboratory_heating_cooling/cryogenic_storage landed on page that is not a product page. 2026-01-24 19:08:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bridges_returns landed on page that is not a product page. 2026-01-24 19:08:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/captor-pen-holder-1-x-1-x-2-3pk>: HTTP status code is not handled or not allowed 2026-01-24 19:08:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/work-benches/workbenches_worktables/automotive_garage_workbenches/mobile_automotive_workbenches landed on page that is not a product page. 2026-01-24 19:08:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers landed on page that is not a product page. 2026-01-24 19:08:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:08:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-3-4-x-2-3-8-5-light-green-shipping-tag>: HTTP status code is not handled or not allowed 2026-01-24 19:08:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock landed on page that is not a product page. 2026-01-24 19:08:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:08:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-edge-2w-12l-1-ply-eye-eye-poly-web-sling-flat-eyes>: HTTP status code is not handled or not allowed 2026-01-24 19:08:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_batteries/snowmobile_batteries already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:08:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-rod-12-l-x-1-3-4-dia-white>: HTTP status code is not handled or not allowed 2026-01-24 19:08:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock landed on page that is not a product page. 2026-01-24 19:08:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-pack-sr-200>: HTTP status code is not handled or not allowed 2026-01-24 19:08:42 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock landed on page that is not a product page. 2026-01-24 19:08:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_batteries/snowmobile_batteries>: HTTP status code is not handled or not allowed 2026-01-24 19:08:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:08:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-72-x-30-adj-height-workbench-w-drawer-riser-black-maple-safety-edge-top>: HTTP status code is not handled or not allowed 2026-01-24 19:08:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-epoxy-wire-shelving-add-on-21305gz-30w-x-21d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 19:08:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-hdpe-bar-48-l-x-6-w-x-1-2-thick-black-marine>: HTTP status code is not handled or not allowed 2026-01-24 19:08:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers landed on page that is not a product page. 2026-01-24 19:08:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gear-oil-pump-mobile-system-basic-5-1>: HTTP status code is not handled or not allowed 2026-01-24 19:08:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-rod-48-l-x-1-3-4-dia-black>: HTTP status code is not handled or not allowed 2026-01-24 19:09:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:09:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buyers-steel-underbody-box-w-ss-door-black-18x18x48>: HTTP status code is not handled or not allowed 2026-01-24 19:09:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:09:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-3-8-16-x-6-threaded-rod-zinc-plated-carbon-steel>: HTTP status code is not handled or not allowed 2026-01-24 19:09:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tempco-finned-strip-240v-t4-14l-1100w>: HTTP status code is not handled or not allowed 2026-01-24 19:09:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 11776 pages (at 83 pages/min), scraped 8160 items (at 52 items/min) 2026-01-24 19:09:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6x9-3-milslide-seal-reclosable-poly-bags>: HTTP status code is not handled or not allowed 2026-01-24 19:09:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:09:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 19:09:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock>: HTTP status code is not handled or not allowed 2026-01-24 19:09:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:09:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-ptfe-bar-24-l-x-2-w-x-1-2-thick-off-white-glass-filled>: HTTP status code is not handled or not allowed 2026-01-24 19:09:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alumi-blast-aluminum-coating-16-oz-6-cans-case-16-055>: HTTP status code is not handled or not allowed 2026-01-24 19:09:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:09:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-24 19:09:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 19:09:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:09:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hon5701ga10t-volt-armless-task-chair-black-polyester>: HTTP status code is not handled or not allowed 2026-01-24 19:09:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pps-mini-size-liners-16348-100-box>: HTTP status code is not handled or not allowed 2026-01-24 19:09:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kendall-howard-27u-standard-19-vented-rear-door-glass-front-door>: HTTP status code is not handled or not allowed 2026-01-24 19:09:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 19:09:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-48-l-x-4-w-x-1-3-4-thick-white>: HTTP status code is not handled or not allowed 2026-01-24 19:09:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 19:09:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kendall-howard-9u-fixed-wall-mount-cabinet>: HTTP status code is not handled or not allowed 2026-01-24 19:09:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyethylene-jerrican-with-spigot-20-liters-5-gallons-screw-cap-3-4-id-spout>: HTTP status code is not handled or not allowed 2026-01-24 19:09:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:09:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plus-astm-like-liquid-in-glass-thermometer-90c-solidification-point-0-to-30c>: HTTP status code is not handled or not allowed 2026-01-24 19:09:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reach-in-refrigerator-horizon-solid-half-solid-doors-52w-hrs2-1s>: HTTP status code is not handled or not allowed 2026-01-24 19:09:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-36-l-x-4-w-x-3-8-thick-black>: HTTP status code is not handled or not allowed 2026-01-24 19:09:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-24 19:09:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_storage_bench_protection/laboratory_bench_racks_stands already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:09:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-36-l-x-5-w-x-3-thick-white>: HTTP status code is not handled or not allowed 2026-01-24 19:09:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/painting_supplies/spray_paint_primer already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:09:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gac-70f-spherical-plain-thrust-bearing-angular-contact-metric>: HTTP status code is not handled or not allowed 2026-01-24 19:09:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:09:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/food_beverages/emergency_food_drink already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:09:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/food_beverages/emergency_food_drink already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:09:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/05-901-outdoor-chili-mac-with-beef-2-servings-pouch-6-pack>: HTTP status code is not handled or not allowed 2026-01-24 19:09:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_storage_bench_protection/laboratory_bench_racks_stands landed on page that is not a product page. 2026-01-24 19:09:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:09:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gx-12f-spherical-plain-thrust-bearing-metric-high-capacity>: HTTP status code is not handled or not allowed 2026-01-24 19:10:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/painting_supplies/spray_paint_primer landed on page that is not a product page. 2026-01-24 19:10:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-24 19:10:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/food_beverages/emergency_food_drink landed on page that is not a product page. 2026-01-24 19:10:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/food_beverages/emergency_food_drink landed on page that is not a product page. 2026-01-24 19:10:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-loop-plastic-pegboard-slatwall-hook-4l-clear>: HTTP status code is not handled or not allowed 2026-01-24 19:10:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:10:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 19:10:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-5-tier-54w-x-14d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 19:10:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 11864 pages (at 88 pages/min), scraped 8217 items (at 57 items/min) 2026-01-24 19:10:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1500-watt-commercial-ceiling-heater-14-3-16-l-120v>: HTTP status code is not handled or not allowed 2026-01-24 19:10:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_desiccators already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:10:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-disposable-plastic-pestle-3-1-2-100pk>: HTTP status code is not handled or not allowed 2026-01-24 19:10:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers landed on page that is not a product page. 2026-01-24 19:10:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/geg-70es-spherical-plain-bearing-metric-extended-inner-ring>: HTTP status code is not handled or not allowed 2026-01-24 19:10:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-54w-x-24d-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 19:10:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-72w-x-30d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 19:10:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/lockers/bulk_storage_lockers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:10:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_desiccators landed on page that is not a product page. 2026-01-24 19:10:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-general-purpose-polyethylene-tray-with-faucet-17-1-2-x-23-1-2-x-6>: HTTP status code is not handled or not allowed 2026-01-24 19:10:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-ptfe-rod-12-l-x-3-1-2-dia-off-white-glass-filled>: HTTP status code is not handled or not allowed 2026-01-24 19:10:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-general-purpose-polyethylene-tray-without-faucet-18-x-22-x-4>: HTTP status code is not handled or not allowed 2026-01-24 19:10:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-bronze-series-pl-75b-pump-1-6hp-115v-1-60>: HTTP status code is not handled or not allowed 2026-01-24 19:10:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:10:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:10:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/six-unit-charger-adapter-for-ksc-35>: HTTP status code is not handled or not allowed 2026-01-24 19:10:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-48w-x-14d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 19:10:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/lockers/bulk_storage_lockers landed on page that is not a product page. 2026-01-24 19:10:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galvanized-steel-outdoor-storage-shed-9-2-w-x-6-1-d-x-7-1-h>: HTTP status code is not handled or not allowed 2026-01-24 19:10:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spinbar-teflon-polygon-magnetic-stirring-bar-2-x-3-8-white-without-pivot-ring>: HTTP status code is not handled or not allowed 2026-01-24 19:10:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-straight-arm-chrome>: HTTP status code is not handled or not allowed 2026-01-24 19:10:42 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 19:10:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:10:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hytrol-stationary-floor-support-24w-max-leg-45-5-8h>: HTTP status code is not handled or not allowed 2026-01-24 19:10:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/laboratory_heating_cooling/laboratory_hotplates_stirrers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:10:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jet-electric-chain-hoist-1-2-ton-15-li-1-phase-115-230v>: HTTP status code is not handled or not allowed 2026-01-24 19:10:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-epoxy-wire-shelving-14727b-72w-x-14d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 19:10:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-5-tier-24w-x-24d-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 19:10:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:10:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-rectangular-clear-strobe-light-4-leds-8891104>: HTTP status code is not handled or not allowed 2026-01-24 19:10:50 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock landed on page that is not a product page. 2026-01-24 19:10:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spinplus-teflon-magnetic-stirring-bar-191-x-95mm-white>: HTTP status code is not handled or not allowed 2026-01-24 19:10:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/laboratory_heating_cooling/laboratory_hotplates_stirrers>: HTTP status code is not handled or not allowed 2026-01-24 19:10:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-24 19:10:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/recyclockout-lockout-cabinet-empty-6062e>: HTTP status code is not handled or not allowed 2026-01-24 19:10:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:10:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:10:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 19:10:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/recyclockout-cable-hasp-lockout-device-10-ft-7133>: HTTP status code is not handled or not allowed 2026-01-24 19:11:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-double-tier-locker-adder-1-wide-12w-x-15d-x-36h-unassembled-gray>: HTTP status code is not handled or not allowed 2026-01-24 19:11:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:11:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:11:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/201002-klp-guard-locking-switchstd-release-12npt-pe>: HTTP status code is not handled or not allowed 2026-01-24 19:11:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 19:11:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock>: HTTP status code is not handled or not allowed 2026-01-24 19:11:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 11948 pages (at 84 pages/min), scraped 8267 items (at 50 items/min) 2026-01-24 19:11:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-three-way-fluoroplastic-stopcock-for-6mm-tubing-4mm-bore>: HTTP status code is not handled or not allowed 2026-01-24 19:11:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 19:11:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 19:11:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-x-48-melamine-shelf-maple>: HTTP status code is not handled or not allowed 2026-01-24 19:11:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-polypropylene-sterilizing-tray-12-x-8-x-5>: HTTP status code is not handled or not allowed 2026-01-24 19:11:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1w-x-5h-grid-panel-semi-gloss-white>: HTTP status code is not handled or not allowed 2026-01-24 19:11:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:11:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/material-handling/ladders_personnel_lifts/step_ladders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:11:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/max-attach-polymagnet-rare-earth-disc-w-adhesive-1-00-dia-x-0-06-thick>: HTTP status code is not handled or not allowed 2026-01-24 19:11:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_label_tape already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:11:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-epdm-roll-720-l-x-12-w-x-3-32-thick-black-60a>: HTTP status code is not handled or not allowed 2026-01-24 19:11:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/statfree-t2-plus-8482-66465-dissipative-rubber-tray-liner-16-d-x-24-w-dark-blue>: HTTP status code is not handled or not allowed 2026-01-24 19:11:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 19:11:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/material-handling/ladders_personnel_lifts/step_ladders landed on page that is not a product page. 2026-01-24 19:11:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-style-7l-shoe-riser-clear>: HTTP status code is not handled or not allowed 2026-01-24 19:11:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_label_tape landed on page that is not a product page. 2026-01-24 19:11:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6l-round-tubing-w-1-1-4-dia-chrome>: HTTP status code is not handled or not allowed 2026-01-24 19:11:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/retail_fixtures_displays_equipment/garment_supplies/hangers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:11:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cm-2019-60-bamboo-condiment-station-15-inch-w-x-4-3-4-inch-d-x-19-3-4-inch-h>: HTTP status code is not handled or not allowed 2026-01-24 19:11:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-neoprene-sheet-36-l-x-36-w-x-3-16-thick-black-high-strength-90a>: HTTP status code is not handled or not allowed 2026-01-24 19:11:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_alarms_signals/strobe_flashing_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:11:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fx9n-refrigerant-air-dryer-1-phase-230v-22-cfm-3-4-npt>: HTTP status code is not handled or not allowed 2026-01-24 19:11:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/retail_fixtures_displays_equipment/displays_display_racks/countertop_displays already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:11:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:11:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/desk_accessories/bookends already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:11:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/osc-170-oil-water-separator-360-cfm-10-ppm-3-4-npt-outlet-organoclay>: HTTP status code is not handled or not allowed 2026-01-24 19:11:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/retail_fixtures_displays_equipment/garment_supplies/hangers landed on page that is not a product page. 2026-01-24 19:11:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/die_cut_labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:11:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-chm364a-5-hp-1140-rpm>: HTTP status code is not handled or not allowed 2026-01-24 19:11:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_alarms_signals/strobe_flashing_lights landed on page that is not a product page. 2026-01-24 19:11:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/174-nyl-red-38pl-ka3pk-safekey-lockout-padlock-nylon-1-5-plastic-shackle-keyed-alike>: HTTP status code is not handled or not allowed 2026-01-24 19:11:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/retail_fixtures_displays_equipment/displays_display_racks/countertop_displays landed on page that is not a product page. 2026-01-24 19:11:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/174-150740-inspection-timer-7-day-10-pack-led-lights-abs-plastic-blue>: HTTP status code is not handled or not allowed 2026-01-24 19:11:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-24 19:11:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-hinged-top-basket-truck-10-bu-tan-vinyl-wood-base-all-swivel>: HTTP status code is not handled or not allowed 2026-01-24 19:11:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:12:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-270fb1248d-incandescentled-bulb-module-blue-12-48v-dc>: HTTP status code is not handled or not allowed 2026-01-24 19:12:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/desk_accessories/bookends landed on page that is not a product page. 2026-01-24 19:12:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-270fg1248d-incandescentled-bulb-module-green-12-48v-dc>: HTTP status code is not handled or not allowed 2026-01-24 19:12:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acid-brushes-0>: HTTP status code is not handled or not allowed 2026-01-24 19:12:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:12:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/no-hub-sealant-1-qt>: HTTP status code is not handled or not allowed 2026-01-24 19:12:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 12038 pages (at 90 pages/min), scraped 8325 items (at 58 items/min) 2026-01-24 19:12:09 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels landed on page that is not a product page. 2026-01-24 19:12:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-270sr12240ad-incandescentled-bulb-module-red-12-240v-acdc>: HTTP status code is not handled or not allowed 2026-01-24 19:12:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-piece-hook-pick-set>: HTTP status code is not handled or not allowed 2026-01-24 19:12:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-270ledg12v-led-bulb-green-12v>: HTTP status code is not handled or not allowed 2026-01-24 19:12:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs landed on page that is not a product page. 2026-01-24 19:12:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_alarms_signals/strobe_flashing_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:12:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-coated-urethane-foam-tape-4016-3-4-x-15-yds-3-mil-off-white>: HTTP status code is not handled or not allowed 2026-01-24 19:12:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-general-purpose-single-phase-odp-motor-3-4-hp-1725-rpm-115-208-230v-odp-56-frame>: HTTP status code is not handled or not allowed 2026-01-24 19:12:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/hand_tools_1/pliers/tongue_groove_pliers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:12:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ecp4100t-5-15-hp-1180-rpm>: HTTP status code is not handled or not allowed 2026-01-24 19:12:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastek-conductive-fluted-sheet-3mm-x-48-x-48-black>: HTTP status code is not handled or not allowed 2026-01-24 19:12:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/work-gloves-15-gauge-gray-nylon-shell-black-hpt-palm-and-fingertips-xs>: HTTP status code is not handled or not allowed 2026-01-24 19:12:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/hand_tools_1/pliers/tongue_groove_pliers>: HTTP status code is not handled or not allowed 2026-01-24 19:12:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_alarms_signals/strobe_flashing_lights landed on page that is not a product page. 2026-01-24 19:12:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-log-book-for-propane-counterbalance-forklift-log-986884>: HTTP status code is not handled or not allowed 2026-01-24 19:12:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-single-phase-tefc-motor-7-1-2-hp-3465-rpm-208-230v-tefc>: HTTP status code is not handled or not allowed 2026-01-24 19:12:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pt-series-heavy-duty-manual-trolley-2200-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-24 19:12:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-rivet-boltless-shelving-48-inch-w-x-30-inch-d-x-84-inch-h-8-levels-add-on-w-o-decking-tan>: HTTP status code is not handled or not allowed 2026-01-24 19:12:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/photo_presentation_paper already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:12:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b-60-charging-hose>: HTTP status code is not handled or not allowed 2026-01-24 19:12:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/photo_presentation_paper already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:12:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-straight-male-5-32-tube-x-1-4-nptf-1-l>: HTTP status code is not handled or not allowed 2026-01-24 19:12:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/photo_presentation_paper already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:12:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/combination-cable-lock-60l-no-8109d>: HTTP status code is not handled or not allowed 2026-01-24 19:12:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/photo_presentation_paper already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:12:56 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/paper_stationery/photo_presentation_paper landed on page that is not a product page. 2026-01-24 19:12:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-anfl3514m-1-5-hp-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-24 19:12:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/paper_stationery/photo_presentation_paper landed on page that is not a product page. 2026-01-24 19:13:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/men-indus-wrk-shirt-shrt-slv-lght-blue-l-sp24>: HTTP status code is not handled or not allowed 2026-01-24 19:13:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-bulkhead-1-4-tube-1-3-8-l>: HTTP status code is not handled or not allowed 2026-01-24 19:13:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/paper_stationery/photo_presentation_paper>: HTTP status code is not handled or not allowed 2026-01-24 19:13:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-rivet-boltless-shelving-36-inch-w-x-36-inch-d-x-84-inch-h-5-levels-add-on-w-o-decking-tan>: HTTP status code is not handled or not allowed 2026-01-24 19:13:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oem-replacement-motor-1-6-hp-1110-rpm-208-230v-teao>: HTTP status code is not handled or not allowed 2026-01-24 19:13:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/paper_stationery/photo_presentation_paper landed on page that is not a product page. 2026-01-24 19:13:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-broom-cabinet-60-w-x-24-d-x-75-h-beige>: HTTP status code is not handled or not allowed 2026-01-24 19:13:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 12129 pages (at 91 pages/min), scraped 8388 items (at 63 items/min) 2026-01-24 19:13:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/definite_purpose_electric_motors/oem_replacement_ac_motors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:13:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-uniform-cabinet-with-4-shelves-36-w-x-24-d-x-75-h-red>: HTTP status code is not handled or not allowed 2026-01-24 19:13:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-swivel-branch-tee-5-32-tube-x-10-32-ml-1-2-7-l>: HTTP status code is not handled or not allowed 2026-01-24 19:13:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oem-replacement-motor-1-8-hp-700-rpm-115v-oao>: HTTP status code is not handled or not allowed 2026-01-24 19:13:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cecp84314t-4-60-hp-1780-rpm>: HTTP status code is not handled or not allowed 2026-01-24 19:13:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-ball-valve-female-1-4-nptf-1-4-5-l>: HTTP status code is not handled or not allowed 2026-01-24 19:13:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-polymer-shelf-with-solid-mat-42-w-x-21-d>: HTTP status code is not handled or not allowed 2026-01-24 19:13:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-l1408t-50-3-hp-1425-rpm>: HTTP status code is not handled or not allowed 2026-01-24 19:13:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/definite_purpose_electric_motors/oem_replacement_ac_motors landed on page that is not a product page. 2026-01-24 19:13:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cable-50-coil>: HTTP status code is not handled or not allowed 2026-01-24 19:13:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-1-4-aro-plug-x-3-8-barb-1-1-2-l>: HTTP status code is not handled or not allowed 2026-01-24 19:13:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-cabinet-with-6-drawers-48-w-x-24-d-x-75-h-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 19:13:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-reducer-1-2-ml-stud-x-3-8-tube-1-2-3-l>: HTTP status code is not handled or not allowed 2026-01-24 19:13:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/replacement_shelves_decks/boltless_shelving_additional_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:13:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/replacement_shelves_decks/boltless_shelving_additional_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:13:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/replacement_shelves_decks/boltless_shelving_additional_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:13:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cabinet-with-electronic-lock-and-card-reader-60-w-x-24-d-x-75-h-red>: HTTP status code is not handled or not allowed 2026-01-24 19:13:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v-grip-pre-configured-wire-shelving-spill-containment-system-48-w-x-18-d-x-84-h-starter-silver>: HTTP status code is not handled or not allowed 2026-01-24 19:13:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-canvas-functional-cargo-pant-black-wp80-30x32-wp80bk3032>: HTTP status code is not handled or not allowed 2026-01-24 19:13:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/replacement_shelves_decks/boltless_shelving_additional_shelves landed on page that is not a product page. 2026-01-24 19:13:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/shelving/replacement_shelves_decks/boltless_shelving_additional_shelves>: HTTP status code is not handled or not allowed 2026-01-24 19:13:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/replacement_shelves_decks/boltless_shelving_additional_shelves landed on page that is not a product page. 2026-01-24 19:13:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/maxiscour-extra-heavy-duty-scouring-pad-40-case-93088-m>: HTTP status code is not handled or not allowed 2026-01-24 19:13:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v-grip-pre-configured-wire-shelving-spill-containment-system-48-w-x-24-d-x-84-h-add-on-blue>: HTTP status code is not handled or not allowed 2026-01-24 19:13:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-x-36-reflective-roll-up-vinyl-sign-transition-symbol-left>: HTTP status code is not handled or not allowed 2026-01-24 19:13:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disinfecting-spray>: HTTP status code is not handled or not allowed 2026-01-24 19:13:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/appliances/cooking_equipment/ovens already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:13:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-union-5mm-tube-7-9-l>: HTTP status code is not handled or not allowed 2026-01-24 19:13:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:13:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medium-rigid-heavyweight-plastic-name-tent-holder-2-1-2-x-8-1-2-clear-25-bx>: HTTP status code is not handled or not allowed 2026-01-24 19:13:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-flow-control-8mm-tube-x-3-8-swift-fit-universal-thread-1-l>: HTTP status code is not handled or not allowed 2026-01-24 19:13:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/appliances/cooking_equipment/ovens landed on page that is not a product page. 2026-01-24 19:14:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-em4102t-20-hp-1180-rpm>: HTTP status code is not handled or not allowed 2026-01-24 19:14:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:14:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-rt-angle-ball-valve-3-8-nptf-1-1-2-l>: HTTP status code is not handled or not allowed 2026-01-24 19:14:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits landed on page that is not a product page. 2026-01-24 19:14:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/badge-holder-adaptor-w-bulldog-clip-25-pk>: HTTP status code is not handled or not allowed 2026-01-24 19:14:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 12218 pages (at 89 pages/min), scraped 8447 items (at 59 items/min) 2026-01-24 19:14:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-gsl3509m-1-5-hp-1750-rpm>: HTTP status code is not handled or not allowed 2026-01-24 19:14:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits landed on page that is not a product page. 2026-01-24 19:14:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-track-steel-floor-conveyor-2-roller-lanes-8-l-8000-lbs>: HTTP status code is not handled or not allowed 2026-01-24 19:14:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/recycling-cube-truck-box-truck-14-bushel>: HTTP status code is not handled or not allowed 2026-01-24 19:14:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_lockout_identification_tags already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:14:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/hvac_controls/pressure_controls_switches already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:14:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/abc2bfs-2-lb-non-sparking-brass-hammer-8-fiberglass-handle>: HTTP status code is not handled or not allowed 2026-01-24 19:14:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/clips_fasteners/push_pins already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:14:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interlake-mecalux-pallet-rack-tear-drop-starter-108-w-x-48-d-x-144-h>: HTTP status code is not handled or not allowed 2026-01-24 19:14:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:14:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interlake-mecalux-pallet-rack-tear-drop-starter-120-w-x-48-d-x-120-h>: HTTP status code is not handled or not allowed 2026-01-24 19:14:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_lockout_identification_tags landed on page that is not a product page. 2026-01-24 19:14:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/44-1-2-h-smart-grid-wall-mount-upright-light-green-proform>: HTTP status code is not handled or not allowed 2026-01-24 19:14:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:14:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/basic_supplies/clips_fasteners/push_pins>: HTTP status code is not handled or not allowed 2026-01-24 19:14:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/hvac_controls/pressure_controls_switches landed on page that is not a product page. 2026-01-24 19:14:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/first-aid-burn-spray-4-oz-pump>: HTTP status code is not handled or not allowed 2026-01-24 19:14:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/ink_toner/laser_toner_cartridges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:14:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats landed on page that is not a product page. 2026-01-24 19:14:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-gas-heater-infrared-positive-pressure-u-tube-tough-guy-175000-btu-40l>: HTTP status code is not handled or not allowed 2026-01-24 19:14:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_labels landed on page that is not a product page. 2026-01-24 19:14:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12920-carboy-hdpe-5-liter>: HTTP status code is not handled or not allowed 2026-01-24 19:14:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultradrain-guard-heavy-metal-model>: HTTP status code is not handled or not allowed 2026-01-24 19:14:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultragutter-guard-9-dia-x-10-l>: HTTP status code is not handled or not allowed 2026-01-24 19:14:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forklift-fan-magnetic-mount-4-5-inch-dc12v-2-speed>: HTTP status code is not handled or not allowed 2026-01-24 19:14:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-x-3-x-3-corrugated-boxes-200lb-test-ect-32>: HTTP status code is not handled or not allowed 2026-01-24 19:14:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/ink_toner/laser_toner_cartridges landed on page that is not a product page. 2026-01-24 19:14:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/big-d-handheld-aerosol-room-deodorant-sunburst>: HTTP status code is not handled or not allowed 2026-01-24 19:14:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/coat_racks_umbrella_stands/coat_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:14:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-lobby-pro-upright-dust-pan-2>: HTTP status code is not handled or not allowed 2026-01-24 19:15:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oil-safe-stumpy-pour-spout-lid-dark-green>: HTTP status code is not handled or not allowed 2026-01-24 19:15:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precoiled-strap-on-pipe-marker-chilled-water-supply-3>: HTTP status code is not handled or not allowed 2026-01-24 19:15:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ejmm2394t-15-hp-3520-rpm>: HTTP status code is not handled or not allowed 2026-01-24 19:15:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/plumbing_parts_accessories/pipe_hose_tubing_accessories/pipe_markers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:15:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/coat_racks_umbrella_stands/coat_racks>: HTTP status code is not handled or not allowed 2026-01-24 19:15:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 12307 pages (at 89 pages/min), scraped 8509 items (at 62 items/min) 2026-01-24 19:15:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pressure-sensitive-pipe-marker-high-pressure-steam-3>: HTTP status code is not handled or not allowed 2026-01-24 19:15:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/door-floor-stop-dome-type-1-5-16-tall-brushed-chrome>: HTTP status code is not handled or not allowed 2026-01-24 19:15:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/plumbing/plumbing_parts_accessories/pipe_hose_tubing_accessories/pipe_markers landed on page that is not a product page. 2026-01-24 19:15:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/400-coverall-zipper-front-elastic-wrist-ankle-stormflap-white-md>: HTTP status code is not handled or not allowed 2026-01-24 19:15:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hemco-scofflaw-fume-hood-48-x-23-x-36>: HTTP status code is not handled or not allowed 2026-01-24 19:15:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precoiled-strap-on-pipe-marker-chilled-water-supply-8>: HTTP status code is not handled or not allowed 2026-01-24 19:15:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-gauge-7-16-construction-stapler>: HTTP status code is not handled or not allowed 2026-01-24 19:15:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-mvm3458c-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-24 19:15:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fellowes-office-suites-monitor-riser>: HTTP status code is not handled or not allowed 2026-01-24 19:15:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/201-bedspread-and-drapery-hanger-19l-metal-chrome-pkg-qty-12>: HTTP status code is not handled or not allowed 2026-01-24 19:15:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ucm713-7-5-10airover-hp-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-24 19:15:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/privacy-screen-designer-cloth-screen-panel-frost>: HTTP status code is not handled or not allowed 2026-01-24 19:15:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/phillips-3-power-bit-x-3-1-2-s2-modified-steel-gunmetal-grey>: HTTP status code is not handled or not allowed 2026-01-24 19:15:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/first_aid_wound_treatment/first_aid_kits/first_aid_kits_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:15:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-x-3-seamless-pipe-nipple-schedule-80-316-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 19:15:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electrical-rated-gloves-black-14-unlined-smooth-finish-beaded-class-0-11>: HTTP status code is not handled or not allowed 2026-01-24 19:15:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torx-174-30-insert-bit-x-1-1-4-5-16-hex-shank-s2-modified-steel-gunmetal-grey>: HTTP status code is not handled or not allowed 2026-01-24 19:15:56 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/first_aid_wound_treatment/first_aid_kits/first_aid_kits_cabinets landed on page that is not a product page. 2026-01-24 19:15:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bridges_returns already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:15:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-vinyl-sign-caution-this-door-must-be-kept-closed-1>: HTTP status code is not handled or not allowed 2026-01-24 19:15:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bridges_returns already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:16:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lever-hoist-with-overload-protection-1-1-2-ton-capacity-20-lift>: HTTP status code is not handled or not allowed 2026-01-24 19:16:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/top-grain-goatskin-leather-nylon-protector-for-novax-gloves-orange-white-size-9>: HTTP status code is not handled or not allowed 2026-01-24 19:16:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2004-w-eglr-6-all-plastic-crowd-control-barricade-w-rubber-feet-interlocking-white>: HTTP status code is not handled or not allowed 2026-01-24 19:16:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bridges_returns landed on page that is not a product page. 2026-01-24 19:16:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 12396 pages (at 89 pages/min), scraped 8576 items (at 67 items/min) 2026-01-24 19:16:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mounted-pipe-guard-46-inch-tall-safety-yellow-protects-10-inch-pipe>: HTTP status code is not handled or not allowed 2026-01-24 19:16:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:16:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-plastic-sign-caution-ear-protection-required-this-area-3>: HTTP status code is not handled or not allowed 2026-01-24 19:16:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bridges_returns landed on page that is not a product page. 2026-01-24 19:16:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-plastic-sign-caution-advise-supervisor-if-equipment-does-not-run-1>: HTTP status code is not handled or not allowed 2026-01-24 19:16:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10x42mm-full-sized-waterproof-binoculars>: HTTP status code is not handled or not allowed 2026-01-24 19:16:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs landed on page that is not a product page. 2026-01-24 19:16:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oil-enamel-gloss-black-gallon-pail-1-case-31100-1>: HTTP status code is not handled or not allowed 2026-01-24 19:16:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:16:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/little-giant-wwc3048-hd6phfl-mobile-heavy-duty-cabinet-workbench-steel-top>: HTTP status code is not handled or not allowed 2026-01-24 19:16:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-foot-patio-umbrella-olefin-hunter-green-silver-pole-venture>: HTTP status code is not handled or not allowed 2026-01-24 19:16:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-foot-patio-umbrella-olefin-terracotta-hardwood-pole-grove>: HTTP status code is not handled or not allowed 2026-01-24 19:16:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ip22-blue-smart-battery-charger-w-bluetooth-12v-30a-1-120v-nema-5-15-abs-plastic>: HTTP status code is not handled or not allowed 2026-01-24 19:16:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs>: HTTP status code is not handled or not allowed 2026-01-24 19:16:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buck-neurological-hammer-7-1-2-l-black-and-chrome>: HTTP status code is not handled or not allowed 2026-01-24 19:16:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_components/pallet_rack_hardware_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:16:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rcb-25-rebar-cutter-bender-5-8-capacity>: HTTP status code is not handled or not allowed 2026-01-24 19:16:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gsm50-heavy-duty-f-clamp-20>: HTTP status code is not handled or not allowed 2026-01-24 19:16:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelf-truck-esd-21w-x-42l-x-80h-polyurethane-4-swivel-2-brake-casters>: HTTP status code is not handled or not allowed 2026-01-24 19:16:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leviton-80455-w-2-gang-1-duplex-1-decora-gfci-device-combo-standard-white>: HTTP status code is not handled or not allowed 2026-01-24 19:16:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xl-flexbollard-52h-adhesive-yellowred-xl-b52-d-yellowred>: HTTP status code is not handled or not allowed 2026-01-24 19:16:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_components/pallet_rack_hardware_accessories landed on page that is not a product page. 2026-01-24 19:16:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oty-32163-pvc-blue-lava-hot-cement-32-oz>: HTTP status code is not handled or not allowed 2026-01-24 19:16:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:16:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-battery-cable-with-3-8-ring-terminals-2-0-awg-6-ft-black>: HTTP status code is not handled or not allowed 2026-01-24 19:16:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gx-touch-50-adapter-for-ccgx-cut-out-blue-abs-plastic>: HTTP status code is not handled or not allowed 2026-01-24 19:16:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:16:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oty-31128-cpvc-medium-orange-cement-4-oz>: HTTP status code is not handled or not allowed 2026-01-24 19:17:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triple-glove-box-dispenser-10-1-2-w-x-7-d-x-10-3-8-h-transparent>: HTTP status code is not handled or not allowed 2026-01-24 19:17:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs landed on page that is not a product page. 2026-01-24 19:17:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:17:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/custom-cut-drawer-liner-roll-black-foam>: HTTP status code is not handled or not allowed 2026-01-24 19:17:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs landed on page that is not a product page. 2026-01-24 19:17:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22-inch-x-14-inch-x-8-inch-corrugated-boxes>: HTTP status code is not handled or not allowed 2026-01-24 19:17:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milwaukee-m12-fuel-8482-cordless-3-8-high-speed-ratchet-tool-only>: HTTP status code is not handled or not allowed 2026-01-24 19:17:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 12490 pages (at 94 pages/min), scraped 8641 items (at 65 items/min) 2026-01-24 19:17:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs landed on page that is not a product page. 2026-01-24 19:17:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-sided-stock-curtain-package-w-hardware-24w-12w-24w-x-10h-white-clear-blue>: HTTP status code is not handled or not allowed 2026-01-24 19:17:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tri-cut-straight-blade-hand-pruning-saw-wood-handle-11-inch-3-pack>: HTTP status code is not handled or not allowed 2026-01-24 19:17:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stock-curtain-w-hardware-24w-x-10h-white-clear-blue>: HTTP status code is not handled or not allowed 2026-01-24 19:17:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mw-40205-ssp-safety-step-platform-40w-x-20d-x-5h>: HTTP status code is not handled or not allowed 2026-01-24 19:17:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/dunnage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:17:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/four-deck-tower-36-inch>: HTTP status code is not handled or not allowed 2026-01-24 19:17:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-series-unit-heater-208v-1-3-phase-5kw-24-amps>: HTTP status code is not handled or not allowed 2026-01-24 19:17:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-20x2-1-4-carriage-bolt-flat-head-1436CF>: HTTP status code is not handled or not allowed 2026-01-24 19:17:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8x2-1-2-flat-head-wood-screws-with-17-point-0840DPF17>: HTTP status code is not handled or not allowed 2026-01-24 19:17:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/individual-character-stencil-36-letter-v>: HTTP status code is not handled or not allowed 2026-01-24 19:17:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/dunnage_racks landed on page that is not a product page. 2026-01-24 19:17:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brown-viton-o-ring-dash-026-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-24 19:17:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pleated-air-filter-merv-8-self-supported-20-w-x20-hx2-d>: HTTP status code is not handled or not allowed 2026-01-24 19:17:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/overhead_clearance_bars_door_guards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:17:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hard-viton-o-ring-dash-230-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 19:17:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8x1-2-modified-truss-head-self-drilling-screws-0808KPMB>: HTTP status code is not handled or not allowed 2026-01-24 19:17:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:17:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wd1306-wheel-barrow-tire-size-530-450-6>: HTTP status code is not handled or not allowed 2026-01-24 19:17:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:17:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brown-viton-o-ring-dash-112-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-24 19:17:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/overhead_clearance_bars_door_guards landed on page that is not a product page. 2026-01-24 19:17:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lennox-tufted-upholstered-full-size-headboard-in-white-vinyl>: HTTP status code is not handled or not allowed 2026-01-24 19:17:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/lathes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:17:50 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs landed on page that is not a product page. 2026-01-24 19:17:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tribeca-twin-size-tufted-upholstered-platform-bed-in-beige>: HTTP status code is not handled or not allowed 2026-01-24 19:17:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/photo_presentation_paper already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:17:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs>: HTTP status code is not handled or not allowed 2026-01-24 19:17:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:17:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chemical-resistant-high-temperature-viton-tubing-14id-x-38od-x-25-ft>: HTTP status code is not handled or not allowed 2026-01-24 19:17:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/lathes landed on page that is not a product page. 2026-01-24 19:18:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ingersoll-rand-activated-carbon-filter-85-cfm>: HTTP status code is not handled or not allowed 2026-01-24 19:18:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/paper_stationery/photo_presentation_paper landed on page that is not a product page. 2026-01-24 19:18:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/riverdale-king-tufted-upholstered-platform-bed-beige-pocket-spring-mattress>: HTTP status code is not handled or not allowed 2026-01-24 19:18:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/55-33-16-5-ton-air-hydraulic-truck-jack-6855>: HTTP status code is not handled or not allowed 2026-01-24 19:18:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards landed on page that is not a product page. 2026-01-24 19:18:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/resilient-anti-slip-tape-gray-6-w-case-of-50-24-l>: HTTP status code is not handled or not allowed 2026-01-24 19:18:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 12579 pages (at 89 pages/min), scraped 8700 items (at 59 items/min) 2026-01-24 19:18:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:18:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stacking-cannabis-container-23-3-8-x-12-x-4-3-8-white>: HTTP status code is not handled or not allowed 2026-01-24 19:18:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:18:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/printed-barricade-tape-caution-chemical-spill-2>: HTTP status code is not handled or not allowed 2026-01-24 19:18:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/snow_ice_removal/ice_melt_rock_salt already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:18:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/snow_ice_removal/ice_melt_rock_salt already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:18:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dp200-36-round-traffic-channelizer-post-white>: HTTP status code is not handled or not allowed 2026-01-24 19:18:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/stamps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:18:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sealless-steel-strapping-tool>: HTTP status code is not handled or not allowed 2026-01-24 19:18:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards landed on page that is not a product page. 2026-01-24 19:18:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/heaters/stoves_fireplaces_log_sets_fire_pits/fireplaces_log_sets_fireboxes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:18:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges>: HTTP status code is not handled or not allowed 2026-01-24 19:18:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/snow_ice_removal/ice_melt_rock_salt landed on page that is not a product page. 2026-01-24 19:18:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-25-11112-300-psikpa-14-cbm-plastic>: HTTP status code is not handled or not allowed 2026-01-24 19:18:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:18:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/snow_ice_removal/ice_melt_rock_salt landed on page that is not a product page. 2026-01-24 19:18:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dividers-for-3-h-drawer-of-modular-drawer-cabinet-36-wx24-d-black>: HTTP status code is not handled or not allowed 2026-01-24 19:18:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/stamps landed on page that is not a product page. 2026-01-24 19:18:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/heaters/stoves_fireplaces_log_sets_fire_pits/fireplaces_log_sets_fireboxes landed on page that is not a product page. 2026-01-24 19:18:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3d-glow-sign-plastic-18x4-fire-alarm>: HTTP status code is not handled or not allowed 2026-01-24 19:18:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/floor_machine_accessories_supplies/vacuum_bags already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:18:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-silicone-foam-with-high-temp-adhesive-18-thick-x-2w-x-8l>: HTTP status code is not handled or not allowed 2026-01-24 19:18:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/headlamps_wearable_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:18:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges landed on page that is not a product page. 2026-01-24 19:18:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hirsh-industries-26in-deep-vertical-file-cabinet-2-drawer-letter-size-putty>: HTTP status code is not handled or not allowed 2026-01-24 19:18:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/print-or-write-removable-color-coding-labels-3-4-dia-yellow-1008-pack-05462>: HTTP status code is not handled or not allowed 2026-01-24 19:18:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sharp-mechanical-pencil-0-9-mm-hb-2-5-black-lead-yellow-barrel-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 19:18:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/automotive-air-condition-flush-1-gallon-4-can>: HTTP status code is not handled or not allowed 2026-01-24 19:18:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/floor_machine_accessories_supplies/vacuum_bags landed on page that is not a product page. 2026-01-24 19:18:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/address_shipping_labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:18:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-side-brush-49-auto-ride-on-sweeper>: HTTP status code is not handled or not allowed 2026-01-24 19:18:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/pencils already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:18:50 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/headlamps_wearable_lights landed on page that is not a product page. 2026-01-24 19:18:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/st1511towerload-15-string-trimmer-kit-w-g3-2-5ah-battery-210w-charger>: HTTP status code is not handled or not allowed 2026-01-24 19:18:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glow-sign-vinyl-fire-extinguisher-graphic>: HTTP status code is not handled or not allowed 2026-01-24 19:18:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/address_shipping_labels>: HTTP status code is not handled or not allowed 2026-01-24 19:18:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/pencils landed on page that is not a product page. 2026-01-24 19:18:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-drawer-cabinet-20-drawers>: HTTP status code is not handled or not allowed 2026-01-24 19:19:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-weatherproof-laser-shipping-labels-3-1-3-x-4-300-pack-5524>: HTTP status code is not handled or not allowed 2026-01-24 19:19:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-dia-x-9-working-length-j-hook-style-a>: HTTP status code is not handled or not allowed 2026-01-24 19:19:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-pocket-special-purpose-literature-rack-tan>: HTTP status code is not handled or not allowed 2026-01-24 19:19:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 12664 pages (at 85 pages/min), scraped 8755 items (at 55 items/min) 2026-01-24 19:19:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/active-seating-stool-fabric-25-h-33-h-black>: HTTP status code is not handled or not allowed 2026-01-24 19:19:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pressure-sensitive-labels-horizontal-drawer-cabinets-sae-cap-screw-sets>: HTTP status code is not handled or not allowed 2026-01-24 19:19:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/task_desk_chairs/task_office_chairs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:19:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/armless-task-chair-fabric-pewter-oss-series>: HTTP status code is not handled or not allowed 2026-01-24 19:19:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-rack-safety-clip>: HTTP status code is not handled or not allowed 2026-01-24 19:19:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dot-placard-four-digit-3257-2>: HTTP status code is not handled or not allowed 2026-01-24 19:19:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-910-half-finger-impact-gloves-wrist-support-black-m-17713>: HTTP status code is not handled or not allowed 2026-01-24 19:19:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/task_desk_chairs/task_office_chairs landed on page that is not a product page. 2026-01-24 19:19:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bronchoscope-drying-cabinet-wall-mount-key-lock-24-w-x-12-3-4-d-x-48-h-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 19:19:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-lift-beam-yellow-42-l-x-36-w-x-12-h-725lbs-80000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 19:19:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dot-placard-blasting-agent-d1-2>: HTTP status code is not handled or not allowed 2026-01-24 19:19:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-series-lightweight-aluminium-crash-cart-5-drawers-30-w-x-22-l-x-37-1-4-h-cream>: HTTP status code is not handled or not allowed 2026-01-24 19:19:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-column-medical-storage-cart-glass-door-22-1-2-w-x-27-d-x-75-3-4-h-cream>: HTTP status code is not handled or not allowed 2026-01-24 19:19:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-2-shelf-container-rack-bk186052n-60w-x-18d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 19:19:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:19:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-twin-hoist-lift-beam-yellow-6000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 19:19:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poly-z-brite-2-shelf-container-rack-bk18306z-30w-x-18d-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 19:19:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silver-epoxy-wire-shelf-platform-p1424ep-24w-x-14d>: HTTP status code is not handled or not allowed 2026-01-24 19:19:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats>: HTTP status code is not handled or not allowed 2026-01-24 19:19:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-container-855-20-22l-x-20w-x-20h-black>: HTTP status code is not handled or not allowed 2026-01-24 19:19:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tall-isolation-cart-3-drawers-standard-key-lock-37-1-2-wx22-lx42-63-100-h-brushed-silver>: HTTP status code is not handled or not allowed 2026-01-24 19:19:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-chamber-cell-culture-slide-glass-sterile-12-pk>: HTTP status code is not handled or not allowed 2026-01-24 19:20:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-series-120-wide-door-3-motor-unheated-460360-obsidian-black>: HTTP status code is not handled or not allowed 2026-01-24 19:20:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1161220-comfort-construction-style-climbing-harness-quick-connect-s>: HTTP status code is not handled or not allowed 2026-01-24 19:20:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/515-tank-heavy-duty-shipping-case-tube-with-wheels-10l-x-10w-x-65h-blk>: HTTP status code is not handled or not allowed 2026-01-24 19:20:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 12763 pages (at 99 pages/min), scraped 8831 items (at 76 items/min) 2026-01-24 19:20:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-height-triple-column-medical-storage-cabinet-60-2-5-wx27-4-5-dx55-3-5-h-cream>: HTTP status code is not handled or not allowed 2026-01-24 19:20:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-28-thin-wall-insert-for-metal-319-428>: HTTP status code is not handled or not allowed 2026-01-24 19:20:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/option-work-glove-hi-vis-yellow-synthetic-leather-palm-l-1-pair-gon-17hvbrl>: HTTP status code is not handled or not allowed 2026-01-24 19:20:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdcc-17kg-dax-carbon-fiber-6-point-suspension-cap-black>: HTTP status code is not handled or not allowed 2026-01-24 19:20:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a24h2006sslp3pt-continuous-hinge-w3point-ltch-type-4x-2400x2000x600in-ss-type-304>: HTTP status code is not handled or not allowed 2026-01-24 19:20:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/emergency_vehicle_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:20:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-profile-gen-3-led-warning-light-bar-clear-lens-amber-leds-7880-a>: HTTP status code is not handled or not allowed 2026-01-24 19:20:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-32-flanged-press-insert-stainless-260-008-cr>: HTTP status code is not handled or not allowed 2026-01-24 19:20:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/patient_support_immobilization/splints_supports already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:20:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hook-style-folding-coat-rack-with-40-triple-prong-hooks-capacity-120-coats>: HTTP status code is not handled or not allowed 2026-01-24 19:20:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/champion-fmpsa4111rc-fmpro-all-stainless-steel-41-wx20-dx42-1-2-h-11-drawer-roller-cabinet>: HTTP status code is not handled or not allowed 2026-01-24 19:20:34 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/emergency_vehicle_lights landed on page that is not a product page. 2026-01-24 19:20:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/full_height_storage_cabinets/stationary_storage_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:20:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-acting-pneumatic-actuator-2210-in-lbs-80psi>: HTTP status code is not handled or not allowed 2026-01-24 19:20:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/surface-mounted-corner-guard-90-degrees-2-ft-ft-wings-12-ft-h-with-caps-silkworm>: HTTP status code is not handled or not allowed 2026-01-24 19:20:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a20h2006sslp-continuous-hinge-wclamps-type-4x-2000x2000x600in-ss-type-304>: HTTP status code is not handled or not allowed 2026-01-24 19:20:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/patient_support_immobilization/splints_supports landed on page that is not a product page. 2026-01-24 19:20:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/cabinets/full_height_storage_cabinets/stationary_storage_cabinets>: HTTP status code is not handled or not allowed 2026-01-24 19:20:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-wire-shelf-stem-caster-truck-black-epoxy-21w-x-60l-x-69h>: HTTP status code is not handled or not allowed 2026-01-24 19:20:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-2-pc-brass-npt-ball-valve-w-nema-4-115vac>: HTTP status code is not handled or not allowed 2026-01-24 19:20:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-frame-coupler-w-2in-ball-0091091>: HTTP status code is not handled or not allowed 2026-01-24 19:20:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:20:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:20:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:20:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/american-lock-solid-steel-rectangular-padlock-no-a5100>: HTTP status code is not handled or not allowed 2026-01-24 19:20:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/composites/composite_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:20:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-button-front-lab-coat-white-polycotton-m>: HTTP status code is not handled or not allowed 2026-01-24 19:21:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nickel-plated-brass-non-drip-coupling-3-4-body-size-red-color-ring-nbr-seals>: HTTP status code is not handled or not allowed 2026-01-24 19:21:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 19:21:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-16-v10-coupling-link>: HTTP status code is not handled or not allowed 2026-01-24 19:21:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vertical-pallet-lifter-fixed-fork-3-ton-capacity-42-fork-length-yellow>: HTTP status code is not handled or not allowed 2026-01-24 19:21:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/composites/composite_bar_stock landed on page that is not a product page. 2026-01-24 19:21:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 19:21:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-frame-coupler-w-2-5-16in-ball-0091097>: HTTP status code is not handled or not allowed 2026-01-24 19:21:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 12852 pages (at 89 pages/min), scraped 8893 items (at 62 items/min) 2026-01-24 19:21:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-bar-1-12-thick-x-2-12-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 19:21:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs landed on page that is not a product page. 2026-01-24 19:21:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-sheet-34-thick-x-8-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 19:21:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-assembly-for-wpx45-pallet-trucks-cl-1120-133000-10-xl>: HTTP status code is not handled or not allowed 2026-01-24 19:21:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:21:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xx-garolite-sheet-116-thick-x-48-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-24 19:21:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-long-sleeve-work-shirt-fluorescent-orange-tall-2xl>: HTTP status code is not handled or not allowed 2026-01-24 19:21:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:21:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-axle-proportional-brake-controller-bc3ap>: HTTP status code is not handled or not allowed 2026-01-24 19:21:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/cart_truck_accessories/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:21:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-x-96-rub-rail-040-thick-gray>: HTTP status code is not handled or not allowed 2026-01-24 19:21:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:21:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/54-bottle-freestanding-wine-rack-6-shelf-14w-x-36l-x-34h-chrome>: HTTP status code is not handled or not allowed 2026-01-24 19:21:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-built-in-combo-lock-for-horizontal-latch-box-locker-metal-dial-rh>: HTTP status code is not handled or not allowed 2026-01-24 19:21:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-24 19:21:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/snap-on-deck-junior-convertible-hand-trucks-987111>: HTTP status code is not handled or not allowed 2026-01-24 19:21:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-24 19:21:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-24 19:21:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gemini-valve-high-duty-cycle-ball-valve-w-double-acting-pneumatic-actuator-1-2-inch>: HTTP status code is not handled or not allowed 2026-01-24 19:21:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/cart_truck_accessories/pallet_truck_parts_accessories/pallet_truck_wheel_kits landed on page that is not a product page. 2026-01-24 19:21:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wallsaver-removable-poster-tape-double-sided-3-4-x-150-w-dispenser>: HTTP status code is not handled or not allowed 2026-01-24 19:21:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-sheet-1-12-thick-x-18-wide-x-18-long>: HTTP status code is not handled or not allowed 2026-01-24 19:21:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:21:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-containment-berm-ultimate-model-12-x-60>: HTTP status code is not handled or not allowed 2026-01-24 19:21:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ballkissen-plus-cushion-15-diameter-blue>: HTTP status code is not handled or not allowed 2026-01-24 19:21:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uhmw-polyethylene-plastic-bar-14-thick-x-34-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-24 19:21:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/spill_control_supplies/spill_containment_berms already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:21:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-24 19:21:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tumbler-nt-12oz-slate-blue>: HTTP status code is not handled or not allowed 2026-01-24 19:21:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-assembly-for-e30-series-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-24 19:21:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:21:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:21:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paper-clip-holder-3-7-8-inch-x-2-1-2-inch-transparent-with-green-tint>: HTTP status code is not handled or not allowed 2026-01-24 19:22:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/spill_control_supplies/spill_containment_berms landed on page that is not a product page. 2026-01-24 19:22:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/marseille-wall-mirror>: HTTP status code is not handled or not allowed 2026-01-24 19:22:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medium-stackable-supply-basket-black>: HTTP status code is not handled or not allowed 2026-01-24 19:22:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pin-for-w-40z-b218-45z-c215-pallet-trucks-hy-302375>: HTTP status code is not handled or not allowed 2026-01-24 19:22:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 19:22:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 19:22:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storage-tote-black-t146-18l-x-10-1-2w-x-2-3-4h>: HTTP status code is not handled or not allowed 2026-01-24 19:22:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:22:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 12946 pages (at 94 pages/min), scraped 8955 items (at 62 items/min) 2026-01-24 19:22:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storage-tote-blue-t146-18l-x-10-1-2w-x-2-3-4h>: HTTP status code is not handled or not allowed 2026-01-24 19:22:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/koch-filter-merv-8-standard-capacity-xl8-pleated-panel-ext-surface-12-inch-w-x-20-inch-h-x-2-inch-d>: HTTP status code is not handled or not allowed 2026-01-24 19:22:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/tool_storage/chests_roller_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:22:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-plastic-bar-18-thick-x-1-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 19:22:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 19:22:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/staplers_staples/staplers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:22:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/press-in-grease-fitting-for-l-2000-l-2300-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-24 19:22:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roller-cabinet-w-7-drawers-34-w-x-25-1-4-d-x-41-h-black>: HTTP status code is not handled or not allowed 2026-01-24 19:22:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:22:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/tool_storage/chests_roller_cabinets landed on page that is not a product page. 2026-01-24 19:22:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x36x74-chrome-wire-shelving-with-24-quickpick-double-open-bins-yellow-2>: HTTP status code is not handled or not allowed 2026-01-24 19:22:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:22:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9u-linier-fixed-wall-mount-cabinet-vented-door>: HTTP status code is not handled or not allowed 2026-01-24 19:22:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/staplers_staples/staplers landed on page that is not a product page. 2026-01-24 19:22:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-uhmw-polyethylene-plastic-sheet-38-thick-x-24-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-24 19:22:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:22:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/marine-emergency-box-11-5-8-l-x-5-1-8-w-x-7-1-8-h-orange>: HTTP status code is not handled or not allowed 2026-01-24 19:22:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/renewable_energy_devices/solar_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:22:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-rectangular-picnic-table-w-seat-cushions-expanded-metal-black>: HTTP status code is not handled or not allowed 2026-01-24 19:22:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 19:22:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:22:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/material-handling/lift_tables_work_positioners/pallet_carousels_positioners already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:22:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 19:22:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-flat-top-griddle-natural-gas-3-burners>: HTTP status code is not handled or not allowed 2026-01-24 19:22:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-5205-double-row-angular-contact-bearing-open-25mm-bore-x-52mm-od-x-20-6mm-w>: HTTP status code is not handled or not allowed 2026-01-24 19:22:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-24 19:22:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/renewable_energy_devices/solar_lights landed on page that is not a product page. 2026-01-24 19:22:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_repair_maintenance/tires_wheels/tire_chains already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:22:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-purpose-container-8-oz-240ml-polyethylene-tall-style-snap-lid-natural-100-pack>: HTTP status code is not handled or not allowed 2026-01-24 19:22:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 19:22:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/material-handling/lift_tables_work_positioners/pallet_carousels_positioners landed on page that is not a product page. 2026-01-24 19:22:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/graduated-tite-rite-container-90ml-3-oz-48mm-opening-white-screwcap-400-pack>: HTTP status code is not handled or not allowed 2026-01-24 19:22:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/composites/composite_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:22:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unisex-mid-length-bib-apron-black-polyestercotton-30-x-33>: HTTP status code is not handled or not allowed 2026-01-24 19:22:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:22:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_repair_maintenance/tires_wheels/tire_chains landed on page that is not a product page. 2026-01-24 19:22:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24x36x74-chrome-wire-shelving-with-33-4-h-shelf-bins-blue>: HTTP status code is not handled or not allowed 2026-01-24 19:22:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/composites/composite_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:22:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/utility-table-72-x-30-black>: HTTP status code is not handled or not allowed 2026-01-24 19:22:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/composites/composite_bar_stock landed on page that is not a product page. 2026-01-24 19:23:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-36-bar-height-breakroom-table-black>: HTTP status code is not handled or not allowed 2026-01-24 19:23:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corner-90-degree-connector-kit-for-46-h-panel-without-cable>: HTTP status code is not handled or not allowed 2026-01-24 19:23:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits>: HTTP status code is not handled or not allowed 2026-01-24 19:23:04 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-24 19:23:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/solenoid-switch-for-mp-mpb-040-ac-pallet-trucks-yl-501270305>: HTTP status code is not handled or not allowed 2026-01-24 19:23:05 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:9 2026-01-24 19:23:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:23:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/office_furniture/literature_mail_sorters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:23:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ice-storage-bin-44w-32-12d-46h-ss-finish>: HTTP status code is not handled or not allowed 2026-01-24 19:23:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 13027 pages (at 81 pages/min), scraped 9003 items (at 48 items/min) 2026-01-24 19:23:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12x36x39-steel-shelving-with-12-6-h-shelf-bins-yellow>: HTTP status code is not handled or not allowed 2026-01-24 19:23:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/composites/composite_sheet_stock landed on page that is not a product page. 2026-01-24 19:23:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mesh-desktop-stackable-letter-tray-black>: HTTP status code is not handled or not allowed 2026-01-24 19:23:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/desk_accessories/desk_organizers_letter_trays already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:23:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:23:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/office_furniture/literature_mail_sorters>: HTTP status code is not handled or not allowed 2026-01-24 19:23:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-24 19:23:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-easyfold-table-48-inch-round-gray-top-chrome-frame>: HTTP status code is not handled or not allowed 2026-01-24 19:23:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:23:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-kit-for-electric-pallet-jack-truck-gwk-8hbe30-lw-fits-toyota-model-8hbe30>: HTTP status code is not handled or not allowed 2026-01-24 19:23:21 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/desk_accessories/desk_organizers_letter_trays landed on page that is not a product page. 2026-01-24 19:23:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/rainwear already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:23:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits>: HTTP status code is not handled or not allowed 2026-01-24 19:23:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-webtex-olive-green-3-piece-suit-pvc-size-medium>: HTTP status code is not handled or not allowed 2026-01-24 19:23:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12x60x74-chrome-wire-shelving-with-118-shelf-bins-red>: HTTP status code is not handled or not allowed 2026-01-24 19:23:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits landed on page that is not a product page. 2026-01-24 19:23:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/rainwear>: HTTP status code is not handled or not allowed 2026-01-24 19:23:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bonita-electric-height-adjustable-desk-30x60-black-w-black-base>: HTTP status code is not handled or not allowed 2026-01-24 19:23:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:23:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/diamond-culture-tube-cap-for-18mm-glass-culture-tubes-polypropylene-red-1000-pack>: HTTP status code is not handled or not allowed 2026-01-24 19:23:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ct-cap-flange-plug-cap-for-ct2-and-cr1-models-13mm-white-1000-pack>: HTTP status code is not handled or not allowed 2026-01-24 19:23:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powr-flite-1-5-hp-20in-pad-1600-rpm-burnisher-metal>: HTTP status code is not handled or not allowed 2026-01-24 19:23:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/material-handling/cranes_hoists/hoists/electric_powered_hoists already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:23:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/axial-exhaust-fan-500-cfm>: HTTP status code is not handled or not allowed 2026-01-24 19:23:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/material-handling/cranes_hoists/hoists/electric_powered_hoists already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:23:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pigskin-leather-drivers-gloves-with-keystone-thumb-size-x2-pkg-qty-12>: HTTP status code is not handled or not allowed 2026-01-24 19:23:44 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 19:23:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ridgeline-vented-full-brim-hard-hat-white-pattern-4-point-ratchet-suspension>: HTTP status code is not handled or not allowed 2026-01-24 19:23:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brass-bristle-brush-tooth-brush-styled-1-1-4>: HTTP status code is not handled or not allowed 2026-01-24 19:23:49 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/material-handling/cranes_hoists/hoists/electric_powered_hoists landed on page that is not a product page. 2026-01-24 19:23:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milwaukee-hole-saw-cutter-2-49-57-8331>: HTTP status code is not handled or not allowed 2026-01-24 19:23:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silent-air-silencer-kit-for-5hp-10hp-with-fittings-and-hoses>: HTTP status code is not handled or not allowed 2026-01-24 19:23:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/material-handling/cranes_hoists/hoists/electric_powered_hoists landed on page that is not a product page. 2026-01-24 19:23:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/energizer-9v-e-nimh-rechargeable-battery>: HTTP status code is not handled or not allowed 2026-01-24 19:23:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:23:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medical-record-drop-box-beige>: HTTP status code is not handled or not allowed 2026-01-24 19:23:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spring-loaded-roll-up-bug-screen-door-with-11-oz-mesh-panels-10-x-10>: HTTP status code is not handled or not allowed 2026-01-24 19:23:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:23:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylon-plastic-sheet-1-thick-x-12-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 19:24:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/flooring/rubber_flooring already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:24:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-24 19:24:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/walls_ceilings/ceiling_tiles already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:24:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-seal-tight-handle-on-ball-valve-lockout>: HTTP status code is not handled or not allowed 2026-01-24 19:24:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 19:24:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/external-power-source-for-rfid-locks-if-battery-life-ends>: HTTP status code is not handled or not allowed 2026-01-24 19:24:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 13117 pages (at 90 pages/min), scraped 9059 items (at 56 items/min) 2026-01-24 19:24:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/flooring/rubber_flooring landed on page that is not a product page. 2026-01-24 19:24:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-vertical-hanger-bar-portable-bin-cart-gray>: HTTP status code is not handled or not allowed 2026-01-24 19:24:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/walls_ceilings/ceiling_tiles landed on page that is not a product page. 2026-01-24 19:24:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/lockers/locker_accessories/locker_locks_keys already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:24:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/omega-ii-cap-with-accessory-slots-and-6-point-mega-ratchet-suspension-hi-viz-orange>: HTTP status code is not handled or not allowed 2026-01-24 19:24:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/omega-ii-full-brim-with-6-point-slide-lock-suspension-hi-viz-yellow>: HTTP status code is not handled or not allowed 2026-01-24 19:24:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:24:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-drawer-bench-high-24-w-cabinet-everest-blue-5>: HTTP status code is not handled or not allowed 2026-01-24 19:24:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/composites/composite_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:24:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/lockers/locker_accessories/locker_locks_keys>: HTTP status code is not handled or not allowed 2026-01-24 19:24:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:24:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anvil-g1190-2-galv-di-floor-flange>: HTTP status code is not handled or not allowed 2026-01-24 19:24:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:24:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:24:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/americana-cap-vented-with-accessory-slots-and-4-point-mega-ratchet-suspension-hi-viz-pink>: HTTP status code is not handled or not allowed 2026-01-24 19:24:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:24:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents landed on page that is not a product page. 2026-01-24 19:24:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=e-1340vs-with-acu-rite-303-dro-with-taper-attachment-and-collet-closer already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:24:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/composites/composite_rod_stock>: HTTP status code is not handled or not allowed 2026-01-24 19:24:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/hvac_service_tools_parts/hvac_thermometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:24:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 19:24:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-drawer-counter-high-36-w-cabinet-black-2>: HTTP status code is not handled or not allowed 2026-01-24 19:24:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 19:24:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges landed on page that is not a product page. 2026-01-24 19:24:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges landed on page that is not a product page. 2026-01-24 19:24:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e-1440vs-with-collet-closer>: HTTP status code is not handled or not allowed 2026-01-24 19:24:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=e-1340vs-with-acu-rite-303-dro-with-taper-attachment-and-collet-closer landed on page that is not a product page. 2026-01-24 19:24:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/hvac_service_tools_parts/hvac_thermometers>: HTTP status code is not handled or not allowed 2026-01-24 19:24:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/vehicle_flood_utility_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:24:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fashion-desk-pad-22-x-17-blue-2016>: HTTP status code is not handled or not allowed 2026-01-24 19:24:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/siesta-octopus-24-round-resin-bar-table-dove-gray>: HTTP status code is not handled or not allowed 2026-01-24 19:24:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/test-tube-17-x-100mm-14ml-polystyrene-1000-pack>: HTTP status code is not handled or not allowed 2026-01-24 19:24:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-swivel-hook-dsh-u-058>: HTTP status code is not handled or not allowed 2026-01-24 19:24:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e-1236vs-with-acu-rite-303-css-dro>: HTTP status code is not handled or not allowed 2026-01-24 19:24:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tong-6-scallop-beige>: HTTP status code is not handled or not allowed 2026-01-24 19:24:56 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/vehicle_flood_utility_lights landed on page that is not a product page. 2026-01-24 19:24:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=jtm-1254rvs-with-newall-dp700-3x-q-dro-x-y-powerfeeds-air-power-drawbar already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:24:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slide-file-folder-20-place-cardboard-10-pack>: HTTP status code is not handled or not allowed 2026-01-24 19:25:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evs-949-mill-3-axis-acu-rite-203-quill-dro-x-y-z-axis-jet-pwrfds-usa-pwrd-drw-br>: HTTP status code is not handled or not allowed 2026-01-24 19:25:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/countertop-refrigerated-merch-ss-36-18w-22-38d-gdm-9-s>: HTTP status code is not handled or not allowed 2026-01-24 19:25:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/19-902-9153-multi-link-asian-interchange-1-4-body-size-safety-esafe-kit-3-out-1-2-fem-inlet>: HTTP status code is not handled or not allowed 2026-01-24 19:25:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60x30-mobile-steel-square-edge-work-bench-adjustable-height-1-1-4-top>: HTTP status code is not handled or not allowed 2026-01-24 19:25:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/55-gallon-drum-640292>: HTTP status code is not handled or not allowed 2026-01-24 19:25:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-x-3-8-barbed-reduction-tee-natural-kynar>: HTTP status code is not handled or not allowed 2026-01-24 19:25:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-8-barbed-equal-tee-natural-kynar>: HTTP status code is not handled or not allowed 2026-01-24 19:25:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wireless-tube-rack-25mm-half-size-16-place-nylon-magenta>: HTTP status code is not handled or not allowed 2026-01-24 19:25:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=jtm-1254rvs-with-newall-dp700-3x-q-dro-x-y-powerfeeds-air-power-drawbar>: HTTP status code is not handled or not allowed 2026-01-24 19:25:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=evs-949-mill-with-x-y-axis-jet-powerfeeds-and-usa-powered-draw-bar already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:25:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=evs-949-mill-with-3-axis-acu-rite-203-knee-dro-and-x-y-z-axis-jet-powerfeeds already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:25:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=evs-949-mill-3-axis-acu-rite-203-quill-dro-x-y-axis-jet-powerfeeds-usa-p-drw-br already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:25:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=894103-etm-949-mill-with-usa-powered-draw-bar already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:25:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ratcheting-headgear-face-shield-with-shade-8-ir-flip-visor-antifog-quad500-series>: HTTP status code is not handled or not allowed 2026-01-24 19:25:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 13200 pages (at 83 pages/min), scraped 9105 items (at 46 items/min) 2026-01-24 19:25:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=etm-949-mill-with-3-axis-acu-rite-303-quill-dro-and-x-y-z-axis-jet-powerfeeds already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:25:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crinkle-latex-coated-gloves-polyester-knit-black-blue-x-large-1-pair>: HTTP status code is not handled or not allowed 2026-01-24 19:25:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/milling_machines already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:25:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/salad-bowl-8-diameter-312-oz-capacity-fiberglass-dark-basketweave>: HTTP status code is not handled or not allowed 2026-01-24 19:25:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=evs-949-mill-with-x-y-axis-jet-powerfeeds-and-usa-powered-draw-bar landed on page that is not a product page. 2026-01-24 19:25:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=evs-949-mill-3-axis-acu-rite-203-quill-dro-x-y-axis-jet-powerfeeds-usa-p-drw-br>: HTTP status code is not handled or not allowed 2026-01-24 19:25:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=evs-949-mill-with-3-axis-acu-rite-203-knee-dro-and-x-y-z-axis-jet-powerfeeds landed on page that is not a product page. 2026-01-24 19:25:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=894103-etm-949-mill-with-usa-powered-draw-bar landed on page that is not a product page. 2026-01-24 19:25:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slide-draining-tray-100-place-for-up-to-200-slides-abs-white-12-pack>: HTTP status code is not handled or not allowed 2026-01-24 19:25:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stackbin-3512-series-esd-w-t-molding-72-w-x-30-d-black>: HTTP status code is not handled or not allowed 2026-01-24 19:25:21 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/milling_machines landed on page that is not a product page. 2026-01-24 19:25:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conveyor-magnet-18in-l>: HTTP status code is not handled or not allowed 2026-01-24 19:25:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=etm-949-mill-with-3-axis-acu-rite-303-quill-dro-and-x-y-z-axis-jet-powerfeeds landed on page that is not a product page. 2026-01-24 19:25:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-x-12-x-8-lf-hopper-front-container-red>: HTTP status code is not handled or not allowed 2026-01-24 19:25:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-ton-bender-and-portable-power-unit-3-phase-460-volt>: HTTP status code is not handled or not allowed 2026-01-24 19:25:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securall-36x18x84-self-latch-industrial-cabinet-yellow>: HTTP status code is not handled or not allowed 2026-01-24 19:25:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/empty-rack-for-use-with-new-1250ul-pipette-tips-and-refill-plates-6-pack>: HTTP status code is not handled or not allowed 2026-01-24 19:25:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-totally-enclosed-direct-drive-duct-fan-3-phase-3-hp-1>: HTTP status code is not handled or not allowed 2026-01-24 19:25:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1847-sc-3l6-3-8-x-1-4-standard-cut-cylindrical-ball-nose-bur>: HTTP status code is not handled or not allowed 2026-01-24 19:25:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-5-step-steel-24w-x-48d-work-platform-ladder>: HTTP status code is not handled or not allowed 2026-01-24 19:25:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/abrasives/points_burrs/burrs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:25:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ice-bin-port-21x23x16>: HTTP status code is not handled or not allowed 2026-01-24 19:25:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/280xl025-standard-timing-belt-xl-1-4-x-28-t140-trapezoidal>: HTTP status code is not handled or not allowed 2026-01-24 19:25:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dura-spike-narrow-w-bond-bird-spikes-24-ft-bp-s224>: HTTP status code is not handled or not allowed 2026-01-24 19:25:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/abrasives/points_burrs/burrs landed on page that is not a product page. 2026-01-24 19:25:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/abrasives/points_burrs/burrs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:25:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1845-sa-3-3-8-x-1-4-double-cut-cylindrical-bur-w-o-end-cut>: HTTP status code is not handled or not allowed 2026-01-24 19:26:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yellow-jacket-adjust-a-valve-opener-14-hose-516-hose>: HTTP status code is not handled or not allowed 2026-01-24 19:26:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tpb4bk-pegboard-back-panel-jet-black-48-inch-x-24-inch>: HTTP status code is not handled or not allowed 2026-01-24 19:26:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/graduated-cylinder-pmp-tpx-molded-graduations-50ml>: HTTP status code is not handled or not allowed 2026-01-24 19:26:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 13293 pages (at 93 pages/min), scraped 9172 items (at 67 items/min) 2026-01-24 19:26:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/banking-vault-safe-interiors-3-compartment-2303-17-i-16-1-4w-x-17d-x-15-11-16h-gray>: HTTP status code is not handled or not allowed 2026-01-24 19:26:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/abrasives/points_burrs/burrs landed on page that is not a product page. 2026-01-24 19:26:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/plumbing_parts_accessories/drain_parts_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:26:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ice-tray-with-lid-4-liter-black>: HTTP status code is not handled or not allowed 2026-01-24 19:26:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-bollard-with-base-removable-plastic-cap-chain-slots-yellow-36-height>: HTTP status code is not handled or not allowed 2026-01-24 19:26:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motorized-hand-truck-pneumatic-tires-and-trailer-hitch-1000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 19:26:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/plumbing/plumbing_parts_accessories/drain_parts_accessories landed on page that is not a product page. 2026-01-24 19:26:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/structural-plastic-solid-shelving-48-x-24-x-75>: HTTP status code is not handled or not allowed 2026-01-24 19:26:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tork-advanced-masterfold-dispenser-napkins-d802a>: HTTP status code is not handled or not allowed 2026-01-24 19:26:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lifting-magnet-1600-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 19:26:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/extra_heavy_boltless_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:26:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50btl36-taper-bushed-58-pitch-36-teeth>: HTTP status code is not handled or not allowed 2026-01-24 19:26:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50a40x34-a-plate-58-pitch-34-bore-40-teeth>: HTTP status code is not handled or not allowed 2026-01-24 19:26:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:26:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kac-111nwi-10-21-yellow>: HTTP status code is not handled or not allowed 2026-01-24 19:26:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:26:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smooth-bollard-post-sleeve-10-hdpe-dome-top-yellow>: HTTP status code is not handled or not allowed 2026-01-24 19:26:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/extra_heavy_boltless_shelves landed on page that is not a product page. 2026-01-24 19:26:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ice-caddies-dark-brown-125-lbs-cap-2-swivel-2-locking>: HTTP status code is not handled or not allowed 2026-01-24 19:26:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-square-plastic-waste-receptacle-w-dome-lid-42-gallon-black>: HTTP status code is not handled or not allowed 2026-01-24 19:26:45 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents landed on page that is not a product page. 2026-01-24 19:26:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hydrofarm-hgbs600-bonsai-shears-60mm-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 19:26:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves landed on page that is not a product page. 2026-01-24 19:26:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pillow-standard-size-cluster-fiber-fill-12-pack>: HTTP status code is not handled or not allowed 2026-01-24 19:26:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securall-breakglass-explosion-proof-exhaust>: HTTP status code is not handled or not allowed 2026-01-24 19:26:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-5-5-8-direct-drive-blower-motor-115-volts-1075-rpm-4>: HTTP status code is not handled or not allowed 2026-01-24 19:26:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80bs16hx158-1-pitch-1-58-finished-bore-16-teeth>: HTTP status code is not handled or not allowed 2026-01-24 19:27:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/prolube-43973-grease-fitting-set-metric-80-piece>: HTTP status code is not handled or not allowed 2026-01-24 19:27:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-full-face-envelopes-packing-list-enclosed-5-1-2-wx4-1-2-l-orange-500-pk>: HTTP status code is not handled or not allowed 2026-01-24 19:27:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_batteries/vehicle_battery_accessories/vehicle_battery_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:27:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-curved-open-top-trash-can-32-gallon-satin-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 19:27:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-c-frame-broan-replacement-motor-120-volts-3000-rpm-1>: HTTP status code is not handled or not allowed 2026-01-24 19:27:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 13387 pages (at 94 pages/min), scraped 9239 items (at 67 items/min) 2026-01-24 19:27:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/definite_purpose_electric_motors/oem_replacement_ac_motors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:27:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-1-4-thermal-paper-6pk>: HTTP status code is not handled or not allowed 2026-01-24 19:27:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_batteries/vehicle_battery_accessories/vehicle_battery_chargers>: HTTP status code is not handled or not allowed 2026-01-24 19:27:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/automotive_shop_equipment/vehicle_lifts_jacks/heavy_duty_floor_jacks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:27:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-pallet-covers-48-x-48-x-96-1-mil-100-pack>: HTTP status code is not handled or not allowed 2026-01-24 19:27:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60a30hx1-a-plate-34-pitch-1-bore-30-teeth>: HTTP status code is not handled or not allowed 2026-01-24 19:27:18 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/definite_purpose_electric_motors/oem_replacement_ac_motors landed on page that is not a product page. 2026-01-24 19:27:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ee-20-ton-air-hydraulic-bottle-jack>: HTTP status code is not handled or not allowed 2026-01-24 19:27:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/79233-forehead-thermometer-100-per-box>: HTTP status code is not handled or not allowed 2026-01-24 19:27:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/vehicle-fleet-maintenance/automotive_shop_equipment/vehicle_lifts_jacks/heavy_duty_floor_jacks landed on page that is not a product page. 2026-01-24 19:27:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-forklift-side-mirror>: HTTP status code is not handled or not allowed 2026-01-24 19:27:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/to-go-rack-silver-ep-24w-x-48l-x-63h-4-wire-shelves-dividers-ledges>: HTTP status code is not handled or not allowed 2026-01-24 19:27:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/hvac_electric_motors/unit_bearing_fan_motors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:27:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/special-order-3-5-bright-yellow-black>: HTTP status code is not handled or not allowed 2026-01-24 19:27:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-reflective-plastic-barrier-chain-2-x-25-ft-red>: HTTP status code is not handled or not allowed 2026-01-24 19:27:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-high-pressure-transfer-water-pump-5-5hp-130-gpm-honda-gx-engine>: HTTP status code is not handled or not allowed 2026-01-24 19:27:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lead-free-rohs-2-2-green-white>: HTTP status code is not handled or not allowed 2026-01-24 19:27:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-dia-black-standard-discs>: HTTP status code is not handled or not allowed 2026-01-24 19:27:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/non-ansi-multi-color-long-sleeve-safety-t-shirt-with-black-bottom-blue-lg>: HTTP status code is not handled or not allowed 2026-01-24 19:27:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/antimicrobial-hi-tech-shelving-36-w-x-12-d-x-87-h-light-gray-11-shelves-adder-unit-open>: HTTP status code is not handled or not allowed 2026-01-24 19:27:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/hvac_electric_motors/unit_bearing_fan_motors landed on page that is not a product page. 2026-01-24 19:27:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-thermostat-guard-includes-one-replacement-key>: HTTP status code is not handled or not allowed 2026-01-24 19:27:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/curbside-truck-chrome-18w-x-48l-x-72h-4-wire-shelves-pneumatic-2-rigid-casters>: HTTP status code is not handled or not allowed 2026-01-24 19:27:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stationary-tip-out-bin-unit-single-sided-24-h-gray>: HTTP status code is not handled or not allowed 2026-01-24 19:27:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-tip-out-bin-unit-double-sided-70-h-white>: HTTP status code is not handled or not allowed 2026-01-24 19:28:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smokers-outpost-wall-mounted-ashtray-locking-with-swivel-black>: HTTP status code is not handled or not allowed 2026-01-24 19:28:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1629-100-100-3-8-id-push-on-and-lock-hose>: HTTP status code is not handled or not allowed 2026-01-24 19:28:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/655-1-4-fnpt-brass-tee-hose-fitting-box-of-5>: HTTP status code is not handled or not allowed 2026-01-24 19:28:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-16bs18hx40-metric-1-pitch-40mm-finished-bore-18-teeth>: HTTP status code is not handled or not allowed 2026-01-24 19:28:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/maxwell-house-house-blend-k-cups-100-count-gr>: HTTP status code is not handled or not allowed 2026-01-24 19:28:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 13485 pages (at 98 pages/min), scraped 9310 items (at 71 items/min) 2026-01-24 19:28:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blue-diamond-low-sodium-lightly-salted-almonds-1-5-oz-12-count-gr>: HTTP status code is not handled or not allowed 2026-01-24 19:28:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-drawer-mobile-cabinet-24-wx21-dx33-1-4-h-everest-blue>: HTTP status code is not handled or not allowed 2026-01-24 19:28:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-plastic-creeper-97102>: HTTP status code is not handled or not allowed 2026-01-24 19:28:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-drawer-mobile-multi-drawer-cabinet-48-wx27-dx37-1-2-h-beige-1>: HTTP status code is not handled or not allowed 2026-01-24 19:28:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-dj2p2hm-2-hp-3-phase-1740-rpm>: HTTP status code is not handled or not allowed 2026-01-24 19:28:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-step-24-w-aluminum-wheelbarrow-ladder-ribbed-tread>: HTTP status code is not handled or not allowed 2026-01-24 19:28:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-step-extra-rolling-safety-ladder-expanded-metal-tread>: HTTP status code is not handled or not allowed 2026-01-24 19:28:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-3783-1-4-hp-1-phase-1075-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:28:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-4242-1-1-2-hp-1-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-24 19:28:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xfs-series-24-exhaust-shutter-fan-1-2-hp-1140-rpm-115v>: HTTP status code is not handled or not allowed 2026-01-24 19:28:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-180-1-3-hp-1-phase-1725-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:28:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/linea-3-shelf-merchandising-table-statuary-bronze>: HTTP status code is not handled or not allowed 2026-01-24 19:28:34 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=us-motor-3783-1-4-hp-1-phase-1075-rpm landed on page that is not a product page. 2026-01-24 19:28:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-d13b2n4z-1-3-hp-1-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-24 19:28:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hgx-led-security-lights-2-round-heads-4000k-2600-lumens-black>: HTTP status code is not handled or not allowed 2026-01-24 19:28:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=us-motor-180-1-3-hp-1-phase-1725-rpm landed on page that is not a product page. 2026-01-24 19:28:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flask-erlenmeyer-globe-glass-narrow-mouth-dual-graduations-astm-e1404-125ml-12-box>: HTTP status code is not handled or not allowed 2026-01-24 19:28:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-knock-down-corrosion-resistant-locker-single-tier-1-wide-12x12x78>: HTTP status code is not handled or not allowed 2026-01-24 19:28:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-ts607-5-hp-1-phase-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-24 19:28:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dandux-rectangular-plastic-box-truck-red-16-bushel-660-lb-cap>: HTTP status code is not handled or not allowed 2026-01-24 19:28:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:28:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/standard_duty_boltless_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:28:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microstat-1-high-120vac-clear-msl1-120c>: HTTP status code is not handled or not allowed 2026-01-24 19:28:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/standard_duty_boltless_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:28:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:28:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-xs14ca2jcr-1-4-hp-1-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-24 19:28:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-fdu5cm2k21z-5-hp-1-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-24 19:28:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-yc5e3b-5-hp-3-phase-1170-rpm>: HTTP status code is not handled or not allowed 2026-01-24 19:29:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves landed on page that is not a product page. 2026-01-24 19:29:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-c3p2g-3-hp-3-phase-1765-rpm>: HTTP status code is not handled or not allowed 2026-01-24 19:29:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/standard_duty_boltless_shelves landed on page that is not a product page. 2026-01-24 19:29:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/standard_duty_boltless_shelves landed on page that is not a product page. 2026-01-24 19:29:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lau-4-blade-condenser-propeller-18-diameter-cw-rotation-2>: HTTP status code is not handled or not allowed 2026-01-24 19:29:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves landed on page that is not a product page. 2026-01-24 19:29:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-carpet-dolly-carpet-cart-500-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 19:29:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wwe-kan67-30-1-h-145tc-cast-iron-helical-bevel-speed-reducer-145tc-input-flange-30-1>: HTTP status code is not handled or not allowed 2026-01-24 19:29:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 13577 pages (at 92 pages/min), scraped 9372 items (at 62 items/min) 2026-01-24 19:29:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/over-work-wall-mounted-pegboard-swing-panel-48-w-x-48-h>: HTTP status code is not handled or not allowed 2026-01-24 19:29:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-7040-1-3-hp-1-phase-825-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:29:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/food-storage-container-18x26x12-17-gallon-capacity-natural-white>: HTTP status code is not handled or not allowed 2026-01-24 19:29:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speed-rail-5-bottle-dark-brown>: HTTP status code is not handled or not allowed 2026-01-24 19:29:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interion-deluxe-mobile-office-partition-panel-with-partial-window-36-1-4-w-x-96-h-blue>: HTTP status code is not handled or not allowed 2026-01-24 19:29:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=us-motor-7040-1-3-hp-1-phase-825-rpm>: HTTP status code is not handled or not allowed 2026-01-24 19:29:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-camcart-heated-food-pan-carrier-front-loading-granite-gray-220v-60-1-ph-23-amp>: HTTP status code is not handled or not allowed 2026-01-24 19:29:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/food-rotation-label-1-1-4-x-2-100-biodegradable-white>: HTTP status code is not handled or not allowed 2026-01-24 19:29:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/safety_cabinets/flammable_materials_cabinets/general_flammable_safety_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:29:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plate-cover-square-fits-8-1-2-antique-parchment>: HTTP status code is not handled or not allowed 2026-01-24 19:29:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camshelving-elements-starter-unit-18-w-x-48-l-x-64-h-4-shelf-brushed-graphite>: HTTP status code is not handled or not allowed 2026-01-24 19:29:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/cabinets/safety_cabinets/flammable_materials_cabinets/general_flammable_safety_cabinets>: HTTP status code is not handled or not allowed 2026-01-24 19:29:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wood-wardrobe-cabinet-36x22x84>: HTTP status code is not handled or not allowed 2026-01-24 19:29:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plug-6mm>: HTTP status code is not handled or not allowed 2026-01-24 19:29:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-control-10mm-tube-x-3-8-metal-release-collet-flow-in-knob-adjustment>: HTTP status code is not handled or not allowed 2026-01-24 19:29:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-electronic-caliper-0-6-inch-0-150mm-range>: HTTP status code is not handled or not allowed 2026-01-24 19:29:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zero-setter-2-inch-h>: HTTP status code is not handled or not allowed 2026-01-24 19:29:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shield-curved-direct-mount-w-o-interlock-20-inch-l-x-10-inch-w>: HTTP status code is not handled or not allowed 2026-01-24 19:29:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/packaging/protective_packaging/roll_dispensers_cutters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:29:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ecp-economy-magnetic-dispenser>: HTTP status code is not handled or not allowed 2026-01-24 19:29:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bird-shock-2-d-battery-charger-green>: HTTP status code is not handled or not allowed 2026-01-24 19:29:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eled-outdoor-universal-metal-light-post-pole-156-inch-h>: HTTP status code is not handled or not allowed 2026-01-24 19:30:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_tables/outdoor_dining_bar_tables already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:30:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:30:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/packaging/protective_packaging/roll_dispensers_cutters landed on page that is not a product page. 2026-01-24 19:30:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/key-largo-silver-lines-slab-door-wood-36w-x-80h-whiskey-oak>: HTTP status code is not handled or not allowed 2026-01-24 19:30:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:30:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/door_frames already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:30:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:30:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:30:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labella-slab-door-wood-24w-x-80h-whiskey-oak>: HTTP status code is not handled or not allowed 2026-01-24 19:30:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:30:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:30:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:30:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:30:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 13666 pages (at 89 pages/min), scraped 9439 items (at 67 items/min) 2026-01-24 19:30:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/privacy-cover-plate-chrome>: HTTP status code is not handled or not allowed 2026-01-24 19:30:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_tables/outdoor_dining_bar_tables landed on page that is not a product page. 2026-01-24 19:30:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:30:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_plates_braces already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:30:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors>: HTTP status code is not handled or not allowed 2026-01-24 19:30:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/door_frames landed on page that is not a product page. 2026-01-24 19:30:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:30:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:30:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors>: HTTP status code is not handled or not allowed 2026-01-24 19:30:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:30:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:30:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:30:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-diamond-quilted-jacket-2xl-black>: HTTP status code is not handled or not allowed 2026-01-24 19:30:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:30:21 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/doors_windows/doors landed on page that is not a product page. 2026-01-24 19:30:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ajm-packaging-corporation-paper-plates-9-dia-white-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 19:30:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reflections-fork-silver-pack-of-320>: HTTP status code is not handled or not allowed 2026-01-24 19:30:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/doors_windows/doors landed on page that is not a product page. 2026-01-24 19:30:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/doors_windows/doors landed on page that is not a product page. 2026-01-24 19:30:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_plates_braces landed on page that is not a product page. 2026-01-24 19:30:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/doors_windows/doors>: HTTP status code is not handled or not allowed 2026-01-24 19:30:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/doors_windows/doors landed on page that is not a product page. 2026-01-24 19:30:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paper-hot-drink-cups-lids-combo-12-oz-pack-of-300>: HTTP status code is not handled or not allowed 2026-01-24 19:30:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats landed on page that is not a product page. 2026-01-24 19:30:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/doors_windows/doors landed on page that is not a product page. 2026-01-24 19:30:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/world-art-compostable-food-container-3-1-16-dia-x-2-3-8-h-pack-of-1000>: HTTP status code is not handled or not allowed 2026-01-24 19:30:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/renewable-wooden-stir-sticks-7-l-pack-of-10000>: HTTP status code is not handled or not allowed 2026-01-24 19:30:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/digital-compact-bench-scale-330-lb-x-0-1-lb-12-in-x-12-in-platform>: HTTP status code is not handled or not allowed 2026-01-24 19:30:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/foodservice_disposables/disposable_lids already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:30:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/colormaxx-paint-primer-12-oz-gloss-gum-drop-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-24 19:30:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-compartment-drawer-organizer-30w-x-18-38d-x-1-38h-black>: HTTP status code is not handled or not allowed 2026-01-24 19:30:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/floor_machine_accessories_supplies/floor_pads_pad_holders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:30:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vinyl-post-op-shoe-w-hook-and-loop-closure-mens-medium>: HTTP status code is not handled or not allowed 2026-01-24 19:30:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/feeding-system-single-syringe-w-iv-pole-bag-60ml-clear-pack-of-30>: HTTP status code is not handled or not allowed 2026-01-24 19:30:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insert-style-wrist-id-band-12l-clear-pack-of-250>: HTTP status code is not handled or not allowed 2026-01-24 19:30:50 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/foodservice_disposables/disposable_lids landed on page that is not a product page. 2026-01-24 19:30:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/floor_machine_accessories_supplies/floor_pads_pad_holders landed on page that is not a product page. 2026-01-24 19:30:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mesh-truck-w-1-adjustable-shelf-steel-2000-lb-cap-66-1-2l-x-32w-x-56-7-16h>: HTTP status code is not handled or not allowed 2026-01-24 19:30:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-standing-rack-for-tubing-or-jump-ropes-23l-x-23w-x-73h-black>: HTTP status code is not handled or not allowed 2026-01-24 19:30:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/physical_therapy/exercise_fitness_equipment/stretch_bands_tubing already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:30:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/physical_therapy/exercise_fitness_equipment/exercise_weights_weight_training already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:30:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pliates-ring-firm-resistance-black>: HTTP status code is not handled or not allowed 2026-01-24 19:31:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-powered-40AH-receiving-cart-w-4-shelves-black>: HTTP status code is not handled or not allowed 2026-01-24 19:31:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/base-cabinet-w-drawer--laminate-top-30w-x-18d-x-36h-pastel-green>: HTTP status code is not handled or not allowed 2026-01-24 19:31:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/physical_therapy/exercise_fitness_equipment/exercise_weights_weight_training>: HTTP status code is not handled or not allowed 2026-01-24 19:31:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bookcase-assembled-36w-x-18d-x-45h-charcoal>: HTTP status code is not handled or not allowed 2026-01-24 19:31:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sterile-sampling-bag-27-oz-capacity-11-1516l-x-4-12w-pack-of-500>: HTTP status code is not handled or not allowed 2026-01-24 19:31:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/counter_height_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:31:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 13753 pages (at 87 pages/min), scraped 9490 items (at 51 items/min) 2026-01-24 19:31:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/testculture-tube-sterile-17-x-100-mm-14-ml-capacity-20-bags-of-25-tubes>: HTTP status code is not handled or not allowed 2026-01-24 19:31:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/physical_therapy/exercise_fitness_equipment/stretch_bands_tubing landed on page that is not a product page. 2026-01-24 19:31:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-premium-heavy-duty-vest-w-multi-pockets-2xl3xl-red>: HTTP status code is not handled or not allowed 2026-01-24 19:31:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/windup-multi-surface-disinfecting-floor-cleaner-fluid-clear>: HTTP status code is not handled or not allowed 2026-01-24 19:31:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/cabinets/counter_height_cabinets>: HTTP status code is not handled or not allowed 2026-01-24 19:31:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forkliftable-truck-w-2-shelves-3600-lb-cap-53-12l-x-30w-x-69h-gray>: HTTP status code is not handled or not allowed 2026-01-24 19:31:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8377ev-winter-bomber-jacket-enhanced-visibility-large-black>: HTTP status code is not handled or not allowed 2026-01-24 19:31:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8915bk-hi-vis-rain-pants-w-black-bottom-class-e-polyester-5xl-lime>: HTTP status code is not handled or not allowed 2026-01-24 19:31:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8280bk-hi-vis-performance-t-shirt-black-bottom-class-2-xl-lime>: HTTP status code is not handled or not allowed 2026-01-24 19:31:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-7001-abrasion-resistant-gloves-nitrile-coated-2xl-black-1-pair>: HTTP status code is not handled or not allowed 2026-01-24 19:31:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/squids-3132-hi-vis-barcode-scanner-lanyard-harness-large-lime>: HTTP status code is not handled or not allowed 2026-01-24 19:31:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chill-its-6630-high-performance-terry-cloth-skull-cap-black>: HTTP status code is not handled or not allowed 2026-01-24 19:31:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8217ba-breakaway-mesh-vest-class-2-sm-lime>: HTTP status code is not handled or not allowed 2026-01-24 19:31:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-bulk-dispenser-tall-12-316w-x-5-34d-x-17-116h-stainless>: HTTP status code is not handled or not allowed 2026-01-24 19:31:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-tier-4-door-digital-locker-12in-w-x-18in-d-x-78in-h-gray-assembled>: HTTP status code is not handled or not allowed 2026-01-24 19:31:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heater-terminal-assembly-for-kf-sealers-w-cutter>: HTTP status code is not handled or not allowed 2026-01-24 19:31:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/sockets_bits/impact_sockets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:31:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silicone-rubber-pad-for-w-300a-w-305a-w-3010a>: HTTP status code is not handled or not allowed 2026-01-24 19:31:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-shaped-bur-single-cut-1-12l-x-18-shank-dia>: HTTP status code is not handled or not allowed 2026-01-24 19:31:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/sockets_bits/impact_sockets>: HTTP status code is not handled or not allowed 2026-01-24 19:31:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bur-box-diamond-cut-sa5-sc3-sc5-sd5-sf3-sf5-sg3-sl4-set-of-8>: HTTP status code is not handled or not allowed 2026-01-24 19:31:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/sockets_bits/power_drill_driver_bits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:31:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nextpette-variable-volume-pipette-sterile-1-100l>: HTTP status code is not handled or not allowed 2026-01-24 19:31:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/basic-magnetic-pull-cord-patient-sensor-alarm-6l-x-2w-x-4h-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 19:31:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-sided-starter-36l-x-18w-x-54h-antique-white>: HTTP status code is not handled or not allowed 2026-01-24 19:32:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/end-unit-w-t-joiner-support-clip-48l-x-14w-x-84h-black>: HTTP status code is not handled or not allowed 2026-01-24 19:32:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/sockets_bits/power_drill_driver_bits landed on page that is not a product page. 2026-01-24 19:32:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reelsafe-spring-retractable-hose-reel-300-psi-12-id-x-50l>: HTTP status code is not handled or not allowed 2026-01-24 19:32:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mmb-ytx14l-bs-12v-12ah-200cca-battery>: HTTP status code is not handled or not allowed 2026-01-24 19:32:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobilecare-electronic-medical-cart-27w-x-27d-x-39h-blue>: HTTP status code is not handled or not allowed 2026-01-24 19:32:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 13850 pages (at 97 pages/min), scraped 9560 items (at 70 items/min) 2026-01-24 19:32:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/medical_devices_instruments/vital_signs_monitors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:32:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/apex-digital-clinical-scale-600-lb-cap-17l-x-17w-platform>: HTTP status code is not handled or not allowed 2026-01-24 19:32:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h1-2-bar-steel-ladder-roof-rack-for-ram-promaster-city-2014-on-black>: HTTP status code is not handled or not allowed 2026-01-24 19:32:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h1-3-bar-steel-ladder-rack-system-for-chevy-express-1996-on-black>: HTTP status code is not handled or not allowed 2026-01-24 19:32:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/military-grade-cargo-lifting-net-4500-lb-capacity-10-opening-360l-x-360w-black>: HTTP status code is not handled or not allowed 2026-01-24 19:32:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=protect-liquid-cooled-standby-generator-150kw-120-240v-1-phase-lp already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:32:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ceramic-bathroom-heater-1500w-gray>: HTTP status code is not handled or not allowed 2026-01-24 19:32:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/medical_devices_instruments/vital_signs_monitors landed on page that is not a product page. 2026-01-24 19:32:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/en-wheeled-salvage-drum-spill-kit-universal-95-gallon>: HTTP status code is not handled or not allowed 2026-01-24 19:32:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-16a26-metric-1-pitch-30mm-bore-26-teeth>: HTTP status code is not handled or not allowed 2026-01-24 19:32:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_supplies_dispensers/paper_towel_dispensers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:32:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-20bs16hx42-metric-1-14-pitch-42mm-finished-bore-16-teeth>: HTTP status code is not handled or not allowed 2026-01-24 19:32:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=protect-liquid-cooled-standby-generator-150kw-120-240v-1-phase-lp landed on page that is not a product page. 2026-01-24 19:32:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-safety-crafted-waterproof-logger-work-boots-steel-toe-size-12-5w-8-inch-h-dark-brown>: HTTP status code is not handled or not allowed 2026-01-24 19:32:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:32:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tuff-job-foodservice-towels-12-inch-x-24-inch-pink-white-200-carton>: HTTP status code is not handled or not allowed 2026-01-24 19:32:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/solar-13w-top-of-pole-24-7-flashing-12-red-led-yellow-housing>: HTTP status code is not handled or not allowed 2026-01-24 19:32:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_supplies_dispensers/paper_towel_dispensers>: HTTP status code is not handled or not allowed 2026-01-24 19:32:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:32:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-100e21h-qd-bushed-1-14-pitch-21-teeth>: HTTP status code is not handled or not allowed 2026-01-24 19:32:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/futuro-night-wrist-support-adjustable-2-pk-6-pk-case>: HTTP status code is not handled or not allowed 2026-01-24 19:32:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-24 19:32:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/back_joint_support/wrist_braces_supports_wraps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:32:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:32:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/groundbreaker-work-boots-w-internal-met-guard-steel-toe-size-7m-6-inch-h-brown>: HTTP status code is not handled or not allowed 2026-01-24 19:32:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:32:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-08a23h-metric-a-plate-12-pitch-12mm-bore-23-teeth>: HTTP status code is not handled or not allowed 2026-01-24 19:32:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/groundbreaker-work-boots-w-external-met-guard-steel-toe-size-10-5w-8-inch-h-brown>: HTTP status code is not handled or not allowed 2026-01-24 19:32:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/back_joint_support/wrist_braces_supports_wraps>: HTTP status code is not handled or not allowed 2026-01-24 19:32:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:32:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:32:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-24 19:32:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-24 19:32:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zig-elusion-heritage-work-men-s-low-cut-sneaker-composite-toe-size-11w-black>: HTTP status code is not handled or not allowed 2026-01-24 19:32:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:32:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-24 19:32:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vitals-skate-inspired-work-shoes-composite-toe-size-7m-wintermoss-whitecap-gray>: HTTP status code is not handled or not allowed 2026-01-24 19:32:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:32:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zig-elusion-heritage-work-women-s-low-cut-sneaker-composite-toe-size-10m-black>: HTTP status code is not handled or not allowed 2026-01-24 19:33:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-24 19:33:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-24 19:33:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80bs36x1316-1-pitch-1-316-finished-bore-36-teeth>: HTTP status code is not handled or not allowed 2026-01-24 19:33:04 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-24 19:33:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/true-skate-inspired-work-shoes-composite-toe-size-13m-navy>: HTTP status code is not handled or not allowed 2026-01-24 19:33:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 19:33:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/truflex-fly-mudguard-slip-on-sneaker-composite-toe-size-12w-black>: HTTP status code is not handled or not allowed 2026-01-24 19:33:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 13940 pages (at 90 pages/min), scraped 9616 items (at 56 items/min) 2026-01-24 19:33:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-crafted-western-work-boots-w-steel-toe-and-internal-met-guard-size-11-5w-brown>: HTTP status code is not handled or not allowed 2026-01-24 19:33:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stone-skate-inspired-work-shoes-composite-toe-size-5-5w-blue-navy>: HTTP status code is not handled or not allowed 2026-01-24 19:33:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stone-skate-inspired-work-shoes-composite-toe-size-9-5w-gray-black>: HTTP status code is not handled or not allowed 2026-01-24 19:33:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:33:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:33:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-crafted-western-work-boots-w-steel-toe-and-internal-met-guard-size-9m-brown>: HTTP status code is not handled or not allowed 2026-01-24 19:33:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-seal-bubble-bags-12-x-11-1-2-250-pack>: HTTP status code is not handled or not allowed 2026-01-24 19:33:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:33:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-bags-3-x-3-2-mil-red-1000-pack>: HTTP status code is not handled or not allowed 2026-01-24 19:33:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:33:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 19:33:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-24 19:33:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ge-caddx-b1245-alarm-replacement-battery-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 19:33:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ademco-alarm-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 19:33:34 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 19:33:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bubble-bags-9-x-12-350-pack>: HTTP status code is not handled or not allowed 2026-01-24 19:33:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 19:33:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-10bs21hx38-metric-58-pitch-38mm-finished-bore-21-teeth>: HTTP status code is not handled or not allowed 2026-01-24 19:33:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simplex-112-112-alarm-replacement-battery-100ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 19:33:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pearl-eraser-medium-3-pk-pink>: HTTP status code is not handled or not allowed 2026-01-24 19:33:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/975-electric-hydraulic-pump>: HTTP status code is not handled or not allowed 2026-01-24 19:33:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/683-screw-type-reel-stand>: HTTP status code is not handled or not allowed 2026-01-24 19:33:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-12a18h-metric-a-plate-34-pitch-14mm-bore-18-teeth>: HTTP status code is not handled or not allowed 2026-01-24 19:33:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:33:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fer-hg9320-patio-cover-kit-10-ft-l-x-20-ft-ft-w-white>: HTTP status code is not handled or not allowed 2026-01-24 19:33:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nano-x1-adventure-athletic-work-shoes-composite-toe-size-10m-silver-gray-clay-black>: HTTP status code is not handled or not allowed 2026-01-24 19:33:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acme-624-home-alarm-alarm-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 19:34:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hook-n-loop-tape-fasteners-black-2-inch-x-15-feet>: HTTP status code is not handled or not allowed 2026-01-24 19:34:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lavante-trail-2-running-work-shoes-composite-toe-size-9m-army-green-black-yellow>: HTTP status code is not handled or not allowed 2026-01-24 19:34:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-41bs10hx34-12-pitch-34-finished-bore-10-teeth>: HTTP status code is not handled or not allowed 2026-01-24 19:34:04 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 19:34:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-40bs16hx58-12-pitch-58-finished-bore-16-teeth>: HTTP status code is not handled or not allowed 2026-01-24 19:34:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:34:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merits-s245-wheelchair-replacement-battery-18ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 19:34:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:34:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 14029 pages (at 89 pages/min), scraped 9677 items (at 61 items/min) 2026-01-24 19:34:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/floatride-energy-3-adventure-women-s-athletic-work-shoes-composite-toe-size-8m-black>: HTTP status code is not handled or not allowed 2026-01-24 19:34:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:34:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ford-rmt66-lawn-and-garden-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 19:34:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/column-protector-30-diameter-round-opening-44-od-x-42h-yellow>: HTTP status code is not handled or not allowed 2026-01-24 19:34:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:34:18 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 19:34:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evolve-skate-inspired-high-top-work-shoes-composite-toe-size-6-5m-triple-black>: HTTP status code is not handled or not allowed 2026-01-24 19:34:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:34:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 19:34:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rotary-screw-air-compressor-wdryer-230v-140-psi-16-cfm-5hp-3ph>: HTTP status code is not handled or not allowed 2026-01-24 19:34:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-tapered-roller-bearing-e32307j-35mm-bore-80mm-od>: HTTP status code is not handled or not allowed 2026-01-24 19:34:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 19:34:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-20l-bs-powersports-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 19:34:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evolve-skate-inspired-work-shoes-composite-toe-size-9-5w-olive-gray>: HTTP status code is not handled or not allowed 2026-01-24 19:34:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 19:34:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 19:34:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kinetik-aptz7s-powersports-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 19:34:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/lubricants/food_grade_lubricants already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:34:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c-flange-kit-pew360tc-prem-eff-364t-365t>: HTTP status code is not handled or not allowed 2026-01-24 19:34:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:34:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extreme-xta20hl-bs-powersports-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 19:34:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:34:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:34:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rack-enclosure-cabinet-cantilever-fixed-shelf-60lb-capacity-2urm-srshelf2p>: HTTP status code is not handled or not allowed 2026-01-24 19:34:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:34:42 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/lubricants/food_grade_lubricants landed on page that is not a product page. 2026-01-24 19:34:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss-washdown-duty-motor-wss13-18-56crd-tefc-round-c-3-ph-1-3-hp-1800-rpm>: HTTP status code is not handled or not allowed 2026-01-24 19:34:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:34:45 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 19:34:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pgo-cp50-50cc-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 19:34:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 19:34:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-24 19:34:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-24 19:34:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 19:34:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-frame-motor-wwht15-12-284tc-gp-tefc-rigid-c-3-ph-284tc-193-fla>: HTTP status code is not handled or not allowed 2026-01-24 19:34:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ft-one-fit-non-belted-full-body-harness-standard-1-d-ring-quick-connect-legs-3x-large>: HTTP status code is not handled or not allowed 2026-01-24 19:34:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/apc-backups-500-bf500-gr-ups-replacement-battery-5ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 19:35:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1000ft-cat5e-cat5-350mhz-bulk-solid-core-plenum-cable-1000>: HTTP status code is not handled or not allowed 2026-01-24 19:35:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/datex-series-37-printer-medical-replacement-battery-1-3ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 19:35:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ft-one-fit-non-belted-full-body-harness-standard-1-d-ring-tongue-buckle-legs-x-small>: HTTP status code is not handled or not allowed 2026-01-24 19:35:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 14121 pages (at 92 pages/min), scraped 9737 items (at 60 items/min) 2026-01-24 19:35:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bb-hr33-12-sealed-lead-acid-replacement-battery-26ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 19:35:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ready-built153-2-tier-6-doors-locker-36w-x-18d-x-78h-black-assembled>: HTTP status code is not handled or not allowed 2026-01-24 19:35:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ready-built153-3-tier-3-doors-locker-12w-x-18d-x-78h-light-gray-assembled>: HTTP status code is not handled or not allowed 2026-01-24 19:35:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/k-k-jump-n-carry-jump-starter-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 19:35:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/boards_easels/dry_erase_whiteboards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:35:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-tech-closed-stand-alone-shelving-8-shelves-500-lb-cap-36w-x-18d-x-87h-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 19:35:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sureway-1003-sealed-lead-acid-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 19:35:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-tech-closed-style-metal-shelving-6-shelves-500-lb-cap-36w-x-18d-x-87h-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 19:35:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-tech-closed-style-metal-shelving-8-shelves-add-on-unit-36w-x-12d-x-87h-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 19:35:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/boards_easels/dry_erase_whiteboards landed on page that is not a product page. 2026-01-24 19:35:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/fans_blowers/cooling_fans/home_office_fans already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:35:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/outdo-ot9-12hr-sealed-lead-acid-replacement-battery-9ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 19:35:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eco-flo-pup61-portable-utility-pump>: HTTP status code is not handled or not allowed 2026-01-24 19:35:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-in-aluminum-wall-exhaust-shutter-rear-flange>: HTTP status code is not handled or not allowed 2026-01-24 19:35:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mighty-products-mn-dn-0814-debris-safety-netting-8-x-14>: HTTP status code is not handled or not allowed 2026-01-24 19:35:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/fans_blowers/cooling_fans/home_office_fans landed on page that is not a product page. 2026-01-24 19:35:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-in-backdraft-damper>: HTTP status code is not handled or not allowed 2026-01-24 19:35:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vision-cp1290-cp-1290-12v-9ah-ups-battery>: HTTP status code is not handled or not allowed 2026-01-24 19:35:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/batterymart-sla-7-sealed-lead-acid-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 19:35:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-16-inch-made-in-usa-jobbers-length-drill-blank>: HTTP status code is not handled or not allowed 2026-01-24 19:35:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/work-height-platform-truck-rnb2-3060-6mr-with-lower-shelf-30-x-60-fixed-height>: HTTP status code is not handled or not allowed 2026-01-24 19:35:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drill-drive-adapter-for-ozwinhnd2-carbon-steel-brake-winch>: HTTP status code is not handled or not allowed 2026-01-24 19:36:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-in-5-1-2-in-black-steel-pipe-nipple-150-psi-lead-free>: HTTP status code is not handled or not allowed 2026-01-24 19:36:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enersys-np2-3-12-sealed-lead-acid-replacement-battery-5ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 19:36:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/exam_patient_room_supplies/exam_room_consumables/table_paper already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:36:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tylenol-extra-strength-tablets-indigestion-heartburn-pain-15-bx>: HTTP status code is not handled or not allowed 2026-01-24 19:36:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powervar-abce420-11-800va-12v-7ah-ups-battery>: HTTP status code is not handled or not allowed 2026-01-24 19:36:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 14214 pages (at 93 pages/min), scraped 9807 items (at 70 items/min) 2026-01-24 19:36:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lavallette-7-piece-outdoor-dining-set>: HTTP status code is not handled or not allowed 2026-01-24 19:36:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/desiccator-non-vacuum-white-base-polypropylenepolycarbonate-6-dia>: HTTP status code is not handled or not allowed 2026-01-24 19:36:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tensacone-yellow-75-ft-black-yellow-chevron-belt>: HTTP status code is not handled or not allowed 2026-01-24 19:36:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/exam_patient_room_supplies/exam_room_consumables/table_paper landed on page that is not a product page. 2026-01-24 19:36:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reagent-bottle-narrow-mouth-hdpe-30ml-capacity-amber-pack-of-72>: HTTP status code is not handled or not allowed 2026-01-24 19:36:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6010zz-deep-groove-ball-bearing-double-shielded-50mm-bore-80mm-od>: HTTP status code is not handled or not allowed 2026-01-24 19:36:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wash-bottle-unitary-ldpe-250ml-capacity-white-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-24 19:36:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/teardrop-bolted-pallet-rack-add-on-108w-x-36d-x-120h>: HTTP status code is not handled or not allowed 2026-01-24 19:36:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/63052rs-p53-deep-groove-ball-bearing-double-sealed-abec-5-25mm-bore-62mm-od>: HTTP status code is not handled or not allowed 2026-01-24 19:36:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fortress-2200-fs-u1-wheelchair-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 19:36:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/500-lb-load-capacity-fiberglass-step-ladder-8-ft-dxl381008>: HTTP status code is not handled or not allowed 2026-01-24 19:36:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:36:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simplicity-20h-lawn-and-garden-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 19:36:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:36:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:36:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelf-truck-30l-x-14w-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 19:36:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:36:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:36:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/permanent-marker-chisel-red-ink-dozen-B785508>: HTTP status code is not handled or not allowed 2026-01-24 19:36:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-24 19:36:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60440226813-cross-pad-341d-3w-x-3l-x-1-80>: HTTP status code is not handled or not allowed 2026-01-24 19:36:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:36:49 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-24 19:36:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tru-ray-heavyweight-construction-paper-12x9-salmon-50-sheets>: HTTP status code is not handled or not allowed 2026-01-24 19:36:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sunworks-construction-paper-9x12-holiday-green-50-sheets>: HTTP status code is not handled or not allowed 2026-01-24 19:36:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-24 19:36:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8020-15-series-1-12-x-3-ultra-lite-six-t-slotted-extrusion-profile-120l-stock-bar>: HTTP status code is not handled or not allowed 2026-01-24 19:36:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-12kse362-emergency-light-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 19:36:56 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-24 19:36:56 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-24 19:36:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers>: HTTP status code is not handled or not allowed 2026-01-24 19:36:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rsvp-ballpoint-retractable-pen-refillable-medium-black-ink-dozen>: HTTP status code is not handled or not allowed 2026-01-24 19:37:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:37:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10309-532-hex-end-power-bit-3-handle-14-stock>: HTTP status code is not handled or not allowed 2026-01-24 19:37:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dyna-ray-dr7497s-emergency-light-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 19:37:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/automatic-battery-isolator-w-optional-override-switch-300-amp>: HTTP status code is not handled or not allowed 2026-01-24 19:37:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 14305 pages (at 91 pages/min), scraped 9869 items (at 62 items/min) 2026-01-24 19:37:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/116ex-b-bracket-wall-elbow-for-116degex-fj>: HTTP status code is not handled or not allowed 2026-01-24 19:37:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-24 19:37:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hubbell-12-800-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 19:37:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x3-8-xhigh-density-poly-acrylic-wool-roller-cover-12-case-18kl038>: HTTP status code is not handled or not allowed 2026-01-24 19:37:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/material-handling/ladders_personnel_lifts/step_ladders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:37:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/material-handling/ladders_personnel_lifts/step_ladders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:37:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hyperlite-extension-ladder-fiberglass-32-type-ia-300-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 19:37:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anti-static-nitrile-gloves-powder-free-5-mil-thickness-xx-large-black-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 19:37:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clip-swinging-doors-pair-36-inch-w-x-85-inch-h-marine-blue>: HTTP status code is not handled or not allowed 2026-01-24 19:37:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/material-handling/ladders_personnel_lifts/step_ladders landed on page that is not a product page. 2026-01-24 19:37:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/material-handling/ladders_personnel_lifts/step_ladders landed on page that is not a product page. 2026-01-24 19:37:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/big-beam-etxfl-emergency-light-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 19:37:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leoch-lp1212-12v-12ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-24 19:37:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/boat-shaped-conference-table-w-elliptical-base-95wx44dx29-1-2h-coastal-gray>: HTTP status code is not handled or not allowed 2026-01-24 19:37:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/umbrellas_awnings_shades/outdoor_umbrellas_bases/umbrellas already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:37:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/big-beam-2cl6s16-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 19:37:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-single-door-panel-class-a-4w-x-12h-white>: HTTP status code is not handled or not allowed 2026-01-24 19:37:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/material-handling/mezzanines_modular_structures/modular-wall-systems already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:37:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-single-door-panel-class-c-4w-x-8h-white>: HTTP status code is not handled or not allowed 2026-01-24 19:37:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/outdoor_furniture/umbrellas_awnings_shades/outdoor_umbrellas_bases/umbrellas landed on page that is not a product page. 2026-01-24 19:37:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-locker-accessory-closed-front-base-12w-x-6h-tan>: HTTP status code is not handled or not allowed 2026-01-24 19:37:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6423-double-flange-linear-bearing-brake-kit-ready>: HTTP status code is not handled or not allowed 2026-01-24 19:37:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/material-handling/mezzanines_modular_structures/modular-wall-systems landed on page that is not a product page. 2026-01-24 19:37:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esd-3-4-length-jacket-snap-cuff-light-blue-small>: HTTP status code is not handled or not allowed 2026-01-24 19:37:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3627-socket-head-cap-screw-114l>: HTTP status code is not handled or not allowed 2026-01-24 19:37:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2418-flanged-mount-caster-base-plate>: HTTP status code is not handled or not allowed 2026-01-24 19:38:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-dual-paper-towel-dispenser-w-gliding-rods-large-clear>: HTTP status code is not handled or not allowed 2026-01-24 19:38:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5750-double-shaft-blank-mounting-plate-15>: HTTP status code is not handled or not allowed 2026-01-24 19:38:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6531-highcycle-single-flange-linear-bearing-brake-kit-ready>: HTTP status code is not handled or not allowed 2026-01-24 19:38:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 14396 pages (at 91 pages/min), scraped 9935 items (at 66 items/min) 2026-01-24 19:38:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/-traffic-safety-sign-egp-aluminum-24-x-24-whiteblue>: HTTP status code is not handled or not allowed 2026-01-24 19:38:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3120-slidein-economy-tslot-stud>: HTTP status code is not handled or not allowed 2026-01-24 19:38:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/diagonal-right-arrow-traffic-safety-sign-hip-aluminum-15-x-21-whiteblue>: HTTP status code is not handled or not allowed 2026-01-24 19:38:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/parking_traffic_signs/parking_traffic_signs_1 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:38:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mojo-remote-control-hw1011-decoy-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 19:38:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ner020cd-20-ner-dual-speed-electric-chain-hoist-2-ton-20-lift-71-ftmin-230v>: HTTP status code is not handled or not allowed 2026-01-24 19:38:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nerp020c-20-ner-electric-chain-hoist-push-trolley-2-ton-20-lift-7-ftmin-230v>: HTTP status code is not handled or not allowed 2026-01-24 19:38:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/parking_traffic_signs/parking_traffic_signs_1 landed on page that is not a product page. 2026-01-24 19:38:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ktm-mxc-racing-525cc-motorcycle-replacement-battery-2000-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 19:38:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nerm020c-l-20-ner-electric-hoist-motor-trolley-2-ton-20-lift-7-ftmin-208v>: HTTP status code is not handled or not allowed 2026-01-24 19:38:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nerm025s-l-15-ner-electric-hoist-motor-trolley-2-12-ton-15-lift-22-ftmin-208v>: HTTP status code is not handled or not allowed 2026-01-24 19:38:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drum-lifter-top-chine-15-23-dia-rimmed-steel-fiber-or-poly-1000lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-24 19:38:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-channel-variable-pipettor-71211109-20-200ul-volume-range>: HTTP status code is not handled or not allowed 2026-01-24 19:38:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u-series-utility-pump-house-heater-with-thermostat-500125w-240120v-gray>: HTTP status code is not handled or not allowed 2026-01-24 19:38:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pgo-t-rex-125-125cc-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 19:38:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96-x-48-slatwall-panel-pearwood-laminate-without-inserts-pkg-qty-4>: HTTP status code is not handled or not allowed 2026-01-24 19:38:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-mx-z-adrenaline-600-ho-sdi-597cc-snowmobile-replacement-battery-2008-12v>: HTTP status code is not handled or not allowed 2026-01-24 19:38:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oxford-28-square-commercial-planter-white>: HTTP status code is not handled or not allowed 2026-01-24 19:38:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lionville-systems-power-rider-sl-250-medical-replacement-battery-9ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 19:38:59 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-24 19:38:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/columnar-pad-11-inch-x-17-inch-24-columns-36-lines-green-100-sheets-pad>: HTTP status code is not handled or not allowed 2026-01-24 19:38:59 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:10 2026-01-24 19:39:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sigmastek-sp12-22-sp12-22hr-sealed-lead-acid-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 19:39:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ibt-technologies-bt10-4-sealed-lead-acid-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 19:39:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/material-handling/lift_tables_work_positioners/pallet_carousels_positioners already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:39:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-source-u1-35-91-225-sealed-lead-acid-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 19:39:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 14494 pages (at 98 pages/min), scraped 10010 items (at 75 items/min) 2026-01-24 19:39:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-pocket-drawer-organizer-plastic-tray-10l-x-3w-x-14h>: HTTP status code is not handled or not allowed 2026-01-24 19:39:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=jtm-4vs-mill3-axis-acu-rite-vue-dro-x-y-axis-powerfeeds already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:39:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bubble-sof-tred-12-thick-safety-anti-fatigue-floor-mat-3x60-black>: HTTP status code is not handled or not allowed 2026-01-24 19:39:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/material-handling/lift_tables_work_positioners/pallet_carousels_positioners landed on page that is not a product page. 2026-01-24 19:39:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90-degree-liquid-tight-connector-alloy-steel-1-size-5-pk>: HTTP status code is not handled or not allowed 2026-01-24 19:39:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/935-1006-935-case-w-foam-220l-x-140w-x-90h-olive>: HTTP status code is not handled or not allowed 2026-01-24 19:39:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=jtm-4vs-mill3-axis-acu-rite-vue-dro-x-y-axis-powerfeeds landed on page that is not a product page. 2026-01-24 19:39:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-step-tall-nestable-plas-step-stand-yell-25-w-25-d-14-3-4-h-nbst-1-14>: HTTP status code is not handled or not allowed 2026-01-24 19:39:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drive-medical-daytona-3-s35006-wheelchair-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 19:39:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electric-mobility-turnabout-heavy-duty-wheelchair-replacement-battery-35ah-12v>: HTTP status code is not handled or not allowed 2026-01-24 19:39:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insect-control-air-curtain-42-inch-w-door-480v-3-4hp-3-ph-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 19:39:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kpex-95-16-200-insulated-cable-shears-1000v-8-oal>: HTTP status code is not handled or not allowed 2026-01-24 19:39:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:39:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triple-grip-anchor-kit-w-combo-hex-head-screw-12-size-101-pieces>: HTTP status code is not handled or not allowed 2026-01-24 19:39:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:39:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1005-standard-class-2-two-tone-mesh-zipper-safety-vest-lime-medium>: HTTP status code is not handled or not allowed 2026-01-24 19:39:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1001-standard-class-2-mesh-zipper-safety-vest-lime-4xl>: HTTP status code is not handled or not allowed 2026-01-24 19:39:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers>: HTTP status code is not handled or not allowed 2026-01-24 19:39:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1801-standard-class-2-5point-breakaway-vest-lime-medium>: HTTP status code is not handled or not allowed 2026-01-24 19:39:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibration-shock-absorption-mount-threaded-stud-156-inch-dia-100-inch-h-5-16-18-thread>: HTTP status code is not handled or not allowed 2026-01-24 19:39:50 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-24 19:39:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lightalarms-8700018-retrofit-emergency-light-replacement-battery-18ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 19:39:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1008-standard-class-2-two-tone-mesh-hook-loop-safety-vest-orange-large>: HTTP status code is not handled or not allowed 2026-01-24 19:39:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/69000-series-box-box-file-mobile-pedestal-16-inchw-x-22-inchd-x-28-1-4-inchh-mahogany>: HTTP status code is not handled or not allowed 2026-01-24 19:40:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grip-24-w-8-step-stl-roll-lad-14-d-top-stp-hndrl-lck-orng-fs083214go>: HTTP status code is not handled or not allowed 2026-01-24 19:40:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oval-head-wall-plate-screw-slotted-almond-6-32-x-34-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 19:40:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/wall_mount_belts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:40:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manual-pneumatic-lockout-valve-with-soft-start-y1523b6102-1-npt>: HTTP status code is not handled or not allowed 2026-01-24 19:40:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/pneumatic_valves/directional_control_valves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:40:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/pneumatic_valves/directional_control_valves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:40:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-piece-knockout-seal-carbon-steel-2-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 19:40:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 14585 pages (at 91 pages/min), scraped 10077 items (at 67 items/min) 2026-01-24 19:40:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:40:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inventory-cone-lime-3-sided-with-dry-erase-decal>: HTTP status code is not handled or not allowed 2026-01-24 19:40:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/peg-perego-santa-fe-train-set-ride-on-toys-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 19:40:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/concrete-screw-anchor-slotted-hex-washer-head-14-x-3-34-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 19:40:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/pneumatic_valves/directional_control_valves landed on page that is not a product page. 2026-01-24 19:40:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/pneumatic_valves/directional_control_valves landed on page that is not a product page. 2026-01-24 19:40:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-head-wood-screw-square-carbon-steel-8-x-2-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 19:40:18 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/wall_mount_belts landed on page that is not a product page. 2026-01-24 19:40:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 19:40:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yamaha-yfm550-grizzly-fi-4x4-auto-eps-550cc-atv-replacement-battery-2009-2014-12v>: HTTP status code is not handled or not allowed 2026-01-24 19:40:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-washer-stainless-steel-0-316-id-0-716-od-8-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 19:40:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-head-wood-screw-phillips-brass-8-x-1-14-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 19:40:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/binders_covers/clipboards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:40:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medium-weight-one-hole-emt-strap-carbon-steel-14-gauge-34-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 19:40:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aprilia-pegaso-trail-650cc-motorcycle-replacement-battery-2009-2012-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 19:40:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lag-shield-38-size-short-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 19:40:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/binders_covers/clipboards>: HTTP status code is not handled or not allowed 2026-01-24 19:40:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plate-strap-carbon-steel-12-gauge-1-12-x-18-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 19:40:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/k-lath-metal-piercing-screw-phillips-wafer-head-8-x-12-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 19:40:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/k-lath-metal-piercing-screw-square-drive-wafer-head-8-x-1-12-size-pk-of-100>: HTTP status code is not handled or not allowed 2026-01-24 19:40:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/strain-relief-connector-steel-34-dia-purple-grommet>: HTTP status code is not handled or not allowed 2026-01-24 19:40:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-strut-strap-carbon-steel-1-14-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 19:40:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/antimicrobial-standard-stapler-with-5000-staples-and-staple-remover-black>: HTTP status code is not handled or not allowed 2026-01-24 19:40:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/bags_cases/computer_bags_briefcases already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:40:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-head-tap-bolt-low-carbon-steel-34-10-x-4-12-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 19:40:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-head-tap-bolt-low-carbon-steel-38-16-x-34-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 19:40:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aprilia-rsv-1000-mille-r-1000cc-motorcycle-replacement-battery-2001-2005-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 19:41:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/american-ironhorse-thunder-1820cc-motorcycle-replacement-battery-2000-2002-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 19:41:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/bags_cases/computer_bags_briefcases landed on page that is not a product page. 2026-01-24 19:41:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ross-5-3-closed-center-double-solenoid-controlled-directional-valve-24vdc-9577k1010w>: HTTP status code is not handled or not allowed 2026-01-24 19:41:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-vis-hooded-sweatshirt-type-o-ansi-class-2-3xl-blackfluorescent-yellowgreen>: HTTP status code is not handled or not allowed 2026-01-24 19:41:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 14678 pages (at 93 pages/min), scraped 10142 items (at 65 items/min) 2026-01-24 19:41:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/protecta-edge-self-retracting-lifeline-galvanized-cable--steel-swivel-snap-hook-33l>: HTTP status code is not handled or not allowed 2026-01-24 19:41:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-sap202-10g-pillow-block-bearing-eccentric-locking-collar-bore-15-875-mm>: HTTP status code is not handled or not allowed 2026-01-24 19:41:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metatarsal-knee-boots-chevron-outsole-steel-toe-size-7-16h-brick-redcream>: HTTP status code is not handled or not allowed 2026-01-24 19:41:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/orthofab-lifestyles-770-mobility-scooter-replacement-battery-75ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 19:41:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jacket-5-channel-cable-protector-42000-lb-capacity-20-x-50-x-1-78-blue>: HTTP status code is not handled or not allowed 2026-01-24 19:41:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/922cr-level-5-cut-resistant-nitriledipped-dir-gloves-gray-xl-1-pair>: HTTP status code is not handled or not allowed 2026-01-24 19:41:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/loki-safety-glassesgoggles-wfogoff-clear-af-lens-black-frame>: HTTP status code is not handled or not allowed 2026-01-24 19:41:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pride-mobility-coltdeluxe-mobility-scooter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 19:41:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flammable-cabinet-manual-close-single-door-54-gal-capacity-23-14w-x-34d-x-65h>: HTTP status code is not handled or not allowed 2026-01-24 19:41:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-row-cylindrical-roller-bearing-n222myc3-110mm-bore-200mm-od>: HTTP status code is not handled or not allowed 2026-01-24 19:41:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-vis-quilt-lined-jacket-type-r-ansi-class-3-m-fluorescent-yellowgreenblack>: HTTP status code is not handled or not allowed 2026-01-24 19:41:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-chrome-wire-shelving-unit-starter-54w-x-24d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 19:41:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electric-mobility-deluxe-cycle-mobility-scooter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 19:41:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bombardier-skandic-tundra-lt-550f-550cc-snowmobile-battery-2008-2011-12v>: HTTP status code is not handled or not allowed 2026-01-24 19:41:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-bolt-open-add-on-oat75-1242-5x-42-inch-w-x-12-inch-d-x-75-inch-h-5-shelves-18-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 19:41:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drive-medical-cirrus-plus-ec-folding-powerchair-mobility-scooter-battery-35ah-12v>: HTTP status code is not handled or not allowed 2026-01-24 19:41:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-adhesive-index-tabs-with-printable-inserts-1-1-2-width-clear-25-tabs-pack>: HTTP status code is not handled or not allowed 2026-01-24 19:42:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss6002zz-deep-groove-ball-bearing-stainless-steel-double-shielded-od-32mm-bore-15mmmetric>: HTTP status code is not handled or not allowed 2026-01-24 19:42:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/52042rs-double-row-angular-contact-ball-bearing-double-sealed-od-47mm-bore-20mm-metric>: HTTP status code is not handled or not allowed 2026-01-24 19:42:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pre-inked-message-stamp-mailed-1-5-8-x-1-2-red>: HTTP status code is not handled or not allowed 2026-01-24 19:42:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/purchase-order-book-3-part-carbonless-5-9-16-x-8-7-16-50-sets-book>: HTTP status code is not handled or not allowed 2026-01-24 19:42:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 14777 pages (at 99 pages/min), scraped 10220 items (at 78 items/min) 2026-01-24 19:42:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/certificate-holder-25-x-8-1-2-dark-blue-5-pack>: HTTP status code is not handled or not allowed 2026-01-24 19:42:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-corrugated-boxes-13-x-9-x-4-200lb-test-ect-32-25-pack>: HTTP status code is not handled or not allowed 2026-01-24 19:42:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bombardier-expedition-sport-600-ace-600cc-snowmobile-battery-2011-2017-12v>: HTTP status code is not handled or not allowed 2026-01-24 19:42:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-chrome-wire-shelving-unit-starter-72w-x-36d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 19:42:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-poly-green-wire-shelving-unit-add-on-72w-x-12d-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 19:42:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bombardier-gsx-le-1200-1170cc-snowmobile-replacement-battery-2014-2015-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 19:42:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-wood-bench-w-stainless-frame-espresso-wood-finish>: HTTP status code is not handled or not allowed 2026-01-24 19:42:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bollardgard-anywhere-yellow-7-x-52-with-25-lb-octagon-rubber-base-bgaw752yw>: HTTP status code is not handled or not allowed 2026-01-24 19:42:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-poly-green-wire-shelving-unit-add-on-24w-x-14d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 19:42:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-tundra-lt-ace-600-600cc-snowmobile-replacement-battery-2012-2018-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 19:42:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-plastic-locker-11-9z1212720-z-style-12-x-12-x-72-1-wide-blue>: HTTP status code is not handled or not allowed 2026-01-24 19:42:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-ski-doo-gsx-600-ho-sdi-593cc-snowmobile-replacement-battery-2004-2005-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 19:42:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-summit-800-ho-adrenaline-800cc-snowmobile-replacement-battery-2006-12v>: HTTP status code is not handled or not allowed 2026-01-24 19:42:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-mx-z-600-adrenaline-600cc-snowmobile-replacement-battery-2004-2010-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 19:42:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kawasaki-mule-1000-utility-vehicle-450cc-utv-replacement-battery-1987-2012-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 19:42:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inverter-duty-motor-y982-254then8058-75hp-230-460v-1800rpm-3ph-254tc-epnv>: HTTP status code is not handled or not allowed 2026-01-24 19:42:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/building_construction_tape/gaffer_tape already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:42:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-712-2-piece-full-port-ball-valve-stainless-steel-38-threaded>: HTTP status code is not handled or not allowed 2026-01-24 19:42:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-534hpnl-lf-brass-heavy-pattern-hose-bibb-34-mnpt>: HTTP status code is not handled or not allowed 2026-01-24 19:42:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-response-2-complete-orange>: HTTP status code is not handled or not allowed 2026-01-24 19:42:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/building_construction_tape/gaffer_tape>: HTTP status code is not handled or not allowed 2026-01-24 19:42:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baxter-healthcare-0007mczz-medical-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 19:42:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/parks-medical-6ca-doppler-medical-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 19:43:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-365di-g-butterfly-valve-with-epdm-seals--gear-operator-5-lug>: HTTP status code is not handled or not allowed 2026-01-24 19:43:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emabfws-rf-bottle-filling-station-retrofit-kit-for-emabf-water-coolers>: HTTP status code is not handled or not allowed 2026-01-24 19:43:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/severe-duty-motor-e490-286tstfna6804-30hp-230-460v-3600rpm-3ph-286ts-fr-tefc>: HTTP status code is not handled or not allowed 2026-01-24 19:43:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 14875 pages (at 98 pages/min), scraped 10293 items (at 73 items/min) 2026-01-24 19:43:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coopower-cp6-4-5-sealed-lead-acid-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 19:43:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-90407-2-hp-220v-60hz-ultra-quiet-oil-free-air-compressor-motor>: HTTP status code is not handled or not allowed 2026-01-24 19:43:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/highlighters_markers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:43:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coopower-cpd12-24-sealed-lead-acid-replacement-battery-26ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 19:43:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fan-blower-motor-pg240-1-20hp-1625-1350rpm-115-200-230v-1ph-48z-fr-tenv>: HTTP status code is not handled or not allowed 2026-01-24 19:43:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fan-blower-motor-k1487-1-3hp-1140-950rpm-208-230-460v-3ph-56-fr-tenv>: HTTP status code is not handled or not allowed 2026-01-24 19:43:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/highlighters_markers landed on page that is not a product page. 2026-01-24 19:43:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60010013989-stikitgold-paper-df-disc-roll-216u-5dia-x-nh-p320-grit>: HTTP status code is not handled or not allowed 2026-01-24 19:43:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kung-long-wp22-12rn-sealed-lead-acid-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 19:43:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coopower-cphl12-18-sealed-lead-acid-replacement-battery-18ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 19:43:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ulb420-15-multi-outlet-power-unit-120v-20a-10-34l-4-outlets-15-cord>: HTTP status code is not handled or not allowed 2026-01-24 19:43:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nm2011-90-flat-elbow-120v-15a-3-316l>: HTTP status code is not handled or not allowed 2026-01-24 19:43:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-shelf-nexelon-blue-wire-shelving-unit-starter-30w-x-14d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 19:43:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ajc-wka12-26nb-12v-26ah-wheelchair-battery>: HTTP status code is not handled or not allowed 2026-01-24 19:43:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/soft-candy-individually-wrapped-465-oz-240box>: HTTP status code is not handled or not allowed 2026-01-24 19:43:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eaglepicher-cf-18-c-sealed-lead-acid-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 19:43:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sunnyway-sw12280-iii-sealed-lead-acid-replacement-battery-26ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 19:43:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-gallon-1-door-manual-left-hinge-fume-hood-acid-cabinet-24x21-5-8x35-3-4-silver>: HTTP status code is not handled or not allowed 2026-01-24 19:43:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/special_purpose_carts_trucks/medical_supply_carts/procedure_carts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:43:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ge-caddx-networx-nx-4-12v-7ah-alarm-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 19:44:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-nexelate-silver-epoxy-wire-shelving-unit-starter-72w-x-18d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 19:44:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fallte-8450-carabiner-medium-twist-lock-1-gate-opening>: HTTP status code is not handled or not allowed 2026-01-24 19:44:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/permobil-chairman-basic-wheelchair-replacement-battery-75ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 19:44:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/special_purpose_carts_trucks/medical_supply_carts/procedure_carts landed on page that is not a product page. 2026-01-24 19:44:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/four-wing-free-air-fan-blade-interchangeable-hub-aluminum-blade-ccw-14-inch-dia-27-pitch>: HTTP status code is not handled or not allowed 2026-01-24 19:44:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/wire_shelving/poly_zbrite_wire_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:44:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 14970 pages (at 95 pages/min), scraped 10365 items (at 72 items/min) 2026-01-24 19:44:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/encore-mc72-prowler-lawn-and-garden-replacement-battery-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 19:44:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:44:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:44:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-shelf-poly-z-brite-wire-shelving-unit-starter-36w-x-14d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 19:44:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:44:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-three-wing-condenser-fan-blade-30-inch-dia-cw-33-pitch-720-850-1000-rpm>: HTTP status code is not handled or not allowed 2026-01-24 19:44:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/bags_cases/computer_bags_briefcases already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:44:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:44:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-poly-z-brite-wire-shelving-unit-starter-30w-x-18d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 19:44:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-24 19:44:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-inlet-blower-wheel-5-3-4-inch-dia-ccw-3600-rpm-1-4-inch-bore-2-inch-w-plastic>: HTTP status code is not handled or not allowed 2026-01-24 19:44:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/wire_shelving/poly_zbrite_wire_shelves landed on page that is not a product page. 2026-01-24 19:44:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers>: HTTP status code is not handled or not allowed 2026-01-24 19:44:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-24 19:44:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers>: HTTP status code is not handled or not allowed 2026-01-24 19:44:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/bags_cases/computer_bags_briefcases landed on page that is not a product page. 2026-01-24 19:44:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/octagonal-sledge-hammer-16-lb-head-36-long-wooden-handle>: HTTP status code is not handled or not allowed 2026-01-24 19:44:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/62-pc-import-cobalt-polished-jobbers-1-60-drill-set>: HTTP status code is not handled or not allowed 2026-01-24 19:44:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-front-for-clip-type-shelving-1-1-2-inch-h-x-36-inch-w-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 19:44:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/york-wide-light-d2e2-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 19:44:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sonnenschein-7191185-emergency-light-replacement-battery-1-3ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 19:44:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/seating/reception_sofas_sectionals already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:44:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/office_furniture/literature_mail_sorters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:44:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/office_furniture/literature_mail_sorters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:44:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wm-cascade-10-pocket-magazine-rack-mahogany>: HTTP status code is not handled or not allowed 2026-01-24 19:44:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/coat_racks_umbrella_stands/luggage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:44:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/best-technologies-r-1-emergency-light-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 19:44:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/medical_chart_file_holders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:44:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transport-storage-cabinet-all-welded-36w-x-24d-x-72h-textured-black>: HTTP status code is not handled or not allowed 2026-01-24 19:44:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/office_furniture/literature_mail_sorters>: HTTP status code is not handled or not allowed 2026-01-24 19:44:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/seating/reception_sofas_sectionals landed on page that is not a product page. 2026-01-24 19:44:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yamaha-yfm66fa-fg-grizzly-660cc-atv-replacement-battery-2005-2008-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 19:44:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/coat_racks_umbrella_stands/luggage_racks>: HTTP status code is not handled or not allowed 2026-01-24 19:44:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/five-shelf-extra-hd-boltless-shelving-add-on-36inw-x-24ind-x-60inh-wood-deck-usa>: HTTP status code is not handled or not allowed 2026-01-24 19:44:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/medical_chart_file_holders>: HTTP status code is not handled or not allowed 2026-01-24 19:44:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/office_furniture/literature_mail_sorters landed on page that is not a product page. 2026-01-24 19:44:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/awsw-partition-80x7-f-navy-blue>: HTTP status code is not handled or not allowed 2026-01-24 19:45:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/awrd-partition-70x84-f-black>: HTTP status code is not handled or not allowed 2026-01-24 19:45:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/filter-monitor-gauge-ab-12000>: HTTP status code is not handled or not allowed 2026-01-24 19:45:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vacuum-filter-element-ab-e11003-2-inch>: HTTP status code is not handled or not allowed 2026-01-24 19:45:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yamaha-yfv600fw-grizzly-600cc-atv-replacement-battery-1998-2001-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 19:45:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 15057 pages (at 87 pages/min), scraped 10421 items (at 56 items/min) 2026-01-24 19:45:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-security-glove-box-holder-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 19:45:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aprilia-rsv4-rf-le-1000cc-motorcycle-replacement-battery-2016-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 19:45:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/imported-plain-angle-plates-ground-finish-8-inch-x-8-inch-x-8-inch>: HTTP status code is not handled or not allowed 2026-01-24 19:45:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/big-dog-chopper-dt-1900cc-motorcycle-replacement-battery-2005-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 19:45:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pam-diabetic-triple-layer-anti-fatigue-insole-mens-1213>: HTTP status code is not handled or not allowed 2026-01-24 19:45:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pic-a-watt-wall-heater-paw1215-1500w-max-120v-compact-white>: HTTP status code is not handled or not allowed 2026-01-24 19:45:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/footwear_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:45:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electric-cove-heater-kcv1206-560w-120v-47l-pearl-white>: HTTP status code is not handled or not allowed 2026-01-24 19:45:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/critical-essential-bleeding-control-kit-1-24-persons-9-pieces-plastic-case>: HTTP status code is not handled or not allowed 2026-01-24 19:45:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/262100000-graduated-beaker-polypropylene-50ml-12-pk-48-case-clear>: HTTP status code is not handled or not allowed 2026-01-24 19:45:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kawasaki-jt1200-stx-r-1200cc-personal-watercraft-battery-2002-2007-12v>: HTTP status code is not handled or not allowed 2026-01-24 19:45:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gssafe-5008-class-3-hi-viz-moisture-wicking-birdeyes-short-sleeve-t-shirt-orange-xl-tall>: HTTP status code is not handled or not allowed 2026-01-24 19:45:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/footwear_accessories landed on page that is not a product page. 2026-01-24 19:45:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/si-1700-orbital-genie-adhering-mat-120v>: HTTP status code is not handled or not allowed 2026-01-24 19:45:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/503-0282-00-large-sample-set-model-h302>: HTTP status code is not handled or not allowed 2026-01-24 19:45:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/appliances/refrigerators_freezers/refrigerator_freezer_combos/undercounter_refrigerator_freezer_combos already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:45:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-roughneck-jumbo-hinged-tote-50-gallon-gray-270148>: HTTP status code is not handled or not allowed 2026-01-24 19:45:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xtreme-power-x-560-electric-scooter-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 19:45:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bollard-cover-cl1386n-7dia-x-72h-yellow-w-red-tape>: HTTP status code is not handled or not allowed 2026-01-24 19:45:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-nexelate-36w-x-30d-x-69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 19:45:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/star-ii-036d-deluxe-electric-scooter-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 19:45:49 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/appliances/refrigerators_freezers/refrigerator_freezer_combos/undercounter_refrigerator_freezer_combos landed on page that is not a product page. 2026-01-24 19:45:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/base-cover-lpgblue-26dia-x-41-1-4h-4-rings-blue>: HTTP status code is not handled or not allowed 2026-01-24 19:45:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-nexelate-48w-x-14d-x-60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 19:45:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sea-dye-marker-yellow-1-case-dx1517m>: HTTP status code is not handled or not allowed 2026-01-24 19:46:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/amigo-mobility-humvee-mobility-scooter-replacement-battery-55ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 19:46:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2du16107-cantilever-rack-double-sided-upright-107d-x-16h>: HTTP status code is not handled or not allowed 2026-01-24 19:46:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2sa42-cantilever-rack-straight-arm-no-lip-42-long>: HTTP status code is not handled or not allowed 2026-01-24 19:46:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 15156 pages (at 99 pages/min), scraped 10494 items (at 73 items/min) 2026-01-24 19:46:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-freeride-e-tec-800r-137-800cc-snowmobile-battery-2012-2017-12v>: HTTP status code is not handled or not allowed 2026-01-24 19:46:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wobbler-sign-holder-4>: HTTP status code is not handled or not allowed 2026-01-24 19:46:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u1474-ns-cantilever-rack-single-sided-upright-74w-x-14h>: HTTP status code is not handled or not allowed 2026-01-24 19:46:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hang-tab-system-516-hole-dia-34w-x-1-5h>: HTTP status code is not handled or not allowed 2026-01-24 19:46:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-piercing-channel-flagsign-holder-1-w-x-1-14h-od>: HTTP status code is not handled or not allowed 2026-01-24 19:46:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/abrasives/points_burrs/mounted_points already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:46:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jumbo-storage-cabinet-j2478su-02-welded-48w-x-24d-x-78h-medium-gray>: HTTP status code is not handled or not allowed 2026-01-24 19:46:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-nexelate-30w-x-14d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 19:46:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/schaefer-stacking-transport-container-14-6-2-h-pl-18-l-x-12-w-x-5-h-blue>: HTTP status code is not handled or not allowed 2026-01-24 19:46:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/abrasives/points_burrs/mounted_points>: HTTP status code is not handled or not allowed 2026-01-24 19:46:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foodservice-shovel-14-inch-white>: HTTP status code is not handled or not allowed 2026-01-24 19:46:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-tier-chair-cart>: HTTP status code is not handled or not allowed 2026-01-24 19:46:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/panel_carts_trucks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:46:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/open-top-dome-receptacles-15-gallon-white-9639wh>: HTTP status code is not handled or not allowed 2026-01-24 19:46:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exit-device-vertical-rod-grade-1-36-rhr>: HTTP status code is not handled or not allowed 2026-01-24 19:46:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8263frhl-hi-vis-fr-safety-vest-class-2-mesh-hook-loop-economy-4xl-5xl-lime>: HTTP status code is not handled or not allowed 2026-01-24 19:46:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8249z-s-hi-vis-surveyors-vest-class-2-zipper-economy-single-size-xs-lime>: HTTP status code is not handled or not allowed 2026-01-24 19:46:49 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/panel_carts_trucks landed on page that is not a product page. 2026-01-24 19:46:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-b62-5l650-v-belt-5-8-x-65>: HTTP status code is not handled or not allowed 2026-01-24 19:46:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/locks_lockout_devices/lockout_tagout_devices/lockout_padlocks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:46:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-b240-v-belt-5-8-x-243>: HTTP status code is not handled or not allowed 2026-01-24 19:46:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/locks_lockout_devices/lockout_tagout_devices/lockout_padlocks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:46:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hvac-motor-em3710t-5g-3-ph-75-hp-575-v-1800-rpm-tefc-213t-frame>: HTTP status code is not handled or not allowed 2026-01-24 19:46:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-motor-208-230-460-v-75-hp-1770-rpm-3-ph-213tc-opsb>: HTTP status code is not handled or not allowed 2026-01-24 19:47:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_microscopes/stereo_microscopes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:47:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3807-belgian-beer-13-oz-12-pack>: HTTP status code is not handled or not allowed 2026-01-24 19:47:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5213-glass-mug-coffee-13-oz-clear-12-pack>: HTTP status code is not handled or not allowed 2026-01-24 19:47:04 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/locks_lockout_devices/lockout_tagout_devices/lockout_padlocks landed on page that is not a product page. 2026-01-24 19:47:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_microscopes/stereo_microscopes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:47:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/locks_lockout_devices/lockout_tagout_devices/lockout_padlocks landed on page that is not a product page. 2026-01-24 19:47:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_microscopes/stereo_microscopes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:47:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelite-72w-x-24d-x-80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 19:47:09 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_microscopes/stereo_microscopes landed on page that is not a product page. 2026-01-24 19:47:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 15245 pages (at 89 pages/min), scraped 10557 items (at 63 items/min) 2026-01-24 19:47:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dolly-for-armless-plastic-stack-chair>: HTTP status code is not handled or not allowed 2026-01-24 19:47:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-black-epoxy-72w-x-14d-x-69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 19:47:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_microscopes/stereo_microscopes landed on page that is not a product page. 2026-01-24 19:47:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_microscopes/stereo_microscopes landed on page that is not a product page. 2026-01-24 19:47:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ken-forging-tbs-10-rh-turnbuckle-stub-1-8-x-9-carbon-right-hand>: HTTP status code is not handled or not allowed 2026-01-24 19:47:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washdown-motor-idcswdm3538-3-phase-05-hp-1750-rpm-208-230-460-volts-tenv-56c-fr>: HTTP status code is not handled or not allowed 2026-01-24 19:47:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-poly-green-42wx14dx60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 19:47:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/k290-scorpion-atv-tire-1108-2s-i-22x1100-8-2ply>: HTTP status code is not handled or not allowed 2026-01-24 19:47:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-motor-460-v-200-hp-1190-rpm-3-ph-449t-tefc>: HTTP status code is not handled or not allowed 2026-01-24 19:47:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unit-handling-motor-vebm3554t-5d-3-ph-15-hp-575-v-1760-rpm-tefc-145tc-frame>: HTTP status code is not handled or not allowed 2026-01-24 19:47:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/prefilled-homogenizer-tubes-2-0ml-silica-glass-beads-1-0mm-acid-washed-50-pk>: HTTP status code is not handled or not allowed 2026-01-24 19:47:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-poly-green-48wx14dx80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 19:47:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magic-clamp-magnetic-clamp-250ml-erlenmeyer>: HTTP status code is not handled or not allowed 2026-01-24 19:47:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/centrifugal-basket-for-sieva-3-500ml-non-perforated>: HTTP status code is not handled or not allowed 2026-01-24 19:47:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machined-chrome-steel-needle-roller-bearing-w-inner-ring-32-x-52-x-27mm>: HTTP status code is not handled or not allowed 2026-01-24 19:47:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washdown-motor-vewdm3554t-3-phase-15-hp-1755-rpm-208-230-460-volts-tenv-145tc-fr>: HTTP status code is not handled or not allowed 2026-01-24 19:47:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-poly-green-30w-x-14d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 19:47:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pistol-safe-wmc-1-059511-ws56-9-1-2-w-x-11-d-x-5-h-2-pistol-capacity-key-lock-sand>: HTTP status code is not handled or not allowed 2026-01-24 19:47:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pistol-safe-wmc-4-231208h-ws56-4-tier-12-w-x-8-d-x-23-h-8-pistol-cap-hasp-lock-sand>: HTTP status code is not handled or not allowed 2026-01-24 19:47:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96-w-x-30-d-maple-butcher-block-square-edge-workbench-adjustable-height-black>: HTTP status code is not handled or not allowed 2026-01-24 19:47:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machined-chrome-steel-needle-roller-bearing-w-inner-ring-22-x-34-x-16mm>: HTTP status code is not handled or not allowed 2026-01-24 19:47:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inverter-vector-motor-zdbrpm281254-3ph-125hp-1780-3555rpm-460v-tebc-fl2898>: HTTP status code is not handled or not allowed 2026-01-24 19:48:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/import-1-4-20-transfer-screw-set>: HTTP status code is not handled or not allowed 2026-01-24 19:48:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dc-scr-nema-4x-control-accessory-bc145-signal-isolator-board>: HTTP status code is not handled or not allowed 2026-01-24 19:48:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/idm-idnm-idwnm-motor-encoder-feedback-kit-enc01nv-a2-avtron-hs35m-tenv-254t-256t-5-24vdc>: HTTP status code is not handled or not allowed 2026-01-24 19:48:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 15346 pages (at 101 pages/min), scraped 10633 items (at 76 items/min) 2026-01-24 19:48:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelon-48w-x-18d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 19:48:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electric-pump-1-hp-2-gallon-3-way-3-position-manual-valve-10-pendant>: HTTP status code is not handled or not allowed 2026-01-24 19:48:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-motor-base-b447t-447t-3-4-x-3-1-2-bolt-size-w-2-adjustable-screw>: HTTP status code is not handled or not allowed 2026-01-24 19:48:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sf-x-2-2-x-46-sf-series-quick-detach-steel-bushing-2-bore>: HTTP status code is not handled or not allowed 2026-01-24 19:48:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/windlass-tensioner-economy-steel-34-1-14-strap-width-023-to-031-strap-gauge>: HTTP status code is not handled or not allowed 2026-01-24 19:48:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/celltreat-scientific-products-25l-pipette-ovation-f2-fixed-volume-two-stroke>: HTTP status code is not handled or not allowed 2026-01-24 19:48:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/pneumatic_hoses_connectors/pneumatic_quick_connect_couplings already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:48:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-strap-sheers-single-hand-operated-38-1-14-strap-width-up-to-031-strap-gauge>: HTTP status code is not handled or not allowed 2026-01-24 19:48:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rippled-hot-cups-12-oz-25-pack-brown>: HTTP status code is not handled or not allowed 2026-01-24 19:48:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/omnicart-cab-36-cap-big-beam-w-locking-panel-woodgrain-49-3-4w-x-17d-x-55-1-8h>: HTTP status code is not handled or not allowed 2026-01-24 19:48:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/definite_purpose_electric_motors/vertical_hollow_shaft_motors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:48:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/pneumatic_hoses_connectors/pneumatic_quick_connect_couplings>: HTTP status code is not handled or not allowed 2026-01-24 19:48:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/celltreat-scientific-products-5ml-pipette-tips-ovation-graduated-vistaclear-box-sterile-60-pack>: HTTP status code is not handled or not allowed 2026-01-24 19:48:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/handhole-gasket-spiral-wound-3-x-5-x-12-0175-thick-elliptical-pk-3>: HTTP status code is not handled or not allowed 2026-01-24 19:48:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/definite_purpose_electric_motors/vertical_hollow_shaft_motors>: HTTP status code is not handled or not allowed 2026-01-24 19:48:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/topoge-series-180-handhole-gasket-4-x-6-x-34-black-rubber-elliptical-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 19:48:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n-ferno-6476-thermal-insulated-freezer-jacket-l-navy>: HTTP status code is not handled or not allowed 2026-01-24 19:48:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roam-cordless-upright-vacuum-13-cleaning-width>: HTTP status code is not handled or not allowed 2026-01-24 19:48:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ex20-100c-15-sw-portable-carpet-extractor-56105416>: HTTP status code is not handled or not allowed 2026-01-24 19:48:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manual-4-way-valve-dc-power-unit-pu310-1-gal-steel-reservoir-250-inch-nptf-outlet>: HTTP status code is not handled or not allowed 2026-01-24 19:48:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smr5bsk-backstop-assembly-fits-reducer-styles-smr5wsmr5>: HTTP status code is not handled or not allowed 2026-01-24 19:49:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-series-clamp-for-hose-ssch075-3-4-inch-id-119-od>: HTTP status code is not handled or not allowed 2026-01-24 19:49:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10wtbk-31516-tapered-bushing-kit-3-1516-fits-reducer-style-smr10>: HTTP status code is not handled or not allowed 2026-01-24 19:49:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/gearboxes_speed_reducers/shaft_mount_reducers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:49:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wsmr6-251-shaft-mount-reducer-size-6-251-ratio-3-716-tapered-bore>: HTTP status code is not handled or not allowed 2026-01-24 19:49:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 15441 pages (at 95 pages/min), scraped 10706 items (at 73 items/min) 2026-01-24 19:49:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7wtbk-21516-tapered-bushing-kit-2-1516-fits-reducer-styles-smr7wsmr7>: HTTP status code is not handled or not allowed 2026-01-24 19:49:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/gearboxes_speed_reducers/speed_reducer_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:49:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bott-power-tool-holders-for-perfo-panels-package-of-5-238-dia>: HTTP status code is not handled or not allowed 2026-01-24 19:49:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sa-whiffle-typical-10-triple-48-gray>: HTTP status code is not handled or not allowed 2026-01-24 19:49:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/gearboxes_speed_reducers/shaft_mount_reducers landed on page that is not a product page. 2026-01-24 19:49:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/infant-icu-crib-e2080cl-standard-30w-x-44l-x-61h-flat-deck-confetti>: HTTP status code is not handled or not allowed 2026-01-24 19:49:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/gearboxes_speed_reducers/speed_reducer_accessories landed on page that is not a product page. 2026-01-24 19:49:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4000-10-ball-valve-flanged-10-cast-iron>: HTTP status code is not handled or not allowed 2026-01-24 19:49:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:49:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hickory-half-zip-pull-over-long-sleeve-shirt-5xl-gray>: HTTP status code is not handled or not allowed 2026-01-24 19:49:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:49:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:49:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-32w-x-29l-navy>: HTTP status code is not handled or not allowed 2026-01-24 19:49:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:49:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/long-sleeve-henley-interlock-knit-flame-resistant-3xl-orange>: HTTP status code is not handled or not allowed 2026-01-24 19:49:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:49:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:49:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:49:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ds-327065w-shrink-wrap-32w-x-65l-7mil-white>: HTTP status code is not handled or not allowed 2026-01-24 19:49:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts landed on page that is not a product page. 2026-01-24 19:49:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:49:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gear-locker-24x18x72-w-top-shelf-security-box-foot-locker-grand-slam-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 19:49:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts landed on page that is not a product page. 2026-01-24 19:49:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts landed on page that is not a product page. 2026-01-24 19:49:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:49:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/digital-caliper-1108-150-0-6-inch-0-150mm-range>: HTTP status code is not handled or not allowed 2026-01-24 19:49:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:49:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats landed on page that is not a product page. 2026-01-24 19:49:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts landed on page that is not a product page. 2026-01-24 19:49:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts landed on page that is not a product page. 2026-01-24 19:49:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts landed on page that is not a product page. 2026-01-24 19:49:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-38w-x-31l-navy>: HTTP status code is not handled or not allowed 2026-01-24 19:49:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:49:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:49:44 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts landed on page that is not a product page. 2026-01-24 19:49:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-38w-x-34l-gray>: HTTP status code is not handled or not allowed 2026-01-24 19:49:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts landed on page that is not a product page. 2026-01-24 19:49:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 19:49:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-galvanized-steel-shelving-189951>: HTTP status code is not handled or not allowed 2026-01-24 19:49:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 19:49:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 19:49:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:49:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-40w-x-34l-green>: HTTP status code is not handled or not allowed 2026-01-24 19:49:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-42w-x-30l-brown>: HTTP status code is not handled or not allowed 2026-01-24 19:49:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:50:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:50:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5463500-48x18x63-add-on-shelving>: HTTP status code is not handled or not allowed 2026-01-24 19:50:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:50:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 19:50:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:50:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-40w-x-33l-gray>: HTTP status code is not handled or not allowed 2026-01-24 19:50:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:50:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/allux-series-mailboxes-mounting-base-pl-for-allux-800-810-wall-mount-mail-parcel-boxes>: HTTP status code is not handled or not allowed 2026-01-24 19:50:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 19:50:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:50:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 15519 pages (at 78 pages/min), scraped 10749 items (at 43 items/min) 2026-01-24 19:50:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-29w-x-32l-black>: HTTP status code is not handled or not allowed 2026-01-24 19:50:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 19:50:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 19:50:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cardboard-single-wall-corrugated-box-17-x-17-x-9-200lb-test-ect-32-kraft-25-pack>: HTTP status code is not handled or not allowed 2026-01-24 19:50:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 19:50:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:50:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:50:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/friedrich-packaged-terminal-air-conditioner12000-btu-cool-292280>: HTTP status code is not handled or not allowed 2026-01-24 19:50:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:50:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:50:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-50w-x-36l-black>: HTTP status code is not handled or not allowed 2026-01-24 19:50:18 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 19:50:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:50:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-46w-x-28l-black>: HTTP status code is not handled or not allowed 2026-01-24 19:50:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 19:50:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 19:50:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:50:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 19:50:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 19:50:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:50:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 19:50:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 19:50:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:50:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:50:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-32w-x-32l-black>: HTTP status code is not handled or not allowed 2026-01-24 19:50:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:50:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b2020-oh-needle-bearing-drawn-cup-full-complement-oil-hole-bore-3175mm>: HTTP status code is not handled or not allowed 2026-01-24 19:50:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 19:50:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vem31104-33hp-1800rpm-56c-frame-3ph-230-460v-odp-c-face-footless-premium-efficiency>: HTTP status code is not handled or not allowed 2026-01-24 19:50:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 19:50:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 19:50:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 19:50:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-bulk-parts-lifter-4-to-5-lbs-pull-4375l-x-3125w-x-3225h-nylon-base-long-handle>: HTTP status code is not handled or not allowed 2026-01-24 19:50:42 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 19:50:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:50:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ps-12-foodslicer-compact-12blade-1-3hp-120v>: HTTP status code is not handled or not allowed 2026-01-24 19:50:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:50:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aqua-tank-aq-40-40-gallons>: HTTP status code is not handled or not allowed 2026-01-24 19:50:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:50:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:50:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mounting-step-3-step-ms-22r-22>: HTTP status code is not handled or not allowed 2026-01-24 19:50:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:50:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:50:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 19:50:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-58w-x-33l-black>: HTTP status code is not handled or not allowed 2026-01-24 19:50:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:50:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:50:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:50:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/friedrich-ptac-15-amp-cord-292291>: HTTP status code is not handled or not allowed 2026-01-24 19:50:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 19:50:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-42w-x-30l-dark-navy>: HTTP status code is not handled or not allowed 2026-01-24 19:50:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 19:50:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 19:50:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24273-duo-edge-roast-slicer-high-carbon-steel-black-handle-12-l>: HTTP status code is not handled or not allowed 2026-01-24 19:51:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 19:51:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 19:51:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/hvac_service_tools_parts/refrigerant_recovery_tools already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:51:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 19:51:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 19:51:04 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 19:51:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:51:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18053-pizza-rocker-high-carbon-steel-black-handle-20-l>: HTTP status code is not handled or not allowed 2026-01-24 19:51:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15303c-cooks-style-paring-knife-blue-handle-high-carbon-steel-3-1-4-l>: HTTP status code is not handled or not allowed 2026-01-24 19:51:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 15595 pages (at 76 pages/min), scraped 10783 items (at 34 items/min) 2026-01-24 19:51:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/hvac_service_tools_parts/refrigerant_recovery_tools>: HTTP status code is not handled or not allowed 2026-01-24 19:51:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-westex-40w-x-36l-black>: HTTP status code is not handled or not allowed 2026-01-24 19:51:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/r4i-tas-115-2301-rotary-screw-air-compressor-1-phase-230-volts-5hp-120-gal>: HTTP status code is not handled or not allowed 2026-01-24 19:51:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 19:51:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wesco-36x24-carpeted-end-hardwood-dolly-3-casters-900-lb-cap-987841>: HTTP status code is not handled or not allowed 2026-01-24 19:51:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-performance-100-series-lan-station-african-mahogany>: HTTP status code is not handled or not allowed 2026-01-24 19:51:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-work-coveralls-48-tall-black>: HTTP status code is not handled or not allowed 2026-01-24 19:51:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/utility-jeans-basket-weave-flame-resistant-8-oz-35w-x-32l-green>: HTTP status code is not handled or not allowed 2026-01-24 19:51:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:51:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-westex-48w-x-unhemmed-tan>: HTTP status code is not handled or not allowed 2026-01-24 19:51:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-jeans-indura-denim-14-oz-35w-x-34l-flame-resistant-blue>: HTTP status code is not handled or not allowed 2026-01-24 19:51:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:51:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:51:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-sheet-union-jack-56008-24-inchl-x-12-inchw-x-02-inchh-silver>: HTTP status code is not handled or not allowed 2026-01-24 19:51:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 19:51:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-westex-46w-x-32l-blue>: HTTP status code is not handled or not allowed 2026-01-24 19:51:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:51:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 19:51:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 19:51:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:51:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/office_machines/shredders/industrial_commercial_shredders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:51:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/x-o-rust-anti-rust-enamel-white-metal-primer-quart-176804>: HTTP status code is not handled or not allowed 2026-01-24 19:51:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:51:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-54w-x-24d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 19:51:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:51:50 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 19:51:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 19:51:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-72w-x-14d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 19:51:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:51:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/office_machines/shredders/industrial_commercial_shredders landed on page that is not a product page. 2026-01-24 19:51:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:51:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:51:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/x-o-rust-anti-rust-enamel-satin-finish-satin-black-gallon-176839>: HTTP status code is not handled or not allowed 2026-01-24 19:51:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 19:51:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-54w-x-21d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 19:51:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:52:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-48w-x-21d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 19:52:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 19:52:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 19:52:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 19:52:04 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 19:52:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bin-divider-for-6-5-8w-x-8h-price-per-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-24 19:52:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_roof_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:52:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 19:52:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wesco-174-plastic-box-truck-4-bushel-gray-272530-8-casters>: HTTP status code is not handled or not allowed 2026-01-24 19:52:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 15681 pages (at 86 pages/min), scraped 10837 items (at 54 items/min) 2026-01-24 19:52:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_bulkheads already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:52:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_storage_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:52:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mount-channel-kit-full-size-mid-high-roof-2113-0-01>: HTTP status code is not handled or not allowed 2026-01-24 19:52:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deep-groove-ball-bearings-inch-1640-zz-shielded-light-duty-087-inch-bore-2-inch-od>: HTTP status code is not handled or not allowed 2026-01-24 19:52:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:52:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swa-50-semi-auto-str-wrap-machine>: HTTP status code is not handled or not allowed 2026-01-24 19:52:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:52:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_storage_kits>: HTTP status code is not handled or not allowed 2026-01-24 19:52:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-16a-1ph-remote-start-reset-button-100-250v-7-6-10a>: HTTP status code is not handled or not allowed 2026-01-24 19:52:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-26a-3ph-direct-online-reset-button-100-250v-16-20a>: HTTP status code is not handled or not allowed 2026-01-24 19:52:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment>: HTTP status code is not handled or not allowed 2026-01-24 19:52:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/screws/socket_screws/socket_head_cap_screws already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:52:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment>: HTTP status code is not handled or not allowed 2026-01-24 19:52:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/washers/lock_washers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:52:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/340150-split-lock-washer-3-8-steel-zinc-yellow-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 19:52:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/screws/ball_point_socket_set_screws already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:52:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/d647rb-osha-sign-danger-high-voltage-authorized-personnel-only-10-x-14-whiteredblack>: HTTP status code is not handled or not allowed 2026-01-24 19:52:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/screws/socket_screws/socket_head_cap_screws landed on page that is not a product page. 2026-01-24 19:52:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/845280-nylon-insert-lock-nut-7-16-14-steel-zinc-yellow-grade-c-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 19:52:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/fasteners/washers/lock_washers>: HTTP status code is not handled or not allowed 2026-01-24 19:52:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/fasteners/screws/ball_point_socket_set_screws>: HTTP status code is not handled or not allowed 2026-01-24 19:52:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1500-watertight-medium-case-with-foam-16-34-x-11-316-x-6-18-black>: HTTP status code is not handled or not allowed 2026-01-24 19:52:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keyed-hasp-u-9951-4w-chrome-plated>: HTTP status code is not handled or not allowed 2026-01-24 19:52:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1001500-wes-150-waterfree-urinal-cartridge-kit>: HTTP status code is not handled or not allowed 2026-01-24 19:52:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vsm-abrasive-belt-104937-silicon-carbide-2-x-72-320-grit>: HTTP status code is not handled or not allowed 2026-01-24 19:52:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-80a-1ph-reset-button-24-60v-75-87a>: HTTP status code is not handled or not allowed 2026-01-24 19:53:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cabinet-exhaust-fan-w-shutter-42-inch-prop-1hp-15455-cfm-3-phase-belt-drive>: HTTP status code is not handled or not allowed 2026-01-24 19:53:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cabinet-exhaust-fan-w-shutter-42-inch-prop-15hp-18145-cfm-3-phase-belt-drive>: HTTP status code is not handled or not allowed 2026-01-24 19:53:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vsm-abrasive-belt-82686-zirconia-alumina-3-4-x-18-60-grit>: HTTP status code is not handled or not allowed 2026-01-24 19:53:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paraclipse-terminator-dual-sd-system-120v-ss-covers-10000-sq-ft-252350>: HTTP status code is not handled or not allowed 2026-01-24 19:53:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 15767 pages (at 86 pages/min), scraped 10900 items (at 63 items/min) 2026-01-24 19:53:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/satin-nickel-victorian-style-knob-set>: HTTP status code is not handled or not allowed 2026-01-24 19:53:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pioneer-americana-2am401-bn-two-handle-kitchen-faucet-with-spray-pvd-brushed-nickel>: HTTP status code is not handled or not allowed 2026-01-24 19:53:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-utility-cut-off-tool-with-guard-sx233a-1-4-npt>: HTTP status code is not handled or not allowed 2026-01-24 19:53:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fao-smartcompliance-refill-emergency-blanket-52-inch-x-84-inch-1-per-bag>: HTTP status code is not handled or not allowed 2026-01-24 19:53:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-48-inch-w-x-21-inch-d-x-74-inch-h-2000-lbs-cap-5-open-grid-shelves>: HTTP status code is not handled or not allowed 2026-01-24 19:53:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-48-inch-w-x-21-inch-d-x-74-inch-h-2000-lbs-cap-4-open-and-1-solid-grid-shelves>: HTTP status code is not handled or not allowed 2026-01-24 19:53:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/95-osha-stairway-stair36o-11-95>: HTTP status code is not handled or not allowed 2026-01-24 19:53:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/106-osha-stairway-stair36o-12-106>: HTTP status code is not handled or not allowed 2026-01-24 19:53:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14mm-locking-assembly-c193-series-c193m14-metric-m4-x-14>: HTTP status code is not handled or not allowed 2026-01-24 19:53:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-frame-folding-tank-stf-4000-183l-x-183w-x-29h-4000-gallon-cap-blue>: HTTP status code is not handled or not allowed 2026-01-24 19:53:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/125-dia-locking-assembly-c193-series-c193e125-steel-m5-x-18>: HTTP status code is not handled or not allowed 2026-01-24 19:53:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-shaft-arbor-al4fsl-lefthand-type-d-214l-thread-fits-12-shaft>: HTTP status code is not handled or not allowed 2026-01-24 19:53:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exam-lamp-13408-goose-neck-dome-style-shade-chome>: HTTP status code is not handled or not allowed 2026-01-24 19:53:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dsv221-3x-ansi-class-2-two-tone-mesh-vest-3xl>: HTTP status code is not handled or not allowed 2026-01-24 19:53:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-48-inch-w-x-24-inch-d-1000-lbs-cap-removable-solid-shelf-mat-polymer-gray>: HTTP status code is not handled or not allowed 2026-01-24 19:53:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5095-33-w-x-244-l-black-resin-ribbon>: HTTP status code is not handled or not allowed 2026-01-24 19:53:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-mechanism-for-use-w-beltrac-stanchions-13l-bronze-belt>: HTTP status code is not handled or not allowed 2026-01-24 19:53:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/crowd_control/barrier_rope_chain already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:53:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medina-left-hand-bridge-48w-x-24d-x-29-1-2h-gray-steel>: HTTP status code is not handled or not allowed 2026-01-24 19:53:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/versatile-4c-mailbox-4c06d-09-23-14h-9-mailboxes-front-loading-aluminum-brown-usps>: HTTP status code is not handled or not allowed 2026-01-24 19:53:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/octagonal-sledge-hammer-1437gfv-8lb-head-36-fiberglass-handle>: HTTP status code is not handled or not allowed 2026-01-24 19:53:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/crowd_control/barrier_rope_chain landed on page that is not a product page. 2026-01-24 19:54:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-basket-truck-with-vinyl-liner-12-bushel-black>: HTTP status code is not handled or not allowed 2026-01-24 19:54:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/antifungal-powder-3-oz-pack-of-24>: HTTP status code is not handled or not allowed 2026-01-24 19:54:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-1-interchangeable-tamper-proof-bit-screwdriver-9315mt-8-1-2-long-15-bits>: HTTP status code is not handled or not allowed 2026-01-24 19:54:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waterproof-bordered-foam-dressing-6l-x-6w-120-pcs>: HTTP status code is not handled or not allowed 2026-01-24 19:54:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 15866 pages (at 99 pages/min), scraped 10974 items (at 74 items/min) 2026-01-24 19:54:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u-sling-w-head-support-x-large>: HTTP status code is not handled or not allowed 2026-01-24 19:54:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tms334j-traffic-sign-reserved-parking-rhode-island-18-x-12-blue>: HTTP status code is not handled or not allowed 2026-01-24 19:54:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rainbow-accents-lockable-wall-cabinet-purplejnc>: HTTP status code is not handled or not allowed 2026-01-24 19:54:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-end-truck-w-removable-shelf-16-x-48>: HTTP status code is not handled or not allowed 2026-01-24 19:54:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tm99k-traffic-sign-no-parking-here-to-corner-18-x-12-white>: HTTP status code is not handled or not allowed 2026-01-24 19:54:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premier-built-in-undercounter-freezer-solid-door-abt-ucbi-0420-4-5-cu-ft>: HTTP status code is not handled or not allowed 2026-01-24 19:54:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sl-half-cylinder-tunable-led-mission-style-outdoor-fixture-12w-800-lumens-black-finish>: HTTP status code is not handled or not allowed 2026-01-24 19:54:29 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-24 19:54:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beveled-welding-mat-black-7-8in-x-4ft-x-75ft-full-roll>: HTTP status code is not handled or not allowed 2026-01-24 19:54:30 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:11 2026-01-24 19:54:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/endura-goatskin-leather-gloves-blended-kevlar-lining-ansi-a6-2xl>: HTTP status code is not handled or not allowed 2026-01-24 19:54:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mc-high-precision-charging-scale-with-blow-molded-case>: HTTP status code is not handled or not allowed 2026-01-24 19:54:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10ft-straight-roller-conveyor-15in-between-frame-3in-roller-centers>: HTTP status code is not handled or not allowed 2026-01-24 19:54:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/keyboards_mice already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:54:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ct-usa-crew-arm-chair-floor-mount-midnight-blue>: HTTP status code is not handled or not allowed 2026-01-24 19:54:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/specialty_cleaners/screen_electronics_wipes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:54:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inner-ring-for-shell-type-needle-roller-bearing-inch-1-bore-114-od-1308mm-width>: HTTP status code is not handled or not allowed 2026-01-24 19:54:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/75302-resin-fiber-disc-3-pak-7-8-arbor-50-grit-7>: HTTP status code is not handled or not allowed 2026-01-24 19:54:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/abrasives/general_purpose_sanding/cloth_sanding_bands already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:54:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/specialty_cleaners/screen_electronics_wipes>: HTTP status code is not handled or not allowed 2026-01-24 19:54:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/76021-angle-grinder-cup-brush-arbor-5-8-x-11-coarse-2-3-4-knot>: HTTP status code is not handled or not allowed 2026-01-24 19:54:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/keyboards_mice landed on page that is not a product page. 2026-01-24 19:54:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:54:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heat-lamp-infrared-single-row-suspension-bar-5-bulb-120v-6006-5>: HTTP status code is not handled or not allowed 2026-01-24 19:54:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbk-athletic-work-shoe-ultraknit-flexweave-black-white-6w>: HTTP status code is not handled or not allowed 2026-01-24 19:54:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/abrasives/general_purpose_sanding/cloth_sanding_bands landed on page that is not a product page. 2026-01-24 19:54:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f42400-2221-lab-companion-clear-polycarbonate-round-style-vacuum-desiccator-20-liter>: HTTP status code is not handled or not allowed 2026-01-24 19:55:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:55:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-24 19:55:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kistt1218-steel-locker-accessory-individual-slope-top-12x18x6-gray>: HTTP status code is not handled or not allowed 2026-01-24 19:55:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compact-portable-scale-cb1001-2-lb-capacity-0-1-gram-readability>: HTTP status code is not handled or not allowed 2026-01-24 19:55:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 15960 pages (at 94 pages/min), scraped 11044 items (at 70 items/min) 2026-01-24 19:55:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-24 19:55:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ironcat-heavyweight-top-grain-cowhide-mig-welding-gloves-ivory-medium-all-leather>: HTTP status code is not handled or not allowed 2026-01-24 19:55:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-chrome-2-tier-wire-shelving-starter-unit-72w-x-14d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 19:55:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-chrome-2-tier-slant-wire-shelving-starter-unit-48w-x-18d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 19:55:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-chrome-3-tier-wire-shelving-starter-unit-1-solid-shelf-36w-x-18d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 19:55:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/international-pulp-hook-t-000-000-0607-hardwood-handle-10>: HTTP status code is not handled or not allowed 2026-01-24 19:55:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lb18-1bh-kit-1-18-ga-blue-steel-square-hole-pegboard-w-18-pc-lochook-set>: HTTP status code is not handled or not allowed 2026-01-24 19:55:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sa-washer-and-heat-pump-dryer-combo-set-4-prong-white>: HTTP status code is not handled or not allowed 2026-01-24 19:55:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/497-crane-lift-kit-solid-polypropylene-hd-black>: HTTP status code is not handled or not allowed 2026-01-24 19:55:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/electric_motor_parts_accessories/electric_motor_power_supplies already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:55:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-gauge-staple-1-1-4-length-15-16-crown-galvanized-steel-pkg-of-10000>: HTTP status code is not handled or not allowed 2026-01-24 19:55:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tt-static-safe-hot-gloves-11-medium>: HTTP status code is not handled or not allowed 2026-01-24 19:55:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/horizon-mfg-safety-glass-holder-with-door-5203-holds-20-glasses-6-3-4l>: HTTP status code is not handled or not allowed 2026-01-24 19:55:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/electric_motor_parts_accessories/electric_motor_power_supplies landed on page that is not a product page. 2026-01-24 19:55:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/horizon-mfg-apron-dispenser-tray-horizontal-or-bagged-packages-4022-15l>: HTTP status code is not handled or not allowed 2026-01-24 19:55:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inline-rectangular-24x14-duct-fan-frd-24-14-115v-2318-cfm>: HTTP status code is not handled or not allowed 2026-01-24 19:55:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/contactor-3-pole-60-75a-120v-for-market-forge-09-6484>: HTTP status code is not handled or not allowed 2026-01-24 19:55:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/contactor-3-pole-40-50a-120v-for-hatco-020101500>: HTTP status code is not handled or not allowed 2026-01-24 19:55:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/tables/folding_tables already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:55:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pf-vacuum-hose-solution-line-w-cuffs-20-feet>: HTTP status code is not handled or not allowed 2026-01-24 19:55:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ajc-elk-elk1280-12v-8ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-24 19:55:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ajc-hkbil-3fm10-6v-10ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-24 19:55:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/tables/folding_tables>: HTTP status code is not handled or not allowed 2026-01-24 19:56:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/27-series-versatrac-5-1-4height-4-hard-rubber-wheel-swivel-caster>: HTTP status code is not handled or not allowed 2026-01-24 19:56:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/27-series-versatrac-5-1-4height-4-polyolefin-wheel-ball-bearing-swivel-caster>: HTTP status code is not handled or not allowed 2026-01-24 19:56:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delta-positioning-construction-harness-3xl-dbi-sala-1101660>: HTTP status code is not handled or not allowed 2026-01-24 19:56:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16in-floor-stripping-pads-black-5-case-gjo90216>: HTTP status code is not handled or not allowed 2026-01-24 19:56:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 16055 pages (at 95 pages/min), scraped 11114 items (at 70 items/min) 2026-01-24 19:56:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lnr-xt-150-waste-oil-heater-pkg-6-inch-chimney-roof-kit-215-gallon-tank-150000-btu>: HTTP status code is not handled or not allowed 2026-01-24 19:56:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/521fl-05-model-521fl-with-flow-rate-of-05-gpm>: HTTP status code is not handled or not allowed 2026-01-24 19:56:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wern-positioning-lanyard-w-chain-2-snaphooks-rebar-hook-18-inch-l>: HTTP status code is not handled or not allowed 2026-01-24 19:56:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-appliance-truck-with-manual-belt-tightener-40710-2-wheel-60h-800-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 19:56:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/252300-single-pocket-tri-fold-countertop-brochure-holder-4-5-x-7-75-10-pack>: HTTP status code is not handled or not allowed 2026-01-24 19:56:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/lockers/gear_lockers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:56:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wern-max-patrol-leading-edge-30-cable-self-retracting-lifeline-w-snap-hook>: HTTP status code is not handled or not allowed 2026-01-24 19:56:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/556011-acrylic-shelf-for-pegboard-slatwall-20-x-2-clear>: HTTP status code is not handled or not allowed 2026-01-24 19:56:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wern-bantam-8-feet-self-retracting-lifeline-le-w-steel-form-hook>: HTTP status code is not handled or not allowed 2026-01-24 19:56:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/miller-h700-construction-comfort-harness-quick-connect-buckle-back-front-side-d-ring-l-xl>: HTTP status code is not handled or not allowed 2026-01-24 19:56:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-efficiency-motor-w-drop-cord-switch-115v-2-speed-yellow>: HTTP status code is not handled or not allowed 2026-01-24 19:56:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vs-stirring-shaft-with-anchor>: HTTP status code is not handled or not allowed 2026-01-24 19:56:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/lockers/gear_lockers landed on page that is not a product page. 2026-01-24 19:56:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/701238-8-flip-scan-quick-back-0-187-diameter-50-pack>: HTTP status code is not handled or not allowed 2026-01-24 19:56:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keystock-1-4-x-1-4-x-1-ft-300-series-stainless-steel-plain-undersize-astm-a484>: HTTP status code is not handled or not allowed 2026-01-24 19:56:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/igg-precision-combination-squares-w-6-inch-blade-pack-of-3>: HTTP status code is not handled or not allowed 2026-01-24 19:56:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/igg-led-100x-measuring-microscope-w-stand>: HTTP status code is not handled or not allowed 2026-01-24 19:56:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/igg-machinist-square-set-w-protective-case-pack-of-3>: HTTP status code is not handled or not allowed 2026-01-24 19:56:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inplant-office-vinyl-cov-hdbd-12x20-4-wall-4-lgts-5-dplx-1-door-white>: HTTP status code is not handled or not allowed 2026-01-24 19:56:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deep-cut-band-saw-kit-dwm120k-10-amps-100-350-fpm-44-7-8-blade>: HTTP status code is not handled or not allowed 2026-01-24 19:56:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90-series-swivel-threaded-stem-caster-900716-polyolefin-5-dia-300-lb-cap>: HTTP status code is not handled or not allowed 2026-01-24 19:57:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/small-esd-component-shipping-and-storage-box-5l-x-1-7-16w-x-1-1-8h>: HTTP status code is not handled or not allowed 2026-01-24 19:57:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-step-aluminum-rolling-ladder-24inw-grip-tread-28ind-top-step-32in-handrails>: HTTP status code is not handled or not allowed 2026-01-24 19:57:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4x4-task-chair-498sl-nvy-navy-fabric-adjustable-arms>: HTTP status code is not handled or not allowed 2026-01-24 19:57:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vacuum-aspirator-collection-system-199170150-polypropylene-1-gal-w-pump-1-pk-blue>: HTTP status code is not handled or not allowed 2026-01-24 19:57:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 16152 pages (at 97 pages/min), scraped 11187 items (at 73 items/min) 2026-01-24 19:57:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/bolts/flange_bolts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:57:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wigan-ceiling-medallion-cm29wg-2934od-x-112d>: HTTP status code is not handled or not allowed 2026-01-24 19:57:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/balance-system-yellow-ball-only-level-1-1-pair>: HTTP status code is not handled or not allowed 2026-01-24 19:57:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/professional-balance-board-weight-rod-only>: HTTP status code is not handled or not allowed 2026-01-24 19:57:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/lockers/locker_accessories/locker_panels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:57:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-wide-designer-wood-locker-22364-double-tier-3-wide-15x24x36-cherry-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 19:57:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/lockers/standard_lockers/wood_standard_lockers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:57:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/bolts/flange_bolts landed on page that is not a product page. 2026-01-24 19:57:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/lockers/standard_lockers/wood_standard_lockers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:57:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cl-ra40-20d-20in-rider-automatic-scrubber-130-ah-wet-batteries-56384073>: HTTP status code is not handled or not allowed 2026-01-24 19:57:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/storage_panels/louvered_panels_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:57:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hand-exerciser-with-16-bands-4-each-tan-yellow-red-green>: HTTP status code is not handled or not allowed 2026-01-24 19:57:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/physical_therapy/exercise_fitness_equipment/resistance_devices already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:57:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/lockers/standard_lockers/wood_standard_lockers>: HTTP status code is not handled or not allowed 2026-01-24 19:57:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/lockers/standard_lockers/wood_standard_lockers landed on page that is not a product page. 2026-01-24 19:57:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/lockers/locker_accessories/locker_panels landed on page that is not a product page. 2026-01-24 19:57:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/storage_panels/louvered_panels_accessories>: HTTP status code is not handled or not allowed 2026-01-24 19:57:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/physical_therapy/exercise_fitness_equipment/balance_training_devices already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:57:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-ton-20-span-301-series-steel-wall-mounted-wall-bracket-jib-crane-tie-rod-design>: HTTP status code is not handled or not allowed 2026-01-24 19:57:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulwark-bracket-bktm02x10x10buss-2w-x-10d-x-10h>: HTTP status code is not handled or not allowed 2026-01-24 19:57:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-wood-extension-pole-brown-24-case-wp-06fmt>: HTTP status code is not handled or not allowed 2026-01-24 19:57:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/physical_therapy/exercise_fitness_equipment/resistance_devices landed on page that is not a product page. 2026-01-24 19:57:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-office-partition-panel-with-pass-thru-cable-24-1-4w-x-65-1-2h-gray>: HTTP status code is not handled or not allowed 2026-01-24 19:57:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-part-reeved-cross-parallel-mount-assembly-model-2993ut>: HTTP status code is not handled or not allowed 2026-01-24 19:57:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9050m-e-f-blk-integra-esd-fabric-upholstered-chair-medium-back-aluminum-base-black>: HTTP status code is not handled or not allowed 2026-01-24 19:57:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/physical_therapy/exercise_fitness_equipment/balance_training_devices landed on page that is not a product page. 2026-01-24 19:57:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stage-pie-set-with-carpet-for-36w-stage-units-8h-16h24h-black>: HTTP status code is not handled or not allowed 2026-01-24 19:57:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-16-18-x-3-4-security-cap-screw-flat-socket-head-18-8-stainless-steel-pkg-of-50>: HTTP status code is not handled or not allowed 2026-01-24 19:57:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/welded-liquid-bag-filter-polyester-felt-7-1-8-d-x-16-1-2-l-50-micron-plastic-flange-pkg-50>: HTTP status code is not handled or not allowed 2026-01-24 19:57:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/liquid-bag-filter-polyester-felt-4-1-8-dia-x-14-l-50-micron-steel-ring-pkg-qty-50>: HTTP status code is not handled or not allowed 2026-01-24 19:57:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-32-x-1-tri-wing-machine-screw-pan-head-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 19:57:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sewn-liquid-bag-filter-polyester-multifilament-8-3-8-x-36-400-micron-steel-ring-pkg-50>: HTTP status code is not handled or not allowed 2026-01-24 19:57:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m4-x-0-7-x-10mm-security-machine-screw-button-torx-head-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 19:58:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-office-partition-panel-with-partial-window-and-pass-thru-cable-36-1-4w-x-77-1-2h-gray>: HTTP status code is not handled or not allowed 2026-01-24 19:58:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/booster-seat-15-1-2-inch-w-x-12-inch-d-x-8-inch-h-dual-heights-wide-base-seat-belt-dark-gray-4-per-case>: HTTP status code is not handled or not allowed 2026-01-24 19:58:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/oven_grill_cleaners already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:58:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chair-guest-leather-my-bk>: HTTP status code is not handled or not allowed 2026-01-24 19:58:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 16243 pages (at 91 pages/min), scraped 11249 items (at 62 items/min) 2026-01-24 19:58:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tripod-stand-dt15s16t27-for-15-oaw-skatewheel-adj-16-to-27h>: HTTP status code is not handled or not allowed 2026-01-24 19:58:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nilfisk-uz-964-upstream-hepa-filter>: HTTP status code is not handled or not allowed 2026-01-24 19:58:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/oven_grill_cleaners landed on page that is not a product page. 2026-01-24 19:58:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelving-24w-x-14d-x-63h-4-shelf-starter-chrome>: HTTP status code is not handled or not allowed 2026-01-24 19:58:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-pc-long-paddle-with-out-holes-white>: HTTP status code is not handled or not allowed 2026-01-24 19:58:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-step-nestable-plastic-step-stand-yel-19-12-w-x-14-d-x-12-h-nst-3-yel>: HTTP status code is not handled or not allowed 2026-01-24 19:58:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tan-tinted-concentrator-bag-on-roll-25w-30l>: HTTP status code is not handled or not allowed 2026-01-24 19:58:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/outdoor_tools_power_equipment/pressure_washers/gas_pressure_washers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:58:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/starburst-fruit-chews-original-variety-50-oz-gr>: HTTP status code is not handled or not allowed 2026-01-24 19:58:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:58:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-platform-for-88880025>: HTTP status code is not handled or not allowed 2026-01-24 19:58:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:58:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/open-platform-kit-8-x-18-x-72>: HTTP status code is not handled or not allowed 2026-01-24 19:58:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-48-w-x-36-d-x-84-h-starter-3-shelf-levels-wood-deck-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 19:58:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/outdoor_tools_power_equipment/pressure_washers/gas_pressure_washers landed on page that is not a product page. 2026-01-24 19:58:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks>: HTTP status code is not handled or not allowed 2026-01-24 19:58:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/legacy-71-bow-front-double-full-pedestal-u-desk-ash-grey>: HTTP status code is not handled or not allowed 2026-01-24 19:58:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:58:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-48-w-x-48-d-x-120-h-add-on-3-shelf-levels-wood-deck-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 19:58:44 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks landed on page that is not a product page. 2026-01-24 19:58:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-72-w-x-48-d-x-96-h-starter-3-shelf-levels-wire-deck-medium-gray>: HTTP status code is not handled or not allowed 2026-01-24 19:58:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-kid-locker-15w-x-15d-x-54h-721-relay-red-red-single-tier-1-wide-knock-down>: HTTP status code is not handled or not allowed 2026-01-24 19:58:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:58:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks>: HTTP status code is not handled or not allowed 2026-01-24 19:58:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:58:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1291233-filter-baffle-20x20-ss-he-fmp>: HTTP status code is not handled or not allowed 2026-01-24 19:58:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/abrasives/grinding_cutoff_wheels/cut_off_wheels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:58:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/igs750-general-protocol-microbiological-incubator-26-4-cu-ft-120v>: HTTP status code is not handled or not allowed 2026-01-24 19:59:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/furniture_parts_accessories/computer_desk_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:59:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alera-aleva316012my-valencia-under-counter-file-organizer-shelf-15-34w-x-10d-x-11hmahogany>: HTTP status code is not handled or not allowed 2026-01-24 19:59:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-inch-x-50-foot-medium-duty-pvc-discharge-hose-coupled-with-cxe-aluminum-cam-and-groove-fittings>: HTTP status code is not handled or not allowed 2026-01-24 19:59:04 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks landed on page that is not a product page. 2026-01-24 19:59:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/electronic_queuing_systems already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:59:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deflect-o-77401-multi-compartment-docuholder-6-compartments-9w-x-7-12d-x-13-34h-clear>: HTTP status code is not handled or not allowed 2026-01-24 19:59:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/gym_fitness_equipment/sports_equipment already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:59:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-pharmacy-vaccine-refrigerator-single-swing-glass-door-26-cu-ft-ph-abt-s26g>: HTTP status code is not handled or not allowed 2026-01-24 19:59:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 16327 pages (at 84 pages/min), scraped 11307 items (at 58 items/min) 2026-01-24 19:59:09 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks landed on page that is not a product page. 2026-01-24 19:59:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-96-w-x-48-d-x-96-h-add-on-3-shelf-levels-wood-deck-medium-gray>: HTTP status code is not handled or not allowed 2026-01-24 19:59:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/furniture_parts_accessories/computer_desk_accessories landed on page that is not a product page. 2026-01-24 19:59:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/abrasives/grinding_cutoff_wheels/cut_off_wheels landed on page that is not a product page. 2026-01-24 19:59:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/credenzas already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:59:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chateau-series-mahogany-laminate-desking-661x295x30-table>: HTTP status code is not handled or not allowed 2026-01-24 19:59:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/electronic_queuing_systems landed on page that is not a product page. 2026-01-24 19:59:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-3907-burner-orifice-43-natural-gas-1-8-inch-mpt-thread>: HTTP status code is not handled or not allowed 2026-01-24 19:59:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/gym_fitness_equipment/sports_equipment landed on page that is not a product page. 2026-01-24 19:59:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/z-creeper-seat-c-2036d>: HTTP status code is not handled or not allowed 2026-01-24 19:59:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/omega-tire-cart-300-lbs-capacity-92030>: HTTP status code is not handled or not allowed 2026-01-24 19:59:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/automotive_shop_equipment/vehicle_lifts_jacks/engine_stands_cradles already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:59:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/credenzas landed on page that is not a product page. 2026-01-24 19:59:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/omega-20-ton-hydraulic-floor-service-jack-22203>: HTTP status code is not handled or not allowed 2026-01-24 19:59:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-28-14w-mobile-cabinet-7-drawers-72-compart-classic-blue-master-keyed>: HTTP status code is not handled or not allowed 2026-01-24 19:59:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-double-triple-expander-kit-green-medium-3-kit>: HTTP status code is not handled or not allowed 2026-01-24 19:59:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-ah21z-poly-z-brite-utility-cart-handle-21w>: HTTP status code is not handled or not allowed 2026-01-24 19:59:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/frm-1120c-horizontal-continuous-band-sealer-with-tilt-head-right-feed-dry-ink-coding>: HTTP status code is not handled or not allowed 2026-01-24 19:59:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/wiring_devices/wire_connectors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:59:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/vehicle-fleet-maintenance/automotive_shop_equipment/vehicle_lifts_jacks/engine_stands_cradles landed on page that is not a product page. 2026-01-24 19:59:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90-bent-magnalugs-6401-050fl-3-8-50-pcs>: HTTP status code is not handled or not allowed 2026-01-24 19:59:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/electrochemistry_meters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:59:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brazed-seam-ring-terminal-copper-lugs-107113-100-4-gauge-1-4-stud>: HTTP status code is not handled or not allowed 2026-01-24 19:59:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/cable_management/cable_tie_tools already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:59:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inplant-office-additional-3070-swing-door-3-w-x-7-h-steel-white>: HTTP status code is not handled or not allowed 2026-01-24 19:59:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/wiring_devices/wire_connectors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:59:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/cable_management/cable_ties_wraps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 19:59:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-exercise-tubing-green-100-roll-box>: HTTP status code is not handled or not allowed 2026-01-24 19:59:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mesh-back-guest-chair-black>: HTTP status code is not handled or not allowed 2026-01-24 19:59:44 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/wiring_devices/wire_connectors landed on page that is not a product page. 2026-01-24 19:59:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/electrochemistry_meters>: HTTP status code is not handled or not allowed 2026-01-24 19:59:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/cable_management/cable_tie_tools landed on page that is not a product page. 2026-01-24 19:59:49 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/wiring_devices/wire_connectors landed on page that is not a product page. 2026-01-24 19:59:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/cable_management/cable_ties_wraps>: HTTP status code is not handled or not allowed 2026-01-24 19:59:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kenda-loadstar-trailer-tire-4-hole-wheel-4-4-480-400-8-lrb>: HTTP status code is not handled or not allowed 2026-01-24 19:59:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eagle-professional-grade-38-air-ratchet-wrench-ega200>: HTTP status code is not handled or not allowed 2026-01-24 19:59:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-174-extra-black-trash-bags-40-to-45-gallon-1-4-mil-100-case>: HTTP status code is not handled or not allowed 2026-01-24 20:00:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-32-x-1-2-security-machine-screw-flat-torx-head-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 20:00:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-16-hex-t-handle-13310>: HTTP status code is not handled or not allowed 2026-01-24 20:00:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 16419 pages (at 92 pages/min), scraped 11367 items (at 60 items/min) 2026-01-24 20:00:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-16-prohold-balldriver-t-handle-75113>: HTTP status code is not handled or not allowed 2026-01-24 20:00:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-office-partition-panel-with-partial-window-and-raceway-36-1-4w-x-47-1-2h-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:00:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ice-block-one-gallon>: HTTP status code is not handled or not allowed 2026-01-24 20:00:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/monitor_filters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:00:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelving-60w-x-24d-x-14h-1-shelf-add-on-chrome>: HTTP status code is not handled or not allowed 2026-01-24 20:00:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8bspp-poly-muffler-145-psi-75-micron-5-to-212-f>: HTTP status code is not handled or not allowed 2026-01-24 20:00:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelving-42w-x-14d-x-63h-4-shelf-starter-chrome>: HTTP status code is not handled or not allowed 2026-01-24 20:00:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nilfisk-alto-attix-19-rubber-crevice-nozzle>: HTTP status code is not handled or not allowed 2026-01-24 20:00:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/hvac_system_parts_components/heating_parts_components/thermocouples_thermopiles already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:00:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25t25000p05a-set-screw-w-full-ball-6-32-thread-x-12-thread-length>: HTTP status code is not handled or not allowed 2026-01-24 20:00:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-control-8mm-tube-x-1-8-metal-release-collet-flow-out-knob-adjustment>: HTTP status code is not handled or not allowed 2026-01-24 20:00:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/needle-valve-5-32-tube-x-10-32-unfscrew-adjustment>: HTTP status code is not handled or not allowed 2026-01-24 20:00:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/monitor_filters landed on page that is not a product page. 2026-01-24 20:00:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelving-30w-x-14d-x-54h-3-shelf-starter-chrome>: HTTP status code is not handled or not allowed 2026-01-24 20:00:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/orange-tint-3-wall-style-specimen-trans-bag-prnt-bio-6w-9l>: HTTP status code is not handled or not allowed 2026-01-24 20:00:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/basic-stirring-hotplate-4-4-ceramic-top-120v>: HTTP status code is not handled or not allowed 2026-01-24 20:00:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/battery-connector-din-a320-female-connector-with-50mm-socket-contacts-handle>: HTTP status code is not handled or not allowed 2026-01-24 20:00:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/hvac_system_parts_components/heating_parts_components/thermocouples_thermopiles>: HTTP status code is not handled or not allowed 2026-01-24 20:00:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelving-48w-x-14d-x-34h-2-shelf-starter-chrome>: HTTP status code is not handled or not allowed 2026-01-24 20:00:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-position-digital-stirring-hotplate-4-postion-120v>: HTTP status code is not handled or not allowed 2026-01-24 20:00:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-double-full-size-mattress-bag-54w-90l>: HTTP status code is not handled or not allowed 2026-01-24 20:00:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hardwound-roll-towel-7-88-inch-x-1000-ft-white-6-rolls-case>: HTTP status code is not handled or not allowed 2026-01-24 20:00:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-reclosable-pharmacy-bag-2-mil-12w-15l>: HTTP status code is not handled or not allowed 2026-01-24 20:00:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-step-plastic-step-stand-gray-20-w-x-33-1-2-d-x-28-1-2-h-st-3-gy>: HTTP status code is not handled or not allowed 2026-01-24 20:00:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/karcher-9-807-723-0-2700psi-3gpm-gas-pressure-washer>: HTTP status code is not handled or not allowed 2026-01-24 20:00:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-wall-reclosable-seal-top-specimen-trans-bag-pink-tint-12w-15l>: HTTP status code is not handled or not allowed 2026-01-24 20:00:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jolly-rancher-chews-candy-in-assorted-fruit-flavors-13-oz-4-count-gr>: HTTP status code is not handled or not allowed 2026-01-24 20:00:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crunch-milk-chocolate-bar-1-55-oz-36-count-gr>: HTTP status code is not handled or not allowed 2026-01-24 20:00:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smooth-platform-kit-12-x-18-x-72>: HTTP status code is not handled or not allowed 2026-01-24 20:00:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-digit-calculator-el2196bl-2-color-printing-8-7-8-x-12-9-10-x-3-1-10-black>: HTTP status code is not handled or not allowed 2026-01-24 20:00:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pegboard-tool-cart-galvanized-48-x-16>: HTTP status code is not handled or not allowed 2026-01-24 20:00:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smooth-tiles-18-x-18>: HTTP status code is not handled or not allowed 2026-01-24 20:00:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/open-tiles-18-x-18>: HTTP status code is not handled or not allowed 2026-01-24 20:00:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2l-erlenmeyer-flask-clamp-30158-for-use-with-maxq-shaker-platforms>: HTTP status code is not handled or not allowed 2026-01-24 20:00:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beam-join-kit-set-of-2>: HTTP status code is not handled or not allowed 2026-01-24 20:00:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-readybuilt-locker-12-w-x-15-d-x-78-h-729-parchment-double-tier-1-wide-assembled>: HTTP status code is not handled or not allowed 2026-01-24 20:00:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-72-w-x-24-d-x-84-h-add-on-3-shelf-levels-wood-deck-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:00:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-shelf-level-for-bulk-storage-rack-48-w-x-48-d-wood-deck-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:00:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/diamond-plate-platform-kit-4-x-36-x-18>: HTTP status code is not handled or not allowed 2026-01-24 20:00:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smooth-platform-kit-4-x-36-x-54>: HTTP status code is not handled or not allowed 2026-01-24 20:00:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-800-series-storage-cabinet-36w-x-24d-x-78h-midnight-ebony-single-tier-assembled>: HTTP status code is not handled or not allowed 2026-01-24 20:00:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smooth-platform-kit-8-x-18-x-72>: HTTP status code is not handled or not allowed 2026-01-24 20:00:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/block-heater-40-x-0-5-ml-tubes>: HTTP status code is not handled or not allowed 2026-01-24 20:00:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-value-max-locker-36-w-x-15-d-x-78-h-729-parchment-triple-tier-3-wide-assembled>: HTTP status code is not handled or not allowed 2026-01-24 20:00:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-value-max-locker-12-w-x-18-d-x-78-h-725-hallowell-gray-single-tier-1-wide-assembled>: HTTP status code is not handled or not allowed 2026-01-24 20:00:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10ml-erlenmeyer-flask-clamp-30150bi-for-use-with-maxq-shaker-platforms>: HTTP status code is not handled or not allowed 2026-01-24 20:00:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/legacy-20-low-box-file-lateral-ash-grey>: HTTP status code is not handled or not allowed 2026-01-24 20:00:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-shelf-level-for-bulk-storage-rack-72-w-x-48-d-steel-deck-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:00:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-72-w-x-48-d-x-72-h-add-on-3-shelf-levels-steel-deck-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:00:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/legacy-60-single-pedestal-desk-ash-grey>: HTTP status code is not handled or not allowed 2026-01-24 20:00:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnetic-1x2-grid-planner-steel-surface-72w-x-48h-ma2792830>: HTTP status code is not handled or not allowed 2026-01-24 20:00:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-series-wheel-20000255-2-1-2-x-1-1-4-polyolefin-3-8-annular-ball-bearing-black>: HTTP status code is not handled or not allowed 2026-01-24 20:00:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/prang-100-crayons-made-with-soy-16-colorsbox>: HTTP status code is not handled or not allowed 2026-01-24 20:00:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-series-wheel-500006929-ws-6-x-2-polyurethane-on-polyolefin-1-2-straight-roller-bearing>: HTTP status code is not handled or not allowed 2026-01-24 20:00:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-series-wheel-5000089497-ws-8-x-2-polyurethane-on-cast-iron-1-2-straight-roller-bearing>: HTTP status code is not handled or not allowed 2026-01-24 20:00:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-hi-tech-metal-bin-shelving-36-w-x-24-d-x-87-h-725-hallowell-gray-add-on-unit-78-bins>: HTTP status code is not handled or not allowed 2026-01-24 20:00:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mold-mildew-stain-remover-32oz>: HTTP status code is not handled or not allowed 2026-01-24 20:00:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-hi-tech-metal-bin-shelving-36-w-x-24-d-x-87-h-725-hallowell-gray-add-on-unit-29-bins>: HTTP status code is not handled or not allowed 2026-01-24 20:00:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-hi-tech-metal-bin-shelving-36-w-x-12-d-x-87-h-725-hallowell-gray-add-on-unit-10-bins>: HTTP status code is not handled or not allowed 2026-01-24 20:00:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chenille-kraft-4091-modeling-clay-assortment-27-12g-each-assorted-neon220-g>: HTTP status code is not handled or not allowed 2026-01-24 20:00:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-hi-tech-metal-bin-shelving-36-w-x-12-d-x-87-h-725-hallowell-gray-add-on-unit-24-bins>: HTTP status code is not handled or not allowed 2026-01-24 20:00:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1181032-belt-pulley-ax25>: HTTP status code is not handled or not allowed 2026-01-24 20:00:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/creativity-street-5923-plastic-paint-trays-10pack>: HTTP status code is not handled or not allowed 2026-01-24 20:00:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/innovera-ivr81910-slim-cd-case-assorted-colors-10pack>: HTTP status code is not handled or not allowed 2026-01-24 20:01:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-48-w-x-48-d-x-120-h-add-on-3-shelf-levels-steel-deck-medium-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:01:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2231016-pin-drive-5x24-mm-pack-10-for-dito-dean>: HTTP status code is not handled or not allowed 2026-01-24 20:01:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/command-17067-general-purpose-hooks-small-holds-12-lb-white-3pack>: HTTP status code is not handled or not allowed 2026-01-24 20:01:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thermo-flask-benchtop-liquid-nitrogen-container-with-lid-and-handle-4-5-liters>: HTTP status code is not handled or not allowed 2026-01-24 20:01:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arctic-express-iata-approved-shipping-system-10-liters>: HTTP status code is not handled or not allowed 2026-01-24 20:01:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fort-knox-workbench-pedestal-3-drawer-18-w-x-24-d-x-32-h-blk-body-red-doors-1-wide-all-welded-1>: HTTP status code is not handled or not allowed 2026-01-24 20:01:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fort-knox-workbench-pedestal-doors-18-w-x-24-d-x-32-h-blk-body-red-doors-1-wide-all-welded-1>: HTTP status code is not handled or not allowed 2026-01-24 20:01:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/321420-o-ring-2-od-for-hobart>: HTTP status code is not handled or not allowed 2026-01-24 20:01:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pacon-37036-bordette-decorative-border-2-14-x-50-roll-flame-red>: HTTP status code is not handled or not allowed 2026-01-24 20:01:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/serrated-2-step-24-alum-rolling-ladder-10-top-step-spring-load-a2sh30g>: HTTP status code is not handled or not allowed 2026-01-24 20:01:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-96-w-x-36-d-x-96-h-add-on-3-shelf-levels-wood-deck-sand>: HTTP status code is not handled or not allowed 2026-01-24 20:01:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cgw-4-12x035x78-t1-za60t-quickie-cut-reinforced-cutoff-wheel>: HTTP status code is not handled or not allowed 2026-01-24 20:01:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ribbed-7-step-24-alum-rolling-ladder-10-top-step-spring-load-a7sh30>: HTTP status code is not handled or not allowed 2026-01-24 20:01:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 16517 pages (at 98 pages/min), scraped 11390 items (at 23 items/min) 2026-01-24 20:01:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cgw-4-12x040x78-t1-za36t-quickie-cut-reinforced-cutoff-wheel>: HTTP status code is not handled or not allowed 2026-01-24 20:01:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bookcase-31wx14dx655h-mocha>: HTTP status code is not handled or not allowed 2026-01-24 20:01:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/241003-burner-nat-lp-4-ci>: HTTP status code is not handled or not allowed 2026-01-24 20:01:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/draw-shave-tmb-10dc-curved-10-steel-blade-with-wood-handles>: HTTP status code is not handled or not allowed 2026-01-24 20:01:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_alarms_signals/audible_alarms already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:01:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t8-fluorescent-lamp-f17t8tl835alto-17w-3500k>: HTTP status code is not handled or not allowed 2026-01-24 20:01:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40-small-capacity-co2-incubator-with-tc-sensor-120v-50-60hz>: HTTP status code is not handled or not allowed 2026-01-24 20:01:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alera-alesd217236bw-double-pedestal-steel-desk-metal-desk-72w-x-36d-x-29-12h-walnutblack>: HTTP status code is not handled or not allowed 2026-01-24 20:01:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-inch-transfer-pump-hose-kits-w-aluminum-couplers-and-fittings>: HTTP status code is not handled or not allowed 2026-01-24 20:01:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-inch-x-25-foot-pvc-lay-flat-discharge-hose-coupled-with-cxe-aluminum-cam-and-groove-fittings>: HTTP status code is not handled or not allowed 2026-01-24 20:01:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-72-w-x-48-d-x-72-h-starter-3-shelf-levels-steel-deck-medium-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:01:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dw-portable-forced-air-propane-heater-dxh125fav-85k-to-125k-btu>: HTTP status code is not handled or not allowed 2026-01-24 20:01:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/85-1130-pink-extra-thick-ptfe-teflon-tape-1-2-inch-x-260-inch>: HTTP status code is not handled or not allowed 2026-01-24 20:01:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_alarms_signals/audible_alarms landed on page that is not a product page. 2026-01-24 20:01:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chateau-series-mahogany-laminate-desking-355x22x30-lateral-file>: HTTP status code is not handled or not allowed 2026-01-24 20:01:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-40-14w-mobile-cabinet-5-drawers-63-compart-bright-blue-master-keyed>: HTTP status code is not handled or not allowed 2026-01-24 20:01:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-40-14w-cabinet-6-drawer-42-compart-classic-blue-individual-lock>: HTTP status code is not handled or not allowed 2026-01-24 20:01:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/essentials-series-cherry-laminate-credenza-28lx1175wx1d>: HTTP status code is not handled or not allowed 2026-01-24 20:01:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-ton-side-pump-bottle-jack-10065b>: HTTP status code is not handled or not allowed 2026-01-24 20:01:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-28-14w-mobile-cabinet-7-drawers-72-compart-light-gray-no-lock>: HTTP status code is not handled or not allowed 2026-01-24 20:01:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/essentials-series-mahogany-laminate-storage-cabinet-22x355x295>: HTTP status code is not handled or not allowed 2026-01-24 20:01:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-40-14w-drawer-cabinet-4-drawer-24-compart-classic-blue-keyed-alike>: HTTP status code is not handled or not allowed 2026-01-24 20:01:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kithvacr-hvac-r-starter-kit-1890995>: HTTP status code is not handled or not allowed 2026-01-24 20:01:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-40-14w-drawer-cabinet-6-drawer-72-compart-light-gray-master-keyed>: HTTP status code is not handled or not allowed 2026-01-24 20:01:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-exercise-band-medium-green>: HTTP status code is not handled or not allowed 2026-01-24 20:01:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-loop-webbing-stirrup-for-exercise-band-tubing-50-each>: HTTP status code is not handled or not allowed 2026-01-24 20:01:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-ae24b-black-epoxy-safety-push-handle-24w>: HTTP status code is not handled or not allowed 2026-01-24 20:01:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tubing-exerciser-black-14>: HTTP status code is not handled or not allowed 2026-01-24 20:01:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tubing-exerciser-blue-14>: HTTP status code is not handled or not allowed 2026-01-24 20:01:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/braided-ground-strap-207110-100-1-0-gauge-100-pcs>: HTTP status code is not handled or not allowed 2026-01-24 20:01:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-general-purpose-primary-wire-230200-2045-18-gauge-45-ft>: HTTP status code is not handled or not allowed 2026-01-24 20:01:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-exercise-tubing-blue-25-roll-box>: HTTP status code is not handled or not allowed 2026-01-24 20:01:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-ah18s-stainless-steel-utility-cart-handle-18w>: HTTP status code is not handled or not allowed 2026-01-24 20:01:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-28-14w-cabinet-9-drawer-111-compart-light-gray-no-lock>: HTTP status code is not handled or not allowed 2026-01-24 20:01:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-al414s-stainless-steel-wire-ledge-14l-x-4h>: HTTP status code is not handled or not allowed 2026-01-24 20:01:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-ae21s-stainless-steel-safety-push-handle-21w>: HTTP status code is not handled or not allowed 2026-01-24 20:01:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tubing-exerciser-black-30>: HTTP status code is not handled or not allowed 2026-01-24 20:01:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1201177-caster-boot-2-inch-f-1-inch-od-w-brk>: HTTP status code is not handled or not allowed 2026-01-24 20:01:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unisex-navy-cotton-polyester-fashion-blend-reversible-set-in-sleeve-scrub-shirt-size-l>: HTTP status code is not handled or not allowed 2026-01-24 20:01:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tisa-452-18-automatic-impulse-sealer-with-2mm-seal-width>: HTTP status code is not handled or not allowed 2026-01-24 20:01:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2801322-corkscrew-blk-plst-hndl-hd>: HTTP status code is not handled or not allowed 2026-01-24 20:01:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unisex-ciel-blue-cotton-polyester-fashion-blend-reversible-scrub-pants-size-l>: HTTP status code is not handled or not allowed 2026-01-24 20:01:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-1069-cast-iron-open-top-burner-head-6-inch>: HTTP status code is not handled or not allowed 2026-01-24 20:01:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1711197-brush-char-medium-for-prince-castle>: HTTP status code is not handled or not allowed 2026-01-24 20:01:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-inplant-office-w-2-windows-class-a-8-w-x-16-d-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:01:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-cable-tie-tool-502903-2001-19-50-lb>: HTTP status code is not handled or not allowed 2026-01-24 20:01:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ironwood-3-compartment-black-grill-door-48-w-cabinet-oiled-cherry-color>: HTTP status code is not handled or not allowed 2026-01-24 20:01:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/44014-detail-brush-extra-stiff-red>: HTTP status code is not handled or not allowed 2026-01-24 20:01:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leviton-2536-slimline-base-single-pin-fluorescent-lampholder-pedestal-white>: HTTP status code is not handled or not allowed 2026-01-24 20:01:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conntek-14303-36-50-amp-rv-camp-power-cord-with-nema-14-50p-male-plug-to-bare-wire>: HTTP status code is not handled or not allowed 2026-01-24 20:01:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leviton-124-d-incandescent-lampholder-medium-base-single-circuit-660w-250v>: HTTP status code is not handled or not allowed 2026-01-24 20:01:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/70474-high-low-brush-medium-red>: HTTP status code is not handled or not allowed 2026-01-24 20:01:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/instrument-cart-2-shelves-1200-lbs-capacity-36-x-24-8-pneumatic-casters>: HTTP status code is not handled or not allowed 2026-01-24 20:01:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010545-grate-broiler-for-garland-range>: HTTP status code is not handled or not allowed 2026-01-24 20:01:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hager-780-157-heavy-duty-full-surface-hinge-fire-rated>: HTTP status code is not handled or not allowed 2026-01-24 20:02:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1711114-cushions-guide-rod-pk-4-for-prince-castle>: HTTP status code is not handled or not allowed 2026-01-24 20:02:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/whiffle153-typical-2-mobile-storage-cabinet-30w-x-19-34d-x-48h-red>: HTTP status code is not handled or not allowed 2026-01-24 20:02:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-buckle-boots-size-9-rubber-black-1-pair>: HTTP status code is not handled or not allowed 2026-01-24 20:02:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/56609-dustpan-black>: HTTP status code is not handled or not allowed 2026-01-24 20:02:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-free-exercise-band-5-strips-yellow-30-strips-box>: HTTP status code is not handled or not allowed 2026-01-24 20:02:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-free-exercise-band-50-yard-roll-box-5-color-set-10-6328>: HTTP status code is not handled or not allowed 2026-01-24 20:02:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-free-exercise-band-silver-50-yard-roll-box>: HTTP status code is not handled or not allowed 2026-01-24 20:02:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/huot-toolscoot-50-taper-cnc-holders>: HTTP status code is not handled or not allowed 2026-01-24 20:02:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/huot-supertower-50-taper-cnc-holders>: HTTP status code is not handled or not allowed 2026-01-24 20:02:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/53684-216-1-6-tube-brush-for-flex-rod-medium-red>: HTTP status code is not handled or not allowed 2026-01-24 20:02:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 16613 pages (at 96 pages/min), scraped 11422 items (at 32 items/min) 2026-01-24 20:02:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/71703-28-single-blade-ultra-hygiene-squeegee-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:02:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-theft-resistant-spindle>: HTTP status code is not handled or not allowed 2026-01-24 20:02:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-exercise-band-6-yard-roll-box-5-color-set>: HTTP status code is not handled or not allowed 2026-01-24 20:02:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-exercise-band-tan-50-yard-roll-box>: HTTP status code is not handled or not allowed 2026-01-24 20:02:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-swing-up-grab-bar-29-w-satin>: HTTP status code is not handled or not allowed 2026-01-24 20:02:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-conturaseries-25-cent-semi-recessed-sanitary-vendor>: HTTP status code is not handled or not allowed 2026-01-24 20:02:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/barnstead-0-2-m-final-filter-d3750-1-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:02:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-welded-frame-mirror-18-w-x-36-h>: HTTP status code is not handled or not allowed 2026-01-24 20:02:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reflective-delineator-post-49-in-h-black>: HTTP status code is not handled or not allowed 2026-01-24 20:02:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-classicseries-heavy-duty-shower-curtain-rod-72-w>: HTTP status code is not handled or not allowed 2026-01-24 20:02:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012411-switch-ilum-rocker-for-woodstone-ovens>: HTTP status code is not handled or not allowed 2026-01-24 20:02:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-classicseries-mop-broom-holder-w-rag-hooks-shelf-34-w>: HTTP status code is not handled or not allowed 2026-01-24 20:02:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-shelf-liner-72w-x-18d-white>: HTTP status code is not handled or not allowed 2026-01-24 20:02:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dandux-vinyl-basket-truck-bulk-truck-20-bushel-green>: HTTP status code is not handled or not allowed 2026-01-24 20:02:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-shelf-liner-60w-x-24d-white>: HTTP status code is not handled or not allowed 2026-01-24 20:02:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-600-series-surface-mounted-double-tissue-dispenser-hoods-satin>: HTTP status code is not handled or not allowed 2026-01-24 20:02:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1111292-aerator-22gpm-vandal-resist-for-ts-brass-bronze-works>: HTTP status code is not handled or not allowed 2026-01-24 20:02:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-battery-jumper-cable-4t-awg-16l-blackred>: HTTP status code is not handled or not allowed 2026-01-24 20:02:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-flat-top-workbench-72-w-x-24-d-1>: HTTP status code is not handled or not allowed 2026-01-24 20:02:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/321444-floor-drain-strainer>: HTTP status code is not handled or not allowed 2026-01-24 20:02:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/64407-small-hand-brush-stiff-orange>: HTTP status code is not handled or not allowed 2026-01-24 20:02:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gn831-1-toggle-latch-110engk-size-110-steel>: HTTP status code is not handled or not allowed 2026-01-24 20:02:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-shelf-liner-72w-x-12d-white>: HTTP status code is not handled or not allowed 2026-01-24 20:02:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/52-bollard-flex-natural-ground-installation-yellow-cover-blue-tapes>: HTTP status code is not handled or not allowed 2026-01-24 20:02:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/76-1201-grinder-attachment-for-22-hubs>: HTTP status code is not handled or not allowed 2026-01-24 20:02:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/62504-lobby-dustpan-w-broom-red>: HTTP status code is not handled or not allowed 2026-01-24 20:02:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-16-18-x-6-machine-screw-round-head-slotted-steel-zinc-ft-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 20:02:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-vandal-resistant-frameless-mirror-11-1-4-w-x-17-1-4-h>: HTTP status code is not handled or not allowed 2026-01-24 20:02:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/85-1048-solderless-terminal-connector-1-4-inch-male-tab-10-stud-hole-10-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:02:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/screws/machine_screws/pan_head_machine_screws already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:02:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/261548-male-branch-tee>: HTTP status code is not handled or not allowed 2026-01-24 20:02:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/52-bollard-flex-with-sign-post-asphalt-installation-yellow-cover-blue-tapes>: HTTP status code is not handled or not allowed 2026-01-24 20:02:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelate-wire-shelf-truck-72x18x69-1200-pound-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:02:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/262634-flat-roller>: HTTP status code is not handled or not allowed 2026-01-24 20:02:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-shelf-liner-42w-x-12d-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:02:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/85-1078-insulated-set-screw-wire-connector-large>: HTTP status code is not handled or not allowed 2026-01-24 20:02:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-polyester-filter-18-w-x-20-h-x-1-d>: HTTP status code is not handled or not allowed 2026-01-24 20:02:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/er70s-6-mild-steel-mig-welding-wire-030-dia-x-33-lb-spool>: HTTP status code is not handled or not allowed 2026-01-24 20:02:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/canon-6432b001-ink-pgi-250xl-chromalife-100-500-page-yield-black>: HTTP status code is not handled or not allowed 2026-01-24 20:02:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-4-sjoow-black-250-ft>: HTTP status code is not handled or not allowed 2026-01-24 20:02:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manhole-sleeve-li-er-two-legs-1000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:02:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/85-1063-ignition-terminal-female-spark-plug-end>: HTTP status code is not handled or not allowed 2026-01-24 20:02:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/fasteners/screws/machine_screws/pan_head_machine_screws>: HTTP status code is not handled or not allowed 2026-01-24 20:02:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conductive-churn-brush-10-4-5in-black-c10546>: HTTP status code is not handled or not allowed 2026-01-24 20:02:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-fiberglass-filter-20-w-x-24-h-x-1-d>: HTTP status code is not handled or not allowed 2026-01-24 20:02:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conductive-workbench-cleaning-set-utility-brush-dust-pan-black-c80500>: HTTP status code is not handled or not allowed 2026-01-24 20:02:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/31946-24-combo-push-broom-soft-stiff-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:02:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/head_face_protection/welding_helmets_lenses already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:02:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/261400-male-connector>: HTTP status code is not handled or not allowed 2026-01-24 20:02:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stronghold-ventilated-all-around-cabinet-36-x-24-x-66>: HTTP status code is not handled or not allowed 2026-01-24 20:02:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/100-45176-flat-elbow-5-14l-x-4-14w-x-3h-white>: HTTP status code is not handled or not allowed 2026-01-24 20:02:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anti-static-dust-pan-21in-black-c80305>: HTTP status code is not handled or not allowed 2026-01-24 20:02:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5380-103-4-216-4-0-pipe-brush-medium-red>: HTTP status code is not handled or not allowed 2026-01-24 20:02:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hinged-duct-access-door-18-x-18>: HTTP status code is not handled or not allowed 2026-01-24 20:02:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2941032-torque-master-reversible-for-anthony-international-california>: HTTP status code is not handled or not allowed 2026-01-24 20:02:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stronghold-stainless-steel-closet-cabinet-48-x-24-x-66>: HTTP status code is not handled or not allowed 2026-01-24 20:02:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011595-knob-for-vulcan-hart>: HTTP status code is not handled or not allowed 2026-01-24 20:02:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stronghold-ss-2-door-independent-lock-cabinet-60-x-24-x-78-8drawers>: HTTP status code is not handled or not allowed 2026-01-24 20:02:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40113-4-stainless-steel-handle-mounted-scraper-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:02:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dinex-6-oz-china-casserole-6-oz-bright-white-dx6cass02a>: HTTP status code is not handled or not allowed 2026-01-24 20:02:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1211121-glide-superlevel-3-8-16>: HTTP status code is not handled or not allowed 2026-01-24 20:02:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/head_face_protection/welding_helmets_lenses>: HTTP status code is not handled or not allowed 2026-01-24 20:02:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-flush-access-door-16-x-16>: HTTP status code is not handled or not allowed 2026-01-24 20:02:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/recessed-valve-box-steel-12-x-12-x-4>: HTTP status code is not handled or not allowed 2026-01-24 20:03:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/battery-powered-li-stacker-truck-1000-lb-capacity-adjustable-25-forks-3>: HTTP status code is not handled or not allowed 2026-01-24 20:03:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/-replacement-12-power-cord-w-gfci-for-apex--jetstream-series-evaporative-coolers>: HTTP status code is not handled or not allowed 2026-01-24 20:03:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/-replacement-pump-for-apex153-500--700-portable-evaporative-coolers>: HTTP status code is not handled or not allowed 2026-01-24 20:03:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:03:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2945-51-aluminum-light-duty-handle>: HTTP status code is not handled or not allowed 2026-01-24 20:03:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-table-with-2-1-4-backsplash-60-w-x-30-d>: HTTP status code is not handled or not allowed 2026-01-24 20:03:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/condensate-pan-drain-opener-1-gallon>: HTTP status code is not handled or not allowed 2026-01-24 20:03:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/41952-narrow-utility-brush-extra-stiff-green>: HTTP status code is not handled or not allowed 2026-01-24 20:03:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bolted-teardrop-pallet-rack-starter-108inw-x-48ind-x-144-inh-6420-lbs-shelf-cap>: HTTP status code is not handled or not allowed 2026-01-24 20:03:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 16708 pages (at 95 pages/min), scraped 11447 items (at 25 items/min) 2026-01-24 20:03:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/261406-male-elbow>: HTTP status code is not handled or not allowed 2026-01-24 20:03:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/41968-ust-long-handle-scrubbing-brush-stiff-purple>: HTTP status code is not handled or not allowed 2026-01-24 20:03:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forkli-able-shipping-container-with-lid-42-x-42-4>: HTTP status code is not handled or not allowed 2026-01-24 20:03:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/five-shelf-x-ray-storage-cabinet-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:03:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forkli-able-shipping-container-49-x-21-without-lid-1>: HTTP status code is not handled or not allowed 2026-01-24 20:03:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits landed on page that is not a product page. 2026-01-24 20:03:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergonomic-sit-stand-desk-monitor-mount-combo-kit-dual-monitor-full-keyboard>: HTTP status code is not handled or not allowed 2026-01-24 20:03:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/indoor-outdoor-quartz-electric-infrared-heater-och-57-480ve-480v-2250w-brown>: HTTP status code is not handled or not allowed 2026-01-24 20:03:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-dolly-with-non-tilt-rollers-36-l-x-36-w-4000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:03:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-dolly-with-non-tilt-rollers-48-l-x-42-w-4000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:03:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-wrist-and-ankle-weight-15-lb-tan>: HTTP status code is not handled or not allowed 2026-01-24 20:03:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stretch-unlined-bib-overall-xs-brown>: HTTP status code is not handled or not allowed 2026-01-24 20:03:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cec10025-socket-cap-screw>: HTTP status code is not handled or not allowed 2026-01-24 20:03:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blue-diamond-non-reinforced-water-activated-tape-3in-x-600ft-tan>: HTTP status code is not handled or not allowed 2026-01-24 20:03:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washington-redskins-helmet-starter-rug-20-inch-x-30-inch>: HTTP status code is not handled or not allowed 2026-01-24 20:03:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/63-h-nexel-chrome-wire-shelving-36-w-x-30-d>: HTTP status code is not handled or not allowed 2026-01-24 20:03:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-coast-guard-44-inch-roung-rug-44-inch-dia>: HTTP status code is not handled or not allowed 2026-01-24 20:03:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industries-handrail-bracket-for-2-tubing-satin-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 20:03:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industries-handrail-bracket-for-2-tubing-polished-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 20:03:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/74-h-nexel-chrome-wire-shelving-60-w-x-14-d>: HTTP status code is not handled or not allowed 2026-01-24 20:03:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/74-h-nexel-chrome-wire-shelving-48-w-x-36-d>: HTTP status code is not handled or not allowed 2026-01-24 20:03:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hand-basket-stand-wheels-28-30-liter-basket>: HTTP status code is not handled or not allowed 2026-01-24 20:03:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ice-dispenser-counter-model-approximately-200-lb-storage-capacity-cube-and-pearl-ice>: HTTP status code is not handled or not allowed 2026-01-24 20:03:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/63-h-nexel-chrome-wire-shelving-add-on-48-w-x-30-d>: HTTP status code is not handled or not allowed 2026-01-24 20:03:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-oz-solid-gray-spoodle>: HTTP status code is not handled or not allowed 2026-01-24 20:03:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-3799-1-4-inch-to-3-8-inch-stuffing-box-adapter-6-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:03:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industries-combination-bracket-for-15-tubing-polished-brass>: HTTP status code is not handled or not allowed 2026-01-24 20:03:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/421759-cover-rocker-switch-plastic-for-star-manufacturing>: HTTP status code is not handled or not allowed 2026-01-24 20:03:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/341960-aluminum-flexible-drainheater>: HTTP status code is not handled or not allowed 2026-01-24 20:03:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-fixed-length-plastic-truck-seal-7-5-green-60lbs-pull-force-100-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:03:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-inch-pom-tong-kool-touch>: HTTP status code is not handled or not allowed 2026-01-24 20:03:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/381733-lamp-for-amana-litton>: HTTP status code is not handled or not allowed 2026-01-24 20:03:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelite-vented-plastic-mat-shelf-truck-60l-x-21w-x-60h-polyurethane-brake-casters>: HTTP status code is not handled or not allowed 2026-01-24 20:03:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ladle-3-oz-1>: HTTP status code is not handled or not allowed 2026-01-24 20:03:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1901195-nozzle-dilution-tea-brewers-for-bunn-o-matic>: HTTP status code is not handled or not allowed 2026-01-24 20:03:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22-1044-slicer-knife-guard-knob-and-stud>: HTTP status code is not handled or not allowed 2026-01-24 20:03:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/relaxed-fit-twill-safety-cargo-utility-pants-32l-inseam-x-38-waist-black>: HTTP status code is not handled or not allowed 2026-01-24 20:03:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bin-divider-fits-7-wx4-h-bins-1>: HTTP status code is not handled or not allowed 2026-01-24 20:03:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelite-vented-plastic-mat-shelf-truck-60l-x-24w-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 20:03:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industries-flush-cross-fitting-for-15-tubing-polished-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 20:03:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22-1126-2-inch-dial-kit-off-100-550>: HTTP status code is not handled or not allowed 2026-01-24 20:03:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/servewell-economy-breath-guard-3>: HTTP status code is not handled or not allowed 2026-01-24 20:03:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-30-adjustable-leg-anti-static-top-workbench-beige>: HTTP status code is not handled or not allowed 2026-01-24 20:03:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-1053-stainless-steel-round-head-10-32-x-3-4-inch-phillips-machine-screw-100-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:03:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1031242-motor-relay>: HTTP status code is not handled or not allowed 2026-01-24 20:03:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72l-flip-top-training-table-cherry-finish>: HTTP status code is not handled or not allowed 2026-01-24 20:03:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/formula-50-cleaner-degreaser-55-gallon-drum>: HTTP status code is not handled or not allowed 2026-01-24 20:03:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/360-stretch-waist-flex-twill-cargo-pants-30l-inseam-x-39-12-40-12-waist-navy>: HTTP status code is not handled or not allowed 2026-01-24 20:03:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/360-stretch-waist-flex-twill-cargo-pants-30l-inseam-x-29-30-waist-navy>: HTTP status code is not handled or not allowed 2026-01-24 20:03:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pop-blind-rivet-3-16-x-6-2-button-head-up-to-1-8-grip-aluminum-steel-pkg-of-500>: HTTP status code is not handled or not allowed 2026-01-24 20:03:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kennedy-5-drawer-roller-friction-brown-1>: HTTP status code is not handled or not allowed 2026-01-24 20:03:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stacking-bin-4-1-8-x-10-7-8-x-4-red>: HTTP status code is not handled or not allowed 2026-01-24 20:03:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/511337-pilot-for-groen>: HTTP status code is not handled or not allowed 2026-01-24 20:03:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pop-blind-rivet-3-16-x-6-16-button-head-up-to-1-grip-aluminum-steel-pkg-of-250>: HTTP status code is not handled or not allowed 2026-01-24 20:03:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/performance-gravity-hvlp-atomizing-head-refill-kit-26730-variety-kit>: HTTP status code is not handled or not allowed 2026-01-24 20:04:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/360-stretch-waist-flex-twill-cargo-pants-30l-inseam-x-39-12-40-12-waist-blk>: HTTP status code is not handled or not allowed 2026-01-24 20:04:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smead-file-folders-13-cut-reinforced-top-tabs-letter-assorted-12pack>: HTTP status code is not handled or not allowed 2026-01-24 20:04:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1881184-link-connecting-model-29-for-bloomfield>: HTTP status code is not handled or not allowed 2026-01-24 20:04:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/megaphone-6-watt-10-431>: HTTP status code is not handled or not allowed 2026-01-24 20:04:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-pressboard-folder-letter-four-section-emerald-green-10box>: HTTP status code is not handled or not allowed 2026-01-24 20:04:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/54-h-nexel-chrome-wire-shelving-add-on-48-w-x-30-d>: HTTP status code is not handled or not allowed 2026-01-24 20:04:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hubbel-industrial-bare-end-12-3-70>: HTTP status code is not handled or not allowed 2026-01-24 20:04:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cct-4-cruiser-bench-counting-scale-8-lb-x-0-0002-lb>: HTTP status code is not handled or not allowed 2026-01-24 20:04:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/passive-filter-427-replacement-lens-polycarbonate-shade-11-black-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-24 20:04:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smead-3-12-accordion-expansion-file-pockets-straight-tab-letter-blue-10box>: HTTP status code is not handled or not allowed 2026-01-24 20:04:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smead-supertab-colored-file-folders-13-cut-letter-assorted-100box>: HTTP status code is not handled or not allowed 2026-01-24 20:04:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hubbel-commercial-16-4-40>: HTTP status code is not handled or not allowed 2026-01-24 20:04:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1421285-tape-masking-2-inch-x60-yard>: HTTP status code is not handled or not allowed 2026-01-24 20:04:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 16802 pages (at 94 pages/min), scraped 11473 items (at 26 items/min) 2026-01-24 20:04:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/relaxed-fit-flex-twill-cargo-pants-30l-inseam-x-38-waist-navy>: HTTP status code is not handled or not allowed 2026-01-24 20:04:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hubbel-industrial-single-outlet-12-3-45>: HTTP status code is not handled or not allowed 2026-01-24 20:04:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1901250-strainer-water-leadfree-lo-flo-for-bunn-o-matic>: HTTP status code is not handled or not allowed 2026-01-24 20:04:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acrylic-coated-fiberglass-welding-blanket-6w-x-6h-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:04:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/file_folders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:04:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/86-h-nexel-chrome-wire-shelving-54-w-x-18-d>: HTTP status code is not handled or not allowed 2026-01-24 20:04:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-stage-hardboard-96l-x-48w-x-8h>: HTTP status code is not handled or not allowed 2026-01-24 20:04:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8ft-long-box-pleat-skirting-32h-stage-red>: HTTP status code is not handled or not allowed 2026-01-24 20:04:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4ft-long-box-pleat-skirting-8h-stage-red>: HTTP status code is not handled or not allowed 2026-01-24 20:04:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-125mm-bore-x-150mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 20:04:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fireproof-3-drawer-vertical-safe-in-file-legal-31-9-16-lx20-13-16-wx40-1-4-h-brown>: HTTP status code is not handled or not allowed 2026-01-24 20:04:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fireproof-3-drawer-vertical-safe-in-file-legal-31-9-16-lx20-13-16-wx40-1-4-h-pewter>: HTTP status code is not handled or not allowed 2026-01-24 20:04:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/54-wall-mount-post-bracket-kit>: HTTP status code is not handled or not allowed 2026-01-24 20:04:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fireproof-4-drawer-vertical-safe-in-file-legal-31-9-16-lx20-13-16-wx52-3-4-h-parchment>: HTTP status code is not handled or not allowed 2026-01-24 20:04:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/650210-end-mill-set-316-34-x-16ths>: HTTP status code is not handled or not allowed 2026-01-24 20:04:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/63-h-nexel-chrome-wire-shelving-add-on-72-w-x-24-d>: HTTP status code is not handled or not allowed 2026-01-24 20:04:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/maxiflex-gloves-black-micro-foam-nitirle-coated-knit-glove-x-large-12-pairs>: HTTP status code is not handled or not allowed 2026-01-24 20:04:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smead-hanging-file-folders-15-tab-11-point-stock-letter-lavender-25box>: HTTP status code is not handled or not allowed 2026-01-24 20:04:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/file_folders landed on page that is not a product page. 2026-01-24 20:04:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/430-1-4-composite-spreader-beam-1-ton-capacity-4-hook-spread>: HTTP status code is not handled or not allowed 2026-01-24 20:04:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/419-2-3-composite-lifting-beam-2-ton-capacity-3-hook-spread>: HTTP status code is not handled or not allowed 2026-01-24 20:04:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1421398-handle-wood-40-inch-w-metal-tip>: HTTP status code is not handled or not allowed 2026-01-24 20:04:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rwg564l-axis-cut-resistant-gloves-foam-nitrile-palm-hi-vis-grn-blk-l-1-pair>: HTTP status code is not handled or not allowed 2026-01-24 20:04:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paraffin-bath-with-6-lb-lavender-paraffin-100-liners-1-mitt-and-1-bootie>: HTTP status code is not handled or not allowed 2026-01-24 20:04:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-ripstop-4-in-1-safety-jacket-s-black>: HTTP status code is not handled or not allowed 2026-01-24 20:04:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-ripstop-4-in-1-safety-jacket-xl-black>: HTTP status code is not handled or not allowed 2026-01-24 20:04:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/type-r-class-2-heavy-duty-engineer-vest-orange-2xl-sv55-2zod-2x>: HTTP status code is not handled or not allowed 2026-01-24 20:04:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paraffin-bath-with-6-lb-unscented-paraffin-14-3l-x-8-2w-x-7-5d>: HTTP status code is not handled or not allowed 2026-01-24 20:04:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1761603-lid-container-mhbh650-650-ce-for-hamilton-beach>: HTTP status code is not handled or not allowed 2026-01-24 20:04:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kit-foot-mount-small-cs-32-for-iso-15552-cylinders>: HTTP status code is not handled or not allowed 2026-01-24 20:04:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fr-type-r-class-2-economy-mesh-vest-5xl-green-sv97e-2vgm-5x>: HTTP status code is not handled or not allowed 2026-01-24 20:04:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/type-r-class-3-heavy-duty-engineer-vest-orange-m-sv55-3zod-m>: HTTP status code is not handled or not allowed 2026-01-24 20:04:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cold-pain-relief-gel-4-oz-tube-case-of-12>: HTTP status code is not handled or not allowed 2026-01-24 20:04:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2531239-battery-size-inch-d-inch-alkaline>: HTTP status code is not handled or not allowed 2026-01-24 20:04:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-80mm-bore-x-720mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 20:04:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-male-connector-60000-04-02-14-tube-x-18-swift-fit-univ-thread-ss>: HTTP status code is not handled or not allowed 2026-01-24 20:04:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kolorcut-cutting-board-15x20x1-2-green>: HTTP status code is not handled or not allowed 2026-01-24 20:04:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-80mm-bore-x-380mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 20:04:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cold-pain-relief-gel-3-oz-roll-on-bottle-case-of-12>: HTTP status code is not handled or not allowed 2026-01-24 20:04:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cold-pain-relief-spray-4-oz-bottle>: HTTP status code is not handled or not allowed 2026-01-24 20:04:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inline-needle-valve-82835-06-06-38-male-nptf-x-38-tube-nickel-plated-brass>: HTTP status code is not handled or not allowed 2026-01-24 20:04:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cep001-o-custom-molded-earplugs-nrr-26db-orange-1-pair>: HTTP status code is not handled or not allowed 2026-01-24 20:04:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/domed-garnish-center>: HTTP status code is not handled or not allowed 2026-01-24 20:04:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-100mm-bore-x-900mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 20:04:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-100mm-bore-x-320mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 20:04:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-80mm-bore-x-120mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 20:04:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speed-rack-holds-10-quart-liter-bottles-single-tier>: HTTP status code is not handled or not allowed 2026-01-24 20:04:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-run-tee-89222-06-02-38-tube-x-18-swift-fit-univ-thread-metallic-collet>: HTTP status code is not handled or not allowed 2026-01-24 20:04:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reusable-hot-and-cold-compress-5-x-6-case-of-48>: HTTP status code is not handled or not allowed 2026-01-24 20:04:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/battery-powered-lift-counter-balanced-stacker-6>: HTTP status code is not handled or not allowed 2026-01-24 20:04:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/capacity-counter-balanced-lift>: HTTP status code is not handled or not allowed 2026-01-24 20:04:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-safety-parka-jacket-3xlt-navy>: HTTP status code is not handled or not allowed 2026-01-24 20:04:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/264558-plate-caster-no-brake-for-american-permanent-ware>: HTTP status code is not handled or not allowed 2026-01-24 20:04:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-1>: HTTP status code is not handled or not allowed 2026-01-24 20:04:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/moist-heat-pack-cover-foam-filled-terry-standard-20-x-24-12-pk>: HTTP status code is not handled or not allowed 2026-01-24 20:04:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-black-urethane-reusable-cold-pack-neck-6-x-23-12-pk>: HTTP status code is not handled or not allowed 2026-01-24 20:04:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-6432-double-acting-magnetic-cylinder-25mm-bore-x-100mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 20:04:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/water-detection-surface-sensor-unsupervised>: HTTP status code is not handled or not allowed 2026-01-24 20:04:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-run-tee-85222-08-06-12-tube-x-38-swift-fit-univ-thread-composite-body>: HTTP status code is not handled or not allowed 2026-01-24 20:04:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6220606-screw-mach-th-ss-100>: HTTP status code is not handled or not allowed 2026-01-24 20:04:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-replacement-cable>: HTTP status code is not handled or not allowed 2026-01-24 20:05:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cold-pack-cover-neck-9-x-25-12-pk>: HTTP status code is not handled or not allowed 2026-01-24 20:05:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-40mm-bore-x-770mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 20:05:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2801293-pourer-liquor-red-plst-12-pk>: HTTP status code is not handled or not allowed 2026-01-24 20:05:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/38-1509-375-watt-infrared-heat-lamp-bulb>: HTTP status code is not handled or not allowed 2026-01-24 20:05:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-control-85953-04-04-screw-adj-14-tube-x-14-swift-fit-composite-body>: HTTP status code is not handled or not allowed 2026-01-24 20:05:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/better-for-you-snack-box-assorted-flavors-37-count>: HTTP status code is not handled or not allowed 2026-01-24 20:05:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-control-85968-04-04-knob-adj-14-tube-x-14-swift-fit-composite-body>: HTTP status code is not handled or not allowed 2026-01-24 20:05:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1341162-strap-set-pw-bi-fold-for-quickserv-corp-drive-thru-windows>: HTTP status code is not handled or not allowed 2026-01-24 20:05:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/centering-pin-for-6-centering-head>: HTTP status code is not handled or not allowed 2026-01-24 20:05:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vollrath-swirlserve-server-20-oz-white>: HTTP status code is not handled or not allowed 2026-01-24 20:05:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 16903 pages (at 101 pages/min), scraped 11504 items (at 31 items/min) 2026-01-24 20:05:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-black-urethane-reusable-cold-pack-standard-10-x-13-1-2-12-pk>: HTTP status code is not handled or not allowed 2026-01-24 20:05:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/moto-vest-4-pockets-cottonpolyester-3xl-navy>: HTTP status code is not handled or not allowed 2026-01-24 20:05:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1132-inch-id-x-1316-inch-od-x-0005-inch-steel-arbor-shim-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 20:05:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/moist-heat-neck-pack-and-cover-set-1-set>: HTTP status code is not handled or not allowed 2026-01-24 20:05:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-inch-id-x-1-12-inch-od-steel-arbor-spacer-assortment>: HTTP status code is not handled or not allowed 2026-01-24 20:05:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-wide-span-rack-60x24x60-gray-with-3-shelves-laminated-deck-1200-lb-cap-per-level>: HTTP status code is not handled or not allowed 2026-01-24 20:05:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2261142-fry-skimmer-inch-fine-mesh-inch>: HTTP status code is not handled or not allowed 2026-01-24 20:05:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adapter-82242n-04-04-14-male-g-thread-x-14-female-nptf-nickel-plated-brass>: HTTP status code is not handled or not allowed 2026-01-24 20:05:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eagle-acid-corrosive-cabinet-manual-close-60-gallon>: HTTP status code is not handled or not allowed 2026-01-24 20:05:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/knee-boots-cleated-outsole-safety-toe-mens-size-5-15h-white>: HTTP status code is not handled or not allowed 2026-01-24 20:05:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-6432-single-acting-magnetic-cylinder-25mm-bore-x-50mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 20:05:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/moto-vest-4-pockets-cottonpolyester-xlt-brown>: HTTP status code is not handled or not allowed 2026-01-24 20:05:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-record-storage-open-3>: HTTP status code is not handled or not allowed 2026-01-24 20:05:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-heavy-duty-shelving-48x24x72-with-5-shelves-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:05:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/44-1387-24v-spark-module-with-wire-harness>: HTTP status code is not handled or not allowed 2026-01-24 20:05:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/male-y-55320-8-18-8mm-tube-x-18-swift-fit-univ-thread-composite-body>: HTTP status code is not handled or not allowed 2026-01-24 20:05:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-strip-10-outlets-surge-protector>: HTTP status code is not handled or not allowed 2026-01-24 20:05:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disc-pad-28772-6-inch-x-3-4-inch-5-16-24-external>: HTTP status code is not handled or not allowed 2026-01-24 20:05:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-heavy-duty-shelving-36x24x72-with-4-shelves-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:05:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-finishing-wheel-8-inch-x-1-inch-x-3-inch-med-grit-silicon-carbide>: HTTP status code is not handled or not allowed 2026-01-24 20:05:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-step-roll-fold-steel-rolling-ladder-grip-strut>: HTTP status code is not handled or not allowed 2026-01-24 20:05:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11-step-roll-fold-steel-rolling-ladder-grip-strut>: HTTP status code is not handled or not allowed 2026-01-24 20:05:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-branch-tee-60210-8-18-8mm-tube-x-18-male-bspt-thread-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 20:05:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-branch-tee-60215-8-14-8mm-tube-x-14-male-bspp-thread-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 20:05:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1981084-holder-knife-assembly-edlund-1-for-edlund>: HTTP status code is not handled or not allowed 2026-01-24 20:05:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/radial-bristle-disc-3-inch-x-nh-ceramic-50-grit>: HTTP status code is not handled or not allowed 2026-01-24 20:05:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/finish-thompson-air-motor-1hp-80-psi>: HTTP status code is not handled or not allowed 2026-01-24 20:05:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2801296-pourer-liquor-chrome-12-pk>: HTTP status code is not handled or not allowed 2026-01-24 20:05:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1901145-faucet-self-serve-orange-hdl-for-bunn-o-matic>: HTTP status code is not handled or not allowed 2026-01-24 20:05:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/32-piece-square-keystock-assortment-zinc-plated-12-inch-length>: HTTP status code is not handled or not allowed 2026-01-24 20:05:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-additional-shelf-level-boltless-wood-deck-34>: HTTP status code is not handled or not allowed 2026-01-24 20:05:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-extra-shelving-48x12x84-gray-with-5-shelves-1500-lb-cap-per-shelf>: HTTP status code is not handled or not allowed 2026-01-24 20:05:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-extra-shelving-48x12x60-gray-with-5-shelves-1500-lb-cap-per-shelf>: HTTP status code is not handled or not allowed 2026-01-24 20:05:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-20-ga-36-wx12-dx73-h-open-clip-style-7-shelf>: HTTP status code is not handled or not allowed 2026-01-24 20:05:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30004-p-plastic-barrier-chain-in-a-pail-hdpe-1-5-x300-6-38mm-green>: HTTP status code is not handled or not allowed 2026-01-24 20:05:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0002-inch-stainless-steel-shim-stock-6-inch-x-50-inch-roll>: HTTP status code is not handled or not allowed 2026-01-24 20:05:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/woodworking-workbench-1500-SM03-cabinet-and-free-accessory-kit>: HTTP status code is not handled or not allowed 2026-01-24 20:05:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultrasonic-thickness-gauge-001-resolution>: HTTP status code is not handled or not allowed 2026-01-24 20:05:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/laminated-aluminum-shim-0062-inch-thick-0002-inch-laminations-24-inch-x-24-inch-sheet>: HTTP status code is not handled or not allowed 2026-01-24 20:05:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/weekend-away-secure-mailbox-with-vault-dvjr0060-front-access-black>: HTTP status code is not handled or not allowed 2026-01-24 20:05:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0702320-4-408-osy-epoxy-coated-gate-valve>: HTTP status code is not handled or not allowed 2026-01-24 20:05:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/341164-warmer-element-240v-1000w-for-seco-products>: HTTP status code is not handled or not allowed 2026-01-24 20:05:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/440-series-premium-high-back-seat-with-lumber-support--slide-rails-vinyl-black>: HTTP status code is not handled or not allowed 2026-01-24 20:05:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-145-tefc-1ph-3450rpm-p145k34fc6gg>: HTTP status code is not handled or not allowed 2026-01-24 20:05:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0003-inch-brass-shim-stock-6-inch-x-60-inch-roll>: HTTP status code is not handled or not allowed 2026-01-24 20:05:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50024-500-plastic-barrier-chain-hdpe-2-x500-8-51mm-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:05:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/51036-100-alternating-heavy-duty-plastic-barrier-chain-hdpe-2-x100-54mm-black-red>: HTTP status code is not handled or not allowed 2026-01-24 20:05:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-20-ga-48-wx30-dx97-h-open-clip-style-7-shelf>: HTTP status code is not handled or not allowed 2026-01-24 20:05:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/top-mount-in-ground-post-for-weekend-away-mail-protector-dvjr0060-dvcs0070-dvjr0060pi-black>: HTTP status code is not handled or not allowed 2026-01-24 20:05:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/professional-infrared-thermometer-37v-2600mah-rechargeable-li-ion-battery-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:05:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/341140-strip-heater120v-500w-12-inch-x-1-1-2>: HTTP status code is not handled or not allowed 2026-01-24 20:05:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/die-rack-shelf-36x24>: HTTP status code is not handled or not allowed 2026-01-24 20:05:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-56-tefc-1ph-1725rpm-m6c17fk78ee>: HTTP status code is not handled or not allowed 2026-01-24 20:05:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/weekend-away-secure-mailbox-with-vault-dvjr0060-front-access-sand>: HTTP status code is not handled or not allowed 2026-01-24 20:05:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-215-tefc-1ph-1740rpm-b-1>: HTTP status code is not handled or not allowed 2026-01-24 20:05:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-rack-mobile-double-sided-rack-1>: HTTP status code is not handled or not allowed 2026-01-24 20:05:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36x18x74-stainless-steel-solid-shelving>: HTTP status code is not handled or not allowed 2026-01-24 20:05:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulkhead-union-60050-6-6mm-tube-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 20:06:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m4-x-0-7-x-10mm-machine-screw-phillips-flat-head-304-stainless-steel-din-965-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 20:06:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/166453-spout-cam-for-cambro>: HTTP status code is not handled or not allowed 2026-01-24 20:06:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-aquatic-exercise-kit-jogger-belt-ankle-cuffs-hand-bars-medium-red>: HTTP status code is not handled or not allowed 2026-01-24 20:06:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classic-kickboard-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:06:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bantam-switch-self-retracting-lifeline-single-leg-steel-rebar-hook-9l>: HTTP status code is not handled or not allowed 2026-01-24 20:06:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30007-100-plastic-barrier-chain-hdpe-1-5-x100-6-38mm-gold>: HTTP status code is not handled or not allowed 2026-01-24 20:06:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50309-10-s-hook-acetal-copolymer-2-gold-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 20:06:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1241442-trim-left-hand-79-inch-for-kolpak-refrigeration>: HTTP status code is not handled or not allowed 2026-01-24 20:06:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 17000 pages (at 97 pages/min), scraped 11535 items (at 31 items/min) 2026-01-24 20:06:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50310-10-s-hook-acetal-copolymer-2-brown-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 20:06:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-tecgen-flame-resistant-henley-shirt-2xl-tall-tan>: HTTP status code is not handled or not allowed 2026-01-24 20:06:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-tecgen-flame-resistant-work-shirt-4xl-tall-tan>: HTTP status code is not handled or not allowed 2026-01-24 20:06:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m6-x-1-0-x-10mm-button-head-socket-cap-screw-304-stainless-steel-din-7380-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 20:06:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012863-heating-element-208-240v-for-hobart>: HTTP status code is not handled or not allowed 2026-01-24 20:06:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/comfy-c-grip-hand-orthosis-adult-large-right-with-1-cover-and-2-soft-rolls>: HTTP status code is not handled or not allowed 2026-01-24 20:06:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ms104ss-all-ss-military-worm-gear-hose-clamp-4-18-inch-7-inch-clamping-diam-10-pcs>: HTTP status code is not handled or not allowed 2026-01-24 20:06:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m16-x-2-0-x-35mm-socket-head-cap-screw-304-stainless-steel-pkg-of-25>: HTTP status code is not handled or not allowed 2026-01-24 20:06:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qr56hs-quick-release-all-ss-hose-clamp-1-inch-4-inch-clamping-diam-10-pcs>: HTTP status code is not handled or not allowed 2026-01-24 20:06:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tex-classic-kinesiology-tape-2-x-4-4-yds-beige-6-rolls>: HTTP status code is not handled or not allowed 2026-01-24 20:06:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-tecgen-flame-resistant-henley-shirt-xs-regular-tan>: HTTP status code is not handled or not allowed 2026-01-24 20:06:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tex-gold-fp-kinesiology-tape-2-x-5-5-yds-red-1-roll>: HTTP status code is not handled or not allowed 2026-01-24 20:06:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-cu-yd-yellow-self-dumping-forkli-hopper-with-heavy-gauge-base>: HTTP status code is not handled or not allowed 2026-01-24 20:06:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-cu-yd-gray-self-dumping-forkli-hopper-with-heavy-gauge-base>: HTTP status code is not handled or not allowed 2026-01-24 20:06:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b16hs-all-ss-worm-gear-hose-clamp-1116-inch-1-12-inch-clamping-diameter-box-of-10>: HTTP status code is not handled or not allowed 2026-01-24 20:06:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-184-tefc-1ph-1740rpm-c184c17fc4cc>: HTTP status code is not handled or not allowed 2026-01-24 20:06:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-tecgen-flame-resistant-work-shirt-m-tall-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:06:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/boardwalk-plastic-tableware-heavy-mediumweight-knife-white-1000carton>: HTTP status code is not handled or not allowed 2026-01-24 20:06:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd36h-916-inch-band-heavy-duty-3-pc-part-ss-hose-clamp-10-pcs>: HTTP status code is not handled or not allowed 2026-01-24 20:06:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m20s-all-ss-worm-gear-hose-clamp-78-inch-1-34-inch-clamping-diameter-box-of-10>: HTTP status code is not handled or not allowed 2026-01-24 20:06:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-184-dp-1ph-3500rpm-c184k34db3bb>: HTTP status code is not handled or not allowed 2026-01-24 20:06:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1481035-trunnion-lid-rail-center-for-true-manufacturing>: HTTP status code is not handled or not allowed 2026-01-24 20:06:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergency-eyewash-plastic-push-handle-replacement>: HTTP status code is not handled or not allowed 2026-01-24 20:06:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blackout-cordless-cellular-shade-72-wide-x-48-drop-snowfall>: HTTP status code is not handled or not allowed 2026-01-24 20:06:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blackout-cordless-cellular-shade-72-wide-x-48-drop-pearl>: HTTP status code is not handled or not allowed 2026-01-24 20:06:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-213-tefc-1ph-3450rpm-c213k34fk1bb>: HTTP status code is not handled or not allowed 2026-01-24 20:06:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-184-dp-1ph-1740rpm-u184c17db20aa>: HTTP status code is not handled or not allowed 2026-01-24 20:06:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b88h-part-ss-worm-gear-hose-clamp-4-332-inch-6-inch-clamping-diameter-box-of-10>: HTTP status code is not handled or not allowed 2026-01-24 20:06:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-56-tefc-1ph-3450rpm-c6c34fk1hh>: HTTP status code is not handled or not allowed 2026-01-24 20:06:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-microwave-food-prep-station-48w-x-24d-x-74h-chrome>: HTTP status code is not handled or not allowed 2026-01-24 20:06:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-flame-resistant-dh-air153-work-shirt-xl-short-navy>: HTTP status code is not handled or not allowed 2026-01-24 20:06:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011659-contactor-for-lincoln-manufacturing>: HTTP status code is not handled or not allowed 2026-01-24 20:06:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-tecgen-flame-resistant-coverall-s-long-tall-navy>: HTTP status code is not handled or not allowed 2026-01-24 20:06:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1901322-cartridge-wtr-flt-sclpro-crtg-for-bunn-o-matic>: HTTP status code is not handled or not allowed 2026-01-24 20:06:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-partition-flexible-nine-panel-15>: HTTP status code is not handled or not allowed 2026-01-24 20:06:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-flame-resistant-helix-work-shirt-xl-tall-dark-khaki>: HTTP status code is not handled or not allowed 2026-01-24 20:06:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/boardwalk-mediumweight-polypropylene-cutlery-fork-white-1000carton>: HTTP status code is not handled or not allowed 2026-01-24 20:06:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1501517-dispenser-water-cup-plst-for-san-jamar>: HTTP status code is not handled or not allowed 2026-01-24 20:06:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/weekly-planner-whiteboard-steel-48-x-36>: HTTP status code is not handled or not allowed 2026-01-24 20:06:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-flame-resistant-dh-air153-work-shirt-xs-regular-khaki>: HTTP status code is not handled or not allowed 2026-01-24 20:06:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-commercial-cooks-scraper-13-12-white>: HTTP status code is not handled or not allowed 2026-01-24 20:06:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metal-bookcase-five-shelf-34-1-2-w-x-12-5-8-d-x-71-h-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:06:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/led_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:06:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-tecgen-flame-resistant-work-shirt-s-short-light-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:06:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/size-b-3-inch-x-3-inch-stainless-steel-slotted-shim-80-piece-shop-kit>: HTTP status code is not handled or not allowed 2026-01-24 20:06:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s9307-13w-ledcfl-replacement-lamp-4-pin-g24q-base-5000k-120-277v-long>: HTTP status code is not handled or not allowed 2026-01-24 20:06:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/woodboring-drill-bits-irwin-speedbor-3046013>: HTTP status code is not handled or not allowed 2026-01-24 20:06:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/threaded-insert-assortment-kit-for-soft-wood-flush-8-32-to-5-16-18-ez-a800>: HTTP status code is not handled or not allowed 2026-01-24 20:06:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009687-screw-on-gasket-185x-17-1-4>: HTTP status code is not handled or not allowed 2026-01-24 20:06:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s9281-3w-led-mr11-gu4-base-3000k-12v>: HTTP status code is not handled or not allowed 2026-01-24 20:06:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s9578-45w-st19-filament-led-medium-base-transparent-amber-2300k>: HTTP status code is not handled or not allowed 2026-01-24 20:06:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-flame-resistant-zip-front-hoodie-xl-tall-navy>: HTTP status code is not handled or not allowed 2026-01-24 20:06:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-auxiliary-air-tank-horizontal-2-gallon-capacity-125-psi>: HTTP status code is not handled or not allowed 2026-01-24 20:06:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-4-gallon-utility-bin>: HTTP status code is not handled or not allowed 2026-01-24 20:06:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/led_bulbs>: HTTP status code is not handled or not allowed 2026-01-24 20:06:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vci-flat-bags-15-x-22-3-mil-blue-500-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:06:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forced-air-wall-heater-efw1230-mw-w-white-120v-3000w>: HTTP status code is not handled or not allowed 2026-01-24 20:06:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/type-321-stainless-steel-tool-wrap-sheet-12-inch-x-24-inch-x-0002-inch-pack-of-20>: HTTP status code is not handled or not allowed 2026-01-24 20:06:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/800va-omni-vs-tower-7-outlet>: HTTP status code is not handled or not allowed 2026-01-24 20:06:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-function-5-1-convertible-hand-truck>: HTTP status code is not handled or not allowed 2026-01-24 20:06:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-tagging-gun>: HTTP status code is not handled or not allowed 2026-01-24 20:06:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-hand-lever-actuated-valve-spring-centered-open-center-g1-8-ports>: HTTP status code is not handled or not allowed 2026-01-24 20:06:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009555-door-gasket-67-7-16-inch-x-29-1-8-for-true-manufacturing>: HTTP status code is not handled or not allowed 2026-01-24 20:06:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/Luxor-Mobile-Bin-Cart-with-Eight-3H-Totes-MBS-SR-8S-Gray-Black-19-1-2L-x-17W-x-37-1-4H>: HTTP status code is not handled or not allowed 2026-01-24 20:06:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carriage-bolt-12-13-5-12-round-head-steel-zinc-cr3-ft-grade-5-unc-pkg-of-25-492100>: HTTP status code is not handled or not allowed 2026-01-24 20:06:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/battery-powered-high-precision-torque-wrench-kit-1-drive-240-1000-ftlb-10-38l>: HTTP status code is not handled or not allowed 2026-01-24 20:06:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carriage-bolt-12-13-1-round-head-steel-zinc-cr3-ft-grade-5-unc-pkg-of-50-492049>: HTTP status code is not handled or not allowed 2026-01-24 20:07:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/decoil-stretch-lanyard-6l-snaphooks>: HTTP status code is not handled or not allowed 2026-01-24 20:07:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1800w-isobar-6-outlet-120v>: HTTP status code is not handled or not allowed 2026-01-24 20:07:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-panel-sheet-mover-truck-2000-lb-capacity-48x24>: HTTP status code is not handled or not allowed 2026-01-24 20:07:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slide-plug-polarized-18-2-spt-2-brown>: HTTP status code is not handled or not allowed 2026-01-24 20:07:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pv-150w-12v-portable-inverter-auto-power-outlet>: HTTP status code is not handled or not allowed 2026-01-24 20:07:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mega-swivel-concrete-anchor-3-4-drill-hole>: HTTP status code is not handled or not allowed 2026-01-24 20:07:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011244-hinge-assembly-for-victory>: HTTP status code is not handled or not allowed 2026-01-24 20:07:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/save-a-stud-rethread-die-14mm-15>: HTTP status code is not handled or not allowed 2026-01-24 20:07:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-2036-sandpaper-36-grit-fits-22-44-model-sanders>: HTTP status code is not handled or not allowed 2026-01-24 20:07:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 17096 pages (at 96 pages/min), scraped 11556 items (at 21 items/min) 2026-01-24 20:07:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evhc12i-06l-led-high-lumen-emergency-unit-white-546-lumens-high-mounts-self-diagnostics>: HTTP status code is not handled or not allowed 2026-01-24 20:07:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-powdered-painted-metal-panel-lock-for-5-panel>: HTTP status code is not handled or not allowed 2026-01-24 20:07:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-square-plastic-waste-receptacle-w-dome-lid-42-gallon-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:07:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ez-cooler-bubbler-cap>: HTTP status code is not handled or not allowed 2026-01-24 20:07:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-inch-x-10-inch-plexiglass-window>: HTTP status code is not handled or not allowed 2026-01-24 20:07:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evcurwdi-compact-led-combo-unit-w-remote-capacity-white-w-red-letters-self-diagnostics>: HTTP status code is not handled or not allowed 2026-01-24 20:07:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/room_dividers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:07:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lab-pack-open-head-poly-drum-metal-lever-lock-30-gal-cap-2-x-2-bung-holes-black>: HTTP status code is not handled or not allowed 2026-01-24 20:07:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1021083-drain-grate-floor-8-inch>: HTTP status code is not handled or not allowed 2026-01-24 20:07:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tote-basket-vented-wall-blue-1>: HTTP status code is not handled or not allowed 2026-01-24 20:07:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/op-920-portable-livestock-scale-2000-lb-x-0-5-lb>: HTTP status code is not handled or not allowed 2026-01-24 20:07:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/06460-1-2-x-2-x-2-plastic-masonry-shim-2212-250pc>: HTTP status code is not handled or not allowed 2026-01-24 20:07:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lexan-174-white-cylinder-3-11-64-screw-fitter6-1-4-in-height>: HTTP status code is not handled or not allowed 2026-01-24 20:07:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/264086-kason-11095000013-door-closer-for-kason>: HTTP status code is not handled or not allowed 2026-01-24 20:07:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traex-squeeze-dispensers-2812-02-single-tip-12-oz>: HTTP status code is not handled or not allowed 2026-01-24 20:07:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stacking-chairs-plastic-midnight-blue-set-of-two-steel-black-frame>: HTTP status code is not handled or not allowed 2026-01-24 20:07:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t8-t12-high-output-fluorescent-socket-plunger>: HTTP status code is not handled or not allowed 2026-01-24 20:07:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/304-stainless-steel-clamp-with-wing-nut-for-quick-clamp-fittings-for-12-and-34-tube>: HTTP status code is not handled or not allowed 2026-01-24 20:07:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bonded-leather-u-shaped-seat-guest-chair-black>: HTTP status code is not handled or not allowed 2026-01-24 20:07:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-panel-display-tower-5-ft-9-inch-high-fabric-cranberry>: HTTP status code is not handled or not allowed 2026-01-24 20:07:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/partitions_dividers/room_dividers landed on page that is not a product page. 2026-01-24 20:07:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11-step-24-steel-safety-angle-orange-rolling-ladder-serrated-sw1132g-o>: HTTP status code is not handled or not allowed 2026-01-24 20:07:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/push-to-connect-tube-fitting-nylon-plastic-straight-adapter-12-tube-od-x-12-npt-female>: HTTP status code is not handled or not allowed 2026-01-24 20:07:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-halogen-socket-gu10-w-mounting-holes>: HTTP status code is not handled or not allowed 2026-01-24 20:07:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012556-spark-ignition-module-for-american-range>: HTTP status code is not handled or not allowed 2026-01-24 20:07:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pasadena-2-outdoor-straight-bench-without-back-surface-mount-green>: HTTP status code is not handled or not allowed 2026-01-24 20:07:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-1-4-in-fitterbrass-finish>: HTTP status code is not handled or not allowed 2026-01-24 20:07:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/push-to-connect-tube-fitting-nylon-90-degree-elbow-adapter-12-tube-od-x-12-npt-female>: HTTP status code is not handled or not allowed 2026-01-24 20:07:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bayonet-base-double-contact-socket-36-in-leads>: HTTP status code is not handled or not allowed 2026-01-24 20:07:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bayonet-base-double-contact-socket-8-in-leads>: HTTP status code is not handled or not allowed 2026-01-24 20:07:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-x-30-super-strong-mobile-workbench>: HTTP status code is not handled or not allowed 2026-01-24 20:07:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washdown-motor-180-tefc-3ph-3505rpm-c184t34wb4aa>: HTTP status code is not handled or not allowed 2026-01-24 20:07:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/phenolic-threaded-candelabra-socket-1-1-4-in-w-shoulder-and-phenolic-ring-8-in-leads>: HTTP status code is not handled or not allowed 2026-01-24 20:07:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washdown-motor-56-tefc-3ph-1725rpm-c6t17wk3jj>: HTTP status code is not handled or not allowed 2026-01-24 20:07:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-two-piece-ladle-46900-1-2-oz-capacity-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 20:07:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/professional-transfer-ladles-and-dippers-4752-64-oz-12-3-4-long>: HTTP status code is not handled or not allowed 2026-01-24 20:07:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bend-glass-14-in-square-white>: HTTP status code is not handled or not allowed 2026-01-24 20:07:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1000ul-extended-length-low-retention-pipette-tips-racked-sterile-960-case>: HTTP status code is not handled or not allowed 2026-01-24 20:07:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/handy-pac-strapping-kit-light-duty>: HTTP status code is not handled or not allowed 2026-01-24 20:07:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1000ml-filter-system-0-22um-pes-filter-90mm-diameter-sterile-12-case>: HTTP status code is not handled or not allowed 2026-01-24 20:07:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-ft-heavy-duty-cord-set-18-3-svt-105-176-c-black>: HTTP status code is not handled or not allowed 2026-01-24 20:07:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-3-ga-sjtw-3-orange-outdoor-extension-cords-100-ft>: HTTP status code is not handled or not allowed 2026-01-24 20:07:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washdown-motor-56h-tenv-1ph-1725rpm-nz6k17vk1bb>: HTTP status code is not handled or not allowed 2026-01-24 20:07:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durham-6-recycled-bench-with-back-portablesurface-mount-brown>: HTTP status code is not handled or not allowed 2026-01-24 20:07:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-flat-washer-sae-1-1-16-i-d-steel-zinc-grade-2-pkg-of-25-cct16>: HTTP status code is not handled or not allowed 2026-01-24 20:07:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ptfe-syringe-filter-0-45um-13mm-sterile-75-case>: HTTP status code is not handled or not allowed 2026-01-24 20:07:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silicone-rubber-push-in-grommet-for-1-34-hole-id-and-116-edge-thickness-1-12-id-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-24 20:07:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-combination-internal-external-tooth-lock-washer-steel-zinc-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 20:07:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylon-syringe-filter-0-22um-30mm-sterile-30-case>: HTTP status code is not handled or not allowed 2026-01-24 20:07:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/infinity-locker-double-tier-12x18x36-6-door-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:07:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cayenne-heavy-duty-induction-hot-plate-912hidc-dual-hob-digital-12-x-13-5-8-x-30>: HTTP status code is not handled or not allowed 2026-01-24 20:07:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50ml-centrifuge-tube-self-standing-resealable-bag-sterile-500-case>: HTTP status code is not handled or not allowed 2026-01-24 20:07:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traex-kondi-keeper-insert-cover-4721-1-quart>: HTTP status code is not handled or not allowed 2026-01-24 20:07:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/adjustable_height_desks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:07:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-square-table-with-2-imme-armless-barstools-coral-seatespresso-top>: HTTP status code is not handled or not allowed 2026-01-24 20:07:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-round-table-with-4-imme-armless-caster-chairs-sky-blue-seatstudio-teak-top>: HTTP status code is not handled or not allowed 2026-01-24 20:07:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iec-metric-motor-132m-ip55-3ph-c132t11fz2cc>: HTTP status code is not handled or not allowed 2026-01-24 20:07:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/261776-baffle-filter-20-x-25-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 20:07:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pla-03274-128oz-3840ml-hdpe-wide-mouth-handled-round-jug-89-400-neck-finish-case-of-60>: HTTP status code is not handled or not allowed 2026-01-24 20:07:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/work-benches/bench_tops_accessories/workbench_risers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:07:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-7-drawer-40-w-shallow-depth-mobile-cabinet-w-butcher-top-gray-master-keyed>: HTTP status code is not handled or not allowed 2026-01-24 20:07:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-5-drawer-standard-width-cabinet-blue-keyed-alike>: HTTP status code is not handled or not allowed 2026-01-24 20:07:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2261143-wendy-s-9-slot-open-fryer-basket>: HTTP status code is not handled or not allowed 2026-01-24 20:07:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blue-please-wait-here-floor-sign>: HTTP status code is not handled or not allowed 2026-01-24 20:07:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/adjustable_height_desks landed on page that is not a product page. 2026-01-24 20:08:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cabinet-shop-desk-with-cabinet-riser-gy>: HTTP status code is not handled or not allowed 2026-01-24 20:08:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-round-table-with-2-imme-armless-barstools-light-gray-seatespresso-top>: HTTP status code is not handled or not allowed 2026-01-24 20:08:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010659-cable-kit-for-alto-shaam>: HTTP status code is not handled or not allowed 2026-01-24 20:08:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mixing-bowl-40769-30-quart-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:08:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-round-table-with-4-imme-armless-chairs-navy-seatmaple-top>: HTTP status code is not handled or not allowed 2026-01-24 20:08:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-round-table-with-4-imme-armless-caster-chairs-black-seatriver-cherry-top>: HTTP status code is not handled or not allowed 2026-01-24 20:08:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-starter-rack-48x48x84-with-3-levels-wood-deck-1500lb-cap-per-shelf-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:08:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/work-benches/bench_tops_accessories/workbench_risers>: HTTP status code is not handled or not allowed 2026-01-24 20:08:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/j-series-pallet-jack-truck-48l-x-20-12w-5500-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:08:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-starter-rack-48x36x84-with-3-level-steel-deck-1500lb-cap-per-shelf-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:08:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-round-table-with-4-imme-armless-caster-chairs-moonbeam-seatriver-cherry-top>: HTTP status code is not handled or not allowed 2026-01-24 20:08:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wear-ever-fry-pan-with-powercoat-2-interior-s4012-8-gauge-9-3-4-bottom-diameter>: HTTP status code is not handled or not allowed 2026-01-24 20:08:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 17193 pages (at 97 pages/min), scraped 11579 items (at 23 items/min) 2026-01-24 20:08:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-round-table-with-4-imme-armless-caster-chairs-black-seatdesigner-white-top>: HTTP status code is not handled or not allowed 2026-01-24 20:08:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1961081-shield-splash-for-frymaster>: HTTP status code is not handled or not allowed 2026-01-24 20:08:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-starter-rack-96x48x84-with-3-levels-wire-deck-800lb-cap-per-shelf-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:08:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-container-without-lid-23-1-2x19-1-2x13>: HTTP status code is not handled or not allowed 2026-01-24 20:08:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8n12a02k-zinc-die-cast-adjustable-lever-w-steel-components-m8-x-12mm-stud>: HTTP status code is not handled or not allowed 2026-01-24 20:08:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/321613-gh-2280zx-large-spindle-bore-lathe-w-acu-rite-300s-dro-collet-closer-10-hp>: HTTP status code is not handled or not allowed 2026-01-24 20:08:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/equipto-vg-bulk-rack-96-w-x-30-d-x-84-h-starter-with-no-decking-textured-regal-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:08:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/equipto-vg-bulk-rack-60-w-x-30-d-x-72-h-starter-with-corrugated-steel-decking-textured-regal-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:08:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1291137-kason-7003-2020-filter-hood-for-kason>: HTTP status code is not handled or not allowed 2026-01-24 20:08:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1291096-filter-baffle-12x20-hd-type-i>: HTTP status code is not handled or not allowed 2026-01-24 20:08:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-round-table-with-4-imme-armless-caster-chairs-sky-blue-seatespresso-top>: HTTP status code is not handled or not allowed 2026-01-24 20:08:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-thick-conductive-anti-static-mat-smooth-surface-3-w-x-custom-cut-length>: HTTP status code is not handled or not allowed 2026-01-24 20:08:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mtm-hydro-5000-psi-14-brass-quick-coupler-plug-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:08:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/lathes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:08:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-add-on-rack-96x24x84-with-3-levels-wood-deck-800-lb-cap-per-level-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:08:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/painting_supplies/paint_brushes_rollers_tools/paint_sprayers_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:08:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/frame-extension-magliner-hand-truck>: HTTP status code is not handled or not allowed 2026-01-24 20:08:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/85-1051-red-wing-type-wire-connectors-100-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:08:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-glass-hard-surface-cleaner-20-pods-jar>: HTTP status code is not handled or not allowed 2026-01-24 20:08:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-round-table-with-4-imme-armless-barstools-navy-seatdesigner-white-top>: HTTP status code is not handled or not allowed 2026-01-24 20:08:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-round-table-with-4-imme-armless-chairs-navy-seatriver-cherry-top>: HTTP status code is not handled or not allowed 2026-01-24 20:08:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/supreme-anti-fatigue-mat-3-4-thick-3x-2-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:08:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hessaire-portable-evaporative-cooler-mc61a-115v-5300-cfm>: HTTP status code is not handled or not allowed 2026-01-24 20:08:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-workmate-stand-ahwl1948>: HTTP status code is not handled or not allowed 2026-01-24 20:08:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/690089-jtm-2-vertical-milling-machine-2-hp-115-230v-1-phase>: HTTP status code is not handled or not allowed 2026-01-24 20:08:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/easy-adjust-boltless-shelf-truck-with-laminate-shelves-9>: HTTP status code is not handled or not allowed 2026-01-24 20:08:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/machining/machinery/lathes>: HTTP status code is not handled or not allowed 2026-01-24 20:08:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/norton-66623325028-bear-tex-non-woven-quick-change-disc-1-inch-dia-fin-grit-aluminum-oxide-type-ii>: HTTP status code is not handled or not allowed 2026-01-24 20:08:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/painting_supplies/paint_brushes_rollers_tools/paint_sprayers_accessories>: HTTP status code is not handled or not allowed 2026-01-24 20:08:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/690120-jtm-1050-variable-speed-milling-machine-w-x-axis-powerfeed-3-hp>: HTTP status code is not handled or not allowed 2026-01-24 20:08:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cable-tie-double-lock-6-30-lb-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 20:08:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-tape-reflective-striped-yellow-black-2>: HTTP status code is not handled or not allowed 2026-01-24 20:08:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6121232-screw-sm-th-ss-100>: HTTP status code is not handled or not allowed 2026-01-24 20:08:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1271109-gasket-ref-36-3-4x78-1-4-3-sd>: HTTP status code is not handled or not allowed 2026-01-24 20:08:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/norton-66243510627-gemini-small-diameter-cut-off-wheel-3-inch-x-035-inch-x-1-4-inch-60-grit-alum-oxide-type-1>: HTTP status code is not handled or not allowed 2026-01-24 20:08:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kyorene-pro-cut-resistant-gloves-polyurethane-coated-ansi-a6-xs-gray-12-pair>: HTTP status code is not handled or not allowed 2026-01-24 20:08:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slat-wall-accessory-8-l-peg-hook>: HTTP status code is not handled or not allowed 2026-01-24 20:08:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-compostable-eco-friendly-round-plates-500-pcs>: HTTP status code is not handled or not allowed 2026-01-24 20:08:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditions-7-piece-patio-dining-set-w-cast-top-dining-table-6-cushioned-swivel-rockers>: HTTP status code is not handled or not allowed 2026-01-24 20:08:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cut-resistant-gloves-hct-nano-foam-nitrile-coated-ansi-a2-s-green-12-pair>: HTTP status code is not handled or not allowed 2026-01-24 20:08:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/appliances/cooking_equipment/griddles already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:08:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manor-7-piece-patio-dining-set-w-6-sling-swivel-rockers-a-cast-top-dining-table>: HTTP status code is not handled or not allowed 2026-01-24 20:08:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cayenne-cast-iron-panini-style-plate-sandwich-press-40795-c-us-canada-2700-3600-watts>: HTTP status code is not handled or not allowed 2026-01-24 20:08:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48w-x-30d-mobile-workbench-shop-top-square-edge-tan>: HTTP status code is not handled or not allowed 2026-01-24 20:08:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-pallet-rack-upright-frame-24d-x-96h>: HTTP status code is not handled or not allowed 2026-01-24 20:08:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5011015-latch-paddle-1-2-inch-studs-for-food-warming-equipment-co-inc>: HTTP status code is not handled or not allowed 2026-01-24 20:08:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cable-tie-4-18lb-1-000-pk>: HTTP status code is not handled or not allowed 2026-01-24 20:08:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/norton-66253410198-gemini-chop-saw-cut-off-wheel-16-inch-x-3-32-inch-x-1-inch-36-grit-aluminum-oxide-type-1>: HTTP status code is not handled or not allowed 2026-01-24 20:08:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esd-tractor-sit-stand-stool-polyurethane-black>: HTTP status code is not handled or not allowed 2026-01-24 20:08:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/appliances/cooking_equipment/griddles>: HTTP status code is not handled or not allowed 2026-01-24 20:08:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gearmotor-ac-parallel-15rpm-tenv>: HTTP status code is not handled or not allowed 2026-01-24 20:08:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-square-table-with-4-imme-armless-barstools-coral-seatdesigner-white-top>: HTTP status code is not handled or not allowed 2026-01-24 20:08:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-1899-10-inch-x-5-1-2-inch-reversible-cam-lift-door-hinge-with-flush-offset>: HTTP status code is not handled or not allowed 2026-01-24 20:08:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plus-cut-resistant-gloves-nano-foam-nitrile-coated-ansi-a5-2xl-yellow-12-pair>: HTTP status code is not handled or not allowed 2026-01-24 20:08:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beige-painted-runway-rail-12ft-for-hubbell-50-lb-overhead-tool-cranes>: HTTP status code is not handled or not allowed 2026-01-24 20:08:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/item-ml1-040-ab3e-enclosed-disconnect-switches>: HTTP status code is not handled or not allowed 2026-01-24 20:08:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traex-safety-mate-insta-chill-food-cooler-7024-2-1-quart-capacity-polycarbonate>: HTTP status code is not handled or not allowed 2026-01-24 20:08:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-dock-board-with-steel-curbs-10>: HTTP status code is not handled or not allowed 2026-01-24 20:08:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-7401-coated-waterproof-winter-work-gloves-large-orange>: HTTP status code is not handled or not allowed 2026-01-24 20:08:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2255002-basket-fry-portion-control-for-prince-castle>: HTTP status code is not handled or not allowed 2026-01-24 20:08:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/generic-white-wall-case-w-label-kit-1-shelf-medium>: HTTP status code is not handled or not allowed 2026-01-24 20:08:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-full-mask-storage-bag>: HTTP status code is not handled or not allowed 2026-01-24 20:08:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-big-ones-8-x-11-50-box>: HTTP status code is not handled or not allowed 2026-01-24 20:08:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-chisel-scaler>: HTTP status code is not handled or not allowed 2026-01-24 20:09:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adesso-720p-hd-usb-webcam-with-built-in-microphone>: HTTP status code is not handled or not allowed 2026-01-24 20:09:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/task_desk_chairs/task_office_chairs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:09:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-sided-napkin-dispenser-6515-06-tabletop-black>: HTTP status code is not handled or not allowed 2026-01-24 20:09:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10x20-canopy-enclosure-kit-wht-w-windows-for-1-3-8-inch-frame>: HTTP status code is not handled or not allowed 2026-01-24 20:09:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/supplied-air-shield-black-welding-helmet-deluxe-2-workers-50-hose-10-lens>: HTTP status code is not handled or not allowed 2026-01-24 20:09:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s-183-turbo-blow-gun-nozzle-kit-3-piece-3-8-npt>: HTTP status code is not handled or not allowed 2026-01-24 20:09:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-use-indoor-outdoor-interior-sensor-bracket-1-red-light-15-cord>: HTTP status code is not handled or not allowed 2026-01-24 20:09:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-x-72-table-with-6-imme-armless-caster-chairs-navy-seatstudio-teak-top>: HTTP status code is not handled or not allowed 2026-01-24 20:09:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flexion153-task-chair-mid-back-14-1316-19-1116h-seat-24h-back-black>: HTTP status code is not handled or not allowed 2026-01-24 20:09:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 17287 pages (at 94 pages/min), scraped 11604 items (at 25 items/min) 2026-01-24 20:09:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1901351-dispenser-hot-water-120v-5gal-for-bunn-o-matic>: HTTP status code is not handled or not allowed 2026-01-24 20:09:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-hp-vac-n-blo-commercial-vacuum-cleaner>: HTTP status code is not handled or not allowed 2026-01-24 20:09:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/task_desk_chairs/task_office_chairs landed on page that is not a product page. 2026-01-24 20:09:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ptp184-floor-stripe>: HTTP status code is not handled or not allowed 2026-01-24 20:09:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6231432-screw-mach-fh-ss-100>: HTTP status code is not handled or not allowed 2026-01-24 20:09:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pivotal-therapy-system-soft-tissue-system-occipivot>: HTTP status code is not handled or not allowed 2026-01-24 20:09:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ignition-20-4-way-stretch-task-chair-blue-adj-lumbar-support-17-21h-seat-basalt>: HTTP status code is not handled or not allowed 2026-01-24 20:09:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tote-box-heat-resistant-40-to-300-degrees-41>: HTTP status code is not handled or not allowed 2026-01-24 20:09:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/framing-nails-fr131-3grs-3-x-131-plastic-collated-galvanized-ring-shank-2000-bx>: HTTP status code is not handled or not allowed 2026-01-24 20:09:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/revel-fidget-stool-with-adjustable-height-backless-plastic-seat-black>: HTTP status code is not handled or not allowed 2026-01-24 20:09:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plus-electrodes-2-x-3-5-rectangle-40-case>: HTTP status code is not handled or not allowed 2026-01-24 20:09:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pivotal-therapy-system-soft-tissue-system-standard-lumbar-pivot>: HTTP status code is not handled or not allowed 2026-01-24 20:09:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flammable-liquid-cabinet-manual-single-door-vertical-storage-2>: HTTP status code is not handled or not allowed 2026-01-24 20:09:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ptp208bk-tough-mark-strip>: HTTP status code is not handled or not allowed 2026-01-24 20:09:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009305-rear-burner-assembly-for-american-range>: HTTP status code is not handled or not allowed 2026-01-24 20:09:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/presentation-gravity-flow-rack-28121-add-on-48w-x-84d-x-72h>: HTTP status code is not handled or not allowed 2026-01-24 20:09:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stackrack-for-12-w-x-20-1-2-d-x-9-1-2-h-bins-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:09:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-x-72-table-with-6-imme-armless-chairs-white-seatriver-cherry-top>: HTTP status code is not handled or not allowed 2026-01-24 20:09:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10x6x20-1-mil-gusseted-poly-bags>: HTTP status code is not handled or not allowed 2026-01-24 20:09:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/task_desk_chairs/task_office_chairs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:09:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-x-72-table-with-6-imme-armless-caster-chairs-coral-seatdesigner-white-top>: HTTP status code is not handled or not allowed 2026-01-24 20:09:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-w-x-30-d-x-12-h-stackbox-black>: HTTP status code is not handled or not allowed 2026-01-24 20:09:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dolly-for-1100-chair-26-chairs-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:09:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8014127-hood-filter-type6-galv-16x20>: HTTP status code is not handled or not allowed 2026-01-24 20:09:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/surface-mount-water-bottle-filling-station-filtered>: HTTP status code is not handled or not allowed 2026-01-24 20:09:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-refrigerated-sushi-display-case>: HTTP status code is not handled or not allowed 2026-01-24 20:09:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10197199-harness-back-hip-d-rings-qwik-fit-leg-straps-extra-small>: HTTP status code is not handled or not allowed 2026-01-24 20:09:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40-watt-hi-pro-spiral-compact-fluorescent-light-bulb-medium-base-B1229109>: HTTP status code is not handled or not allowed 2026-01-24 20:09:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x4x4-corrugated-mailers>: HTTP status code is not handled or not allowed 2026-01-24 20:09:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sliding-glass-door-merchandiser-78-long-3-door>: HTTP status code is not handled or not allowed 2026-01-24 20:09:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/task_desk_chairs/task_office_chairs>: HTTP status code is not handled or not allowed 2026-01-24 20:09:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10197365-construction-harness-back-hip-d-ring-tongue-buckle-leg-straps-xl>: HTTP status code is not handled or not allowed 2026-01-24 20:09:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coast-20865-model-px1-aluminum-tactical-3aaa-315-lumen-flashlight>: HTTP status code is not handled or not allowed 2026-01-24 20:09:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blickman-howard-instrument-shelf-table-48l-x-24w-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 20:09:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12x2x2-corrugated-mailers>: HTTP status code is not handled or not allowed 2026-01-24 20:09:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cantilever-rack-single-sided-upright-8>: HTTP status code is not handled or not allowed 2026-01-24 20:09:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-mat-ii-reversible-drainage-mat-3-ft-x-2-ft-red>: HTTP status code is not handled or not allowed 2026-01-24 20:09:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/professional-dishwashing-liquid-original-scent-5-gallon-pail>: HTTP status code is not handled or not allowed 2026-01-24 20:09:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electrical-tape-1-2-x-20-assorted-colors>: HTTP status code is not handled or not allowed 2026-01-24 20:09:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-liquid-electrical-tape>: HTTP status code is not handled or not allowed 2026-01-24 20:09:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enclosure-kit-slide-door-24-x-24-x-18-light-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:09:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enclosure-kit-slide-door-24-x-24-x-18-grey>: HTTP status code is not handled or not allowed 2026-01-24 20:09:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bristol-ridge-scraper-carpet-mat-4-ft-x-6-ft-midnight>: HTTP status code is not handled or not allowed 2026-01-24 20:09:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/armless-mesh-dra-ing-stool-vinyl-black>: HTTP status code is not handled or not allowed 2026-01-24 20:09:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cantilever-rack-horizontal-brace-set-5>: HTTP status code is not handled or not allowed 2026-01-24 20:09:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tubular-light-mounting-bracket-long-l-qty-2-required-when-mounting-tube-lights>: HTTP status code is not handled or not allowed 2026-01-24 20:09:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/28-divider-rotating-parts-center>: HTTP status code is not handled or not allowed 2026-01-24 20:09:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ratchet-lever-hoist-6-ton-capacity-20-lift>: HTTP status code is not handled or not allowed 2026-01-24 20:09:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-wide-boltless-shelving-starter-48in-w-x-48in-d-x-60in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:09:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010373-pressure-switch-for-groen>: HTTP status code is not handled or not allowed 2026-01-24 20:09:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/packard-contactor-2-pole-150125100-208240>: HTTP status code is not handled or not allowed 2026-01-24 20:09:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-21-x-36-light-grey-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:09:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galvanized-shelf-truck-36x24x80-1200-pound-capacity-with-brakes>: HTTP status code is not handled or not allowed 2026-01-24 20:09:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-24-x-24-light-blue-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:09:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tutco-crankcase-70-120-37-38>: HTTP status code is not handled or not allowed 2026-01-24 20:09:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24x17x3-flat-corrugated-boxes>: HTTP status code is not handled or not allowed 2026-01-24 20:09:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tutco-crankcase-70-240-38>: HTTP status code is not handled or not allowed 2026-01-24 20:09:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-wide-boltless-shelving-starter-60in-w-x-36in-d-x-72in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:09:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dynamic-2-1-2-fluid-glycerine-filled-pressure-gage-center-back-15-psi>: HTTP status code is not handled or not allowed 2026-01-24 20:09:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tutco-crankcase-75-240-48-34>: HTTP status code is not handled or not allowed 2026-01-24 20:10:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/wiring_devices/relays_sequencers/relays already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:10:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-add-on-96in-w-x-24in-d-x-60in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:10:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flipak-attached-lid-container-13>: HTTP status code is not handled or not allowed 2026-01-24 20:10:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bravo-3-gal-carpet-spotter-55-psi-pump-106-inch-waterlift-heat-ready-300mh>: HTTP status code is not handled or not allowed 2026-01-24 20:10:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-18-x-60-black-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:10:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ap-mechanical-dynamometer-with-shackles-5-dial-15-000-lb-x-100-lb>: HTTP status code is not handled or not allowed 2026-01-24 20:10:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-heavy-duty-boltless-shelving-add-on-42in-w-x-30in-d-x-60in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:10:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-1-4-circular-saw-15a-4300-rpm-5-8-arbor-bare-tool>: HTTP status code is not handled or not allowed 2026-01-24 20:10:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/security-work-center-storage-cabinet-with-bins-14>: HTTP status code is not handled or not allowed 2026-01-24 20:10:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 17381 pages (at 94 pages/min), scraped 11631 items (at 27 items/min) 2026-01-24 20:10:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-x1450-3-mil-poly-tubing>: HTTP status code is not handled or not allowed 2026-01-24 20:10:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/wiring_devices/relays_sequencers/relays>: HTTP status code is not handled or not allowed 2026-01-24 20:10:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-x1450-3-mil-poly-tubing>: HTTP status code is not handled or not allowed 2026-01-24 20:10:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/outdoor-enclosed-bulletin-board-cabinet-3-door-4-feet-h-x-8-feet-w-silver-trim-natural-cork>: HTTP status code is not handled or not allowed 2026-01-24 20:10:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-30-x-60-dark-yellow-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:10:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gks-jack-roll-block-12l-x-4w-x-4h-40000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:10:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012309-toggle-switch-for-carter-hoffmann>: HTTP status code is not handled or not allowed 2026-01-24 20:10:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/special_purpose_carts_trucks/media_carts/audio_visual_cabinets_carts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:10:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/461008-thermostat-w-diald1-d18-3-8-x-4-1-2-60>: HTTP status code is not handled or not allowed 2026-01-24 20:10:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-2-rubber-voltage-gloves-red-size-9-dwh1429r>: HTTP status code is not handled or not allowed 2026-01-24 20:10:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011656-low-pressure-control>: HTTP status code is not handled or not allowed 2026-01-24 20:10:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xtreme-48-oz-blender-electronic-keypad-bpa-free-copolyester-jar-2-speeds>: HTTP status code is not handled or not allowed 2026-01-24 20:10:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-7in-regular-cut>: HTTP status code is not handled or not allowed 2026-01-24 20:10:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-cal-flame-resistant-coveralls-m-gray-tcg02150861>: HTTP status code is not handled or not allowed 2026-01-24 20:10:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enclosure-kit-swing-door-18-x-18-x-18-grey>: HTTP status code is not handled or not allowed 2026-01-24 20:10:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wide-span-rack-60x48x96-with-3-shelves-wire-deck-1200-lb-capacity-per-level>: HTTP status code is not handled or not allowed 2026-01-24 20:10:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durastripe-12x9-rectangle-emergency-stop>: HTTP status code is not handled or not allowed 2026-01-24 20:10:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-30-x-36-white-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:10:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tan-wall-bracket>: HTTP status code is not handled or not allowed 2026-01-24 20:10:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-starter-96inw-x-36ind-x-72inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:10:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/special_purpose_carts_trucks/media_carts/audio_visual_cabinets_carts landed on page that is not a product page. 2026-01-24 20:10:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compact-battery-operated-work-positioner-5>: HTTP status code is not handled or not allowed 2026-01-24 20:10:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/265645-output-small-gear-for-dynamic-international-hand-mixers>: HTTP status code is not handled or not allowed 2026-01-24 20:10:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-hi-vis-leg-gaiters-class-e-osfm-fluorescent-yellow-l05tvce>: HTTP status code is not handled or not allowed 2026-01-24 20:10:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/snaplocs-with-fasteners-e-track-single-strap-anchors-slcsz4fa-zinc-4-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:10:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-dbl-solenoid-vlv-ext-pilot-pres-cntr-spr-cntred-g1-4-24v-ac-5va-coil-black>: HTTP status code is not handled or not allowed 2026-01-24 20:10:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enclosure-kit-slide-door-18-x-36-x-18-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:10:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-open-cntr-dbl-solenoid-valve-pilot-spr-cntred-1-8-nptf-12v-dc-3w-coil-black-conn>: HTTP status code is not handled or not allowed 2026-01-24 20:10:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scott-embossed-premium-bathroom-tissue-605-sheets-roll>: HTTP status code is not handled or not allowed 2026-01-24 20:10:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enclosure-kit-swing-door-18-x-18-x-18-dark-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:10:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-high-capacity-bulk-rack-with-wood-decking-96x48x120-starter>: HTTP status code is not handled or not allowed 2026-01-24 20:10:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2000-respirator-storage-bag-with-zipper-14-x-16>: HTTP status code is not handled or not allowed 2026-01-24 20:10:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/421136-switch>: HTTP status code is not handled or not allowed 2026-01-24 20:10:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/facial-tissues-in-pop-up-dispenser-box-100-sheets-box-36-boxes-ctn>: HTTP status code is not handled or not allowed 2026-01-24 20:10:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-dbl-solenoid-valve-pilot-spring-closed-cntr-1-2-nptf-24v-dc-3w-coil-led-conn>: HTTP status code is not handled or not allowed 2026-01-24 20:10:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-dbl-solenoid-valve-pilot-spr-cntred-closed-cntr-g1-8-24v-ac-5va-coil-black-conn>: HTTP status code is not handled or not allowed 2026-01-24 20:10:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4400-da-defibrillator-wall-case-with-alarm-plastic>: HTTP status code is not handled or not allowed 2026-01-24 20:10:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-visibility-safety-vise-5-inch-jaw-width-5-1-4-inch-jaw-opening>: HTTP status code is not handled or not allowed 2026-01-24 20:10:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/turbo-air-tbb-4sg-back-bar-cooler-90-3-8-w-x-27-1-4-d-x-37-1-8-h>: HTTP status code is not handled or not allowed 2026-01-24 20:10:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/st-drawer-layout-8-compartments-5-h>: HTTP status code is not handled or not allowed 2026-01-24 20:10:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72007-titan-18-hvls-fan-18-ft-dia-3-ph-200-240v>: HTTP status code is not handled or not allowed 2026-01-24 20:10:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-2-single-solenoid-valve-pilot-spr-return-1-2-nptf-110v-ac-5va-coil-led-connection>: HTTP status code is not handled or not allowed 2026-01-24 20:10:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-high-capacity-bulk-rack-without-decking-72x36x96-add-on>: HTTP status code is not handled or not allowed 2026-01-24 20:10:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-high-capacity-bulk-rack-without-decking-60x48x120-add-on>: HTTP status code is not handled or not allowed 2026-01-24 20:10:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enclosure-kit-slide-door-14-x-48-x-18-grey>: HTTP status code is not handled or not allowed 2026-01-24 20:10:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-16-x-2-hex-head-cap-screw-carbon-steel-zinc-unc-grade-5-usa-100-pack-457146>: HTTP status code is not handled or not allowed 2026-01-24 20:10:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-inch-round-10-led-clear-backup-light>: HTTP status code is not handled or not allowed 2026-01-24 20:10:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tffn-16-gauge-building-wire-stranded-type-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:10:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2551011-blade-4-8-section-2-for-shaver-specialty-king-cutter-vegetable-potato-cutter>: HTTP status code is not handled or not allowed 2026-01-24 20:10:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-high-capacity-bulk-rack-with-wire-decking-96x48x120-starter>: HTTP status code is not handled or not allowed 2026-01-24 20:10:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/on-off-switch-panel-w-flash-pattern-control>: HTTP status code is not handled or not allowed 2026-01-24 20:10:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-high-capacity-bulk-rack-with-wire-decking-72x24x96-add-on>: HTTP status code is not handled or not allowed 2026-01-24 20:10:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metromax-4-42-w-x-24-d-x-63-h-mobile-unit>: HTTP status code is not handled or not allowed 2026-01-24 20:10:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/68671-s-blade-r2-dice-for-robot-coupe>: HTTP status code is not handled or not allowed 2026-01-24 20:10:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12940-250-piece-woodruff-key-assortment-maintenance-kit>: HTTP status code is not handled or not allowed 2026-01-24 20:10:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1381234-tester-cooking-oil-w-case>: HTTP status code is not handled or not allowed 2026-01-24 20:10:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dot-9-hole-round-stainless-steel-light-cabinet>: HTTP status code is not handled or not allowed 2026-01-24 20:10:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72519-laminated-stainless-steel-arbor-shim-1-id-x-1-1-2-od-x-0062-thick>: HTTP status code is not handled or not allowed 2026-01-24 20:10:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011357-sensor-defrost-freezer-for-turbo-air-refrigeration>: HTTP status code is not handled or not allowed 2026-01-24 20:10:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-powered-2-inch-bung-mount-agitator-1-hp-2-6-inch-cl-propellers>: HTTP status code is not handled or not allowed 2026-01-24 20:10:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-extra-heavy-duty-boltless-shelving-add-on-96inw-x-36ind-x-72inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:11:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/amana-06-cu-ft-2100-watt-hd-commercial-microwave>: HTTP status code is not handled or not allowed 2026-01-24 20:11:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012165-ignitor-sensor-for-vulcan-hart>: HTTP status code is not handled or not allowed 2026-01-24 20:11:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/761145-portable-4-range-brnr>: HTTP status code is not handled or not allowed 2026-01-24 20:11:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kit2cv11b3x12-12-cal-ultrasoft-arc-flash-kit-coverall-balaclava-3xl-glove-size-12>: HTTP status code is not handled or not allowed 2026-01-24 20:11:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drink-mat-13982-nfl-cleveland-browns-3-1-4-x-24-x-1>: HTTP status code is not handled or not allowed 2026-01-24 20:11:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b4-compact-flat-clinch-executive-stapler-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:11:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-add-on-48inw-x-48ind-x-96inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:11:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-32-x-1-2-phillips-pan-head-machine-screw-pkg-of-100-dyas03016>: HTTP status code is not handled or not allowed 2026-01-24 20:11:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-1-2-x-2-1-2-x-36-160-edge-protectors-cased>: HTTP status code is not handled or not allowed 2026-01-24 20:11:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kit4sc40xl09-40-cal-compliance-arc-flash-kit-w-short-coat-bib-overall-xl-sz-09>: HTTP status code is not handled or not allowed 2026-01-24 20:11:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-add-on-60inw-x-24ind-x-72inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:11:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4600sp-self-propelled-airless-paint-striper>: HTTP status code is not handled or not allowed 2026-01-24 20:11:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 17478 pages (at 97 pages/min), scraped 11656 items (at 25 items/min) 2026-01-24 20:11:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-pressure-cntr-g1-4-dbl-solenoid-valve-pilot-spr-cntred-220v-ac-5va-coil-led-conn>: HTTP status code is not handled or not allowed 2026-01-24 20:11:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4mm-x-40mm-phillips-pan-head-metric-machine-screw-pkg-of-100-cnc04040>: HTTP status code is not handled or not allowed 2026-01-24 20:11:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-x-2-x-12-225-edge-protectors-cased>: HTTP status code is not handled or not allowed 2026-01-24 20:11:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/av-base-layer-shirt-s-gray-bstavlssm>: HTTP status code is not handled or not allowed 2026-01-24 20:11:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-g-1-2-double-solenoid-valve-pilot-12v-dc-3w-coil-black-connection>: HTTP status code is not handled or not allowed 2026-01-24 20:11:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/three-way-connector-kit-for-64-in-h-panel-with-1-pass-through-cable>: HTTP status code is not handled or not allowed 2026-01-24 20:11:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-8-x-24-x-550-feet-3-anti-static-air-foam-rolls>: HTTP status code is not handled or not allowed 2026-01-24 20:11:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-add-on-96inw-x-24ind-x-72inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:11:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conical-lip-plastic-anchor-4-6-8-x-3-4-plain-finish-pkg-of-100-078011>: HTTP status code is not handled or not allowed 2026-01-24 20:11:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-hose-plain-hose-nozzle-572-2521-2-1-2-nh-brass>: HTTP status code is not handled or not allowed 2026-01-24 20:11:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machine-screw-anchor-w-setting-tool-1-4-20-steel-zinc-cone-lead-sleeve-100-pk-275502>: HTTP status code is not handled or not allowed 2026-01-24 20:11:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-dbl-solenoid-vlv-ext-pilot-closed-cntr-spr-cntred-1-8-nptf-24v-dc-2w-coil-led>: HTTP status code is not handled or not allowed 2026-01-24 20:11:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2491050-switch-interlock-assembly-for-amana-litton>: HTTP status code is not handled or not allowed 2026-01-24 20:11:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-hose-double-male-adapter-378-1521524-1-1-2-nh-x-1-1-2-nh-aluminum>: HTTP status code is not handled or not allowed 2026-01-24 20:11:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-dbl-solenoid-vlv-ext-pilot-closed-cntr-spr-cntred-g1-4-24v-dc-3w-coil-led-conn>: HTTP status code is not handled or not allowed 2026-01-24 20:11:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/antique-trolley-for-street-vendor-poppers>: HTTP status code is not handled or not allowed 2026-01-24 20:11:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-stacking-bin-16-1-2-x-18-x-11-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:11:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultrawedge-wedge-anchor-1-2-13-x-4-1-2-steel-zinc-pkg-of-25-157210>: HTTP status code is not handled or not allowed 2026-01-24 20:11:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/canola-oil>: HTTP status code is not handled or not allowed 2026-01-24 20:11:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-open-single-solenoid-valve-ext-pilot-1-4-nptf-220v-ac-5va-coil-black-connection>: HTTP status code is not handled or not allowed 2026-01-24 20:11:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-flame-resistant-work-shirt-m-navy-tcgsswn00116mdrg00>: HTTP status code is not handled or not allowed 2026-01-24 20:11:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-153-chrome-hamper-stand-with-foot-pedal-poly-coated-steel-lid>: HTTP status code is not handled or not allowed 2026-01-24 20:11:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-basket-279001-12-nickel-plated-steel-16l-x-13w-x-6h>: HTTP status code is not handled or not allowed 2026-01-24 20:11:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-starter-72inw-x-48ind-x-84inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:11:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009524-slate-blue-10-inch-for-caddy-corp-of-america>: HTTP status code is not handled or not allowed 2026-01-24 20:11:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009527-pump-rebuild-kit-for-server-products>: HTTP status code is not handled or not allowed 2026-01-24 20:11:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-dbl-solenoid-vlv-ext-pilot-open-cntr-spr-cntred-1-8-nptf-110v-ac-5va-coil-black>: HTTP status code is not handled or not allowed 2026-01-24 20:11:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-ft-x12-ft-aluminum-frame-vinyl-tackboard-stone>: HTTP status code is not handled or not allowed 2026-01-24 20:11:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-starter-96inw-x-48ind-x-72inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:11:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010949-spring-carriage-lock-for-hobart>: HTTP status code is not handled or not allowed 2026-01-24 20:11:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-rack-wall-rack-1>: HTTP status code is not handled or not allowed 2026-01-24 20:11:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-wood-coat-rack-with-chrome-top-bars-1-hanging-rod-mahogany>: HTTP status code is not handled or not allowed 2026-01-24 20:11:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wax-resin-thermal-transfer-ribbon-65mm-x-450m-apr6-ouside-ink-wax-resin->: HTTP status code is not handled or not allowed 2026-01-24 20:11:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/luxury-high-back-walnut-luggage-rack-w-back-strap-black-straps-1-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:11:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011848-chain-for-turbochef>: HTTP status code is not handled or not allowed 2026-01-24 20:11:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grip-24w-6-step-steel-rolling-ladder-21d-top-step-lock-type-b-wcal-osha-handrail-ca>: HTTP status code is not handled or not allowed 2026-01-24 20:11:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25-drawcord-nylon-white-straight>: HTTP status code is not handled or not allowed 2026-01-24 20:11:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-3296-lower-bearing-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:11:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-heavy-duty-boltless-shelving-add-on-48inw-x-12ind-x-72inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:11:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/divider-18-d-x-10-h-premium-stacking-bin-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-24 20:11:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grip-24w-6-step-steel-rolling-ladder-14d-top-step-lock-type-b-wcal-osha-handrail-ca>: HTTP status code is not handled or not allowed 2026-01-24 20:11:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-add-on-72inw-x-24ind-x-72inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:11:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-dbl-solenoid-vlv-ext-pilot-closed-cntr-spr-cntred-1-4-nptf-24v-dc-3w-coil-led>: HTTP status code is not handled or not allowed 2026-01-24 20:11:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/open-sided-wire-exchange-truck-four-wire-one-galvanized-shelf-1000-lb-cap-3>: HTTP status code is not handled or not allowed 2026-01-24 20:11:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-performance-utility-pants-36-x-30-navy-pnt9su36x30>: HTTP status code is not handled or not allowed 2026-01-24 20:11:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-24w-13-step-steel-rolling-ladder-21d-top-step-wcal-osha-handrail-ca>: HTTP status code is not handled or not allowed 2026-01-24 20:11:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-58-enclosed-blower-115v-1075rpm-16hp>: HTTP status code is not handled or not allowed 2026-01-24 20:11:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-add-on-72inw-x-30ind-x-84inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:11:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-add-on-72inw-x-36ind-x-60inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:11:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-18-ga-36-wx18-dx73-h-open-clip-style-4-shelf>: HTTP status code is not handled or not allowed 2026-01-24 20:11:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-single-leg-support-15-23h-for-curved-conveyors>: HTTP status code is not handled or not allowed 2026-01-24 20:11:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-s-brass-big-flo-kettle-pot-sink-faucet>: HTTP status code is not handled or not allowed 2026-01-24 20:11:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1180-2-drum-steel-spill-pallet-68-gallon-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:11:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-work-shirt-4x-light-blue-tcg01190231>: HTTP status code is not handled or not allowed 2026-01-24 20:11:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jrs-5l-18w-straight-gal-steel-roller-conveyor-1-3-8-roller-dia-16-bf-3-axle-center>: HTTP status code is not handled or not allowed 2026-01-24 20:11:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/york-replacement-1050rpm-230v>: HTTP status code is not handled or not allowed 2026-01-24 20:11:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/direct-replacement-hill-refrigeration-208-230v-1625rpm-13hp>: HTTP status code is not handled or not allowed 2026-01-24 20:11:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shop-desk-w-pigeonhole-compartments-flat-top-34-1-2-w-x-30-d-x-38-to-42-1-2-h-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:11:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-add-on-48inw-x-12ind-x-96inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:11:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-power-cleaner-1-quart-bottle-citrus-scent-2112>: HTTP status code is not handled or not allowed 2026-01-24 20:11:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-utility-shirt-3xl-t-tan-df2-324ls-kh-3xlt>: HTTP status code is not handled or not allowed 2026-01-24 20:11:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jrs-16w-1-3-8-dia-gal-steel-roller-for-18-roller-conveyors>: HTTP status code is not handled or not allowed 2026-01-24 20:11:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010096-misc-fittings-kit-for-h-p-chicken-fryer>: HTTP status code is not handled or not allowed 2026-01-24 20:11:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-starter-96inw-x-48ind-x-60inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:11:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-shaft-1550rpm-115v-110-115-125hp>: HTTP status code is not handled or not allowed 2026-01-24 20:11:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-to-55-gallon-solvent-resistant-drum-pump>: HTTP status code is not handled or not allowed 2026-01-24 20:11:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/haws-pedestal-mounted-stainless-bowl-eyeface-wash-axion-msr-eyeface-wash-head>: HTTP status code is not handled or not allowed 2026-01-24 20:11:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-18-ga-48-wx24-dx85-h-open-clip-style-6-shelf>: HTTP status code is not handled or not allowed 2026-01-24 20:11:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-open-single-solenoid-valve-pilot-spr-return-1-4-nptf-110v-ac-5va-coil-led-conn>: HTTP status code is not handled or not allowed 2026-01-24 20:11:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-58-3-amp-208-230v-1075rpm-reversible>: HTTP status code is not handled or not allowed 2026-01-24 20:11:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelvimg-18-ga-48-wx30-dx85-h-open-clip-style-7-shelf>: HTTP status code is not handled or not allowed 2026-01-24 20:11:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-18-ga-48-wx18-dx97-h-open-clip-style-4-shelf>: HTTP status code is not handled or not allowed 2026-01-24 20:11:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/complete-set-of-7-drum-pump-nx-adapters>: HTTP status code is not handled or not allowed 2026-01-24 20:12:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/little-buddy-manual-bead-breaker-71600>: HTTP status code is not handled or not allowed 2026-01-24 20:12:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012973-decal-ctrl-panl-vent-kfc-for-henny-penny>: HTTP status code is not handled or not allowed 2026-01-24 20:12:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/outdoor-ball-fan-208-230460v-1075rpm-1hp>: HTTP status code is not handled or not allowed 2026-01-24 20:12:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-58-outdoor-ball-fan-a-208-230v-1075rpm-34hp>: HTTP status code is not handled or not allowed 2026-01-24 20:12:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/264630-hasp-lock-assembly-for-cres-cor>: HTTP status code is not handled or not allowed 2026-01-24 20:12:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-heavy-duty-boltless-shelving-add-on-60inw-x-24ind-x-72inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:12:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lightweight-flame-resistant-t-shirt-s-desert-sand-df2-cm-446ts-ds-sm>: HTTP status code is not handled or not allowed 2026-01-24 20:12:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lyon-locker-double-tier-12x12x36-6-door-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:12:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-jumbo-welded-storage-cabinet-48w-x-24d-x-42h-sand>: HTTP status code is not handled or not allowed 2026-01-24 20:12:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-safety-view-locker-single-tier-12x18x72-1-door-assembled-parchment>: HTTP status code is not handled or not allowed 2026-01-24 20:12:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-58-high-efficiency-indoor-blower-208-230v-1075rpm-12hp>: HTTP status code is not handled or not allowed 2026-01-24 20:12:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 17579 pages (at 101 pages/min), scraped 11673 items (at 17 items/min) 2026-01-24 20:12:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fr-control-2-0-long-sleeve-t-shirt-l-navy-c52fksrlslg>: HTTP status code is not handled or not allowed 2026-01-24 20:12:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-cal-flame-resistant-coveralls-3xl-ln-navy-tcg02160878>: HTTP status code is not handled or not allowed 2026-01-24 20:12:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-metal-pegboard-red-96-x-32-x-3-4>: HTTP status code is not handled or not allowed 2026-01-24 20:12:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-sterile-microbiological-filtration-monitors-100ml-50-pk>: HTTP status code is not handled or not allowed 2026-01-24 20:12:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-58-enclosed-outdoor-ball-fan-230460v-1075rpm-34hp>: HTTP status code is not handled or not allowed 2026-01-24 20:12:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-closed-single-solenoid-valve-pilot-spr-return-1-8-nptf-220v-ac-5va-coil-led-conn>: HTTP status code is not handled or not allowed 2026-01-24 20:12:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triple-tier-locker-with-legs-3-wide-12w-x-15d-x-24h-assembled-light-grey>: HTTP status code is not handled or not allowed 2026-01-24 20:12:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-w-server-station-shelf>: HTTP status code is not handled or not allowed 2026-01-24 20:12:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-cabinet-shelf-for-24-deep-combination-cabinet-light-grey>: HTTP status code is not handled or not allowed 2026-01-24 20:12:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kitchen-pegboard-pack-storage-organization-kit-green-blue-32-x-32-x-6>: HTTP status code is not handled or not allowed 2026-01-24 20:12:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-32-x-2-security-machine-screw-pan-spanner-head-18-8-stainless-steel-ft-unf-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 20:12:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kitchen-pegboard-pack-storage-organization-kit-galvanized-blue-32-x-32-x-6>: HTTP status code is not handled or not allowed 2026-01-24 20:12:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/label-holder-1-1-2-in-48-in-clear-strip>: HTTP status code is not handled or not allowed 2026-01-24 20:12:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-metal-pegboard-gray-48-x-32-x-3-4>: HTTP status code is not handled or not allowed 2026-01-24 20:12:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pegboard-basic-tool-organizer-kit-galvanized-black-48-x-32-x-9>: HTTP status code is not handled or not allowed 2026-01-24 20:12:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-starter-60inw-x-48ind-x-84inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:12:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quick-adjust-wire-shelving-36x18x86>: HTTP status code is not handled or not allowed 2026-01-24 20:12:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zip-seal-vinyl-pouches-9-12-in-self-adhesive>: HTTP status code is not handled or not allowed 2026-01-24 20:12:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-double-solenoid-valve-ext-pilot-1-4-nptf-220v-ac-5va-coil-black-connection>: HTTP status code is not handled or not allowed 2026-01-24 20:12:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/five-tier-box-locker-with-legs-3-wide-12w-x-12d-x-12h-assembled-sand>: HTTP status code is not handled or not allowed 2026-01-24 20:12:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/soft-buna-n-foam-sheet-with-acrylic-adhesive-1-16-x-12-x-24>: HTTP status code is not handled or not allowed 2026-01-24 20:12:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyurethane-foam-sheet-no-adhesive-4-x-39-x-39>: HTTP status code is not handled or not allowed 2026-01-24 20:12:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quick-adjust-shelf-18x60-poly-z-brite>: HTTP status code is not handled or not allowed 2026-01-24 20:12:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-person-box-locker-no-legs-12w-x-18d-x-12h-assembled-sand>: HTTP status code is not handled or not allowed 2026-01-24 20:12:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/laser-ink-jet-insert-letter-pref-2-1-4-in-2-5-8-in-600-per-package>: HTTP status code is not handled or not allowed 2026-01-24 20:12:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/viton-foam-sheet-no-adhesive-1-4-x-12-x-12>: HTTP status code is not handled or not allowed 2026-01-24 20:12:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/viton-foam-strip-with-high-temp-adhesive-1-8-x-1-2-x-56>: HTTP status code is not handled or not allowed 2026-01-24 20:12:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-16-x-3-4-security-machine-screw-flat-hex-socket-head-alloy-steel-black-oxide-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 20:12:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-24-x-1-1-2-security-machine-screw-flat-hex-socket-head-302hq-18-8-ss-ft-unc-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 20:12:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-32-x-1-1-4-security-machine-screw-button-torx-head-alloy-steel-black-oxide-ft-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 20:12:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pegboard-garden-tool-board-organizer-blue-red-48-x-32-x-9>: HTTP status code is not handled or not allowed 2026-01-24 20:12:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/80-063-200-vertical-small-cpu-holder-black>: HTTP status code is not handled or not allowed 2026-01-24 20:12:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pegboard-garden-tool-board-organizer-black-blue-48-x-32-x-9>: HTTP status code is not handled or not allowed 2026-01-24 20:12:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-strip-curtain-door-bulk-roll-standard-grade-clear-12-x-200-x-120>: HTTP status code is not handled or not allowed 2026-01-24 20:12:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-open-cntr-g-1-4-dbl-solenoid-valve-pilot-spring-cntred-24v-ac-5va-coil-led-conn>: HTTP status code is not handled or not allowed 2026-01-24 20:12:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clearone-unite150-ptz-hd-camera>: HTTP status code is not handled or not allowed 2026-01-24 20:12:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dry-erase-marker-and-eraser-kit-assorted>: HTTP status code is not handled or not allowed 2026-01-24 20:12:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classico-ground-coffee-032-oz-pouch-pack-of-76>: HTTP status code is not handled or not allowed 2026-01-24 20:12:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magliner-aluminum-platform-truck-with-diamond-deck-48-x-30-1000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:12:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carson-optical-rd-050-rd-series-10x50mm-open-bridge-waterproof-binoculars>: HTTP status code is not handled or not allowed 2026-01-24 20:12:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perfetto-ground-coffee-032-oz-pouch-pack-of-76>: HTTP status code is not handled or not allowed 2026-01-24 20:12:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bar-mat-12-x-18-long-one-piece-brown>: HTTP status code is not handled or not allowed 2026-01-24 20:12:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spice-womens-safety-work-boots-steel-toe-7h-size-9-mystic-dune-brown>: HTTP status code is not handled or not allowed 2026-01-24 20:12:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-king-sk3000-pallet-rack-96w-x-36d-x-96h-4950-lbs-cap-per-level-starter-unit>: HTTP status code is not handled or not allowed 2026-01-24 20:12:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-king-sk3000-pallet-rack-108w-x-36d-x-96h-4040-lbs-cap-per-level-add-on-unit>: HTTP status code is not handled or not allowed 2026-01-24 20:12:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/decorative_lighting/pendant_lighting already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:12:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-x-1-security-sheet-metal-screw-oval-one-way-head-18-8-stainless-steel-ft-usa-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 20:12:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-unit-12v-24w-lead-calcium12w-heads>: HTTP status code is not handled or not allowed 2026-01-24 20:12:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-24-removable-handle-magliner-aluminum-platform-trucks>: HTTP status code is not handled or not allowed 2026-01-24 20:12:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-30-removable-handle-magliner-aluminum-platform-trucks>: HTTP status code is not handled or not allowed 2026-01-24 20:12:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-x-2-security-sheet-metal-screw-oval-one-way-head-steel-zinc-ft-usa-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 20:12:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8ab-x-1-1-4-security-sheet-metal-screw-flat-hex-socket-head-302hq-18-8-ss-ft-usa-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 20:12:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/escort-emergency-light-6v-18w-diagnostics>: HTTP status code is not handled or not allowed 2026-01-24 20:12:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/horizon-womens-safety-work-boots-alloy-toe-8h-size-10-desert-tan>: HTTP status code is not handled or not allowed 2026-01-24 20:13:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/lighted_exit_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:13:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-steel-emergency-light-12v-110w>: HTTP status code is not handled or not allowed 2026-01-24 20:13:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wide-span-rack-96x48x60-with-3-shelves-no-deck-800-lb-capacity-per-level>: HTTP status code is not handled or not allowed 2026-01-24 20:13:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regal-rechargeable-motorized-outdoor-sun-shade-w-protective-valance-10-x-8-elder>: HTTP status code is not handled or not allowed 2026-01-24 20:13:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/decorative_lighting/pendant_lighting>: HTTP status code is not handled or not allowed 2026-01-24 20:13:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wide-span-rack-60x24x84-with-3-shelves-wood-deck-1200-lb-capacity-per-level>: HTTP status code is not handled or not allowed 2026-01-24 20:13:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-4003-parker-3-light-vanity-wall-sandstone-etched-polished-chrome-18-75w-x-9-5h>: HTTP status code is not handled or not allowed 2026-01-24 20:13:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clearview-3-16-polycarbonate-industrial-work-station-partition-36-w-x-48-h>: HTTP status code is not handled or not allowed 2026-01-24 20:13:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/omega-womens-safety-work-boots-steel-toe-6h-size-11-bourbon-brown>: HTTP status code is not handled or not allowed 2026-01-24 20:13:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 17676 pages (at 97 pages/min), scraped 11709 items (at 36 items/min) 2026-01-24 20:13:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/lighted_exit_signs>: HTTP status code is not handled or not allowed 2026-01-24 20:13:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-inch-buna-n-cam-and-groove-gasket>: HTTP status code is not handled or not allowed 2026-01-24 20:13:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/airgo-ultra-lightweight-boot-plain-toe-cleated-outsole-8-h-black-size-7>: HTTP status code is not handled or not allowed 2026-01-24 20:13:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-2-inch-c-polypropylene-cam-and-groove-coupler-x-hose-shank>: HTTP status code is not handled or not allowed 2026-01-24 20:13:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/77-140-1-light-wall-mount-w-white-glass-globe-black-6w-x-7-25h>: HTTP status code is not handled or not allowed 2026-01-24 20:13:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/76-500-add-on-motion-sensor-white>: HTTP status code is not handled or not allowed 2026-01-24 20:13:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/surveyor-vest-c2l2-solid-lime-s-m>: HTTP status code is not handled or not allowed 2026-01-24 20:13:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pulsar-knee-boot-plain-toe-chevron-plus-174-outsole-15-h-blk-gray-size-3>: HTTP status code is not handled or not allowed 2026-01-24 20:13:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/welding-cylinder-cart-pneumatic-wheels-33-1-4-x-23-1-16-x-57-11-16>: HTTP status code is not handled or not allowed 2026-01-24 20:13:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/76-635-1-light-3-louver-pagoda-garden-light-old-bronze-9-625w-x-7-125h>: HTTP status code is not handled or not allowed 2026-01-24 20:13:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/77-879-1-light-flush-mount-carport-frosted-acrylic-panels-white-10-25w-x-5-5h>: HTTP status code is not handled or not allowed 2026-01-24 20:13:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/wire_shelving/high_density_shelving already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:13:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reflective-t-shirt-long-sleeve-1-pocket-silver-tape-type-r-class-3-fl-lime-xl>: HTTP status code is not handled or not allowed 2026-01-24 20:13:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-539-1-light-die-cast-linen-acrylic-lens-white-acrylic-textured-black-7-75w-x-9-75h>: HTTP status code is not handled or not allowed 2026-01-24 20:13:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wide-span-rack-72x48x60-with-3-shelves-wood-deck-900-lb-capacity-per-level>: HTTP status code is not handled or not allowed 2026-01-24 20:13:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/76-644-mounting-post-verdi-2w-x-18h>: HTTP status code is not handled or not allowed 2026-01-24 20:13:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aware-wear-ansi-107-class-headwear-61705-lime>: HTTP status code is not handled or not allowed 2026-01-24 20:13:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reflective-long-sleeve-shirt-silver-tape-type-r-class-3-fl-lime-3xl>: HTTP status code is not handled or not allowed 2026-01-24 20:13:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/signs-with-safety-message-legend-danger-flammable-material>: HTTP status code is not handled or not allowed 2026-01-24 20:13:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/signs-with-safety-message-legend-danger-electrical-hazard-2>: HTTP status code is not handled or not allowed 2026-01-24 20:13:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sgna030gv-self-closing-safety-gate-28-5-inch-to-32-inch-length-galvanized>: HTTP status code is not handled or not allowed 2026-01-24 20:13:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-shelving-48x12x60-with-4-shelves-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:13:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/shelving/wire_shelving/high_density_shelving>: HTTP status code is not handled or not allowed 2026-01-24 20:13:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pulsar-knee-boot-plain-toe-chevron-plus-174-outsole-15-h-blk-gray-size-8>: HTTP status code is not handled or not allowed 2026-01-24 20:13:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spray-suction-hose-kit-for-b-250-scrubber-2-640-938-0>: HTTP status code is not handled or not allowed 2026-01-24 20:13:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/category-m-flat-pleated-filter-cellulose-6-907-276-0>: HTTP status code is not handled or not allowed 2026-01-24 20:13:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-shelving-36x24x84-with-5-shelves-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:13:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-shelving-48x24x84-with-5-shelves-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:13:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-4971-boxwood-1-light-outdoor-wall-clear-beveled-white-8w-x-22h>: HTTP status code is not handled or not allowed 2026-01-24 20:13:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-steel-wheel-riser-66-x-24-x-12>: HTTP status code is not handled or not allowed 2026-01-24 20:13:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plc-03501-64oz-natural-hdpe-handled-round-jug-with-38-400-black-phenolic-cap-case-of-40>: HTTP status code is not handled or not allowed 2026-01-24 20:13:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plc-03391-2oz-natural-hdpe-cylinder-bottle-with-24-410-white-pp-cap-case-of-48>: HTTP status code is not handled or not allowed 2026-01-24 20:13:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/valence-womens-safety-work-boots-steel-toe-7h-size-105-chestnut-brown>: HTTP status code is not handled or not allowed 2026-01-24 20:13:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-drum-lock-for-55-gallon-drums-aluminum>: HTTP status code is not handled or not allowed 2026-01-24 20:13:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/77-059-1-light-ceiling-flush-mount-small-mushroom-polished-brass-7-5w-x-5h>: HTTP status code is not handled or not allowed 2026-01-24 20:13:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/240134-white-hdpe-sr-rieke-pail-lid-for-3-5-5-and-6-gallon-pails-case-of-32>: HTTP status code is not handled or not allowed 2026-01-24 20:13:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/traffic_parking_lot_safety/delineator_posts_channelizer_cones already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:13:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gravity-womens-safety-work-boots-steel-toe-4h-size-95-black>: HTTP status code is not handled or not allowed 2026-01-24 20:13:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-shelf-level-boltless-wire-deck-10>: HTTP status code is not handled or not allowed 2026-01-24 20:13:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hydraulic-toe-saddle-jack-15-ton-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:13:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storability-top-track-wall-frame-33w-x-12d-x-1-34h-white>: HTTP status code is not handled or not allowed 2026-01-24 20:13:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-shelving-36x12x96-with-5-shelves-wire-deck-2>: HTTP status code is not handled or not allowed 2026-01-24 20:13:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/valence-womens-safety-work-boots-steel-toe-7h-size-75w-chestnut-brown>: HTTP status code is not handled or not allowed 2026-01-24 20:13:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2661057-blade-scraper-plastic-17-inch-l-for-taylor-freezer>: HTTP status code is not handled or not allowed 2026-01-24 20:13:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mnt246ptp-safety-tag-notice-potable-drinking-water-only-575-h-x-325-w-plastic-25-pk>: HTTP status code is not handled or not allowed 2026-01-24 20:13:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp65ss-eye-face-wash-head-ss-inverted-iaminar-and-integral-flow-control>: HTTP status code is not handled or not allowed 2026-01-24 20:13:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-stem-caster-4-dia-stainless-disc-with-tech-lock-1-1-2-h-stem>: HTTP status code is not handled or not allowed 2026-01-24 20:13:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/we15d33a-wall-air-conditioner-w-elec-heat-14500-btu-cool-230-v>: HTTP status code is not handled or not allowed 2026-01-24 20:13:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-381-standard-kneeling-pad-1-2-thick-14-x-21-black>: HTTP status code is not handled or not allowed 2026-01-24 20:13:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/traffic_parking_lot_safety/delineator_posts_channelizer_cones landed on page that is not a product page. 2026-01-24 20:13:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-acrylic-desk-divider-48-x-24-mounted>: HTTP status code is not handled or not allowed 2026-01-24 20:13:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-dry-erase-marker-4-assorted-colors-pack-of-5>: HTTP status code is not handled or not allowed 2026-01-24 20:13:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8407320-blades-clipless-new-style-for-taylor-freezer>: HTTP status code is not handled or not allowed 2026-01-24 20:13:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-x-6-portable-wire-grid-wall-panel-semi-gloss-white>: HTTP status code is not handled or not allowed 2026-01-24 20:14:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-8482-10-x-10-light-duty-2-9-oz-tarp-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:14:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-4954-cove-neck-1-light-outdoor-hang-clear-seeded-white-7w-x-12h>: HTTP status code is not handled or not allowed 2026-01-24 20:14:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-3259-1-light-pendant-frosted-white-brushed-nickel-9-75w-x-20-5h>: HTTP status code is not handled or not allowed 2026-01-24 20:14:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-high-letter-starter-empty-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:14:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interlocking-flap-file-storage-box-fsb500-letter-24l-x-12w-x-10h-white-price-each>: HTTP status code is not handled or not allowed 2026-01-24 20:14:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1381314-thermometer-w-probe>: HTTP status code is not handled or not allowed 2026-01-24 20:14:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-file-storage-box-w-lid-fsb300-letter-legal-15l-x-12w-x-10h-kraft-price-each>: HTTP status code is not handled or not allowed 2026-01-24 20:14:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-boltless-shelving-add-on-36inw-x-24ind-x-60inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:14:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grip-16-w-3-step-steel-rolling-ladder-10-d-top-step-1>: HTTP status code is not handled or not allowed 2026-01-24 20:14:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 17770 pages (at 94 pages/min), scraped 11742 items (at 33 items/min) 2026-01-24 20:14:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-polyurethane-barb-x-1-4-male-npt-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 20:14:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lakeside-homed-classic-5-drawer-treatment-cart>: HTTP status code is not handled or not allowed 2026-01-24 20:14:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-boltless-shelving-add-on-36inw-x-18ind-x-72inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:14:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-rubber-load-bearing-pad-10-long-10-wide-1h>: HTTP status code is not handled or not allowed 2026-01-24 20:14:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-maroon-x-stripping-pad-10-per-case>: HTTP status code is not handled or not allowed 2026-01-24 20:14:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-boltless-shelving-add-on-36inw-x-24ind-x-96inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:14:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esd-safe-vinyl-clean-room-stool-with-nylon-base-with-drag-chain-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:14:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lincoln-steel-plug-1-4-x-1-4-female-npt-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-24 20:14:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-copper-press-x-fpt-w-female-adapter>: HTTP status code is not handled or not allowed 2026-01-24 20:14:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grip-16-w-5-step-steel-rolling-ladder-21-d-top-step>: HTTP status code is not handled or not allowed 2026-01-24 20:14:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grip-16-w-7-step-steel-rolling-ladder-21-d-top-step>: HTTP status code is not handled or not allowed 2026-01-24 20:14:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/osha-handrail-kit-5-to-10-steps-21-d-step>: HTTP status code is not handled or not allowed 2026-01-24 20:14:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009707-goodyear-84220-belt>: HTTP status code is not handled or not allowed 2026-01-24 20:14:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/outdoor-ratio-led-floodlight-3200l-26w-40k-wide-dist-knuckle-mt-120-277v>: HTTP status code is not handled or not allowed 2026-01-24 20:14:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fuel-delivery-1-50-ft-250-psi-120-lbs-fuel-with-hose-120-lbs>: HTTP status code is not handled or not allowed 2026-01-24 20:14:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/meteorlite-2-low-profile-strobe-light-12-80-volts-permanent-mount-amber>: HTTP status code is not handled or not allowed 2026-01-24 20:14:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3200-series-safe-t-alert-back-up-alarm-107-db-12-24-volts-standard>: HTTP status code is not handled or not allowed 2026-01-24 20:14:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010166-blade-plastic-one-hole-for-taylor-freezer>: HTTP status code is not handled or not allowed 2026-01-24 20:14:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/decorative_lighting/vanity_bath_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:14:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grounding-50-ft-grounding-with-cable-y-clip>: HTTP status code is not handled or not allowed 2026-01-24 20:14:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-boltless-shelving-starter-36inw-x-24ind-x-60inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:14:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-cleaner-gallon-bottle-four-bottles>: HTTP status code is not handled or not allowed 2026-01-24 20:14:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alliance-hose-thermoplastic-snubber-hose-3-8x24-with-1-4-male-and-male-swivel>: HTTP status code is not handled or not allowed 2026-01-24 20:14:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2271218-speaker-assy-hp-timer-31153-for-henny-penny>: HTTP status code is not handled or not allowed 2026-01-24 20:14:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/sneeze_guards_distancing_barriers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:14:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/serrated-24-w-10-step-steel-rolling-ladder-10-d-top-step-kdsr110242>: HTTP status code is not handled or not allowed 2026-01-24 20:14:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-boltless-shelving-starter-36inw-x-18ind-x-96inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:14:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/three-conductor-grounded-temp-flex-35-illuminated-plug-300v-7>: HTTP status code is not handled or not allowed 2026-01-24 20:14:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/decorative_lighting/vanity_bath_lights>: HTTP status code is not handled or not allowed 2026-01-24 20:14:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-pallet-lifter-4000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:14:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/38-style-b-j-hook>: HTTP status code is not handled or not allowed 2026-01-24 20:14:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/emergency_lighting_signage/emergency_lighting already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:14:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-emergency-unit-sq-heads-remote-capacity-self-diagnostics-nicad-battery120-277v>: HTTP status code is not handled or not allowed 2026-01-24 20:14:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-boltless-shelving-add-on-48inw-x-24ind-x-72inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:14:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/partitions_dividers/sneeze_guards_distancing_barriers>: HTTP status code is not handled or not allowed 2026-01-24 20:14:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-72-lift-beam-adjustable-length-6000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:14:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-cabinet-with-removable-bins-5>: HTTP status code is not handled or not allowed 2026-01-24 20:14:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/461521-t-stat-bi-metal>: HTTP status code is not handled or not allowed 2026-01-24 20:14:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-vinyl-gloves-powder-free-large>: HTTP status code is not handled or not allowed 2026-01-24 20:14:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vertical-plate-clamp-lifting-attachment-4000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:14:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/881467-valve-body-expansion-valve-for-parker-hannifin>: HTTP status code is not handled or not allowed 2026-01-24 20:14:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/emergency_lighting_signage/emergency_lighting landed on page that is not a product page. 2026-01-24 20:14:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-x-30-metal-shelf-for-security-truck-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:14:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-3-shelf-boltless-shelving-add-on-48inw-x-24ind-x-60inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:14:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/strapping-attachment-for-hvac-hand-truck>: HTTP status code is not handled or not allowed 2026-01-24 20:14:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-uncaged-fixed-access-ladder-yellow-6>: HTTP status code is not handled or not allowed 2026-01-24 20:14:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/security-truck-48l-x-24w-x-66h-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:14:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/59803-premium-resin-fibre-discs-4-1-2-inch-x-7-8-inch-60-grit-ceramic>: HTTP status code is not handled or not allowed 2026-01-24 20:14:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/front-loader-tricycle-500-capacity-w-platform-37-in-axle-orange>: HTTP status code is not handled or not allowed 2026-01-24 20:14:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medium-duty-service-door-single-panel-beige-3ft-6in-x-7ft-kickplate>: HTTP status code is not handled or not allowed 2026-01-24 20:14:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-34-inch-high-post-poly-green-4-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:14:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caged-walk-through-fixed-access-ladder-yellow-9>: HTTP status code is not handled or not allowed 2026-01-24 20:14:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caged-walk-through-fixed-access-ladder-gray-10>: HTTP status code is not handled or not allowed 2026-01-24 20:14:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010271-front-panel-assembly-for-manitowoc-machines>: HTTP status code is not handled or not allowed 2026-01-24 20:14:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelfmax-nestable-plastic-shelf-bin-11-1-8w-x-23-5-8d-x-6h-clear>: HTTP status code is not handled or not allowed 2026-01-24 20:14:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-nsf-plastic-bus-tub-20-l-x-15-w-x-5-h-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:14:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25t4-cl-g8-25w-halogen-w-bi-pin-base>: HTTP status code is not handled or not allowed 2026-01-24 20:14:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hotel-laundry-bag-with-draw-tape-closure-pkg-qty-1000>: HTTP status code is not handled or not allowed 2026-01-24 20:14:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-nsf-plastic-bus-tub-21-1-2-l-x-17-3-4-w-x-7-h-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:14:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tuf-r-heavy-lldpe-flat-bag-8in-x-12in-2-mil-pkg-qty-1000>: HTTP status code is not handled or not allowed 2026-01-24 20:14:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/62-1020-pressure-gauge-0-200-psi-1-4-inch-mpt-bottom-mount>: HTTP status code is not handled or not allowed 2026-01-24 20:14:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/datum-tekstak-laptop-storage-locker-8-tier-key-lock-9>: HTTP status code is not handled or not allowed 2026-01-24 20:14:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wide-truck-ramps-20-ton-47-l-x-18-1-4-w-x-9-1-4-h-51-overall-length>: HTTP status code is not handled or not allowed 2026-01-24 20:15:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-literature-mailers-6l-x-6w-x-2h-white-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-24 20:15:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dispense-rite-optional-side-straw-section-ctld-model-dispensers>: HTTP status code is not handled or not allowed 2026-01-24 20:15:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torque-limiting-extension-1-2-drive-110-ft-lbs>: HTTP status code is not handled or not allowed 2026-01-24 20:15:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3x5-heritage-rib-mat-charcoal>: HTTP status code is not handled or not allowed 2026-01-24 20:15:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/federal-signal-two-way-intercom-standard>: HTTP status code is not handled or not allowed 2026-01-24 20:15:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torque-limiting-extension-1-2-drive-45-ft-lbs>: HTTP status code is not handled or not allowed 2026-01-24 20:15:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/35mr11-fth-s-c-35w-halogen-w-bi-pin-base>: HTTP status code is not handled or not allowed 2026-01-24 20:15:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/federal-signal-4-vibrating-bell-120-08-120-98>: HTTP status code is not handled or not allowed 2026-01-24 20:15:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-3-wall-specimen-transfer-bag-pkg-qty-1000>: HTTP status code is not handled or not allowed 2026-01-24 20:15:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/federal-signal-weatherproof-vibratone-bell-08-120-98>: HTTP status code is not handled or not allowed 2026-01-24 20:15:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-train-lift-table-2-500-capacity-lbs>: HTTP status code is not handled or not allowed 2026-01-24 20:15:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tuf-r-standard-lldpe-flat-bag-7in-x-24in-1-mil-pkg-qty-1000>: HTTP status code is not handled or not allowed 2026-01-24 20:15:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/893-filament-tape-1-2-x-60-yds-6-mil-clear>: HTTP status code is not handled or not allowed 2026-01-24 20:15:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 17868 pages (at 98 pages/min), scraped 11767 items (at 25 items/min) 2026-01-24 20:15:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-profile-steady-burning-led-120-12-pipe-amber>: HTTP status code is not handled or not allowed 2026-01-24 20:15:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergo-iws-24-d-pedestrian-detection-24in-led-sign-combo>: HTTP status code is not handled or not allowed 2026-01-24 20:15:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/floor-crane-2000-lb-capacity-6>: HTTP status code is not handled or not allowed 2026-01-24 20:15:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/light-duty-floor-crane-500-lb-capacity-9>: HTTP status code is not handled or not allowed 2026-01-24 20:15:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fc40t5-830-40w-fluorescent-w-2gx13-base-warm-white>: HTTP status code is not handled or not allowed 2026-01-24 20:15:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8015526-probe-kit-for-true-manufacturing>: HTTP status code is not handled or not allowed 2026-01-24 20:15:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/homak-rd04004193-rs-pro-series-41-w-x-24-d-x-39-h-9-drawer-red-roller-tool-cabinet>: HTTP status code is not handled or not allowed 2026-01-24 20:15:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7a19-27-7w-compact-fluorescent-w-medium-base-soft-white>: HTTP status code is not handled or not allowed 2026-01-24 20:15:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/homak-og02072120-rs-pro-series-71-1-2-w-x-23-1-2-d-x-23-3-8-h-12-drawer-orange-tool-chest>: HTTP status code is not handled or not allowed 2026-01-24 20:15:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/13t2c-50-13w-compact-fluorescent-w-candelabra-base-natural-light>: HTTP status code is not handled or not allowed 2026-01-24 20:15:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-w-x-36-d-plastic-laminate-square-edge-workbench-adjustable-height-black>: HTTP status code is not handled or not allowed 2026-01-24 20:15:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-locker-number-plate-kit-pkg-of-200-numbered-1900-2099>: HTTP status code is not handled or not allowed 2026-01-24 20:15:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/surge_protectors_extension_cords/surge_protectors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:15:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pic-gauges-2-1-2-pressure-gauge-liquid-filled-300-psi-ss-case-center-back-mount-pro-202l-254h>: HTTP status code is not handled or not allowed 2026-01-24 20:15:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bookcases_hutches already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:15:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hoop-jump-natural-tbark-430n>: HTTP status code is not handled or not allowed 2026-01-24 20:15:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ft18hl-841-env-18w-compact-fluorescent-w-2g11-base-cool-white>: HTTP status code is not handled or not allowed 2026-01-24 20:15:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pic-gauges-4-pressure-gauge-liquid-filled-2000-psi-stainless-case-lower-mount-201l-402o>: HTTP status code is not handled or not allowed 2026-01-24 20:15:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5t2c-50-5w-compact-fluorescent-w-candelabra-base-natural-light>: HTTP status code is not handled or not allowed 2026-01-24 20:15:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8013200-control-low-pressure>: HTTP status code is not handled or not allowed 2026-01-24 20:15:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/500q-f-500w-halogen-w-mini-can-base>: HTTP status code is not handled or not allowed 2026-01-24 20:15:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-easy-fold-mailers-12-18l-x-9-18w-x-4h-kraft-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-24 20:15:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/surge_protectors_extension_cords/surge_protectors landed on page that is not a product page. 2026-01-24 20:15:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010662-temperature-gauge-for-alto-shaam>: HTTP status code is not handled or not allowed 2026-01-24 20:15:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cp-hd-5-5-channel-heavy-duty-cable-protector>: HTTP status code is not handled or not allowed 2026-01-24 20:15:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bookcases_hutches landed on page that is not a product page. 2026-01-24 20:15:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hydraulic-bottle-jack-20-ton-air>: HTTP status code is not handled or not allowed 2026-01-24 20:15:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-double-wall-corrugated-boxes-30l-x-15w-x-15h-kraft-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 20:15:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/de-flex-niru-anti-fatigue-modular-mat-tile-18-by-18-black-8-pk>: HTTP status code is not handled or not allowed 2026-01-24 20:15:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-easy-fold-mailers-9-58l-x-6-58w-x-3-12h-kraft-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-24 20:15:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cfd13w-841-13w-compact-fluorescent-w-gx23-2-base-cool-white>: HTTP status code is not handled or not allowed 2026-01-24 20:15:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torque-stick-13-16-1-drive-475-ft-lbs-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:15:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-easy-fold-mailers-11l-x-8-12w-x-3h-white-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-24 20:15:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cushion-stat-dyna-shield-anti-static-mat-3-by-5-black>: HTTP status code is not handled or not allowed 2026-01-24 20:15:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/combination_metalworking_machines already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:15:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1381052-thermometer-60-140f-125-inch-od>: HTTP status code is not handled or not allowed 2026-01-24 20:15:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waste-oil-drain-w-pump-18-gallon-low-profile-plastic>: HTTP status code is not handled or not allowed 2026-01-24 20:15:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/combination_metalworking_machines already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:15:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bin-nestable-9>: HTTP status code is not handled or not allowed 2026-01-24 20:15:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-plastic-tote-without-lid-18-1-2x13-1-4x12-white>: HTTP status code is not handled or not allowed 2026-01-24 20:15:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nut-buddy-lug-nut-loosener-58-1-gear-ratio-2-950-ft-lbs-torque>: HTTP status code is not handled or not allowed 2026-01-24 20:15:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/column-protector-10-in-opening-24-in-h-red>: HTTP status code is not handled or not allowed 2026-01-24 20:15:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tote-box-dividable-19>: HTTP status code is not handled or not allowed 2026-01-24 20:15:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/travira-32-bar-table-tekwood-natural>: HTTP status code is not handled or not allowed 2026-01-24 20:15:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/282400-breaker-strip-for-perlick-refrigeration>: HTTP status code is not handled or not allowed 2026-01-24 20:15:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jackson-safety-a-5500sn-dielectric-cap-style-non-slotted-hard-hat-adapter-blue-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-24 20:15:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/travira-armchair-black-sling-tekwood-natural-armcaps-2-pk>: HTTP status code is not handled or not allowed 2026-01-24 20:15:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-decorative-light-fixture-9w-3000k-120v-4-12l-venice-lensbrushed-nickel>: HTTP status code is not handled or not allowed 2026-01-24 20:15:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_seating/patio_dining_chairs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:15:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/combination_metalworking_machines>: HTTP status code is not handled or not allowed 2026-01-24 20:15:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/combination_metalworking_machines>: HTTP status code is not handled or not allowed 2026-01-24 20:15:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergotec-pro-wndw-squge-rplcemnt-rbr-12-soft>: HTTP status code is not handled or not allowed 2026-01-24 20:15:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-3-4-thick-x-4-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 20:15:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/krp-150fr-fire-rated-access-door-for-wall-ceil-paddle-handle-48wx48h-krpp4848ph>: HTTP status code is not handled or not allowed 2026-01-24 20:15:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uhmw-polyethylene-plastic-rod-2-diameter-x-6-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 20:15:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-back-aluminum-beverage-hand-truck-16-w-x-10-d-shoe-with-brakes>: HTTP status code is not handled or not allowed 2026-01-24 20:15:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rlph1ns-long-sleeve-pullover-hoodie-with-uv-protection-3xl-hi-vis-orange>: HTTP status code is not handled or not allowed 2026-01-24 20:15:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25a15-cl-25w-incandescent-w-medium-base>: HTTP status code is not handled or not allowed 2026-01-24 20:15:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guardian-booth-6-x-8-guard-booth-economy-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:15:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/indoor-vertical-fat-bike-rack-with-4-mounts>: HTTP status code is not handled or not allowed 2026-01-24 20:15:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guardian-booth-4-x-6-guard-booth-deluxe-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:15:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-strip-door-curtain-12-w-x-10-h>: HTTP status code is not handled or not allowed 2026-01-24 20:15:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_seating/patio_dining_chairs>: HTTP status code is not handled or not allowed 2026-01-24 20:15:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uhmw-polyethylene-plastic-sheet-1-4-thick-x-24-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 20:15:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6424v-5w-miniature-sealed-beams-w-festoon-base>: HTTP status code is not handled or not allowed 2026-01-24 20:15:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25g25-w-25w-incandescent-w-medium-base-120v>: HTTP status code is not handled or not allowed 2026-01-24 20:16:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25ca10-f-25w-incandescent-w-medium-base-130v>: HTTP status code is not handled or not allowed 2026-01-24 20:16:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tripower-classical-section-molded-notch-vbelt-gates-ax50>: HTTP status code is not handled or not allowed 2026-01-24 20:16:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-2-thick-x-2-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 20:16:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drums-up-jr7-1-2h-x-21-1-2-top-diameter-75-gallon-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:16:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8013586-thermostat-bjwa-36-inch-capillary-for-southbend-range>: HTTP status code is not handled or not allowed 2026-01-24 20:16:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poly-collector-shell-only-66-gallon-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:16:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibco-silent-pneumatic-turbine-vibrator-mlt-19>: HTTP status code is not handled or not allowed 2026-01-24 20:16:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-step-stairway-slope-work-platform-24in-x-60in-platform>: HTTP status code is not handled or not allowed 2026-01-24 20:16:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibco-silent-pneumatic-turbine-vibrator-vs-510>: HTTP status code is not handled or not allowed 2026-01-24 20:16:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibco-silent-pneumatic-turbine-vibrator-vs-510ht>: HTTP status code is not handled or not allowed 2026-01-24 20:16:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibco-silent-pneumatic-turbine-vibrator-mltss-19-ht>: HTTP status code is not handled or not allowed 2026-01-24 20:16:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jackson-safety-dielectric-cap-style-slotted-hard-hat-adapter-with-crown-black-pack-of-40>: HTTP status code is not handled or not allowed 2026-01-24 20:16:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 17962 pages (at 94 pages/min), scraped 11788 items (at 21 items/min) 2026-01-24 20:16:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-1104-ultimate-blower-wheel-puller-for-5-8-inch-belt-drive>: HTTP status code is not handled or not allowed 2026-01-24 20:16:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poly-spill-pad-with-grate-24-x-24-x-2-5600-ye>: HTTP status code is not handled or not allowed 2026-01-24 20:16:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:16:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-12-x-9-standard-clear-strip-strip-curtain-doors>: HTTP status code is not handled or not allowed 2026-01-24 20:16:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/incandescent_light_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:16:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibco-hd-silent-pneumatic-turbine-vibrator-ccw-5000>: HTTP status code is not handled or not allowed 2026-01-24 20:16:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibco-12-volt-electric-vibrator-dc-200>: HTTP status code is not handled or not allowed 2026-01-24 20:16:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/incandescent_light_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:16:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uhmw-polyethylene-plastic-bar-2-thick-x-2-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 20:16:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uhmw-polyethylene-plastic-bar-1-4-thick-x-2-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 20:16:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/epr30-3-23w-incandescent-w-flange-base>: HTTP status code is not handled or not allowed 2026-01-24 20:16:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/13491-water-filter-cartridgehf40-for-cuno-inc>: HTTP status code is not handled or not allowed 2026-01-24 20:16:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009430-drain-valve-120v-for-cma-dishmachines>: HTTP status code is not handled or not allowed 2026-01-24 20:16:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 20:16:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/933-series-airtight-watertight-case-with-foam-933-1004-19-7-8-l-x-16-1-8-w-x-10-1-8-h-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:16:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-double-round-decorative-light-fixture-20w-304050k-120v-13l-blackgold>: HTTP status code is not handled or not allowed 2026-01-24 20:16:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/incandescent_light_bulbs landed on page that is not a product page. 2026-01-24 20:16:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-wood-cubicle-cabinet-10-shelves-wlocks-solid-door-48-x-22-14-x-72-natural-oakbrown>: HTTP status code is not handled or not allowed 2026-01-24 20:16:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/incandescent_light_bulbs>: HTTP status code is not handled or not allowed 2026-01-24 20:16:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lodestar-vs-electric-chain-hoist-2-ton-10-foot-lift-1-3-8-fpm-115-1-60-2-step-control>: HTTP status code is not handled or not allowed 2026-01-24 20:16:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010986-thermocouple-h-limit-for-henny-penny>: HTTP status code is not handled or not allowed 2026-01-24 20:16:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibco-silent-pneumatic-turbine-vibrator-bbs-190>: HTTP status code is not handled or not allowed 2026-01-24 20:16:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/respirator-cartridge-accessory-vapor-particulates-filter-3>: HTTP status code is not handled or not allowed 2026-01-24 20:16:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chipbuddie-pack-of-replacement-blades>: HTTP status code is not handled or not allowed 2026-01-24 20:16:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/omni-pir-ultrasonic-low-voltage-ceiling-sensor-with-2000-sq-ft-range-off-white>: HTTP status code is not handled or not allowed 2026-01-24 20:16:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-flush-mount-ceiling-light-fixture-15w-1120-lumens-12-dia-brushed-nickel>: HTTP status code is not handled or not allowed 2026-01-24 20:16:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-drum-with-lining>: HTTP status code is not handled or not allowed 2026-01-24 20:16:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unisan-clip-on-dust-mop-frame-zinc-plated>: HTTP status code is not handled or not allowed 2026-01-24 20:16:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012265-field-kit-probe-moldedstmchef-1-pc-con-for-cleveland-range>: HTTP status code is not handled or not allowed 2026-01-24 20:16:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jackson-safety-sc-6-cap-style-hard-hat-4-point-ratchet-suspension-brown-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-24 20:16:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ushio-hpl-575-115v-js115v-575wc>: HTTP status code is not handled or not allowed 2026-01-24 20:16:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fibre-metal-174-speedy-loop-153-headband-blue-fully-adjustable-plastic>: HTTP status code is not handled or not allowed 2026-01-24 20:16:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:16:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2000-heated-eyewash-station-cover-vinyl>: HTTP status code is not handled or not allowed 2026-01-24 20:16:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/halogen_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:16:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milton-s-1836w-h-style-industrial-coupler-3-8-mnpt>: HTTP status code is not handled or not allowed 2026-01-24 20:16:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-container-forkli-able-with-double-wall-red-1>: HTTP status code is not handled or not allowed 2026-01-24 20:16:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-g1sc-018-ss>: HTTP status code is not handled or not allowed 2026-01-24 20:16:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rapidwire-wood-wall-connection-hardware-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:16:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polycarbonate-plastic-sheet-1-8-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 20:16:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-two-piece-clamping-collar-gm2c-14-b>: HTTP status code is not handled or not allowed 2026-01-24 20:16:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/76-1207-scaleout-filter-cartridge>: HTTP status code is not handled or not allowed 2026-01-24 20:16:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-b10-torpedo-clear-light-bulb-e14-base-5w-470-lumens-27k-warm-white-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-24 20:16:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/halogen_bulbs>: HTTP status code is not handled or not allowed 2026-01-24 20:16:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/840-style-2-sided-driver-cage-4w-x-6d-x-8h>: HTTP status code is not handled or not allowed 2026-01-24 20:16:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jlc-electric-chain-hoist-2-ton-cap-20-ft-lift-8-fpm-115230v>: HTTP status code is not handled or not allowed 2026-01-24 20:16:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012357-electric-thermostat250-500f-for-garland-range>: HTTP status code is not handled or not allowed 2026-01-24 20:16:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-g1sc-168-b>: HTTP status code is not handled or not allowed 2026-01-24 20:16:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/photoluminescent-flexible-nosing-strip-1-x-4-feet>: HTTP status code is not handled or not allowed 2026-01-24 20:16:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/p-skfl5-petroleum-sorbent-spill-kit5-gallon-capacity-3-kitscase>: HTTP status code is not handled or not allowed 2026-01-24 20:16:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-rack-tear-drop-beam-120-l>: HTTP status code is not handled or not allowed 2026-01-24 20:16:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/import-individual-transfer-punch-23-64>: HTTP status code is not handled or not allowed 2026-01-24 20:16:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c-skfl31-chemical-sorbent-spill-kit-31-gallon-1-case>: HTTP status code is not handled or not allowed 2026-01-24 20:16:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/088trblklar-base-layer-top-black-large>: HTTP status code is not handled or not allowed 2026-01-24 20:16:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2221429-sensor-temperature-for-waring-products>: HTTP status code is not handled or not allowed 2026-01-24 20:16:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 20:16:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-gsc-187-z>: HTTP status code is not handled or not allowed 2026-01-24 20:16:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0490rnavlar-insulated-softshell-jacket-navy-large>: HTTP status code is not handled or not allowed 2026-01-24 20:16:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/381657-ignition-cable>: HTTP status code is not handled or not allowed 2026-01-24 20:16:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/self_luminous_emergency_exit_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:16:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-g1sc-137-ss>: HTTP status code is not handled or not allowed 2026-01-24 20:16:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72x30-shop-top-safety-edge-power-apron-production-bench-tan-32313709>: HTTP status code is not handled or not allowed 2026-01-24 20:16:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/led_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:16:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-inch-speed-squeegyepdm-rubber-with-steel-curved-frame>: HTTP status code is not handled or not allowed 2026-01-24 20:17:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-24-w-8-step-steel-rolling-ladder-20-d-top-step>: HTTP status code is not handled or not allowed 2026-01-24 20:17:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-w-x-30-d-mobile-production-workbench-with-power-apron-maple-butcher-block-square-edge-blue-32313736>: HTTP status code is not handled or not allowed 2026-01-24 20:17:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-t8-tube-light-bulb-med-bi-pin-g13-base-18w-2300-lumens-65k-white-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-24 20:17:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grip-24-w-12-step-steel-rolling-ladder-10-d-top-step>: HTTP status code is not handled or not allowed 2026-01-24 20:17:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-g1sc-243-b>: HTTP status code is not handled or not allowed 2026-01-24 20:17:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-gsc-237-z>: HTTP status code is not handled or not allowed 2026-01-24 20:17:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/self_luminous_emergency_exit_signs landed on page that is not a product page. 2026-01-24 20:17:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-gallon-self-evacuating-oil-drain>: HTTP status code is not handled or not allowed 2026-01-24 20:17:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/led_bulbs>: HTTP status code is not handled or not allowed 2026-01-24 20:17:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 18056 pages (at 94 pages/min), scraped 11816 items (at 28 items/min) 2026-01-24 20:17:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-gsc-300-z>: HTTP status code is not handled or not allowed 2026-01-24 20:17:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-set-screw-collar-gmc-06-ss>: HTTP status code is not handled or not allowed 2026-01-24 20:17:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/led_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:17:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-a19-household-light-bulb-e26-base-9w-800-lumens-3000k-warm-white-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-24 20:17:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iron-ageia0163-mens-6-internal-met-guard-work-boot-brown-size-85-w>: HTTP status code is not handled or not allowed 2026-01-24 20:17:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-gsc-125-b>: HTTP status code is not handled or not allowed 2026-01-24 20:17:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-gsc-206-ss>: HTTP status code is not handled or not allowed 2026-01-24 20:17:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-disc-series-chair-caster-with-nylon-locking-wheel-stem-type-e>: HTTP status code is not handled or not allowed 2026-01-24 20:17:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/garage_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:17:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m3-0x-50-d3-spiral-pt-plug-chamfer-iso-metric-hss-tmx-hand-tap-ground-rh-2-flutes>: HTTP status code is not handled or not allowed 2026-01-24 20:17:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m5-0-x-80-d4-plug-chamfer-iso-metric-hss-tm-x-hand-tap-ground-rh-4-flutes>: HTTP status code is not handled or not allowed 2026-01-24 20:17:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-g2sc-037-ss>: HTTP status code is not handled or not allowed 2026-01-24 20:17:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/led_bulbs>: HTTP status code is not handled or not allowed 2026-01-24 20:17:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m5x-5-hss-iso-metric-tmx-round-adjustable-die-fine-thread-1-inch-od>: HTTP status code is not handled or not allowed 2026-01-24 20:17:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-press-3-4-ton-pneumatic-arbor-press-w-4-bore-x-6-stroke-cylinder-mounted>: HTTP status code is not handled or not allowed 2026-01-24 20:17:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/venture-patio-umbrella-with-silver-pole-132w-sunbrella-1a-natural-fabric>: HTTP status code is not handled or not allowed 2026-01-24 20:17:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-gal-lab-pack-with-screw-top-lid>: HTTP status code is not handled or not allowed 2026-01-24 20:17:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-hand-ctrl-w-cv-100-washdown>: HTTP status code is not handled or not allowed 2026-01-24 20:17:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sts030134-3-hp-oil-less-scroll-comp-30-gal-horiz-116-psi-3-ph-208-230v-refrigerated-dryer>: HTTP status code is not handled or not allowed 2026-01-24 20:17:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/cabinets/garage_cabinets>: HTTP status code is not handled or not allowed 2026-01-24 20:17:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-g2sc-218-b>: HTTP status code is not handled or not allowed 2026-01-24 20:17:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/residential_decorative_lighting/recessed_lighting already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:17:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/front-brim-hard-hat-sun-shield-universal-fit-a-s3>: HTTP status code is not handled or not allowed 2026-01-24 20:17:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/solid-galvanized-shelf-72x18>: HTTP status code is not handled or not allowed 2026-01-24 20:17:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/molded-black-plastic-hangers-6-per-package>: HTTP status code is not handled or not allowed 2026-01-24 20:17:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-mission-style-lantern-outdoor-vanity-light-fixture-9w-600-lumens-6-18l-black>: HTTP status code is not handled or not allowed 2026-01-24 20:17:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-channel-cable-guard>: HTTP status code is not handled or not allowed 2026-01-24 20:17:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-channel-cable-guard>: HTTP status code is not handled or not allowed 2026-01-24 20:17:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010616-ignition-electrode-incandescence-for-cleveland-range>: HTTP status code is not handled or not allowed 2026-01-24 20:17:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a335-manual-sealless-combination-tool-1-2-w-regular-duty-steel-strapping>: HTTP status code is not handled or not allowed 2026-01-24 20:17:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/465-adhesive-transfer-tape-12-x-60-yds-2-mil-clear>: HTTP status code is not handled or not allowed 2026-01-24 20:17:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/residential_decorative_lighting/recessed_lighting>: HTTP status code is not handled or not allowed 2026-01-24 20:17:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-fiberglass-step-ladder-w-plastic-tool-tray-1>: HTTP status code is not handled or not allowed 2026-01-24 20:17:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010380-thermostat-for-accu-temp>: HTTP status code is not handled or not allowed 2026-01-24 20:17:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-step-24-w-perforated-all-directional-steel-rolling-ladder-w-cal-osha-handrail-cal-pip-7-32p>: HTTP status code is not handled or not allowed 2026-01-24 20:17:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/13-step-steel-safety-rolling-ladder-w-weight-actuated-lock-24-w-expanded-step-orange-w-cal-osha-handrail-cal-wa134014x-o>: HTTP status code is not handled or not allowed 2026-01-24 20:17:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-bin-storage-unit-double-row-with-4-large-8-small-clear-bins-20-lbs-bin-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:17:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-safety-view-locker-triple-tier-12x15x24-9-doors-assembled-tan>: HTTP status code is not handled or not allowed 2026-01-24 20:17:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/venture-patio-umbrella-with-silver-pole-132w-olefin-navy-fabric>: HTTP status code is not handled or not allowed 2026-01-24 20:17:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e545-gfci-duplex-outlet-double-pole-20a-120v-ac-white-pack-of-3>: HTTP status code is not handled or not allowed 2026-01-24 20:17:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shutoff-relief-valve-1-8-npt-inlet-1-4-npt-exhbracket>: HTTP status code is not handled or not allowed 2026-01-24 20:17:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fs-36-h-grn-storage-cabinet-all-welded-36-wx18-dx72-h-hunter-green>: HTTP status code is not handled or not allowed 2026-01-24 20:17:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-step-24-wx83-d-stainless-steel-rolling-safety-ladder-serrated-grating-w-cal-osha-handrail-cal-ss103221g>: HTTP status code is not handled or not allowed 2026-01-24 20:17:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bca-367213-bl-bookcase-all-welded-36-wx13-dx72-h-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:17:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gendouble-black-rolling-basket-55-liter-with-4-swivel-wheels>: HTTP status code is not handled or not allowed 2026-01-24 20:17:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/venture-patio-umbrella-with-silver-pole-132w-sunbrella-1a-canvas-fabric>: HTTP status code is not handled or not allowed 2026-01-24 20:17:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergotron-100-series-pivot-double>: HTTP status code is not handled or not allowed 2026-01-24 20:17:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aah-36rbmag2-c-storage-cabinet-all-welded-36-wx18-dx78-h-charcoal>: HTTP status code is not handled or not allowed 2026-01-24 20:17:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/graphic-signs-danger-hard-hat-area-vinyl-7-w-x-10-h>: HTTP status code is not handled or not allowed 2026-01-24 20:17:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergotron-mouse-holder-light-grey>: HTTP status code is not handled or not allowed 2026-01-24 20:17:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bca-365218-n-bookcase-all-welded-36-wx18-dx52-h-navy>: HTTP status code is not handled or not allowed 2026-01-24 20:17:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regulator-5211d6017-1-npt-0100-psig-regulated-pressure-range-640-scfm>: HTTP status code is not handled or not allowed 2026-01-24 20:17:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-wood-cutting-t-shank-jigsaw-blade-t101b-10-tpi-5-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:17:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-wood-cutting-t-shank-jigsaw-blade-t101d-6-tpi-5-piece>: HTTP status code is not handled or not allowed 2026-01-24 20:17:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fs-42-l-grn-storage-cabinet-all-welded-42-wx18-dx72-h-leaf-green>: HTTP status code is not handled or not allowed 2026-01-24 20:17:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rl136-portion-control-scale-12-inches-x-12-inches-platform-13-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:17:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bronze-hose-braid-w-copper-sweat-ends-2x10-5>: HTTP status code is not handled or not allowed 2026-01-24 20:18:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fixed-tv-wall-mount-with-built-in-level-for-32-in-84-in-tvs>: HTTP status code is not handled or not allowed 2026-01-24 20:18:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-extinguisher-cabinet-surface-mount-fits-2-to-5-lb-extinguisher>: HTTP status code is not handled or not allowed 2026-01-24 20:18:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25-w-x-20-h-x-4-d-pleated-merv-10-high-capacity-air-filter-global-industrial>: HTTP status code is not handled or not allowed 2026-01-24 20:18:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/attix-19-as-e-xc-19-gal-wet-dry-vac-w-tool-start-auto-filter-cleaning>: HTTP status code is not handled or not allowed 2026-01-24 20:18:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-step-easy-turn-rolling-ladder-standard-angle>: HTTP status code is not handled or not allowed 2026-01-24 20:18:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-step-easy-turn-rolling-ladder-safety-angle>: HTTP status code is not handled or not allowed 2026-01-24 20:18:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss-hose-braid-w-male-npt-nipple-0-75-x-24>: HTTP status code is not handled or not allowed 2026-01-24 20:18:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lightweight-aluminum-transport-wheelchair-black-frame-17-inch>: HTTP status code is not handled or not allowed 2026-01-24 20:18:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aah-36rbmag3-n-storage-cabinet-all-welded-36-wx24-dx78-h-navy>: HTTP status code is not handled or not allowed 2026-01-24 20:18:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 18151 pages (at 95 pages/min), scraped 11848 items (at 32 items/min) 2026-01-24 20:18:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-dual-access-fiberglass-step-ladder-2>: HTTP status code is not handled or not allowed 2026-01-24 20:18:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/porous-tape-2-x-10-yds-12-pcs>: HTTP status code is not handled or not allowed 2026-01-24 20:18:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compact-flammable-storage-cabinet-12-gallon-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:18:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-w-x-20-h-x-4-d-pleated-merv-11-high-capacity-air-filter-global-industrial>: HTTP status code is not handled or not allowed 2026-01-24 20:18:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eercise-peddler-with-attractive-silver-vein-finish>: HTTP status code is not handled or not allowed 2026-01-24 20:18:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/infinity-locker-single-tier-12x12x60-1-door-assembled-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:18:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-w-x-16-h-x-2-d-pleated-merv-8-high-capacity-air-filter-global-industrial>: HTTP status code is not handled or not allowed 2026-01-24 20:18:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-step-steel-stock-picking-ladder-expanded-tread-w-cal-osha-handrail-cal-spl-7-14nx>: HTTP status code is not handled or not allowed 2026-01-24 20:18:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-pilaster-w-shoe-22w-x-82h-charcoal>: HTTP status code is not handled or not allowed 2026-01-24 20:18:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-w-x-20-h-x-1-d-paint-booth-ring-panel-merv-8-air-filter-global-industrial>: HTTP status code is not handled or not allowed 2026-01-24 20:18:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5900552-cable-chocker-fixed-310-cap-lbs>: HTTP status code is not handled or not allowed 2026-01-24 20:18:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5901583-flexible-counterweight-for-rope-lifeline>: HTTP status code is not handled or not allowed 2026-01-24 20:18:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-locker-double-tier-12x18x36-with-legs-2-doors-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:18:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-inch-sentra-ec-h-d-detachable-full-arm-dual-cross-brace-elevating-legrest>: HTTP status code is not handled or not allowed 2026-01-24 20:18:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-w-x-20-h-x-4-d-carbon-pleated-merv-6-standard-performance-air-filter-global-industrial>: HTTP status code is not handled or not allowed 2026-01-24 20:18:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zoneaire-select-packaged-terminal-air-conditioner-with-heat-pump-7200-btu-cool-208230v>: HTTP status code is not handled or not allowed 2026-01-24 20:18:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fs-30-c-storage-cabinet-all-welded-30-wx18-dx72-h-charcoal>: HTTP status code is not handled or not allowed 2026-01-24 20:18:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polymer-pilaster-w-shoe-22w-x-82h-azure>: HTTP status code is not handled or not allowed 2026-01-24 20:18:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-outward-swing-partition-door-w-hardware-36w-pepper-dust>: HTTP status code is not handled or not allowed 2026-01-24 20:18:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-pilaster-w-shoe-10w-x-82h-white>: HTTP status code is not handled or not allowed 2026-01-24 20:18:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/web-sling-2-ply-eye-eye-3-long-3-es-wide>: HTTP status code is not handled or not allowed 2026-01-24 20:18:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s601002-wire-rope-sling-3-8-x-3-eye-and-eye-2000-2400-4800-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-24 20:18:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-closed-linear-bearing-wseals-sws16uu-1id-225l>: HTTP status code is not handled or not allowed 2026-01-24 20:18:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-squeegee-rear-a-for-factory-cat-28-754l>: HTTP status code is not handled or not allowed 2026-01-24 20:18:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polymer-pilaster-w-shoe-12w-x-82h-ivory-essence-speckle>: HTTP status code is not handled or not allowed 2026-01-24 20:18:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polymer-pilaster-w-shoe-24w-x-82h-white>: HTTP status code is not handled or not allowed 2026-01-24 20:18:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-inward-swing-partition-door-w-hardware-24w-almond>: HTTP status code is not handled or not allowed 2026-01-24 20:18:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hydraulic-hand-dynamometer-300-lb-analog-output-signal-no-gauge-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:18:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forney-ground-clamp-brass-200a>: HTTP status code is not handled or not allowed 2026-01-24 20:18:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-pad-lok-pad-driver-complete-assembly-for-nobles-tennant-397918-nobles-tennant-1220200>: HTTP status code is not handled or not allowed 2026-01-24 20:18:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-3-1-convertible-hand-truck>: HTTP status code is not handled or not allowed 2026-01-24 20:18:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ball-transfer-table-with-4-centers-5400-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:18:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plain-magnetic-roll-stock-100l-x-3h-x-003-thick-black>: HTTP status code is not handled or not allowed 2026-01-24 20:18:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropelene-strapping-kit-postal-kit-plus-1-2-inches-x-0-018-inches-x-3000-feet-white-4-inches-x-6-inches-core>: HTTP status code is not handled or not allowed 2026-01-24 20:18:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bishamon-mobilift-double-scissor-lift-table-660lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:18:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacment-wheel-for-nilfisk-advance-56315115nilfisk-advance-56315050>: HTTP status code is not handled or not allowed 2026-01-24 20:18:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1833-9-16-12in-oal-hss-h-d-sand-blasted-118-point-carbide-tipped-sds-plus-3-masonry-drill>: HTTP status code is not handled or not allowed 2026-01-24 20:18:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s152075-6-poly-web-sling-endless-9800-12200-24400-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-24 20:18:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b151073-8-single-leg-chain-sling-w-sling-hook>: HTTP status code is not handled or not allowed 2026-01-24 20:18:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-w-x-30-d-x-1-3-4-thick-steel-square-edge-workbench-top-12-gauge-steel-black>: HTTP status code is not handled or not allowed 2026-01-24 20:18:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smedley-spring-dynamometer-adult-220-lb-capacity-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:18:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blue-elevated-plastic-box-truck-6-bushel-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:18:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1833-5-16-12in-oal-hss-h-d-sand-blasted-118-point-carbide-tipped-sds-plus-3-masonry-drill>: HTTP status code is not handled or not allowed 2026-01-24 20:18:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-squeegee-rear-a-for-nobles-tennant-1025341>: HTTP status code is not handled or not allowed 2026-01-24 20:18:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-signs-danger-eye-protection-rigid-plastic-7-h-x-10-w>: HTTP status code is not handled or not allowed 2026-01-24 20:18:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leverset-single-step-roses-passage-lock-plsh-brass>: HTTP status code is not handled or not allowed 2026-01-24 20:18:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/digital-door-lock-2200-surface-rim-mount-with-key-override-jet-black>: HTTP status code is not handled or not allowed 2026-01-24 20:18:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-pilaster-w-shoe-7w-x-82h-black>: HTTP status code is not handled or not allowed 2026-01-24 20:18:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-brush-kit-mal-grit-xtra-for-nilfisk-advance-56505771-nilfisk-advance-56505801>: HTTP status code is not handled or not allowed 2026-01-24 20:18:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1821-3-8-6in-oal-hss-h-d-sand-blasted-118-point-carbide-tipped-sds-plus-2-masonry-drill>: HTTP status code is not handled or not allowed 2026-01-24 20:18:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-climax-2c-250>: HTTP status code is not handled or not allowed 2026-01-24 20:18:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-signs-danger-area-rigid-plastic-10-h-x-14-w>: HTTP status code is not handled or not allowed 2026-01-24 20:18:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-wall-exhaust-shutter-rear-flange-14-eas-mf-14>: HTTP status code is not handled or not allowed 2026-01-24 20:18:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-industry-standard-clamping-couplings-climax-2iscc-125-100-s>: HTTP status code is not handled or not allowed 2026-01-24 20:18:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b151093-8-double-leg-chain-sling-w-grab-hook>: HTTP status code is not handled or not allowed 2026-01-24 20:18:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-brush-kit-for-factory-cat-14-421c>: HTTP status code is not handled or not allowed 2026-01-24 20:18:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1872-7-hss-heavy-duty-black-gold-135-hss-heavy-duty-parabolic-jobber-length-drill>: HTTP status code is not handled or not allowed 2026-01-24 20:18:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/post-floor-glide-package-of-4>: HTTP status code is not handled or not allowed 2026-01-24 20:18:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-inch-rubber-overboots-mens-black-xl-1-pair>: HTTP status code is not handled or not allowed 2026-01-24 20:19:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1889-3-8-4in-oal-hss-heavy-duty-bright-118-point-fast-helix-carbide-tipped-masonry-drill>: HTTP status code is not handled or not allowed 2026-01-24 20:19:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-headrail-w-screws-polymer-partitions-120>: HTTP status code is not handled or not allowed 2026-01-24 20:19:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-safe-step-on-plastic-container-23-gallon-red-fg614600red>: HTTP status code is not handled or not allowed 2026-01-24 20:19:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wide-mouth-straight-sided-ppco-jars-with-closure-125ml-case-of-36>: HTTP status code is not handled or not allowed 2026-01-24 20:19:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-with-keyway-2c-kw-series-climax-2c-150-kw>: HTTP status code is not handled or not allowed 2026-01-24 20:19:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/three-sided-frame-18x30-nexelon>: HTTP status code is not handled or not allowed 2026-01-24 20:19:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rectangular-hdpe-carboy-with-steel-handle-20-liter-case-of-4>: HTTP status code is not handled or not allowed 2026-01-24 20:19:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-safegrip153-footwear-traction-aid-s-black>: HTTP status code is not handled or not allowed 2026-01-24 20:19:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dunnage-shelf-18x30-2>: HTTP status code is not handled or not allowed 2026-01-24 20:19:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dunnage-shelf-24x48-nexelon>: HTTP status code is not handled or not allowed 2026-01-24 20:19:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacment-vacuum-bag-for-nilfisk-advance-107413584-nilfisk-advance-1471058500>: HTTP status code is not handled or not allowed 2026-01-24 20:19:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1875r-5-32-hss-h-d-black-gold-135-split-point-3-flatted-shank-mechanics-length-drill>: HTTP status code is not handled or not allowed 2026-01-24 20:19:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-squeegee-rear-a-for-ipc-eagle-mpvr08035>: HTTP status code is not handled or not allowed 2026-01-24 20:19:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1501-tt-9-32-cobalt-heavy-duty-tin-tipped-135-split-point-jobber-length-drill>: HTTP status code is not handled or not allowed 2026-01-24 20:19:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 18250 pages (at 99 pages/min), scraped 11874 items (at 26 items/min) 2026-01-24 20:19:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-vac-motor-gasket-for-ipc-eagle-gugo00272-windsor-86003760>: HTTP status code is not handled or not allowed 2026-01-24 20:19:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-cylindrical-brush-18-sr-for-nobles-tennant-222304>: HTTP status code is not handled or not allowed 2026-01-24 20:19:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polymer-pilaster-w-shoe-3w-x-82h-moss>: HTTP status code is not handled or not allowed 2026-01-24 20:19:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacment-vacuum-bag-for-nilfisk-advance-56391185>: HTTP status code is not handled or not allowed 2026-01-24 20:19:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pob-g-oval-fast-food-baskets>: HTTP status code is not handled or not allowed 2026-01-24 20:19:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polymer-partition-panel-wo-brackets-5912w-burgundy>: HTTP status code is not handled or not allowed 2026-01-24 20:19:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-cotton-flame-resistant-henley-shirt-2xl-hi-vis-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:19:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/550-3-32-cobalt-heavy-duty-straw-135-split-point-jobber-length-drill>: HTTP status code is not handled or not allowed 2026-01-24 20:19:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tkps-7-bakery-spatulas-7-3-4l-blade-black-polypropylene-handle>: HTTP status code is not handled or not allowed 2026-01-24 20:19:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tmt-cdf3-dial-deep-fryer-thermometer>: HTTP status code is not handled or not allowed 2026-01-24 20:19:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pgw-1018-pan-grate>: HTTP status code is not handled or not allowed 2026-01-24 20:19:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40903-2x-dog-life-vest-yellow-2x-large>: HTTP status code is not handled or not allowed 2026-01-24 20:19:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-le-garolite-sheet-1-4-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 20:19:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-x-8-x-4-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 20:19:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/psh-14-scraper-600f-heat-resistant-14l-silicone>: HTTP status code is not handled or not allowed 2026-01-24 20:19:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-carbon-fiber-sheet-twill-weave-1-16-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 20:19:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/polymer_bathroom_partitions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:19:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/psm-mw-medium-well-steak-markers>: HTTP status code is not handled or not allowed 2026-01-24 20:19:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-x-12-x-9-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 20:19:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-x-12-x-20-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 20:19:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polymer-partition-panel-wo-brackets-5912w-olive>: HTTP status code is not handled or not allowed 2026-01-24 20:19:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-two-piece-industry-standard-clamping-couplings-climax-2miscc-35-35>: HTTP status code is not handled or not allowed 2026-01-24 20:19:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/phenolic-black-core-partition-panel-wo-brackets-5912w-almond>: HTTP status code is not handled or not allowed 2026-01-24 20:19:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-industry-standard-clamping-couplings-w-keyway-climax-2iscc062-050skw>: HTTP status code is not handled or not allowed 2026-01-24 20:19:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/13-1-2-x-13-1-2-x-7-1-2-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 20:19:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ratchet-trailer-stabilizing-jack-100-100000-lb-static-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:19:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/protective-hat-staz-on-suspension-lamp-bracket-cord-holder-std-natural-tan>: HTTP status code is not handled or not allowed 2026-01-24 20:19:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/abc-industries-thermohose-1399-heater-duct-wire-rope-exit-20-dia-x-25l-x-6-pitch-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:19:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/abc-industries-thermohose-1399-heater-duct-16-dia-x-16l-x-6-pitch-yellowblack>: HTTP status code is not handled or not allowed 2026-01-24 20:19:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plp-5w-perforated-bus-box>: HTTP status code is not handled or not allowed 2026-01-24 20:19:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pmb-18-magnetic-bar>: HTTP status code is not handled or not allowed 2026-01-24 20:19:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-couplings-recessed-screw-climax-cc-100-075>: HTTP status code is not handled or not allowed 2026-01-24 20:19:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/polymer_bathroom_partitions>: HTTP status code is not handled or not allowed 2026-01-24 20:19:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thermoplastic-shallow-well-jet-pump-3-4-hp>: HTTP status code is not handled or not allowed 2026-01-24 20:19:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/accessory-forestry-kit>: HTTP status code is not handled or not allowed 2026-01-24 20:19:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disinfectant-bathroom-cleaners-liquid-island-breeze-22-oz-trigger-spray-bottle-6carton>: HTTP status code is not handled or not allowed 2026-01-24 20:19:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o-ir-higher-performance-polyolefin-cf-shrink-film-45-ga-20w-x-5830l-clear-1-roll>: HTTP status code is not handled or not allowed 2026-01-24 20:19:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-climax-2c-075-a>: HTTP status code is not handled or not allowed 2026-01-24 20:19:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-scrubber-12carton>: HTTP status code is not handled or not allowed 2026-01-24 20:19:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/squids-3760-phone-pouch-and-trap-clear-standard-size>: HTTP status code is not handled or not allowed 2026-01-24 20:19:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cast-iron-centrifugal-pump-1-2-hp>: HTTP status code is not handled or not allowed 2026-01-24 20:19:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/staz-on-skullgard-cap-or-hat-suspension-staz-on-bag>: HTTP status code is not handled or not allowed 2026-01-24 20:19:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-laminate-pilaster-w-shoe-24w-x-82h-khaki-brown>: HTTP status code is not handled or not allowed 2026-01-24 20:19:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/visor-pc-shade-3-ir-molded-8-inch-w-x-17-inch-l-x-098-inch-h>: HTTP status code is not handled or not allowed 2026-01-24 20:19:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-laminate-pilaster-w-shoe-7w-x-82h-mission-white>: HTTP status code is not handled or not allowed 2026-01-24 20:19:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/squids-3184-concrete-finishers-and-masons-tool-tethering-kit>: HTTP status code is not handled or not allowed 2026-01-24 20:19:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/skullerz-8955-lightweight-bump-cap-long-brim-navy>: HTTP status code is not handled or not allowed 2026-01-24 20:19:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o-ir-higher-performance-polyolefin-cf-shrink-film-60-ga-18w-x-4375l-clear-1-roll>: HTTP status code is not handled or not allowed 2026-01-24 20:19:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sct-10f-stackable-creamer-w-flat-cover-10-oz-stainless-steel-stackable>: HTTP status code is not handled or not allowed 2026-01-24 20:19:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-with-keyway-2c-kw-series-climax-2c-125-kw>: HTTP status code is not handled or not allowed 2026-01-24 20:19:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mega-scoop-128-oz-orange>: HTTP status code is not handled or not allowed 2026-01-24 20:19:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pprc-1c-pp-round-cover-polypropelyne-translucent>: HTTP status code is not handled or not allowed 2026-01-24 20:19:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v-gard-h2-mips-safety-helmet-vented-climbing-class-c-type-2-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:19:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-laminate-pilaster-w-shoe-8w-x-82h-dove-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:19:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hand-scoop-32-oz-orange>: HTTP status code is not handled or not allowed 2026-01-24 20:19:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/optional-adjustable-height-shelf-for-rc2963-nashville-wire-cart>: HTTP status code is not handled or not allowed 2026-01-24 20:19:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/21-inch-w-stacking-church-chair-fabric-brown-hercules-series>: HTTP status code is not handled or not allowed 2026-01-24 20:19:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/plastic_laminate_bathroom_partitions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:19:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-industry-standard-clamping-couplings-w-keyway-climax-2iscc-075-075kw>: HTTP status code is not handled or not allowed 2026-01-24 20:19:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/patriot-fully-framed-ta-50-locker-wsolid-doors-sides-36wx24dx78h-grayunassembled>: HTTP status code is not handled or not allowed 2026-01-24 20:19:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-laminate-outward-swing-door-w-hardware-26w-graphite-grafix>: HTTP status code is not handled or not allowed 2026-01-24 20:19:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-laminate-inward-swing-door-w-hardware-26w-almond>: HTTP status code is not handled or not allowed 2026-01-24 20:20:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pmc-14k-single-menu-cover-8-1-2w-14h-plastic-w-vinyl-binding-single-page-black>: HTTP status code is not handled or not allowed 2026-01-24 20:20:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/phenolic-black-core-outward-swing-partition-door-w-hardware-26w-graphite-grafix>: HTTP status code is not handled or not allowed 2026-01-24 20:20:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-5-sprinkler-water-pressure-gauge-1-4-npt-0-300-psi-lower-mount-sprinkler-water>: HTTP status code is not handled or not allowed 2026-01-24 20:20:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-x-5-x-5-3-multi-depth-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 20:20:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-5-forged-brass-pressure-gauge-1-4-npt-0-5000-psi-glycerine-filled-lm-601l-254r>: HTTP status code is not handled or not allowed 2026-01-24 20:20:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-linen-le-phenolic-sheet-0-093-x-36-000-x-48-000>: HTTP status code is not handled or not allowed 2026-01-24 20:20:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-climax-1c-287-z>: HTTP status code is not handled or not allowed 2026-01-24 20:20:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/plastic_laminate_bathroom_partitions>: HTTP status code is not handled or not allowed 2026-01-24 20:20:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/80-grit-sand-paper-for-use-with-6-5in-pad-holder-493317-50-pack-493236>: HTTP status code is not handled or not allowed 2026-01-24 20:20:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-climax-2c-243-s>: HTTP status code is not handled or not allowed 2026-01-24 20:20:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tkp-42-turner-4l-blade-8-w-black-polypropylene-handle>: HTTP status code is not handled or not allowed 2026-01-24 20:20:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-lb-machine-weight-for-16in-surfacing-machines-491691>: HTTP status code is not handled or not allowed 2026-01-24 20:20:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 18345 pages (at 95 pages/min), scraped 11897 items (at 23 items/min) 2026-01-24 20:20:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-canvas-ce-phenolic-sheet-0-750-x-36-000-x-48-000>: HTTP status code is not handled or not allowed 2026-01-24 20:20:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-laminate-outward-swing-door-w-hardware-24w-storm-solidz>: HTTP status code is not handled or not allowed 2026-01-24 20:20:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-x-2-solid-rubber-wheel-sl10-34-axel-size-3-4>: HTTP status code is not handled or not allowed 2026-01-24 20:20:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-x-24-kra-crimped-end-tubes-case>: HTTP status code is not handled or not allowed 2026-01-24 20:20:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-duty-tilt-truck-w-handle-1-cu-yd-cap-1200-lb-cap-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:20:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/patriot-fully-framed-ta-50-locker-wexpanded-metal-sides-36wx24dx78h-grunassembled>: HTTP status code is not handled or not allowed 2026-01-24 20:20:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tos-7-blade-offset-spatula-7-3-4l-blade-wooden-handle>: HTTP status code is not handled or not allowed 2026-01-24 20:20:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pmsb-4-molcajete-salsa-bowls-4-oz-plastic-resin>: HTTP status code is not handled or not allowed 2026-01-24 20:20:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/equipment-stand-w-adjustable-undershelf-30-x-36-24-high-ss-top-w-utensil-rack>: HTTP status code is not handled or not allowed 2026-01-24 20:20:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stretch-wrap-film-for-hand-dispenser-blown-120-gauge-18w-x-1000l-clear-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-24 20:20:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/work-table-24-wide-top-108-long-16-gauge-1-12-backsplash>: HTTP status code is not handled or not allowed 2026-01-24 20:20:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-xxx-paper-phenolic-rod-1-750-dia-x-48-000-inch>: HTTP status code is not handled or not allowed 2026-01-24 20:20:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-set-screw-collar-climax-mc-24>: HTTP status code is not handled or not allowed 2026-01-24 20:20:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sbs-24-silicone-baking-mat-24-1-2l-16-5-8w-rectangular>: HTTP status code is not handled or not allowed 2026-01-24 20:20:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-coupling-w-keyway-climax-rc-050-kw>: HTTP status code is not handled or not allowed 2026-01-24 20:20:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-x-18-x-18-white-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 20:20:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/banding-stretch-wrap-with-reusable-dispenser-cast-150-gauge-5w-x-600l-clear-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-24 20:20:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-set-screw-collar-climax-mc-32-s>: HTTP status code is not handled or not allowed 2026-01-24 20:20:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wsecurity-bxfootlockervented18x18x76blall-welded>: HTTP status code is not handled or not allowed 2026-01-24 20:20:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-two-piece-clamping-collar-climax-m2c-17-s>: HTTP status code is not handled or not allowed 2026-01-24 20:20:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0030-05-shangarila-dinner-fork>: HTTP status code is not handled or not allowed 2026-01-24 20:20:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-set-screw-collar-climax-mc-36>: HTTP status code is not handled or not allowed 2026-01-24 20:20:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stretch-wrap-film-for-hand-dispenser-cast-80-gauge-18w-x-1500l-red-tint-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-24 20:20:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelf-security-box-24wx21dx72h-gray-all-welded>: HTTP status code is not handled or not allowed 2026-01-24 20:20:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-7-8-x-9-7-8-stock-pad>: HTTP status code is not handled or not allowed 2026-01-24 20:20:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:20:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-couplings-recessed-screw-cc-series-climax-2cc-100-075>: HTTP status code is not handled or not allowed 2026-01-24 20:20:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelfsecurity-boxvented24wx24dx72hblall-welded>: HTTP status code is not handled or not allowed 2026-01-24 20:20:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0385l2-hivis-iron-tuff-high-bib-overall-regular-hivis-orange-5xl>: HTTP status code is not handled or not allowed 2026-01-24 20:20:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rx-500c-1-0-rx-500-magnifier-safety-glasses-1-0-magnifier-clear-lens>: HTTP status code is not handled or not allowed 2026-01-24 20:20:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-paddle-36-in-long>: HTTP status code is not handled or not allowed 2026-01-24 20:20:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelf-footlocker-vented-24wx21dx78hblall-welded>: HTTP status code is not handled or not allowed 2026-01-24 20:20:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vanguard-series-gas-engine-3-532-vertical-shaft-26-hp>: HTTP status code is not handled or not allowed 2026-01-24 20:20:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-x-3-x-48-225-edge-protectors-cased>: HTTP status code is not handled or not allowed 2026-01-24 20:20:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wsecurity-box-footlocker-24wx21dx76hrdall-welded>: HTTP status code is not handled or not allowed 2026-01-24 20:20:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-z-link-maginfying-lens-holder>: HTTP status code is not handled or not allowed 2026-01-24 20:20:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-poly-bags-8x28-4-mil-500-ctn>: HTTP status code is not handled or not allowed 2026-01-24 20:20:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 20:20:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-sided-bulk-truck-slat-sides-2-shlvs-lip-3600-lbs-cap-24-60-poly-wheels>: HTTP status code is not handled or not allowed 2026-01-24 20:20:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cylindrical-roller-brush-for-kira-b50-b-60-w-bp-hard-green>: HTTP status code is not handled or not allowed 2026-01-24 20:20:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-2-inch-fr-belt-and-buckle>: HTTP status code is not handled or not allowed 2026-01-24 20:20:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelf-footlocker-vented-24wx21dx72hgrall-welded>: HTTP status code is not handled or not allowed 2026-01-24 20:20:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jtm-1254vs-with-2-axis-acu-rite-g-2-millpower-cnc>: HTTP status code is not handled or not allowed 2026-01-24 20:20:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tb600-stainless-steel-hydraulic-gate-closer>: HTTP status code is not handled or not allowed 2026-01-24 20:20:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tb-175-garden-gate-closer-black>: HTTP status code is not handled or not allowed 2026-01-24 20:20:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0021-09-continental-demitasse-spoon>: HTTP status code is not handled or not allowed 2026-01-24 20:20:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelf-security-box-24wx18dx72h-blue-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 20:20:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pb1100al-panic-bar-ul-listed-aluminum>: HTTP status code is not handled or not allowed 2026-01-24 20:20:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-co-monitor-50-cable-assembly-plug>: HTTP status code is not handled or not allowed 2026-01-24 20:20:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-one-piece-industry-standard-clamping-couplings-w-keyway-climax-miscc-14-14-skw>: HTTP status code is not handled or not allowed 2026-01-24 20:20:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s2172s-stainless-steel-wire-shelf-72w-x-21d>: HTTP status code is not handled or not allowed 2026-01-24 20:20:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ghs-corrosive-pictogram-label-4-x-4-500-roll>: HTTP status code is not handled or not allowed 2026-01-24 20:20:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-radex-filter-cartridge>: HTTP status code is not handled or not allowed 2026-01-24 20:20:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-coupling-w-keyway-climax-rc-125-kw>: HTTP status code is not handled or not allowed 2026-01-24 20:20:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interion-174-36-round-steel-mesh-outdoor-caf-table-bronze>: HTTP status code is not handled or not allowed 2026-01-24 20:20:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ghs-skull-crossbones-pictogram-placard-10-3-4-x-10-3-4>: HTTP status code is not handled or not allowed 2026-01-24 20:20:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/btm-16ac-microfiber-towels>: HTTP status code is not handled or not allowed 2026-01-24 20:20:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vertical-7-inch-reel-storage-container-37568-8l-x-8w-x-9h-black>: HTTP status code is not handled or not allowed 2026-01-24 20:20:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/azar-252377-four-tier-brochure-holder-9-1-4-w-x-7-d-x-13-1-4-h>: HTTP status code is not handled or not allowed 2026-01-24 20:20:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-14x48-chrome>: HTTP status code is not handled or not allowed 2026-01-24 20:20:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-set-screw-collar-climax-mc-025>: HTTP status code is not handled or not allowed 2026-01-24 20:20:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paddle-sign-stop-slow-telescoping-handle-ps6>: HTTP status code is not handled or not allowed 2026-01-24 20:20:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dxs2153-metric-standard-impact-socket-15mm-drive-black>: HTTP status code is not handled or not allowed 2026-01-24 20:21:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rigid-coupling-4-set-screws-2-90-rc4h-series-climax-rc-137-kw4h90>: HTTP status code is not handled or not allowed 2026-01-24 20:21:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rigid-coupling-4-set-screws-2-90-rc4h-series-climax-rc-175-kw4h90>: HTTP status code is not handled or not allowed 2026-01-24 20:21:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-cu-yd-yellow-self-dumping-forkli-hopper>: HTTP status code is not handled or not allowed 2026-01-24 20:21:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hardwood-laminate-floor-cleaner-gallon-bottle-4-bottles-case-1041692>: HTTP status code is not handled or not allowed 2026-01-24 20:21:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alo-16-chinese-style-colander-16-qt-16d-aluminum>: HTTP status code is not handled or not allowed 2026-01-24 20:21:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-threaded-clamping-collar-recessed-screw-climax-tc-112-12>: HTTP status code is not handled or not allowed 2026-01-24 20:21:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vertical-back-metal-restaurant-barstool-black-vinyl-seat>: HTTP status code is not handled or not allowed 2026-01-24 20:21:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-l4-light-w-strap-mount-unit>: HTTP status code is not handled or not allowed 2026-01-24 20:21:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mold-stain-mildew-stain-remover-gallon-bottle-4-bottles-case-1041694>: HTTP status code is not handled or not allowed 2026-01-24 20:21:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-partition-flexible-eleven-panel-30>: HTTP status code is not handled or not allowed 2026-01-24 20:21:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0341-iron-tuff-inspector-coat-regular-navy-xl>: HTTP status code is not handled or not allowed 2026-01-24 20:21:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-l-x-30-w-16-gauge-stainless-steel-workbench-w-undershelf-le-sink>: HTTP status code is not handled or not allowed 2026-01-24 20:21:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wok-16n-wok-w-riveted-handle-16-d-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 20:21:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0925-econo-tuff-jacket-regular-navy-5xl>: HTTP status code is not handled or not allowed 2026-01-24 20:21:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 18442 pages (at 97 pages/min), scraped 11918 items (at 21 items/min) 2026-01-24 20:21:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-flow-control-valve>: HTTP status code is not handled or not allowed 2026-01-24 20:21:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/re-machinable-couplings-r2cc-series-climax-r2cc-075-075-s>: HTTP status code is not handled or not allowed 2026-01-24 20:21:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/95-gallon-rubbermaid-large-mobile-waste-receptacle-gray-with-lid>: HTTP status code is not handled or not allowed 2026-01-24 20:21:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-t150-inhalation-valves>: HTTP status code is not handled or not allowed 2026-01-24 20:21:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-climax-c-200-bo>: HTTP status code is not handled or not allowed 2026-01-24 20:21:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-water-filter-cartridge>: HTTP status code is not handled or not allowed 2026-01-24 20:21:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lifting-builders-hoist-hanger-for-oz05pbta-obh-500-1000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:21:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-px5-fan-housing>: HTTP status code is not handled or not allowed 2026-01-24 20:21:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-px5-battery-door-with-hinge>: HTTP status code is not handled or not allowed 2026-01-24 20:21:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40x32x34-1-2-blue-nylon-cover>: HTTP status code is not handled or not allowed 2026-01-24 20:21:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wwb-6-woven-salad-bowl>: HTTP status code is not handled or not allowed 2026-01-24 20:21:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stingray-5-refillable-system-kit-50-1-2-handle-black-srkb2>: HTTP status code is not handled or not allowed 2026-01-24 20:21:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-recessed-screw-climax-h1c-293-s>: HTTP status code is not handled or not allowed 2026-01-24 20:21:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-inch-x-5-yards-self-adhesive-bandage-non-latex-white-pack-of-20>: HTTP status code is not handled or not allowed 2026-01-24 20:21:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lightweight-set-screw-collar-l-climax-lc-100>: HTTP status code is not handled or not allowed 2026-01-24 20:21:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cpr-face-shield-refill-0155-0262>: HTTP status code is not handled or not allowed 2026-01-24 20:21:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fft-1826-plastic-sheet-tray>: HTTP status code is not handled or not allowed 2026-01-24 20:21:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jlh-900wo-10-lever-hoist-with-overload-protection-9-ton-capacity-10-lift>: HTTP status code is not handled or not allowed 2026-01-24 20:21:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-low-flow-industrial-spray-nozzle-black-brass-adapter>: HTTP status code is not handled or not allowed 2026-01-24 20:21:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n11s-swivel-hose-adapter-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 20:21:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/twp-41-steak-burger-turner-4l-blade-3-3-4w-white-polypropylene-handle>: HTTP status code is not handled or not allowed 2026-01-24 20:21:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0211-value-string-knit-liner-natural-large>: HTTP status code is not handled or not allowed 2026-01-24 20:21:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/digital-temperature-controller-a421gbf-02c-24-vac-spdt-nema-1>: HTTP status code is not handled or not allowed 2026-01-24 20:21:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bstop-08bp-bleed-stop-active-shooter-backpack-system-multi-use-multi-injury>: HTTP status code is not handled or not allowed 2026-01-24 20:21:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/respiratory_protection/powered_air_purifying_respirator_systems already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:21:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-x-30-esd-safety-edge-top-pedestal-workbench-w-4-drawers>: HTTP status code is not handled or not allowed 2026-01-24 20:21:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s90-150-20-hand-chain-hoist-1-12-ton-capacity-20-lift>: HTTP status code is not handled or not allowed 2026-01-24 20:21:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-recessed-screw-climax-h2c-100-s>: HTTP status code is not handled or not allowed 2026-01-24 20:21:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-inch-x-5-yards-self-adhesive-bandage-non-latex-dark-purple>: HTTP status code is not handled or not allowed 2026-01-24 20:21:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motorola-cls-series-2-way-radio-4-channel>: HTTP status code is not handled or not allowed 2026-01-24 20:21:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glc-01010-pre-cleaned-clean-glass-screw-thread-sample-vials-8-dram-30ml-case-of-144>: HTTP status code is not handled or not allowed 2026-01-24 20:21:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-ft-by-16-ft-stage-package-24-inch-height-black-carpet-box-pleat-black-skirting>: HTTP status code is not handled or not allowed 2026-01-24 20:21:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1101m-deluxe-filterregulatorlubricator-metal-bowl-38-npt-250-psi-40-micron>: HTTP status code is not handled or not allowed 2026-01-24 20:21:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/respiratory_protection/powered_air_purifying_respirator_systems>: HTTP status code is not handled or not allowed 2026-01-24 20:21:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-inch-x-5-yards-self-adhesive-bandage-non-latex-black-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-24 20:21:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/epdm-rubber-strip-no-adhesive-60a-14-thick-x-14-wide-x-10-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 20:21:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1111-5piggyback-filterregulatorpolycarbonate-bowl-wmetal-guard12-npt150-psi40-micron>: HTTP status code is not handled or not allowed 2026-01-24 20:21:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kobi-32-in-tall-planter-white>: HTTP status code is not handled or not allowed 2026-01-24 20:21:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-locker-single-tier-12x12x72-without-legs-1-door-assembled-sand>: HTTP status code is not handled or not allowed 2026-01-24 20:21:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/utph-9b-utility-tong-heat-resistant-heat-resistant-9l-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:21:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1308200-universal-blade-guide>: HTTP status code is not handled or not allowed 2026-01-24 20:21:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clw11-1205wmbrp-led-wall-pack-120w-16800-lumens-5000k>: HTTP status code is not handled or not allowed 2026-01-24 20:21:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:21:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/23152-cut-off-wheel-type-1-saitech-3-inch-x-035-inch-x-1-4-inch-ceramic-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-24 20:21:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/23105-cut-off-wheel-type-1-a60s-6-inch-x-045-inch-x-5-8-inch-60-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-24 20:21:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gh-2280zx-large-spindle-bore-lathe-10-hp-3ph-230460v>: HTTP status code is not handled or not allowed 2026-01-24 20:21:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cd-64-coffee-decanter-64-oz-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 20:21:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1304005-meyer-angle-cylinder-05810>: HTTP status code is not handled or not allowed 2026-01-24 20:21:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-tier-shelving-starter-unit-solid-galvanized-steel-54w-x-18d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 20:21:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-threaded-clamping-collar-climax-istc-062-18>: HTTP status code is not handled or not allowed 2026-01-24 20:21:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ut-16hp-b-utility-tong-cold-food-service-cold-food-service-16l-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:21:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mbm-g-deluxe-neon-marker-green>: HTTP status code is not handled or not allowed 2026-01-24 20:21:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips landed on page that is not a product page. 2026-01-24 20:21:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-tier-shelving-add-on-unit-solid-galvanized-steel-42w-x-18d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 20:21:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/combination_metalworking_machines already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:21:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrosion-resistant-set-screw-collar-cr-climax-crc-043-s>: HTTP status code is not handled or not allowed 2026-01-24 20:21:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/viton-rubber-roll-no-adhesive-75a-332-thick-x-36-wide-x-7-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 20:21:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/room-divider-6-height-7-sections-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:21:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50236-7a-s-fiber-disc-7-inch-x-7-8-inch-50-grit-ceramic>: HTTP status code is not handled or not allowed 2026-01-24 20:21:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/37902-4s-disc-roll-6-inch-dia-120-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-24 20:22:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/70110-2a-flap-wheel-3-inch-x-2-inch-x-1-4-inch-40-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-24 20:22:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-hardware-kit-for-rubbermaid-utility-carts>: HTTP status code is not handled or not allowed 2026-01-24 20:22:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-double-wide-climax-d2c-075>: HTTP status code is not handled or not allowed 2026-01-24 20:22:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-cord-spring-rewind-reels-duplex-industrial-receptacle-35-cord-12-awg>: HTTP status code is not handled or not allowed 2026-01-24 20:22:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reducer-sleeve-2-to-3>: HTTP status code is not handled or not allowed 2026-01-24 20:22:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:22:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/splice-fitting-3-to-3>: HTTP status code is not handled or not allowed 2026-01-24 20:22:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-pocket-poly-portfolio-folder-with-3-hole-punch-yellow-25-set>: HTTP status code is not handled or not allowed 2026-01-24 20:22:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/combination_metalworking_machines landed on page that is not a product page. 2026-01-24 20:22:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-handle-for-rubbermaid-cleaning-cart>: HTTP status code is not handled or not allowed 2026-01-24 20:22:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnetic-safety-striped-document-frame-assorted-9-x-12-16-box>: HTTP status code is not handled or not allowed 2026-01-24 20:22:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quick-adjust-wire-shelving-48x14x54>: HTTP status code is not handled or not allowed 2026-01-24 20:22:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 18536 pages (at 94 pages/min), scraped 11944 items (at 26 items/min) 2026-01-24 20:22:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quick-adjust-wire-shelving-36x14x74>: HTTP status code is not handled or not allowed 2026-01-24 20:22:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-climax-c-181>: HTTP status code is not handled or not allowed 2026-01-24 20:22:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/l-322cn-carbon-hook-swivel-w-latch-7-1-2t-wll>: HTTP status code is not handled or not allowed 2026-01-24 20:22:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-climax-c-068-dt>: HTTP status code is not handled or not allowed 2026-01-24 20:22:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls>: HTTP status code is not handled or not allowed 2026-01-24 20:22:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-hole-set-screw-collar-c-2h-series-climax-c-075-2h-90>: HTTP status code is not handled or not allowed 2026-01-24 20:22:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-roll-no-adhesive-60a-116-thick-x-36-wide-x-5-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 20:22:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-brake-w-4-way-die-for-open-cavity-65t-120t-11065t-elite-ironworkers>: HTTP status code is not handled or not allowed 2026-01-24 20:22:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/body-harness-pass-thru-leg-strap>: HTTP status code is not handled or not allowed 2026-01-24 20:22:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pcb-8-8-compartment-cutlery-basket-with-handle>: HTTP status code is not handled or not allowed 2026-01-24 20:22:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-cord-spring-rewind-reels-12-awg-75-less-cord-accessory>: HTTP status code is not handled or not allowed 2026-01-24 20:22:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22380-depressed-center-wheel-t27-ultimate-cut-4-1-2-inch-x-045-inch-x-7-8-inch-blended>: HTTP status code is not handled or not allowed 2026-01-24 20:22:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50231-ao-fiber-disc-4-1-2-inch-x-7-8-inch-50-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-24 20:22:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-recessed-screw-climax-h2c-162-s>: HTTP status code is not handled or not allowed 2026-01-24 20:22:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-recessed-screw-climax-h2c-168>: HTTP status code is not handled or not allowed 2026-01-24 20:22:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rustic-z-frame-wardrobe-black-48-x-17-3-16-x-68>: HTTP status code is not handled or not allowed 2026-01-24 20:22:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dolly-17-7-8x-10-5-8-x-5-tote-red>: HTTP status code is not handled or not allowed 2026-01-24 20:22:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cantilever-rack-straight-arm-48-with-lip-300-pound>: HTTP status code is not handled or not allowed 2026-01-24 20:22:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/bins_totes_containers/baskets_organizers/baskets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:22:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-industry-standard-clamping-couplings-climax-iscc-112-100-s>: HTTP status code is not handled or not allowed 2026-01-24 20:22:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/bins_totes_containers/baskets_organizers/closet_garment_organizers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:22:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/locking-hitch-pin-for-all-terrain-vise153>: HTTP status code is not handled or not allowed 2026-01-24 20:22:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-strip-with-acrylic-adhesive-50a-12-thick-x-2-wide-x-5-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 20:22:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g-429-galvanized-clip-fist-grip-1>: HTTP status code is not handled or not allowed 2026-01-24 20:22:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/din-580-lifting-eye-bolts-stainless-steel-shoulder-matte-m8-1-7-16-h-5-16-t-1-2-l>: HTTP status code is not handled or not allowed 2026-01-24 20:22:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leeson-single-phase-general-purpose-motor-50hz-1-2hp-37kw-1425rpm-56-ip54-manual-c-face>: HTTP status code is not handled or not allowed 2026-01-24 20:22:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-rack-tear-drop-beam-pair-96-l-13>: HTTP status code is not handled or not allowed 2026-01-24 20:22:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-ring-economy-vinyl-view-binder-1-1-2-capacity-black>: HTTP status code is not handled or not allowed 2026-01-24 20:22:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n-7328-closet-door-finger-pull-4-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:22:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/din-508-t-slot-nuts-steel-blackened-m6-1-2-w-x-3-8-h>: HTTP status code is not handled or not allowed 2026-01-24 20:22:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/binders_covers/binders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:22:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tabloid-clearvue-slant-d-ring-binder-3-capacity-11-x-17-white>: HTTP status code is not handled or not allowed 2026-01-24 20:22:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:22:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/bins_totes_containers/baskets_organizers/baskets>: HTTP status code is not handled or not allowed 2026-01-24 20:22:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/bins_totes_containers/baskets_organizers/closet_garment_organizers>: HTTP status code is not handled or not allowed 2026-01-24 20:22:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speed-bump-with-cable-protection-6-foot-long>: HTTP status code is not handled or not allowed 2026-01-24 20:22:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-high-capacity-bulk-rack-with-wire-decking-96x18x72-starter>: HTTP status code is not handled or not allowed 2026-01-24 20:22:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/210-quad-ring-xring-34id-x-1od-70-duro-round-black-pkg-100>: HTTP status code is not handled or not allowed 2026-01-24 20:22:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/record-storage-rack-gray-72x30x84>: HTTP status code is not handled or not allowed 2026-01-24 20:22:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-strip-with-acrylic-adhesive-60a-116-thick-x-4-w-x-5-ft-l>: HTTP status code is not handled or not allowed 2026-01-24 20:22:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bash-2-lb-steel-dead-blow-hammer-14l-handle>: HTTP status code is not handled or not allowed 2026-01-24 20:22:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chemotherapy-drug-transport-bags-6x9-1000-case>: HTTP status code is not handled or not allowed 2026-01-24 20:22:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/binders_covers/binders>: HTTP status code is not handled or not allowed 2026-01-24 20:22:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leeson-210-tefc-d-flange-endbell-w-lb>: HTTP status code is not handled or not allowed 2026-01-24 20:22:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls>: HTTP status code is not handled or not allowed 2026-01-24 20:22:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cortech-usa-7603bk-endurance-night-table-black-fr-pack-of-1>: HTTP status code is not handled or not allowed 2026-01-24 20:22:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-shelf-cart-poly-green-24l-x-18w-x-40h-polyurethane-rigid-casters>: HTTP status code is not handled or not allowed 2026-01-24 20:22:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gn-749-threaded-pipe-plugs-steel-zinc-plated-w-nbr-synthetic-rubber-seal-m-24-x-1-5>: HTTP status code is not handled or not allowed 2026-01-24 20:22:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-hook-stake-bright-white>: HTTP status code is not handled or not allowed 2026-01-24 20:22:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/capital-locker-single-tier-12x18x72-3-door-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:22:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leeson-6-lb-ft-coupler-brake-56c-143-5tc-nema3-ip23-230-460v-1ph-aluminum-stearns>: HTTP status code is not handled or not allowed 2026-01-24 20:22:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gear-driven-band-saw-1-1-2-hp-single-phase-220v-bs-260m>: HTTP status code is not handled or not allowed 2026-01-24 20:22:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n-7175-closet-door-hanger-dial-action-18-inch-offset-34-inch-nylonpack-of-2>: HTTP status code is not handled or not allowed 2026-01-24 20:22:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tune-up-kit-for-dxs-2236-series-air-impact-wrench>: HTTP status code is not handled or not allowed 2026-01-24 20:22:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-caster-with-1square-swivel-stem-2-1-16h-stem-325-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:22:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paramount-locker-single-tier-15x18x72-3-door-tan>: HTTP status code is not handled or not allowed 2026-01-24 20:22:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-roll-no-adhesive-70a-316-thick-x-36-wide-x-6-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 20:22:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milwaukee-3697-27-m18-fuel-7-tool-combo-kit>: HTTP status code is not handled or not allowed 2026-01-24 20:22:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kitchen-stepper-2-step-adult-folding-step-stool-19-78l-x-18-38w-x-34-12h-green>: HTTP status code is not handled or not allowed 2026-01-24 20:22:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stinger-spillpal-foam-wall-spill-containment-berm-30l-x-12w-x-4h-black>: HTTP status code is not handled or not allowed 2026-01-24 20:22:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stinger-spillpal-foam-wall-spill-containment-berm-40l-x-12w-x-4h-black>: HTTP status code is not handled or not allowed 2026-01-24 20:22:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=straight-hand-racers-brake-72-l-22-gauge already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:22:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/laptop-tablet-holder-10-17>: HTTP status code is not handled or not allowed 2026-01-24 20:22:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/d-island-71w-x-36d-x-29-1-2h-espresso>: HTTP status code is not handled or not allowed 2026-01-24 20:22:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-purpose-manual-metal-shear-12-blade-length-1875-mild-steel-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:23:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metal-working-dust-collector-220v-single-phase-3-hp-1450-cfm-mdc-1800-1-0>: HTTP status code is not handled or not allowed 2026-01-24 20:23:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-mesh-sliding-gate-10x4>: HTTP status code is not handled or not allowed 2026-01-24 20:23:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elephantwall-topple-barrier-16-ft-h-x-12-ft-l>: HTTP status code is not handled or not allowed 2026-01-24 20:23:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrislynn-m22x15x330-free-running-helical-inserts-bulk-pack-of-15-inserts>: HTTP status code is not handled or not allowed 2026-01-24 20:23:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=multi-purpose-manual-sheet-metal-shear-14-gauge-3600-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:23:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cnc-press-brake-3-phase-220v-bp-3305cnc>: HTTP status code is not handled or not allowed 2026-01-24 20:23:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-x-30-shop-top-pedestal-workbench-w-4-drawers-2>: HTTP status code is not handled or not allowed 2026-01-24 20:23:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=straight-hand-racers-brake-72-l-22-gauge landed on page that is not a product page. 2026-01-24 20:23:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-giant-stackable-storage-bin-16-1-2x18x11-green>: HTTP status code is not handled or not allowed 2026-01-24 20:23:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-manually-operated-hand-straight-brake-10-l-14-gauge>: HTTP status code is not handled or not allowed 2026-01-24 20:23:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 18630 pages (at 94 pages/min), scraped 11969 items (at 25 items/min) 2026-01-24 20:23:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrislynn-m4x07x80-free-running-helical-inserts-bulk-pack-of-100-inserts>: HTTP status code is not handled or not allowed 2026-01-24 20:23:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=pipe-notcher-single-phase-220v-tn-200e already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:23:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-non-resettable-combination-padlocks-no-1500kalf>: HTTP status code is not handled or not allowed 2026-01-24 20:23:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:23:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/94-l-j-trim-in-louvre-granite-5-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:23:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:23:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-high-security-brass-solid-body-padlocks-no-6840lj>: HTTP status code is not handled or not allowed 2026-01-24 20:23:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-mesh-partition-security-room-10x10x8-without-roof-2-sides-w-window>: HTTP status code is not handled or not allowed 2026-01-24 20:23:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dock-bumper-24-w-x-4-5-d-x-10-h>: HTTP status code is not handled or not allowed 2026-01-24 20:23:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-high-security-steel-weather-resistant-covered-laminated-padlocks-no-6125>: HTTP status code is not handled or not allowed 2026-01-24 20:23:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-black-double-foam-standard-duty-floor-squeegee-0736>: HTTP status code is not handled or not allowed 2026-01-24 20:23:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/94-l-l-trim-in-oak-mocha-5-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:23:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/291-acrylic-carton-sealing-tape-2-x-55-yds-26-mil-clear-6pack>: HTTP status code is not handled or not allowed 2026-01-24 20:23:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-general-security-brass-solid-body-padlocks-no-4120ka>: HTTP status code is not handled or not allowed 2026-01-24 20:23:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/35111-workbrutes-hitop-work-overshoes-black-cleated-outsole-md>: HTTP status code is not handled or not allowed 2026-01-24 20:23:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=pipe-notcher-single-phase-220v-tn-200e>: HTTP status code is not handled or not allowed 2026-01-24 20:23:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-part-stub-style-inventory-tag-pre-wired-1000-1999-8-6-14l-x-3-18w-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 20:23:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls>: HTTP status code is not handled or not allowed 2026-01-24 20:23:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-mesh-partition-security-room-20x15x8-without-roof-4-sides-w-window>: HTTP status code is not handled or not allowed 2026-01-24 20:23:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets>: HTTP status code is not handled or not allowed 2026-01-24 20:23:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/23-2-l-x-11-1-w-vinyl-wall-tile-wind-gust-10-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:23:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-strip-no-adhesive-50a-132-thick-x-6-wide-x-5-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 20:23:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iteq1520-duplex-circuit-breaker-type-qt-class-ctl-twin-15a-20a>: HTTP status code is not handled or not allowed 2026-01-24 20:23:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=multi-purpose-manual-sheet-metal-shear-14-gauge-3600-rpm landed on page that is not a product page. 2026-01-24 20:23:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-part-carbonless-stub-style-inv-tag-pre-wired-2500-2999-8-6-14lx3-18w-500pk>: HTTP status code is not handled or not allowed 2026-01-24 20:23:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vpkwb2020-circuit-breaker-type-b-twin-1-pole-20a-20a-clamshell-packaged>: HTTP status code is not handled or not allowed 2026-01-24 20:23:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-shelving-open-shelving-36x12x75-2>: HTTP status code is not handled or not allowed 2026-01-24 20:23:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-x15-fall-safety-net-with-debris-liner-black>: HTTP status code is not handled or not allowed 2026-01-24 20:23:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-sheet-with-acrylic-adhesive-50a-18-thick-x-12-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 20:23:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-sheet-with-acrylic-adhesive-50a-332-thick-x-12-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 20:23:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/welded-jumbo-desk-height-cabinet-48-w-x-18-d-x-30-h-sand>: HTTP status code is not handled or not allowed 2026-01-24 20:23:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hytrol-3-ft-ball-transfer-table-33-inch-between-frame-3-inch-ball-center>: HTTP status code is not handled or not allowed 2026-01-24 20:23:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-shelving-closed-shelving-36x12x39-8>: HTTP status code is not handled or not allowed 2026-01-24 20:23:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluidity-02f-direct-acting-solenoid-valve-3-2-nc-nbr-seal-1-2-nptf-3-mm-24v-dc>: HTTP status code is not handled or not allowed 2026-01-24 20:23:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-x30-safety-barrier-net-net-only-orange>: HTTP status code is not handled or not allowed 2026-01-24 20:23:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluidity-03f-hung-diaphragm-solenoid-valve-2-2-nc-fkm-seal-1-2-nptf-12-mm-12v-dc>: HTTP status code is not handled or not allowed 2026-01-24 20:23:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1800-weather-fashions-trim-rubber-overshoes-black-md>: HTTP status code is not handled or not allowed 2026-01-24 20:23:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paramount-locker-single-tier-12x15x60-1-door-assembled-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:23:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1350-wintertuff-ice-traction-stretch-overshoes-black-studded-outsole-3xl>: HTTP status code is not handled or not allowed 2026-01-24 20:23:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-delrin-homopolymer-sheet-0-250-thick-x-24-w-x-24-l>: HTTP status code is not handled or not allowed 2026-01-24 20:23:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-part-carbonless-stub-style-inv-tag-pre-wired-0000-0499-8-6-14lx3-18w-500pk>: HTTP status code is not handled or not allowed 2026-01-24 20:23:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:23:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s78122-class-3-hooded-sweatshirt-fluorescent-lime-4xl>: HTTP status code is not handled or not allowed 2026-01-24 20:23:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-320-nipple-3-8-male-npt-european-interchange>: HTTP status code is not handled or not allowed 2026-01-24 20:23:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluidity-03f-hung-diaphragm-solenoid-valve-2-2-nc-epdm-seal-1-4-nptf-10-mm-12v-dc>: HTTP status code is not handled or not allowed 2026-01-24 20:23:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/j12148-magnaprene-hooded-jacket-green-inner-cuffs-md>: HTTP status code is not handled or not allowed 2026-01-24 20:23:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lightweight-portable-hand-operated-li-truck-500-pounds-straddle-legs>: HTTP status code is not handled or not allowed 2026-01-24 20:23:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-af-blend-sheet-1-250-thick-x-24-w-x-24-l>: HTTP status code is not handled or not allowed 2026-01-24 20:23:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wireway-husky-tear-drop-pallet-rack-108x42x144>: HTTP status code is not handled or not allowed 2026-01-24 20:23:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o53107-35mm-industrial-work-snap-fly-front-overall-yellow-3xl>: HTTP status code is not handled or not allowed 2026-01-24 20:23:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o56107-durascrim-snap-fly-front-overall-yellow-3xl>: HTTP status code is not handled or not allowed 2026-01-24 20:23:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-roll-no-adhesive-50a-38-thick-x-36-wide-x-7-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 20:23:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tyvek174-shipping-tag-pre-wired-3-34l-x-1-78w-green-100pack>: HTTP status code is not handled or not allowed 2026-01-24 20:23:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets>: HTTP status code is not handled or not allowed 2026-01-24 20:23:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heat-wagon-50ft-long-high-pressure-gas-hose-1in-diameter>: HTTP status code is not handled or not allowed 2026-01-24 20:23:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/j22208-iron-eagle-storm-fly-front-jacket-green-hood-snaps-xl>: HTTP status code is not handled or not allowed 2026-01-24 20:23:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-large-nesting-tote-23-3-4-x-17-1-4x12-white>: HTTP status code is not handled or not allowed 2026-01-24 20:23:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tyvek174-shipping-tag-pre-wired-4-34l-x-2-38w-yellow-100pack>: HTTP status code is not handled or not allowed 2026-01-24 20:23:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esd-office-chair-high-height-value-line-fabric-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:23:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-circle-tag-3-dia-green-100pack>: HTTP status code is not handled or not allowed 2026-01-24 20:23:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3m-048011-27605-6x12-radial-bristle-brsh-50-grit>: HTTP status code is not handled or not allowed 2026-01-24 20:23:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-sheet-with-high-temp-adhesive-50a-14-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 20:23:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-sheet-with-high-temp-adhesive-50a-18-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 20:23:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-ldpe-rod-0-625-dia>: HTTP status code is not handled or not allowed 2026-01-24 20:24:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1730s08-osha-stairway-30width-8-stairs>: HTTP status code is not handled or not allowed 2026-01-24 20:24:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:24:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-delrin-homopolymer-rod-5-500-dia>: HTTP status code is not handled or not allowed 2026-01-24 20:24:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/61500121423-clean-finish-roll-4w-x-30l-vfn>: HTTP status code is not handled or not allowed 2026-01-24 20:24:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s63219-comforttuff-2-pc-suit-blaze-orange-attached-hood-md>: HTTP status code is not handled or not allowed 2026-01-24 20:24:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gn471-1-crank-handles-zinc-die-cast-revolving-handle-thru-square-bore-5-16-1-97-l>: HTTP status code is not handled or not allowed 2026-01-24 20:24:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/queueway-11-inch-by-14-inch-acrylic-sign-please-wait-here-double-sided>: HTTP status code is not handled or not allowed 2026-01-24 20:24:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-ldpe-rod-1-000-dia>: HTTP status code is not handled or not allowed 2026-01-24 20:24:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s63217-comforttuff-2-pc-suit-yellow-attached-hood-3xl>: HTTP status code is not handled or not allowed 2026-01-24 20:24:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s63217-comforttuff-2-pc-suit-yellow-attached-hood-lg>: HTTP status code is not handled or not allowed 2026-01-24 20:24:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-roll-no-adhesive-60a-316-thick-x-36-wide-x-7-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 20:24:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-heater-mh360faet-portable-electric-forced-air-heater-garage-space-heater-3600w-240v>: HTTP status code is not handled or not allowed 2026-01-24 20:24:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 18723 pages (at 93 pages/min), scraped 11991 items (at 22 items/min) 2026-01-24 20:24:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-round-outdoor-bar-height-table-42-h-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:24:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1x1x1-4-alum-oxide-80-grit-flap-wheel>: HTTP status code is not handled or not allowed 2026-01-24 20:24:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-mirror-extruded-acrylic-film-masked-sheet-0-220-thick-x-48-w-x-96-ll-each>: HTTP status code is not handled or not allowed 2026-01-24 20:24:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 20:24:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylatron-nsm-rod-7-000-dia>: HTTP status code is not handled or not allowed 2026-01-24 20:24:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/briskheat-dhcs11-silicone-heater-for-15-gallon-metal-drum-120-volts-50-425-f-heavy-duty>: HTTP status code is not handled or not allowed 2026-01-24 20:24:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/abrasives/grinding_cutoff_wheels/bench_pedestal_grinding_wheels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:24:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/woodruff-key-3-8-w-x-1-1-2-outer-dia-carbon-steel-plain-key-no-1212-made-in-usa>: HTTP status code is not handled or not allowed 2026-01-24 20:24:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/abrasives/grinding_cutoff_wheels/bench_pedestal_grinding_wheels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:24:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/booster-gun-w-rigid-end>: HTTP status code is not handled or not allowed 2026-01-24 20:24:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/z-swivel>: HTTP status code is not handled or not allowed 2026-01-24 20:24:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/abrasives/sanding_discs/quick_change_sanding_discs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:24:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tygon-b44-3-tube-aab00027-0-375-id-x-500-od-x-50-l>: HTTP status code is not handled or not allowed 2026-01-24 20:24:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:24:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-sheet-no-adhesive-50a-38-thick-x-24-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 20:24:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/bin_shelving/steel_bin_shelving already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:24:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-profile-oil-drain-w-pump-25-gallon>: HTTP status code is not handled or not allowed 2026-01-24 20:24:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:24:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perm-liner-truck-20-bu-blue-vinyl-wire-base-all-swivel>: HTTP status code is not handled or not allowed 2026-01-24 20:24:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perm-liner-truck-16-bu-yellow-vinyl-wire-base-all-swivel>: HTTP status code is not handled or not allowed 2026-01-24 20:24:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylatron-gs-sheet-extruded-0-500-thick-x-24-w-x-24-l>: HTTP status code is not handled or not allowed 2026-01-24 20:24:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/abrasives/grinding_cutoff_wheels/bench_pedestal_grinding_wheels>: HTTP status code is not handled or not allowed 2026-01-24 20:24:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tyvek174-shipping-tag-6-14l-x-3-18w-orange-100pack>: HTTP status code is not handled or not allowed 2026-01-24 20:24:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tygon-silver-s3-tube-as600017-0-250-id-x-375-od-x-50-l>: HTTP status code is not handled or not allowed 2026-01-24 20:24:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-locker-bench-hardwood-top-w-steel-tube-pedestals-bolt-down-36-x-9-1-2-x-17>: HTTP status code is not handled or not allowed 2026-01-24 20:24:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/abrasives/sanding_discs/quick_change_sanding_discs>: HTTP status code is not handled or not allowed 2026-01-24 20:24:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylatron-gsm-sheet-cast-1-500-thick-x-24-w-x-48-l>: HTTP status code is not handled or not allowed 2026-01-24 20:24:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets>: HTTP status code is not handled or not allowed 2026-01-24 20:24:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/abrasives/grinding_cutoff_wheels/bench_pedestal_grinding_wheels landed on page that is not a product page. 2026-01-24 20:24:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tygon-b-44-4x-tube-aaa00074-1-500-id-x-2-000-od-x-50-l>: HTTP status code is not handled or not allowed 2026-01-24 20:24:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sensor-ntc-for-volcano-unit-heaters>: HTTP status code is not handled or not allowed 2026-01-24 20:24:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/bin_shelving/steel_bin_shelving landed on page that is not a product page. 2026-01-24 20:24:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets>: HTTP status code is not handled or not allowed 2026-01-24 20:24:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:24:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-oil-filled-cast-nylon-tube-4-id-x-5-500-od-x-52-l>: HTTP status code is not handled or not allowed 2026-01-24 20:24:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rock-wall-stop-concave-2-1-2-dia-chromium-plated>: HTTP status code is not handled or not allowed 2026-01-24 20:24:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o22007-iron-eagle-plain-front-overall-gold-snaplock-buckles-lg>: HTTP status code is not handled or not allowed 2026-01-24 20:24:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rock-pull-plate-8-l-x-16-h-x-3-4-oxidized-satin-bronze-8-ctc>: HTTP status code is not handled or not allowed 2026-01-24 20:24:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-tag-5-4-34l-x-2-38w--fluorescent-orange-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 20:24:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-tag-pre-wired-1-2-34l-x-1-38w-fluorescent-red-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 20:24:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perm-liner-truck-16-bu-black-vinyl-wood-base-2-rg-2-sw>: HTTP status code is not handled or not allowed 2026-01-24 20:24:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/accuglide8482-2-taping-head-upper-taping-head-2>: HTTP status code is not handled or not allowed 2026-01-24 20:24:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/expandable-add-on-rack-96x48x84-gray-with-3-level-wire-deck-800lb-cap-per-level>: HTTP status code is not handled or not allowed 2026-01-24 20:24:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-machine-table-16-x-30-x-36>: HTTP status code is not handled or not allowed 2026-01-24 20:24:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-welded-storage-locker-1-center-shelf-30-x-60>: HTTP status code is not handled or not allowed 2026-01-24 20:24:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tyvek174-shipping-tag-pre-wired-4-14l-x-2-18w-orange-100pack>: HTTP status code is not handled or not allowed 2026-01-24 20:24:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-24 20:24:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/welded-workbench-butcher-block-top-fixed-height-30-x-72>: HTTP status code is not handled or not allowed 2026-01-24 20:24:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-tag-pre-strung-12-8l-x-4w-pink-500pack>: HTTP status code is not handled or not allowed 2026-01-24 20:24:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/35113-workbrutes-hitop-work-overshoes-yellow-cleated-outsole-2xl>: HTTP status code is not handled or not allowed 2026-01-24 20:24:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-tag-pre-strung-4-4-14l-x-2-18w-fluorescent-orange-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 20:24:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/53-natural-rubber-tape-3-x-55-yds-21-mil-clear-24pack>: HTTP status code is not handled or not allowed 2026-01-24 20:24:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-gift-box-bottoms-6l-x-6w-x-3h-white-50pack>: HTTP status code is not handled or not allowed 2026-01-24 20:24:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machine-table-3-shelves-18-x-32>: HTTP status code is not handled or not allowed 2026-01-24 20:24:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/del-serrated-nozzle-8-l-red>: HTTP status code is not handled or not allowed 2026-01-24 20:24:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-30-glass-filled-nylon-6-6-sheet-0-750-thick-x-24-w-x-48-l>: HTTP status code is not handled or not allowed 2026-01-24 20:24:49 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-24 20:24:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o24129-icon-snap-fly-front-overall-fluorescent-orange-2xl>: HTTP status code is not handled or not allowed 2026-01-24 20:24:49 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:12 2026-01-24 20:24:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-18-ga-36-wx12-dx73-h-closed-clip-style-5-shelf-starter>: HTTP status code is not handled or not allowed 2026-01-24 20:24:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gorbel-hd-free-standing-jib-crane-12-ft-span-and-10-ft-height-under-boom-500-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:24:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gorbel-hd-free-standing-jib-crane-10-ft-span-and-14-ft-height-under-boom-4000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:24:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2525-flatback-tape-34-x-60-yds-95-mil-orange-48pack>: HTTP status code is not handled or not allowed 2026-01-24 20:24:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cubby-locker-welded-34-1-2w-x-13-1-2d-x-66h-15-compartments-medium-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:24:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reverse-tuck-folding-cartons-3l-x-3w-x-10h-white-250pack>: HTTP status code is not handled or not allowed 2026-01-24 20:24:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/handy-horn-hand-held-gas-air-horn-490>: HTTP status code is not handled or not allowed 2026-01-24 20:24:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reinforced-vinyl-tubing-bulk-coil-2-i-d-100-feet-coil>: HTTP status code is not handled or not allowed 2026-01-24 20:24:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-fair-cherry-deodorant-1-quart-bottle-12-bottles-case>: HTTP status code is not handled or not allowed 2026-01-24 20:24:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hang-stack-bins-w-two-12-rails-six-bins-4-1-8w-x-5-3-8d-x-3h-red>: HTTP status code is not handled or not allowed 2026-01-24 20:25:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hang-stack-bins-w-two-12-rails-six-bins-4-1-8w-x-7-3-8d-x-3h-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:25:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hardwood-platform-truck-72x36-1000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:25:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jewelry-boxes-5-14l-x-3-34w-x-78h-kraft-100pack>: HTTP status code is not handled or not allowed 2026-01-24 20:25:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reverse-tuck-folding-cartons-3l-x-3w-x-3h-white-250pack>: HTTP status code is not handled or not allowed 2026-01-24 20:25:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nomex-410-sheet-0-007-thick-x-36-000-inch-w-x-100-l>: HTTP status code is not handled or not allowed 2026-01-24 20:25:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-vinyl-tubing-2-i-d-x-1-4-wall-x-50-feet>: HTTP status code is not handled or not allowed 2026-01-24 20:25:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aspect-peel-stick-wood-tile-weathered-barn-a70-02>: HTTP status code is not handled or not allowed 2026-01-24 20:25:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/great-lakes-tin-jamestown-2-x-2-nail-up-tin-ceiling-tile-in-argento-t51-07>: HTTP status code is not handled or not allowed 2026-01-24 20:25:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ntep-pallet-scale-with-led-indicator-4-x-4-10-000-lb-x-2-lb>: HTTP status code is not handled or not allowed 2026-01-24 20:25:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-top-machine-table-24-x-48-x-42-swivel-casters-w-brakes>: HTTP status code is not handled or not allowed 2026-01-24 20:25:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-top-machine-table-24-x-48-x-42-w-accessory-drawer>: HTTP status code is not handled or not allowed 2026-01-24 20:25:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 18814 pages (at 91 pages/min), scraped 12011 items (at 20 items/min) 2026-01-24 20:25:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/great-lakes-tin-toledo-2-x-2-nail-up-tin-ceiling-tile-in-unfinished-t55-03>: HTTP status code is not handled or not allowed 2026-01-24 20:25:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/accessory-data-cards-black-3x-1-3-4-black-10-pack-fm2630>: HTTP status code is not handled or not allowed 2026-01-24 20:25:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foam-lined-cd-mailers-5-18l-x-5w-white-100pack>: HTTP status code is not handled or not allowed 2026-01-24 20:25:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oversized-hammock-with-pillow-tropical-fruit>: HTTP status code is not handled or not allowed 2026-01-24 20:25:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quartz-heat-treated-high-purity-sio2-binderless-glass-microfiber-filter-4-25-cm-dia-100pk>: HTTP status code is not handled or not allowed 2026-01-24 20:25:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gift-boxes-6l-x-6w-x-6h-black-100pack>: HTTP status code is not handled or not allowed 2026-01-24 20:25:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/libman-commercial-dust-pan-and-counter-brush-set>: HTTP status code is not handled or not allowed 2026-01-24 20:25:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aspect-honeycomb-matted-12-x-4-brushed-stainless-metal-decorative-tile-backsplash-a98-50>: HTTP status code is not handled or not allowed 2026-01-24 20:25:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-z-brite-wire-shelving-72wx21dx54h>: HTTP status code is not handled or not allowed 2026-01-24 20:25:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/solaira-sicr40240g-infrared-heater-4-kw-208-240v-grey>: HTTP status code is not handled or not allowed 2026-01-24 20:25:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-container-solid-wall-48x40x31-long-side-runners-white-1500-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:25:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-z-brite-wire-shelving-add-on-24wx18dx54h>: HTTP status code is not handled or not allowed 2026-01-24 20:25:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/health-gards-toilet-seat-covers-15-x-17-white-3000-case>: HTTP status code is not handled or not allowed 2026-01-24 20:25:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/libman-commercial-24-inch-polymer-rough-surface-industrial-push-broom>: HTTP status code is not handled or not allowed 2026-01-24 20:25:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/49-56-9629-2-5-16-bi-metal-hole-saw>: HTTP status code is not handled or not allowed 2026-01-24 20:25:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/54h-nexel-chrome-wire-shelving-54wx21d>: HTTP status code is not handled or not allowed 2026-01-24 20:25:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5262-21-7-8-sds-plus-rotary-hammer-kit>: HTTP status code is not handled or not allowed 2026-01-24 20:25:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fresh-products-wave-3d-urinal-deodorizer-screen-red-spiced-apple-fragrance-10-screens-box>: HTTP status code is not handled or not allowed 2026-01-24 20:25:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/49-56-0177-3-1-8-bi-metal-hole-saw>: HTTP status code is not handled or not allowed 2026-01-24 20:25:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/great-lakes-tin-syracuse-2-x-2-lay-in-tin-ceiling-tile-in-gloss-white-y50-00>: HTTP status code is not handled or not allowed 2026-01-24 20:25:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20ft-hyperlite-sumostance-375lb-capacity-type-iaa-fiberglass-extension-ladder-17220>: HTTP status code is not handled or not allowed 2026-01-24 20:25:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-32-4511-6-magnetic-bit-tip-holder>: HTTP status code is not handled or not allowed 2026-01-24 20:25:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sunset-20-square-outdoor-aluminum-low-table-fusion-bronze>: HTTP status code is not handled or not allowed 2026-01-24 20:25:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-20-7497-installation-kit-sds-plus-7-pc>: HTTP status code is not handled or not allowed 2026-01-24 20:25:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-32-4461-2-power-bit-phillips-1>: HTTP status code is not handled or not allowed 2026-01-24 20:25:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/great-lakes-tin-jamestown-2-x-2-lay-in-tin-ceiling-tile-in-antique-white-y51-02>: HTTP status code is not handled or not allowed 2026-01-24 20:25:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2221vs-shaft-5510107>: HTTP status code is not handled or not allowed 2026-01-24 20:25:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/walls_ceilings/ceiling_tiles already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:25:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bowl-clip-cucumber-melon-green-72-case>: HTTP status code is not handled or not allowed 2026-01-24 20:25:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-vinyl-bag-hotel-cart>: HTTP status code is not handled or not allowed 2026-01-24 20:25:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/walls_ceilings/ceiling_tiles already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:25:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bright-air-super-odor-eliminator-mandarin-orange-and-fresh-lemon-14-oz-6-case>: HTTP status code is not handled or not allowed 2026-01-24 20:25:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8201-key-5513859>: HTTP status code is not handled or not allowed 2026-01-24 20:25:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8201-knob-5513846>: HTTP status code is not handled or not allowed 2026-01-24 20:25:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/great-lakes-tin-ludington-2-x-2-lay-in-tin-ceiling-tile-in-matte-white-y57-01>: HTTP status code is not handled or not allowed 2026-01-24 20:25:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-dunnage-rack-48x20x12>: HTTP status code is not handled or not allowed 2026-01-24 20:25:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/illinois-engineered-products-double-folding-gate-6w-to-8w-and-76h>: HTTP status code is not handled or not allowed 2026-01-24 20:25:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/structural-shelving-4-solid-dura-shelves-3>: HTTP status code is not handled or not allowed 2026-01-24 20:25:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cone-bowl-mop-with-10-handle-2-diameter-head>: HTTP status code is not handled or not allowed 2026-01-24 20:25:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sunset-single-unit-towel-valet-madras-blue-sling-with-glacier-white-frame>: HTTP status code is not handled or not allowed 2026-01-24 20:25:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/walls_ceilings/ceiling_tiles>: HTTP status code is not handled or not allowed 2026-01-24 20:25:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jamaica-beach-adj-sling-chaise-lounge-blue-sling-w-white-frame-pack-of-16>: HTTP status code is not handled or not allowed 2026-01-24 20:25:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o-ring-30-36-3-1230r-5234741>: HTTP status code is not handled or not allowed 2026-01-24 20:25:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-qt-aluminum-pot>: HTTP status code is not handled or not allowed 2026-01-24 20:25:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/walls_ceilings/ceiling_tiles>: HTTP status code is not handled or not allowed 2026-01-24 20:25:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/panel-kit-72-w-workbench-with-18-w-whiteboard-36-w-pegboard-mounting-rail-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:25:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylon-brush-w-quick-connect-for-1316-id-tubes-blue-pack-of-15>: HTTP status code is not handled or not allowed 2026-01-24 20:25:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4c-horizontal-mailbox-41h-doubleparcel-box-3-pl5-1-pl6-front-load-aluminum-usps>: HTTP status code is not handled or not allowed 2026-01-24 20:25:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sigma-34-square-aluminum-table-w-umbrella-hole-fusion-bronze>: HTTP status code is not handled or not allowed 2026-01-24 20:25:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-surface-disinfectant-degreaser-herbal-32-oz-spray-bottle>: HTTP status code is not handled or not allowed 2026-01-24 20:25:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merion-vented-stack-chairs-with-arms-rests-black>: HTTP status code is not handled or not allowed 2026-01-24 20:25:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/y-connection-with-valves>: HTTP status code is not handled or not allowed 2026-01-24 20:25:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4b-horizontal-mailbox-24-a-doors-23-usable-front-loading-bronze-usps-access>: HTTP status code is not handled or not allowed 2026-01-24 20:25:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wiwax-cleaning-and-maintenance-solution-liquid-gallon-bottle-4-case>: HTTP status code is not handled or not allowed 2026-01-24 20:25:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/suma-freeze-d2-9-floor-cleaner-liquid-1-gal-4-per-case>: HTTP status code is not handled or not allowed 2026-01-24 20:25:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2232ac-bearing-ball-5510143>: HTTP status code is not handled or not allowed 2026-01-24 20:25:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4c-horizontal-mailbox-41h-single-parcel-box-1-pl5-1-pl6-front-load-aluminum-usps>: HTTP status code is not handled or not allowed 2026-01-24 20:25:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-plastic-trash-container-dome-lid-garbage-can-dome-lid-44-gallon-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:25:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4210-screw-316x12-5513058>: HTTP status code is not handled or not allowed 2026-01-24 20:25:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-48x24x60-with-5-shelves-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:25:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/560-ergodeck-heavy-duty-outside-corner-6x15-black>: HTTP status code is not handled or not allowed 2026-01-24 20:25:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-rubber-skirts-85x820-641245-floor-scrubber>: HTTP status code is not handled or not allowed 2026-01-24 20:25:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyethylene-container-30-lx24-wx15-h-2>: HTTP status code is not handled or not allowed 2026-01-24 20:25:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-integral-fil-reg-with-gauge-metal-bowl-float-drain>: HTTP status code is not handled or not allowed 2026-01-24 20:25:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25816-crank-table-raiser-5507519>: HTTP status code is not handled or not allowed 2026-01-24 20:25:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fda-plastic-storage-container-with-lid-44-1-2-x-44-1-2-x-36>: HTTP status code is not handled or not allowed 2026-01-24 20:25:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swa-70-semi-automatic-stretch-wrap-machine-70-diameter>: HTTP status code is not handled or not allowed 2026-01-24 20:25:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swa-5070lp-pmo-rf-powered-mast-option-retro-fit-115v-50-x-70>: HTTP status code is not handled or not allowed 2026-01-24 20:25:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swa-70-r-4848-approach-ramp-for-swa-70-wrap-machine-48>: HTTP status code is not handled or not allowed 2026-01-24 20:25:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carpet-spot-and-stain-remover-liquid-24-oz-6-per-case>: HTTP status code is not handled or not allowed 2026-01-24 20:25:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o-rings-for-arrow-modulars-components>: HTTP status code is not handled or not allowed 2026-01-24 20:26:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lp-ng-dual-fuel-blue-flame-vent-free-thermostatic-garage-heater-gbf30dtdg-2-30000-btu>: HTTP status code is not handled or not allowed 2026-01-24 20:26:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-intermediate-particulate-filter-w-metal-bowl-float-drain>: HTTP status code is not handled or not allowed 2026-01-24 20:26:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8ft-picnic-table-aluminum-planking-hd-galvanized-steel-frame>: HTTP status code is not handled or not allowed 2026-01-24 20:26:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-mil-heavy-duty-polyethylene-boot-covers-clear-lg-50box>: HTTP status code is not handled or not allowed 2026-01-24 20:26:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dishwashing-liquid-ultra-strength-original-scent-20-oz-bottle-9-case>: HTTP status code is not handled or not allowed 2026-01-24 20:26:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7020-washer-38-ref-141-5714041>: HTTP status code is not handled or not allowed 2026-01-24 20:26:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-storage-drawer-cabinet-36x24x32-2>: HTTP status code is not handled or not allowed 2026-01-24 20:26:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crew-easy-paks-toilet-bowl-cleaner-0-5-oz-packet-90-packets-tub>: HTTP status code is not handled or not allowed 2026-01-24 20:26:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyethylene-bouffant-cap-34-mil-waterproof-100-latex-free-clear-21-100bag>: HTTP status code is not handled or not allowed 2026-01-24 20:26:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-sintered-muffler>: HTTP status code is not handled or not allowed 2026-01-24 20:26:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-muffler-1-8-female>: HTTP status code is not handled or not allowed 2026-01-24 20:26:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/765-deep-freeze-mat-38-thick-4w-x-8l-black>: HTTP status code is not handled or not allowed 2026-01-24 20:26:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-1600-standard-elastic-back-support-black-x-large>: HTTP status code is not handled or not allowed 2026-01-24 20:26:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dispatcher-locker-single-front-door-full-height-rear-panel-36w-x-30d-x-79-1-2h>: HTTP status code is not handled or not allowed 2026-01-24 20:26:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 18911 pages (at 97 pages/min), scraped 12025 items (at 14 items/min) 2026-01-24 20:26:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/410-airug-anti-fatigue-mat-38-thick-2w-x-up-to-60l-blackyellow-border>: HTTP status code is not handled or not allowed 2026-01-24 20:26:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/lockers/locker_accessories/locker_shelves_rods already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:26:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-drawer-unit-divider-kit-8-compartment-4>: HTTP status code is not handled or not allowed 2026-01-24 20:26:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylon-hairnet-honeycomb-116-hole-100-latex-free-brown-24-100bag>: HTTP status code is not handled or not allowed 2026-01-24 20:26:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/470-marble-sof-tyle153-anti-fatigue-mat-12-thick-4w-x-up-to-75l-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:26:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25-ton-indoor-outdoor-portable-air-conditioner-kpo25-43-270k-btu-cool-only-460-volt>: HTTP status code is not handled or not allowed 2026-01-24 20:26:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powermig-360mp-multi-process-welder-one-pak-360a-5060-hz-red>: HTTP status code is not handled or not allowed 2026-01-24 20:26:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guide-bl-holder-5631031>: HTTP status code is not handled or not allowed 2026-01-24 20:26:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/408-achilles153-anti-fatigue-mat-38-thick-2w-x-60l-black>: HTTP status code is not handled or not allowed 2026-01-24 20:26:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8205hl-class-2-super-econo-vest-orange-l-xl>: HTTP status code is not handled or not allowed 2026-01-24 20:26:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sport-half-frame-safety-glasses-anti-fog-smoke-lens-red-frame>: HTTP status code is not handled or not allowed 2026-01-24 20:26:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/lockers/locker_accessories/locker_shelves_rods landed on page that is not a product page. 2026-01-24 20:26:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hndl-5710761>: HTTP status code is not handled or not allowed 2026-01-24 20:26:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cannes-outdoor-stackable-armchair-terra-cotta-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-24 20:26:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reflective-bollard-sleeve-6-dia-x-52-h-yellow-with-red-tape>: HTTP status code is not handled or not allowed 2026-01-24 20:26:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-mobile-dunnage-rack-54-lx24-wx8-h>: HTTP status code is not handled or not allowed 2026-01-24 20:26:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tennsco-capstone-additional-shelf-level-42-w-x-24-d-medium-grey>: HTTP status code is not handled or not allowed 2026-01-24 20:26:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-solid-adjustable-brute-shelf-15-wx30-l>: HTTP status code is not handled or not allowed 2026-01-24 20:26:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-solid-brute-shelf-15-wx42-l>: HTTP status code is not handled or not allowed 2026-01-24 20:26:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cbo16-single-flue-chimney-cap-bolt-on-16-round-black-galvanized-steel>: HTTP status code is not handled or not allowed 2026-01-24 20:26:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/full-pedestal-file-file-26wx22d-driftwood>: HTTP status code is not handled or not allowed 2026-01-24 20:26:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8330z-class-3-two-tone-vest-lime-4xl-5xl>: HTTP status code is not handled or not allowed 2026-01-24 20:26:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/control-panel-hbs-1321w-1321w-151>: HTTP status code is not handled or not allowed 2026-01-24 20:26:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ivy-league-executive-captains-chair-with-casters-vinyl-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:26:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergodyne-600-single-layer-neoprene-knee-sleeve-black-small>: HTTP status code is not handled or not allowed 2026-01-24 20:26:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/java-wicker-stacking-armchair-charcoal-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-24 20:26:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-drawer-workbench-maple>: HTTP status code is not handled or not allowed 2026-01-24 20:26:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-cabinet-workbench-plastic>: HTTP status code is not handled or not allowed 2026-01-24 20:26:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/380560-high-resolution-precision-milliohm-meter-110>: HTTP status code is not handled or not allowed 2026-01-24 20:26:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-36w-x-36l-1-mil-clear-250pack>: HTTP status code is not handled or not allowed 2026-01-24 20:26:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-36w-x-42l-1-mil-clear-250pack>: HTTP status code is not handled or not allowed 2026-01-24 20:26:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2oz-clear-straight-sided-round-with-53-400-green-thermoset-f217-ptfe-lined-cap-24pk>: HTTP status code is not handled or not allowed 2026-01-24 20:26:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o-ring-jsg-0716n-00-4107>: HTTP status code is not handled or not allowed 2026-01-24 20:26:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/warn-w-rigging-kit>: HTTP status code is not handled or not allowed 2026-01-24 20:26:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-40w-x-48l-1-mil-clear-250pack>: HTTP status code is not handled or not allowed 2026-01-24 20:26:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/capacitor-holder-12t1t2t1ph-1et-1c-509>: HTTP status code is not handled or not allowed 2026-01-24 20:26:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jet-jtm-1050evs2-230-mill-newall-dp700-dro-x-y-and-z-axis-powerfeeds-690641>: HTTP status code is not handled or not allowed 2026-01-24 20:26:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:26:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triumph-style-t6hh-hss-aircraft-extension-drill-black-oxide-332-6>: HTTP status code is not handled or not allowed 2026-01-24 20:26:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-20-x-38-sckt-set-scre-6714159>: HTTP status code is not handled or not allowed 2026-01-24 20:26:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/tool_storage/chests_roller_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:26:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/898-filament-tape-1-x-60-yds-66-mil-clear-36pack>: HTTP status code is not handled or not allowed 2026-01-24 20:26:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-300hl-rounded-cap-knee-pad-gray-cap-one-size>: HTTP status code is not handled or not allowed 2026-01-24 20:26:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-72-post-with-plastic-caps>: HTTP status code is not handled or not allowed 2026-01-24 20:26:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/128oz-amber-jug-w-38-400-black-phenolic-w-pulp-vinyl-cap-6pk>: HTTP status code is not handled or not allowed 2026-01-24 20:26:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-9w-x-10l-1-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 20:26:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-wall-mount-outdoor-ashtray-9-w-x-5-d-x-13-h>: HTTP status code is not handled or not allowed 2026-01-24 20:26:51 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits landed on page that is not a product page. 2026-01-24 20:26:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-9w-x-24l-1-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 20:26:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cart-cover-400-denier-nylon-for-18-x-60-x-63-unit-hook-loop-fastener-closure-white>: HTTP status code is not handled or not allowed 2026-01-24 20:26:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/weathermaster-twin-yellow-post-retracting-belt-barrier-ada-compliant-11-ft-yellow-authorized-belt>: HTTP status code is not handled or not allowed 2026-01-24 20:26:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/tool_storage/chests_roller_cabinets landed on page that is not a product page. 2026-01-24 20:26:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anti-static-reclosable-poly-bags-9w-x-12l-2-mil-pink-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 20:26:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/loud-one-universal-replacement-disc-horn-low-tone-310-2t>: HTTP status code is not handled or not allowed 2026-01-24 20:26:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8008-hi-vis-leg-gaiters-lime-osfa>: HTTP status code is not handled or not allowed 2026-01-24 20:26:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/38-24-hex-nut-plated-6566002>: HTTP status code is not handled or not allowed 2026-01-24 20:26:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprng-18209-6813026>: HTTP status code is not handled or not allowed 2026-01-24 20:26:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classic-cotton-high-visibility-long-sleeve-t-shirt-with-pocket-orange-xl>: HTTP status code is not handled or not allowed 2026-01-24 20:27:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8365-class-3-rain-jacket-lime-3-x-large>: HTTP status code is not handled or not allowed 2026-01-24 20:27:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bushing-pf3-62-6288962>: HTTP status code is not handled or not allowed 2026-01-24 20:27:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xd-smooth-aluminum-underbody-18x18x24-1706400>: HTTP status code is not handled or not allowed 2026-01-24 20:27:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/64oz-clear-jug-w-38-400-black-phenolic-polycone-cap-6pk>: HTTP status code is not handled or not allowed 2026-01-24 20:27:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-of-2-blue-side-shelves-security-audio-visual-cart-1>: HTTP status code is not handled or not allowed 2026-01-24 20:27:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drive-screw-25-x-5-5625491>: HTTP status code is not handled or not allowed 2026-01-24 20:27:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/binders_covers/reports_covers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:27:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/128oz-clear-jug-w-38-400-black-phenolic-solid-pe-cap-4pk>: HTTP status code is not handled or not allowed 2026-01-24 20:27:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bellman-hotel-cart-41x24-satin-aluminum-with-burgundy-carpet-rubber-wheels>: HTTP status code is not handled or not allowed 2026-01-24 20:27:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-round-message-stamp-star-pre-inked-re-inkable-red>: HTTP status code is not handled or not allowed 2026-01-24 20:27:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insul-tube-6l-1-2-thick-3-1-2>: HTTP status code is not handled or not allowed 2026-01-24 20:27:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 19007 pages (at 96 pages/min), scraped 12056 items (at 31 items/min) 2026-01-24 20:27:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-one-extended-insert-indexes-clear-5-tab-letter-buff-24-sets-box>: HTTP status code is not handled or not allowed 2026-01-24 20:27:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insul-tube-6l-3-8-thick-3-8>: HTTP status code is not handled or not allowed 2026-01-24 20:27:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pin-jsg-0750-28-0009>: HTTP status code is not handled or not allowed 2026-01-24 20:27:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulb-switch-2852d55723>: HTTP status code is not handled or not allowed 2026-01-24 20:27:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/web-sling-2-ply-eye-eye-6-long-2-es-wide>: HTTP status code is not handled or not allowed 2026-01-24 20:27:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-economical-insertable-index-clear-tabs-5-tab-letter-buff-6-sets-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:27:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/128oz-amber-jug-w-38-400-black-phenolic-w-solid-pe-cap-6pk>: HTTP status code is not handled or not allowed 2026-01-24 20:27:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/binders_covers/reports_covers>: HTTP status code is not handled or not allowed 2026-01-24 20:27:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washer-mdl-ghbbdb-1340a-32a5719>: HTTP status code is not handled or not allowed 2026-01-24 20:27:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aspect-backsplash-trim-kit-vinyl-peel-stick-black-953-07>: HTTP status code is not handled or not allowed 2026-01-24 20:27:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-16w-x-20l-4-mil-clear-500pack>: HTTP status code is not handled or not allowed 2026-01-24 20:27:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/38-gallon-square-waste-receptacle-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:27:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washer-9180-216-1>: HTTP status code is not handled or not allowed 2026-01-24 20:27:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ring-valve-jco-series-21a101300>: HTTP status code is not handled or not allowed 2026-01-24 20:27:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-11w-x-15l-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 20:27:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-one-table-of-contents-dividers-assorted-color-5-tab-1-5-letter-white-6-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:27:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-two-pocket-portfolio-embossed-leather-grain-paper-dark-blue-25-box>: HTTP status code is not handled or not allowed 2026-01-24 20:27:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/k-flex-elastomeric-foam-tape-1-8-thick2-wide30>: HTTP status code is not handled or not allowed 2026-01-24 20:27:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-12w-x-36l-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 20:27:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sparkplugs-plugstation-disp-pack-1500-per-cs-6644>: HTTP status code is not handled or not allowed 2026-01-24 20:27:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/miracool-bandanas-blue-denim-12-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:27:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/recycled-easel-pads-unruled-27-x-34-white-50sheet-2carton>: HTTP status code is not handled or not allowed 2026-01-24 20:27:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storage-cabinet-36x18x72-tan-1>: HTTP status code is not handled or not allowed 2026-01-24 20:27:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stow-away-hard-hat-shade-khaki>: HTTP status code is not handled or not allowed 2026-01-24 20:27:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/panel_carts_trucks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:27:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-wire-shelf-platform-30-w-x-24-d>: HTTP status code is not handled or not allowed 2026-01-24 20:27:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-liner-24-bushel-yellow-vinyl-basket-truck-bulk-truck>: HTTP status code is not handled or not allowed 2026-01-24 20:27:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/410kas3blu-safety-padlock-1-12h-shackle-blue-3set>: HTTP status code is not handled or not allowed 2026-01-24 20:27:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/cash_register_pos_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:27:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s33wht-snap-lock-safety-padlock-1-38w-x-1-12h-wht>: HTTP status code is not handled or not allowed 2026-01-24 20:27:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/219009-10-alphatec-nitrile-knit-lined-58-535-10>: HTTP status code is not handled or not allowed 2026-01-24 20:27:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/danger-no-trespassing-7x10-pressure-sensitive-vinyl>: HTTP status code is not handled or not allowed 2026-01-24 20:27:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-32w-x-32l-4-mil-clear-100pack>: HTTP status code is not handled or not allowed 2026-01-24 20:27:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-5w-x-18l-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 20:27:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classic-35mm-pvc-polyester-49-coat-yellow-200cxl>: HTTP status code is not handled or not allowed 2026-01-24 20:27:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classic-35mm-pvc-polyester-49-coat-yellow-200cx5>: HTTP status code is not handled or not allowed 2026-01-24 20:27:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/not-an-exit-aluminum>: HTTP status code is not handled or not allowed 2026-01-24 20:27:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/locks_lockout_devices/lockout_tagout_devices/lockout_padlocks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:27:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/carts-trucks/panel_carts_trucks>: HTTP status code is not handled or not allowed 2026-01-24 20:27:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gear--8201-5783981>: HTTP status code is not handled or not allowed 2026-01-24 20:27:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-ton-capacity-counterstay-internal-extractors-21-0-thru-21-5>: HTTP status code is not handled or not allowed 2026-01-24 20:27:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/paper_stationery/cash_register_pos_rolls>: HTTP status code is not handled or not allowed 2026-01-24 20:27:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/410kaprp-safety-padlock-1-12h-shackle-purple->: HTTP status code is not handled or not allowed 2026-01-24 20:27:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/117274-8-sol-vex-unsupported-nitrile-line-37-175-8>: HTTP status code is not handled or not allowed 2026-01-24 20:27:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-l-x-30-w-production-bench-esd-square-edge-blue-2>: HTTP status code is not handled or not allowed 2026-01-24 20:27:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reserved-parking-for-18x12-063-aluminum>: HTTP status code is not handled or not allowed 2026-01-24 20:27:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/locks_lockout_devices/lockout_tagout_devices/lockout_padlocks>: HTTP status code is not handled or not allowed 2026-01-24 20:27:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-square-plastic-height-adjustable-activity-table-set-with-4-chairs-red>: HTTP status code is not handled or not allowed 2026-01-24 20:27:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cambridge-tufted-upholstered-queen-size-headboard-fabric-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:27:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-size-stacking-bin-12-3-8-x20-x-8-red>: HTTP status code is not handled or not allowed 2026-01-24 20:27:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classic-cotton-high-visibility-t-shirt-orange-xl>: HTTP status code is not handled or not allowed 2026-01-24 20:27:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/key-5x5x35-bdb1340-h63>: HTTP status code is not handled or not allowed 2026-01-24 20:27:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-premium-locker-triple-tier-12x18x24-3-door-assembled-parchment>: HTTP status code is not handled or not allowed 2026-01-24 20:27:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chain-plastic-2-links-white-100-feet-trade-size-8>: HTTP status code is not handled or not allowed 2026-01-24 20:27:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s1106orj-aluminum-safety-padlock-1-12w-x-1-12h-shackle-orange>: HTTP status code is not handled or not allowed 2026-01-24 20:27:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fry-basketss-28-2040-tuc100011>: HTTP status code is not handled or not allowed 2026-01-24 20:27:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/probes-44-1235-vul353589-1>: HTTP status code is not handled or not allowed 2026-01-24 20:27:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/p100-nuisance-level-acidgas-relief-respirator-2096>: HTTP status code is not handled or not allowed 2026-01-24 20:27:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-13w-x-20l-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 20:27:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/r-1005-a-rebuild-kit-urinal-1-0-gpf>: HTTP status code is not handled or not allowed 2026-01-24 20:27:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-premium-locker-double-tier-12x18x30-6-door-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:27:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3m-aluminum-foil-tape-4380-silver-1-in-x-55-yds-36-rolls-per-case>: HTTP status code is not handled or not allowed 2026-01-24 20:27:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3t-lug-mount-assembly-3ss-3c-lma>: HTTP status code is not handled or not allowed 2026-01-24 20:27:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-plug-body-with-industrial-profile-1-2-mnpt-connection-used-with-isi-coupling>: HTTP status code is not handled or not allowed 2026-01-24 20:27:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/econ-kev-18-thslot-sleeve-9378te>: HTTP status code is not handled or not allowed 2026-01-24 20:27:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-10-gauge-dyneema-sleeve-9318d10>: HTTP status code is not handled or not allowed 2026-01-24 20:28:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-safety-quick-release-coupling-1-4-body-w-iso-6150-profile-3-8-mnpt>: HTTP status code is not handled or not allowed 2026-01-24 20:28:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thermostats-46-1496-hostb0041>: HTTP status code is not handled or not allowed 2026-01-24 20:28:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/push-to-connect-fittings-1-2x1-4-mnpt>: HTTP status code is not handled or not allowed 2026-01-24 20:28:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/power_tools/planers_jointers/woodworking_jointers_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:28:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-step-75-incline-ladder-24in-w-grip-strut-tread-ucl7504242>: HTTP status code is not handled or not allowed 2026-01-24 20:28:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30319562>: HTTP status code is not handled or not allowed 2026-01-24 20:28:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powermatic-pm1800b-18in-bandsaw-5hp-1ph-230v>: HTTP status code is not handled or not allowed 2026-01-24 20:28:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-static-shielding-bags-3w-x-3l-28-mil-transparent-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 20:28:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scale-hbs814gh-800-49>: HTTP status code is not handled or not allowed 2026-01-24 20:28:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 19103 pages (at 96 pages/min), scraped 12081 items (at 25 items/min) 2026-01-24 20:28:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-unit-5-tier-poly-z-brite-24w-x-48l-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 20:28:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-step-24in-w-steel-safety-angle-rolling-ladder-perforated-tread-safety-yellow-kdec103246-y>: HTTP status code is not handled or not allowed 2026-01-24 20:28:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-step-16in-w-steel-safety-angle-rolling-ladder-perforated-tread-safety-yellow-kdec106166-y>: HTTP status code is not handled or not allowed 2026-01-24 20:28:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-l-x-4-3-8-w-tilesetting-trowel-wood-handles>: HTTP status code is not handled or not allowed 2026-01-24 20:28:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/power_tools/planers_jointers/woodworking_jointers_accessories>: HTTP status code is not handled or not allowed 2026-01-24 20:28:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-step-24in-w-steel-safety-angle-rolling-ladder-perforated-tread-gray-kdec108246>: HTTP status code is not handled or not allowed 2026-01-24 20:28:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-18w-x-18l-6-mil-clear-250pack>: HTTP status code is not handled or not allowed 2026-01-24 20:28:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ball-peen-hammers-107g>: HTTP status code is not handled or not allowed 2026-01-24 20:28:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-shelves-48-w-storage-cabinet-gray-3>: HTTP status code is not handled or not allowed 2026-01-24 20:28:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gusseted-poly-bags-10w-x-4d-x-24h-3-mil-clear-500pack>: HTTP status code is not handled or not allowed 2026-01-24 20:28:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-fnpt-filter-regulator>: HTTP status code is not handled or not allowed 2026-01-24 20:28:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-jaw-solid-independent-chuck-semi-steel-body-camlock-d-10-inch-d1-6>: HTTP status code is not handled or not allowed 2026-01-24 20:28:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pinch-bars-p-8>: HTTP status code is not handled or not allowed 2026-01-24 20:28:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ball-valve-14-cos18-229>: HTTP status code is not handled or not allowed 2026-01-24 20:28:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pump-plunger-jbj22-15>: HTTP status code is not handled or not allowed 2026-01-24 20:28:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/penco-locker-single-tier-12x15x60-3-door-assembled-champagne>: HTTP status code is not handled or not allowed 2026-01-24 20:28:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/55-136-forged-hexagonal-steep-ripping-bar-36-inch-long>: HTTP status code is not handled or not allowed 2026-01-24 20:28:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/penco-locker-single-tier-12x18x72-3-door-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:28:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gusseted-poly-bags-12w-x-8d-x-30h-3-mil-clear-250pack>: HTTP status code is not handled or not allowed 2026-01-24 20:28:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/key-6x55-mdl-ghb-1340a-ghb1340a-150>: HTTP status code is not handled or not allowed 2026-01-24 20:28:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/push-to-connect-fittings-1-4-90d-elbow-w-tube>: HTTP status code is not handled or not allowed 2026-01-24 20:28:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/hand_tools_1/hand_tool_kits_sets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:28:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-step-steel-easy-turn-rolling-ladder-serrated-tread-safety-angle-kdad109242>: HTTP status code is not handled or not allowed 2026-01-24 20:28:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-step-aluminum-rolling-ladder-24in-w-ribbed-tread-14in-d-top-step-32in-handrails-wlar102244>: HTTP status code is not handled or not allowed 2026-01-24 20:28:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/needle-nose-pliers-w-side-cutter-coil-spring-5-9-16-inch>: HTTP status code is not handled or not allowed 2026-01-24 20:28:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/hand_tools_1/pliers/tongue_groove_pliers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:28:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/side-shelf-kit-for-computer-cabinet-dark-gray-set-of-2>: HTTP status code is not handled or not allowed 2026-01-24 20:28:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-unit-5-tier-nexelon-21w-x-72l-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 20:28:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-10w-x-18l-6-mil-clear-500pack>: HTTP status code is not handled or not allowed 2026-01-24 20:28:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-unit-5-tier-nexelon-24w-x-30l-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 20:28:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0-1-x-00005-0-001mm-digimatic-lite-micrometer-ratchet-no-spc-output>: HTTP status code is not handled or not allowed 2026-01-24 20:28:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-step-24in-w-10in-d-top-step-steel-rolling-ladder-grip-strut-tread-no-handrail-kdsr002242>: HTTP status code is not handled or not allowed 2026-01-24 20:28:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/hand_tools_1/hand_tool_kits_sets>: HTTP status code is not handled or not allowed 2026-01-24 20:28:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/screw-pitch-gage-4-42-tpi-and-0-4-7mm-51-leaves-inch-metric>: HTTP status code is not handled or not allowed 2026-01-24 20:28:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/afk2416-floor-stand-kit-qty-2-2400x1606-steelgray>: HTTP status code is not handled or not allowed 2026-01-24 20:28:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/wire_shelving/poly_zbrite_wire_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:28:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-step-aluminum-rolling-ladder-24in-w-grip-strut-14in-d-top-step-32in-handrails-wlar109245>: HTTP status code is not handled or not allowed 2026-01-24 20:28:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flammable-cabinet-drum-storage-with-self-close-doors>: HTTP status code is not handled or not allowed 2026-01-24 20:28:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/hand_tools_1/pliers/tongue_groove_pliers>: HTTP status code is not handled or not allowed 2026-01-24 20:28:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a18p16al-panel-junction-box-1675x1488-fits-18x16-aluminum>: HTTP status code is not handled or not allowed 2026-01-24 20:28:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-poly-z-brite-24w-x-72l-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 20:28:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-ventilated-locker-six-tier-12x18x12-6-door-assembled-1>: HTTP status code is not handled or not allowed 2026-01-24 20:28:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-nexelon-24w-x-36l-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 20:28:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-step-mobile-work-platform-40in-w-x-74in-l-36in-handrails-safety-yellow-wlwp143648sl-y>: HTTP status code is not handled or not allowed 2026-01-24 20:28:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-chrome-24w-x-36l-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 20:28:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-36x12x72-with-5-shelves-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:28:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-speed-2-button-pendant-station-cable-gland-style>: HTTP status code is not handled or not allowed 2026-01-24 20:28:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-gas-infrared-heater-091540ngs-straight-configuration-40-l-75000-btu>: HTTP status code is not handled or not allowed 2026-01-24 20:28:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/shelving/wire_shelving/poly_zbrite_wire_shelves>: HTTP status code is not handled or not allowed 2026-01-24 20:28:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a8n8p-panel-625x625-fits-8x8-steelwhite>: HTTP status code is not handled or not allowed 2026-01-24 20:28:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/model-no-bth-9-ball-transfer-head-svj-12-max-pipe-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:28:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22mm-non-metallic-non-illuminated-push-pull-maintained-square-mushroom-button-red>: HTTP status code is not handled or not allowed 2026-01-24 20:28:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1500-strapping-tape-1-x-60-yds-51-mil-clear-36pack>: HTTP status code is not handled or not allowed 2026-01-24 20:28:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-shelving-48x18x72-with-4-shelves-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:28:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-gas-infrared-heater-0922r10ngu-u-configuration-10-l-40000-btu>: HTTP status code is not handled or not allowed 2026-01-24 20:28:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-plastic-dunnage-rack-1500-lbs-capacity-2>: HTTP status code is not handled or not allowed 2026-01-24 20:28:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/knob-40-b-12-13-black-3406206>: HTTP status code is not handled or not allowed 2026-01-24 20:28:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-profile-shipping-floor-scale-22-w-x-42-d>: HTTP status code is not handled or not allowed 2026-01-24 20:28:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-deck-narrow-aisle-high-end-platform-truck-63-l-x-16-w>: HTTP status code is not handled or not allowed 2026-01-24 20:28:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foot-pound-ratchet-head-torque-wrenches-6016c>: HTTP status code is not handled or not allowed 2026-01-24 20:29:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mighty-products-mn-ms70-b0824-70-shade-cloth-shade-tarp-8-x-24>: HTTP status code is not handled or not allowed 2026-01-24 20:29:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mighty-products-mn-ms70-b1010-70-shade-cloth-shade-tarp-10-x-10>: HTTP status code is not handled or not allowed 2026-01-24 20:29:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clamp-together-adjustable-sleeve-w-o-ring-3-inch-diameter-304-stianless-steel-24-gage>: HTTP status code is not handled or not allowed 2026-01-24 20:29:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/open-end-static-shielding-bags-6w-x-26l-28-mil-transparent-100pack>: HTTP status code is not handled or not allowed 2026-01-24 20:29:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blue-cap-style-4-point-ratchet-suspension-hard-hat>: HTTP status code is not handled or not allowed 2026-01-24 20:29:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-3w-x-6l-8-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 20:29:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/table-4-chair-set-navy-polypropylene-cafe-chairs-42w-x-29h-round-natural-table>: HTTP status code is not handled or not allowed 2026-01-24 20:29:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/model-no-2-strap-wrench-11-34l-1-18-strap-width-3-12-od-tube-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:29:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-step-roll-fold-steel-rolling-ladder-grip-strut>: HTTP status code is not handled or not allowed 2026-01-24 20:29:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-back-black-leathersoft-executive-swivel-office-chair-with-molded-foam-seat-and-adjustable-arms>: HTTP status code is not handled or not allowed 2026-01-24 20:29:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mid-back-armless-white-leathersoft-contemporary-ribbed-executive-swivel-office-chair>: HTTP status code is not handled or not allowed 2026-01-24 20:29:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/support-for-locking-lever-1821161>: HTTP status code is not handled or not allowed 2026-01-24 20:29:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/183001-acrylic-refrigerator-lock-box-keyed-alike-clear>: HTTP status code is not handled or not allowed 2026-01-24 20:29:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/le51a-1-2-hp-115v-19ft-shutoff-120-gpm-2-discharge-sewage-pump>: HTTP status code is not handled or not allowed 2026-01-24 20:29:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-shrink-bags-80-ga-12w-x-16l-clear-250pack>: HTTP status code is not handled or not allowed 2026-01-24 20:29:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 19195 pages (at 92 pages/min), scraped 12101 items (at 20 items/min) 2026-01-24 20:29:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-shrink-bags-100-ga-12w-x-18l-clear-250pack>: HTTP status code is not handled or not allowed 2026-01-24 20:29:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cast-iron-grip-handle-weights-nist-class-f-20-lbs>: HTTP status code is not handled or not allowed 2026-01-24 20:29:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-cylindrical-weights-nist-class-f-with-traceable-certificate-500-grams>: HTTP status code is not handled or not allowed 2026-01-24 20:29:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leadhead-passive-welding-helmet-with-ir-10-plate>: HTTP status code is not handled or not allowed 2026-01-24 20:29:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8934-filament-tape-1-x-60-yds-4-mil-clear-12pack>: HTTP status code is not handled or not allowed 2026-01-24 20:29:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/locking-nut-bdb-1340-4743>: HTTP status code is not handled or not allowed 2026-01-24 20:29:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-w-x-36-d-production-workbench-maple-butcher-block-square-edge-with-drawers-shelf-tan>: HTTP status code is not handled or not allowed 2026-01-24 20:29:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-clear-laminating-pouches-3-mil-9-x-11-1-2-25-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:29:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/44195-double-diaphragm-fuel-transfer-pump>: HTTP status code is not handled or not allowed 2026-01-24 20:29:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v2g-eyewear-5-0-ir-filter-lens-with-black-strap-temples>: HTTP status code is not handled or not allowed 2026-01-24 20:29:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-woodcase-pencil-hb-2-yellow-barrel-144pack>: HTTP status code is not handled or not allowed 2026-01-24 20:29:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-cabinets-af-361-b-combination-cabinet-assembled-36x21x72-black>: HTTP status code is not handled or not allowed 2026-01-24 20:29:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/umbrellas_awnings_shades/outdoor_umbrellas_bases/umbrellas already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:29:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/upright-assembly-7>: HTTP status code is not handled or not allowed 2026-01-24 20:29:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/orange-cap-style-4-point-snap-lock-suspension-hard-hat>: HTTP status code is not handled or not allowed 2026-01-24 20:29:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/resealable-4-size-assortment-warning-poly-bags-15-mil-clear-100pack>: HTTP status code is not handled or not allowed 2026-01-24 20:29:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sidetracker-wall-mount-hose-reel-15-34-x-11-12>: HTTP status code is not handled or not allowed 2026-01-24 20:29:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-cylinder-small-cylinder-rescuecart-4500-psi-hansen-mp-4r>: HTTP status code is not handled or not allowed 2026-01-24 20:29:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hon-310-series-4-drawer-vertical-file-26-d-letter-putty>: HTTP status code is not handled or not allowed 2026-01-24 20:29:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-foot-x-20-foot-heavy-duty-silver-tarp>: HTTP status code is not handled or not allowed 2026-01-24 20:29:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-chain-1-link-100-l-hdpe-brown>: HTTP status code is not handled or not allowed 2026-01-24 20:29:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-w-x-36-d-production-workbench-maple-butcher-block-safety-edge-complete-bench-tan>: HTTP status code is not handled or not allowed 2026-01-24 20:29:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-hanford-triple-post-with-support-brace-no-base-lewiston-mailbox>: HTTP status code is not handled or not allowed 2026-01-24 20:29:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-chain-3-4-link-25-l-hdpe-traffic-orange>: HTTP status code is not handled or not allowed 2026-01-24 20:29:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-foam-tape-34-x-36-yds-625-mil-black-16pack>: HTTP status code is not handled or not allowed 2026-01-24 20:29:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-cabinets-bca-368413-p-bookcase-assembled-36x13x84-putty>: HTTP status code is not handled or not allowed 2026-01-24 20:29:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-foot-x-50-foot-heavy-duty-white-tarp>: HTTP status code is not handled or not allowed 2026-01-24 20:29:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stronghold-stainless-steel-storage-cabinet-60-x-24-x-60>: HTTP status code is not handled or not allowed 2026-01-24 20:29:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/outdoor_furniture/umbrellas_awnings_shades/outdoor_umbrellas_bases/umbrellas landed on page that is not a product page. 2026-01-24 20:29:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1400-strapping-tape-38-x-60-yds-43-mil-clear-12pack>: HTTP status code is not handled or not allowed 2026-01-24 20:29:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-chain-3-4-link-100-l-hdpe-traffic-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:29:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-pocket-oak-wall-rack-light-oak>: HTTP status code is not handled or not allowed 2026-01-24 20:29:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/015-027bk-10-rj45-pvc-slip-on-boots-for-cat-5e-cat-6-black-10-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:29:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/complete-electronic-packaging-workbench-maple-butcher-block-safety-edge-72-x-30>: HTTP status code is not handled or not allowed 2026-01-24 20:29:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uattend-rfid-wifi-time-clock>: HTTP status code is not handled or not allowed 2026-01-24 20:29:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-foot-x-30-foot-super-heavy-duty-brown-tarp>: HTTP status code is not handled or not allowed 2026-01-24 20:29:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-precision-ansi-roller-chain-200-1r-2-1-2-pitch-connecting-link>: HTTP status code is not handled or not allowed 2026-01-24 20:29:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/resealable-warning-poly-bags-5w-x-7l-15-mil-clear-100pack>: HTTP status code is not handled or not allowed 2026-01-24 20:29:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6878-winter-hard-hat-liner-kit-3-layer-liner-mouthpiece-black>: HTTP status code is not handled or not allowed 2026-01-24 20:29:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-foot-x-20-foot-super-heavy-duty-brown-tarp>: HTTP status code is not handled or not allowed 2026-01-24 20:29:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/300-j2507-wh-cat-3-v-max-u-style-keystone-jack-white>: HTTP status code is not handled or not allowed 2026-01-24 20:29:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electronic-packaging-workbench-plastic-safety-edge-60-x-30-with-riser-kit>: HTTP status code is not handled or not allowed 2026-01-24 20:29:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/full_height_storage_cabinets/stationary_storage_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:29:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fibre-disc-782c-36-tr-4-in-die-r400bb-25-per-inner-100-per-case>: HTTP status code is not handled or not allowed 2026-01-24 20:29:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-x-36-x-30-double-shelf-shop-stand-maple-black>: HTTP status code is not handled or not allowed 2026-01-24 20:29:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/packaging-workbench-maple-butcher-block-safety-edge-72-x-30-with-lower-shelf-kit>: HTTP status code is not handled or not allowed 2026-01-24 20:29:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anvil-cllr--seal-jsm-403-40303>: HTTP status code is not handled or not allowed 2026-01-24 20:29:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/100-009org-r-abs-plastic-bowl-12-orange-replacement>: HTTP status code is not handled or not allowed 2026-01-24 20:29:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-foot-x-12-foot-forest-green-tarp>: HTTP status code is not handled or not allowed 2026-01-24 20:29:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-foot-x-40-foot-blue-tarp>: HTTP status code is not handled or not allowed 2026-01-24 20:29:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/complete-mobile-packaging-workbench-plastic-square-edge-72-x-30>: HTTP status code is not handled or not allowed 2026-01-24 20:29:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usda-fda-rackable-pallet-40-x-48-blue-5000-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:29:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/074-828-5rd-cat-6-utp-mold-injection-snagless-patch-cord-5-ft-red>: HTTP status code is not handled or not allowed 2026-01-24 20:29:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spindle-shaft-ghd-2020pf-6117>: HTTP status code is not handled or not allowed 2026-01-24 20:29:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gib-adjusting-screw-ghb-1340-7237>: HTTP status code is not handled or not allowed 2026-01-24 20:29:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/cabinets/full_height_storage_cabinets/stationary_storage_cabinets>: HTTP status code is not handled or not allowed 2026-01-24 20:29:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washer-jtm-1050evs-ivs-066>: HTTP status code is not handled or not allowed 2026-01-24 20:29:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flap-disc-769f-05917-t29-5-x-7-8-in-60-yf-weight-10-per-case>: HTTP status code is not handled or not allowed 2026-01-24 20:29:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-paper-clips-vinyl-coated-wire-jumbo-assorted-colors-250pack>: HTTP status code is not handled or not allowed 2026-01-24 20:29:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-bands-size-84-312-x-12-155-bands1lb-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:29:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/dump_truck_inserts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:29:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wormtext-jvm836-42t>: HTTP status code is not handled or not allowed 2026-01-24 20:29:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-bands-size-54-assorted-length-sizes-1lb-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:29:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/front-blade-reference-blade-730371-wi-31-2-b-tg>: HTTP status code is not handled or not allowed 2026-01-24 20:29:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-foot-x-40-foot-camouflage-green-tarp>: HTTP status code is not handled or not allowed 2026-01-24 20:29:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/31-5in-x-55in-rectangular-tempered-glass-metal-table>: HTTP status code is not handled or not allowed 2026-01-24 20:29:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-electronic-packaging-workbench-plastic-safety-edge-60-x-30-with-riser-kit>: HTTP status code is not handled or not allowed 2026-01-24 20:29:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-5w-x-15l-4-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 20:29:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dial-jtm-124vs-lk-033>: HTTP status code is not handled or not allowed 2026-01-24 20:29:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergency-shower-decontamination-booth-with-finished-side-panels-40w-x-37d-x-93h>: HTTP status code is not handled or not allowed 2026-01-24 20:29:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-w-5-drawer-binder-lateral-file-gray-1>: HTTP status code is not handled or not allowed 2026-01-24 20:30:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-plunger-can-4-quart-polyethylene>: HTTP status code is not handled or not allowed 2026-01-24 20:30:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/dump_truck_inserts>: HTTP status code is not handled or not allowed 2026-01-24 20:30:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o-ring-dia-16x24-slt330fh-30-26>: HTTP status code is not handled or not allowed 2026-01-24 20:30:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wood-crate-four-way-entry-72-l-x-48-w-x-55-12-h>: HTTP status code is not handled or not allowed 2026-01-24 20:30:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/linuraedge-slim-profile-led-light-22-26v-dc-9w-acrylic-187>: HTTP status code is not handled or not allowed 2026-01-24 20:30:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40143-air-jet-washers-3-pack-rubber>: HTTP status code is not handled or not allowed 2026-01-24 20:30:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/41421-nylon-blast-suit-blue-medium-nylon-cotton>: HTTP status code is not handled or not allowed 2026-01-24 20:30:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polypropylene-bags-10l-x-6w-15-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 20:30:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-step-turn-platform-24-inch-w-grip-strut>: HTTP status code is not handled or not allowed 2026-01-24 20:30:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/halux-universal-led-treatment-light-n50-1-p-fx-double-arm-floor-stand>: HTTP status code is not handled or not allowed 2026-01-24 20:30:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 19287 pages (at 92 pages/min), scraped 12115 items (at 14 items/min) 2026-01-24 20:30:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30320su-f21t5835-21w-fluorescent-t5-bulb-mini-bipin-neutral-white>: HTTP status code is not handled or not allowed 2026-01-24 20:30:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40081-spot-blaster-round-adapter-rubber>: HTTP status code is not handled or not allowed 2026-01-24 20:30:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/halux-universal-led-treatment-light-n50-1-p-fx-double-arm-wall-mount>: HTTP status code is not handled or not allowed 2026-01-24 20:30:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bar-rack-horizontal-stacking-8>: HTTP status code is not handled or not allowed 2026-01-24 20:30:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/05010su-fc8t9ww-22w-fluorescent-t9-circline-bulb-4pin-warm-white>: HTTP status code is not handled or not allowed 2026-01-24 20:30:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-10w-x-18l-4-mil-clear-500pack>: HTTP status code is not handled or not allowed 2026-01-24 20:30:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40357-7-64-nozzles-3-pack-ceramic>: HTTP status code is not handled or not allowed 2026-01-24 20:30:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poly-tubing-8w-x-725l-4-mil-black-1-roll>: HTTP status code is not handled or not allowed 2026-01-24 20:30:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-connection-jsg1052-23>: HTTP status code is not handled or not allowed 2026-01-24 20:30:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-cal-recycled-rubber-flooring-1-4-x-4-x-4-rolls-black-rubber-mats>: HTTP status code is not handled or not allowed 2026-01-24 20:30:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss-tong-6>: HTTP status code is not handled or not allowed 2026-01-24 20:30:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wood-weave-plate-6>: HTTP status code is not handled or not allowed 2026-01-24 20:30:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/minigrip174-reclosable-poly-bags-6w-x-10l-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 20:30:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss-dipper-32-oz-solder-seal>: HTTP status code is not handled or not allowed 2026-01-24 20:30:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-barnstormer-variable-speed-with-std-l-bracket>: HTTP status code is not handled or not allowed 2026-01-24 20:30:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40337-replacement-hood-lens-pack-of-100-plexiglass>: HTTP status code is not handled or not allowed 2026-01-24 20:30:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-brake-cable-l-980-641244-floor-scrubber>: HTTP status code is not handled or not allowed 2026-01-24 20:30:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50-typhoon-exhaust-slant-wall-belt-drive-no-cone>: HTTP status code is not handled or not allowed 2026-01-24 20:30:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paper-clamp-jsg-0416-102139>: HTTP status code is not handled or not allowed 2026-01-24 20:30:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yoke-jsg-0735-hd-12033>: HTTP status code is not handled or not allowed 2026-01-24 20:30:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/41164-sealing-block-3-pack-rubber>: HTTP status code is not handled or not allowed 2026-01-24 20:30:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smart-extruder-for-the-z18>: HTTP status code is not handled or not allowed 2026-01-24 20:30:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ast44-swivel-tilt-581x400x400-aluminum>: HTTP status code is not handled or not allowed 2026-01-24 20:30:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/method-support-extruder>: HTTP status code is not handled or not allowed 2026-01-24 20:30:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40002-90-lb-portable-pressure-blaster-deadman-steel>: HTTP status code is not handled or not allowed 2026-01-24 20:30:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-block-reclosable-poly-bags-4w-x-6l-6-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 20:30:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-block-reclosable-poly-bags-4w-x-8l-4-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 20:30:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e19fwm12u20-fixed-wall-mount-rack-19in-12u-steelblack>: HTTP status code is not handled or not allowed 2026-01-24 20:30:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speakman-anystream-hotel-5-jet-3-1-2-dia-showerhead-polished-chrome-finish-2-5-gpm>: HTTP status code is not handled or not allowed 2026-01-24 20:30:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-ceiling-diffuser-2>: HTTP status code is not handled or not allowed 2026-01-24 20:30:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-block-reclosable-poly-bags-5w-x-7l-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 20:30:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-comp-toe-athletic-sneakers-size-8m-black>: HTTP status code is not handled or not allowed 2026-01-24 20:30:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-athletic-sneakers-water-and-oil-resistant-size-85m-black>: HTTP status code is not handled or not allowed 2026-01-24 20:30:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vandal-resistant-aerator>: HTTP status code is not handled or not allowed 2026-01-24 20:30:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/13-blue-scrubbing-pad-5-per-case>: HTTP status code is not handled or not allowed 2026-01-24 20:30:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bin-nestable-4-1-8-w-x23-5-8-d-x-6-h-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:30:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vl22-syspend-vl-motion-arm-22-in-672x255x35-alumgray>: HTTP status code is not handled or not allowed 2026-01-24 20:30:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-slip-on-shoes-size-10m-white>: HTTP status code is not handled or not allowed 2026-01-24 20:30:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-remover-burnishing-pad-5-per-case>: HTTP status code is not handled or not allowed 2026-01-24 20:30:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/minigrip-white-block-reclosable-poly-bags-8w-x-10l-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 20:30:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-comp-toe-athletic-sneakers-size-105m-black>: HTTP status code is not handled or not allowed 2026-01-24 20:30:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/minigrip-anti-static-reclosable-poly-bags-2-12w-x-3l-4-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 20:30:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-cartridge-filtratation-system-0-5-micron-rating-2-gpm>: HTTP status code is not handled or not allowed 2026-01-24 20:30:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jaw-ins1280-verti-lk-2908030>: HTTP status code is not handled or not allowed 2026-01-24 20:30:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mediumweight-plastic-teaspoons-white-1-000-per-carton>: HTTP status code is not handled or not allowed 2026-01-24 20:30:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/250-gallon-fuelcube-diesel-fuel-tank-12v-pump-kit-fuel-gauge>: HTTP status code is not handled or not allowed 2026-01-24 20:30:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-floor-mount-safety-bollard-w-chain-slots-cap-5-1-2-dia-x-48-h-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:30:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/planet-gear-80223-14>: HTTP status code is not handled or not allowed 2026-01-24 20:30:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-block-reclosable-poly-bags-10w-x-12l-6-mil-clear-500pack>: HTTP status code is not handled or not allowed 2026-01-24 20:30:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/minigrip174-reclosable-poly-bags-5w-x-7l-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 20:30:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-steel-stack-on-desk-hutch-72w-black>: HTTP status code is not handled or not allowed 2026-01-24 20:30:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/csd12126lg-concept-wall-mount-encl-1200x1200x600-18-body-ga>: HTTP status code is not handled or not allowed 2026-01-24 20:30:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dinex-translucent-lid-w-straw-slot-fits-dxft1607-1000-cs-clear-dx5600st8714>: HTTP status code is not handled or not allowed 2026-01-24 20:30:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/side-guide-replacement-part-for-rt18-rt26-rt37s-and-rt51s-paper-trimmers>: HTTP status code is not handled or not allowed 2026-01-24 20:30:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-packing-workbench-esd-safety-edge-60-x-36-with-lower-shelf-kit>: HTTP status code is not handled or not allowed 2026-01-24 20:30:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o-ring-285x2-jsm5130-29>: HTTP status code is not handled or not allowed 2026-01-24 20:30:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/atex74p55-zonex-panel-atex-695x505mm-fits-740x550mm-steelwhite>: HTTP status code is not handled or not allowed 2026-01-24 20:30:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/folding-door-gate-48-w-x-43-h-986578>: HTTP status code is not handled or not allowed 2026-01-24 20:30:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/janitorial-shadow-board-combo-kit-blue-on-black-general-purpose-composite-sbk101acp>: HTTP status code is not handled or not allowed 2026-01-24 20:30:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/janitorial-shadow-board-yellow-on-white-pro-series-acrylic-sb108fg>: HTTP status code is not handled or not allowed 2026-01-24 20:30:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/janitorial-shadow-board-red-on-white-pro-series-acrylic-sb106fg>: HTTP status code is not handled or not allowed 2026-01-24 20:30:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rx-pro-power-workstation-hutch-roller-cabinet-72w-x-25d-blue-w-chrome-dr-pulls>: HTTP status code is not handled or not allowed 2026-01-24 20:30:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nilotron-metered-air-fresheners-red-clover-tea-scent-7-oz-refill-12-case>: HTTP status code is not handled or not allowed 2026-01-24 20:31:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/valve-stem-jsm-842-sm-84205>: HTTP status code is not handled or not allowed 2026-01-24 20:31:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nilotron-metered-air-fresheners-tango-mango-scent-7-oz-refill-12-case>: HTTP status code is not handled or not allowed 2026-01-24 20:31:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waterhog-fashion-mat-charcoal-6-x-16>: HTTP status code is not handled or not allowed 2026-01-24 20:31:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/650-butoflex-chemical-resistant-butyl-gloves-supported-14-l-size-10-650310>: HTTP status code is not handled or not allowed 2026-01-24 20:31:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/janitorial-shadow-board-combo-kit-yellow-on-black-industrial-grade-aluminum-sbk107al>: HTTP status code is not handled or not allowed 2026-01-24 20:31:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-ss-304-cast-hex-head-plug-14-npt-male>: HTTP status code is not handled or not allowed 2026-01-24 20:31:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cheez-it-cracker-1-1-2-oz-single-serving-snack-pack-eight-packs-box>: HTTP status code is not handled or not allowed 2026-01-24 20:31:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-ss-316-cast-tee-34-npt-female>: HTTP status code is not handled or not allowed 2026-01-24 20:31:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c00-series-threaded-stem-caster-4>: HTTP status code is not handled or not allowed 2026-01-24 20:31:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nl52-stanzoil-neoprene-gloves-14-l-medium-weight-1-pair-size-9-337429>: HTTP status code is not handled or not allowed 2026-01-24 20:31:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 19386 pages (at 99 pages/min), scraped 12141 items (at 26 items/min) 2026-01-24 20:31:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waterhog-fashion-diamond-mat-evergreen-3-x-4>: HTTP status code is not handled or not allowed 2026-01-24 20:31:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a1200nect-ct-cabinet1200a-w-lugs-galvanizedgray>: HTTP status code is not handled or not allowed 2026-01-24 20:31:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rx-roller-cabinet-16-drawers-84w-x-30d-x-47-38h-black-with-red-drawer-pulls>: HTTP status code is not handled or not allowed 2026-01-24 20:31:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e6pbx-push-button-encl-extra-deep-6pb-x-305mm>: HTTP status code is not handled or not allowed 2026-01-24 20:31:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/valley-craft-a-frame-shallow-tray-lip-down>: HTTP status code is not handled or not allowed 2026-01-24 20:31:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waterhog-fashion-mat-navy-4-x-12>: HTTP status code is not handled or not allowed 2026-01-24 20:31:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rx-roller-cabinet-16-drawers-84w-x-30d-x-47-38h-black-with-blue-drawer-pulls>: HTTP status code is not handled or not allowed 2026-01-24 20:31:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/certified-174-2001-153-extraction-cleaner-5-gallon>: HTTP status code is not handled or not allowed 2026-01-24 20:31:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e9pby25-push-button-encl-extra-lgpanel-9pb-x-305mm>: HTTP status code is not handled or not allowed 2026-01-24 20:31:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprng-jwbs20-jwbs20-84>: HTTP status code is not handled or not allowed 2026-01-24 20:31:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/locking-nut-84-0028-00>: HTTP status code is not handled or not allowed 2026-01-24 20:31:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speedi-boot-90-175-register-vent-boot-w-adj-hangers-for-hvac-duct-work-4-x-12-x-8>: HTTP status code is not handled or not allowed 2026-01-24 20:31:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a1412nf-j-box-clamp-cover-type-4-1400x1200x600-steelgray>: HTTP status code is not handled or not allowed 2026-01-24 20:31:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-5-tier-72w-x-18d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 20:31:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manual-hi-gate-locked-exit-stainless>: HTTP status code is not handled or not allowed 2026-01-24 20:31:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/135-176-stainless-steel-corner-guard-16-ga-rd-corners-24-lx3-wx3-h>: HTTP status code is not handled or not allowed 2026-01-24 20:31:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90-176-ss-corner-guard-16-ga-drilled-holes-sq-corners-48-lx3-wx3-h>: HTTP status code is not handled or not allowed 2026-01-24 20:31:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speedi-grille-round-ceiling-register-field-installable-8-round-bowtie-volume-damper-12-x-12>: HTTP status code is not handled or not allowed 2026-01-24 20:31:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a16126t1pp-control-box-type-1-w-perf-panel-1600x1200x600-steelgray>: HTTP status code is not handled or not allowed 2026-01-24 20:31:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speedi-boot-torpedo-end-register-vent-boot-w-adj-hangers-for-hvac-duct-work-6-x-12-x-6>: HTTP status code is not handled or not allowed 2026-01-24 20:31:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3m-duct-tape-6969-silver-72-mm-x-54-8-m-10-7-mil-12-per-case-bulk>: HTTP status code is not handled or not allowed 2026-01-24 20:31:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/worm-shaft-6290592>: HTTP status code is not handled or not allowed 2026-01-24 20:31:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ase16x12x6nk-pull-box-screw-cover-1600x1200x600-steelgray>: HTTP status code is not handled or not allowed 2026-01-24 20:31:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sv10-non-powered-tablet-cart-white-aluminum>: HTTP status code is not handled or not allowed 2026-01-24 20:31:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-unit-5-tier-60w-x-24d-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 20:31:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rx-side-locker-3-drawers-4-shelves-24w-x-30d-x-60-78h-green-w-black-handles>: HTTP status code is not handled or not allowed 2026-01-24 20:31:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-green-wire-shelving-starter-unit-5-tier-36w-x-24d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 20:31:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bright-star-amber-2>: HTTP status code is not handled or not allowed 2026-01-24 20:31:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ring-table-25a-6295394>: HTTP status code is not handled or not allowed 2026-01-24 20:31:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speedi-boot-straight-register-vent-boot-w-adj-hangers-for-hvac-duct-work-6-x-12-x-6>: HTTP status code is not handled or not allowed 2026-01-24 20:31:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speedi-boot-torpedo-end-register-vent-boot-w-adj-hangers-for-hvac-duct-work-4-x-12-x-6>: HTTP status code is not handled or not allowed 2026-01-24 20:31:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rx-pro-power-workstation-hutch-roller-cabinet-55w-x-25d-blue-with-black-dr-pulls>: HTTP status code is not handled or not allowed 2026-01-24 20:31:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90-176-ss-corner-guard-16-ga-sq-corners-60-lx2-wx2-h-standard>: HTTP status code is not handled or not allowed 2026-01-24 20:31:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-equal-tee>: HTTP status code is not handled or not allowed 2026-01-24 20:31:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12x18-inch-acrylic-rectangle-convex-safety-mirror-plastic-back>: HTTP status code is not handled or not allowed 2026-01-24 20:31:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wrench-spanner-12-14-6295507>: HTTP status code is not handled or not allowed 2026-01-24 20:31:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-s2148b-black-epoxy-wire-shelf-48-w-x-21-d-with-clips>: HTTP status code is not handled or not allowed 2026-01-24 20:31:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lcd-industrial-computer-cabinet-black-unassembled-249190jbk>: HTTP status code is not handled or not allowed 2026-01-24 20:31:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-ft-natural-gas-rated-hose-with-quick-connect>: HTTP status code is not handled or not allowed 2026-01-24 20:31:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-reducing-tee-x-1-mnpt>: HTTP status code is not handled or not allowed 2026-01-24 20:31:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-outlet-with-shutoff-1-inlet>: HTTP status code is not handled or not allowed 2026-01-24 20:31:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-tubing-90-degree-elbow>: HTTP status code is not handled or not allowed 2026-01-24 20:31:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rx-pro-power-workstation-hutch-roller-cabinet-72w-x-30d-matte-blk-wblk-dr-pulls>: HTTP status code is not handled or not allowed 2026-01-24 20:31:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/temperature-sensor-part-replacement-for-pleasant-hearth-vent-free-fireplace-systems>: HTTP status code is not handled or not allowed 2026-01-24 20:31:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26x12x72-solar-star-greenhouse-w-solid-polycarbonate>: HTTP status code is not handled or not allowed 2026-01-24 20:31:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10l-x-16w-horizontal-bandsaw-2-hp-230v-3-phase>: HTTP status code is not handled or not allowed 2026-01-24 20:31:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-kc-thermaflex-flexible-hvac-duct-6-diameter-r4-2>: HTTP status code is not handled or not allowed 2026-01-24 20:31:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/13l-x-40w-straight-bed-engine-lathe-3-hp-230v-3-phase>: HTTP status code is not handled or not allowed 2026-01-24 20:31:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-green-wire-shelving-starter-unit-5-tier-54w-x-18d-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 20:31:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30x12x96-solar-star-greenhouse-w-solid-polycarbonate-gas-heater>: HTTP status code is not handled or not allowed 2026-01-24 20:31:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-u-channel-72-l-x-1-4-w-clear>: HTTP status code is not handled or not allowed 2026-01-24 20:31:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bollard-safety-net-4-foot-l-by-16-foot-w-fits-8-3-4-inches-to-9-5-8-inches-dia-brsn-416-10>: HTTP status code is not handled or not allowed 2026-01-24 20:31:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelate-silver-epoxy-wire-shelving-add-on-unit-5-tier-72w-x-21d-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 20:31:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/armorultra-job-on-pallet-3-layer-epoxy-system-5000-sq-ft-desert-tan>: HTTP status code is not handled or not allowed 2026-01-24 20:31:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/small-animal-hut-61w-x-62h-x-82l>: HTTP status code is not handled or not allowed 2026-01-24 20:31:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enclosed-bottom-computer-cart-30w-x-24d-x-40h-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:31:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cateraide-ice-caddy-200-lb-of-iceblack>: HTTP status code is not handled or not allowed 2026-01-24 20:31:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pulley-cover-jms12scms-241>: HTTP status code is not handled or not allowed 2026-01-24 20:31:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2ex-nut-12-inch-wss3-031a>: HTTP status code is not handled or not allowed 2026-01-24 20:31:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-foam-buna-n-pvc-strip-120-l-x-1-2-w-x-1-2-thick-black-acrylic-adhesive>: HTTP status code is not handled or not allowed 2026-01-24 20:31:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clp-series-swivel-earpiece-with-inline-push-to-talk-436644>: HTTP status code is not handled or not allowed 2026-01-24 20:31:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelate-silver-epoxy-wire-shelving-add-on-unit-5-tier-72w-x-18d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 20:31:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bollard-safety-net-4-foot-l-by-8-foot-w-fits-3-1-2-inches-to-4-1-2-inches-dia-brsn-48-5>: HTTP status code is not handled or not allowed 2026-01-24 20:32:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/index-card-holder-5-x-8-black>: HTTP status code is not handled or not allowed 2026-01-24 20:32:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bollard-safety-net-4-foot-l-by-8-foot-w-fits-7-3-4-inches-to-8-5-8-inches-dia-brsn-48-9>: HTTP status code is not handled or not allowed 2026-01-24 20:32:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-foam-epdm-tube-60-l-x-7-8-id-x-1-5-16-od-black>: HTTP status code is not handled or not allowed 2026-01-24 20:32:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shaft-model-201planer-6012213>: HTTP status code is not handled or not allowed 2026-01-24 20:32:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classical-a-v-belt-a43>: HTTP status code is not handled or not allowed 2026-01-24 20:32:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8090253157-iron-series-two-stage-vertical-air-compressor-5hp-80-gallon>: HTTP status code is not handled or not allowed 2026-01-24 20:32:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bollard-safety-net-4-foot-l-by-24-foot-l-5-5-8-inches-to-6-5-8-inches-dia-brsn-424-7>: HTTP status code is not handled or not allowed 2026-01-24 20:32:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/34x12x72-solar-star-greenhouse-w-poly-top-and-ends-drop-down-sides-gas-heater>: HTTP status code is not handled or not allowed 2026-01-24 20:32:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/supratile-interlocking-edge-7mm-thick-20-12-x-2-12-black-4-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:32:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 19484 pages (at 98 pages/min), scraped 12167 items (at 26 items/min) 2026-01-24 20:32:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-solid-galvanized-steel-shelving-starter-unit-5-tier-36w-x-24d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 20:32:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8090250054-single-stage-vertical-air-compressor-35hp-60-gallon>: HTTP status code is not handled or not allowed 2026-01-24 20:32:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-dominator-extra-stripping-pad-5-per-case>: HTTP status code is not handled or not allowed 2026-01-24 20:32:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/krylon-osha-paint-safety-red>: HTTP status code is not handled or not allowed 2026-01-24 20:32:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ground-stake>: HTTP status code is not handled or not allowed 2026-01-24 20:32:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/krylon-industrial-work-day-enamel-paint-gloss-pink>: HTTP status code is not handled or not allowed 2026-01-24 20:32:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/small-animal-hut-10w-x-65h-x-122l>: HTTP status code is not handled or not allowed 2026-01-24 20:32:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_lockout_identification_tags already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:32:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classical-b-v-belt-b180>: HTTP status code is not handled or not allowed 2026-01-24 20:32:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/majestic-greenhouse-28w-x-24l-w-roll-up-sides>: HTTP status code is not handled or not allowed 2026-01-24 20:32:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cast-acrylic-sheet-48-l-x-48-w-x-1-4-thick-black>: HTTP status code is not handled or not allowed 2026-01-24 20:32:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/env100-env-oil-only-pads15w-x-19lheavy-weightbale100>: HTTP status code is not handled or not allowed 2026-01-24 20:32:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classical-cogged-ax-v-belt-ax34>: HTTP status code is not handled or not allowed 2026-01-24 20:32:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labels-with-static-safe-print-2-12l-x-1-34w-blackyellow-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-24 20:32:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:32:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cast-acrylic-bar-24-l-x-4-w-x-3-16-thick-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:32:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labels-with-rush-medical-supplies-print-4l-x-4w-fluorescent-red-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-24 20:32:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vinyl-ezd-ring-reference-binder-1-capacity-red>: HTTP status code is not handled or not allowed 2026-01-24 20:32:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelate-silver-epoxy-wire-shelving-starter-unit-5-tier-60w-x-21d-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 20:32:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mailing-tubes-with-caps-3dia-x-18l-0070-thick-red-24pack>: HTTP status code is not handled or not allowed 2026-01-24 20:32:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/94150-do-not-enter-sign-high-intensity-prismatic-reflective-sign-aluminum-30w-x-30h>: HTTP status code is not handled or not allowed 2026-01-24 20:32:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/weathershield-commercial-canopy-18w-x-50l-tan>: HTTP status code is not handled or not allowed 2026-01-24 20:32:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-green-wire-shelving-add-on-unit-5-tier-42w-x-21d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 20:32:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/65350-blank-accident-prevention-tag-no-grommet-yellow-polyester-3w-x-5-34h>: HTTP status code is not handled or not allowed 2026-01-24 20:32:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_lockout_identification_tags landed on page that is not a product page. 2026-01-24 20:32:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bubble-lined-poly-mailers-8-12w-x-12l-2-mil-white-100pack>: HTTP status code is not handled or not allowed 2026-01-24 20:32:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-green-wire-shelving-add-on-unit-5-tier-24w-x-21d-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 20:32:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 20:32:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/foam/foam_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:32:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/65504-lockout-tag-danger-do-not-operate-electricians-at-work-vinyl-25pack>: HTTP status code is not handled or not allowed 2026-01-24 20:32:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:32:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30x12x96-solar-star-greenhouse-w-poly-ends-and-drop-down-sides>: HTTP status code is not handled or not allowed 2026-01-24 20:32:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_lockout_identification_tags already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:32:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/34x12x48-solar-star-greenhouse-w-poly-ends-and-drop-down-sides-gas-heater>: HTTP status code is not handled or not allowed 2026-01-24 20:32:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galaxy-pastry-brush-3white>: HTTP status code is not handled or not allowed 2026-01-24 20:32:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:32:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stayflats-plus-peel-seal-mailers-6w-x-8l-kraft-100pack>: HTTP status code is not handled or not allowed 2026-01-24 20:32:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/148140-bilingual-texas-gun-sign-3007-polyester-28w-x-20h>: HTTP status code is not handled or not allowed 2026-01-24 20:32:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/bouffants_hairnets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:32:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pointer-model-201planer-6012239>: HTTP status code is not handled or not allowed 2026-01-24 20:32:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-standard-cold-forged-4x4-12-rigid-6x2-plastex-roller-900lb>: HTTP status code is not handled or not allowed 2026-01-24 20:32:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/foam/foam_sheets>: HTTP status code is not handled or not allowed 2026-01-24 20:32:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-seal-envelopes-12w-x-15-12l-white-500pack>: HTTP status code is not handled or not allowed 2026-01-24 20:32:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 20:32:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-u-channel-24-l-x-3-8-w-clear>: HTTP status code is not handled or not allowed 2026-01-24 20:32:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_lockout_identification_tags>: HTTP status code is not handled or not allowed 2026-01-24 20:32:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pd439g-large-ear-plug-dispenser-acrylic-12w-x-13h>: HTTP status code is not handled or not allowed 2026-01-24 20:32:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gusseted-merchandise-bags-10w-x-2d-x-15h-kraft-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 20:32:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs>: HTTP status code is not handled or not allowed 2026-01-24 20:32:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-90-degree-angle-12-l-x-1-w-clear>: HTTP status code is not handled or not allowed 2026-01-24 20:32:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/bouffants_hairnets>: HTTP status code is not handled or not allowed 2026-01-24 20:32:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-x-15-x-8-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 20:32:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700256clr-pegboard-floor-stand-w5-c-channel-sliding-16w-x-60h-clr-1-pc>: HTTP status code is not handled or not allowed 2026-01-24 20:32:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/panel-face-envelopes-packing-list-enclosed-print-7l-x-5-12w-orange-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 20:32:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelite-vented-plastic-shelving-starter-unit-5-tier-48w-x-24d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 20:32:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:32:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4dle-sprocket-model-201-plnr-6012276>: HTTP status code is not handled or not allowed 2026-01-24 20:32:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700250pnk-pegboard-floor-stand-16w-x-60h-pk-1-pc>: HTTP status code is not handled or not allowed 2026-01-24 20:32:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/104347-toughstripe-floor-marking-tape-polyester-3w-x-100l-blackyellow>: HTTP status code is not handled or not allowed 2026-01-24 20:32:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700258pur-pegboard-floor-stand-w5-c-channel-sliding-16w-x-66h-pl-1-pc>: HTTP status code is not handled or not allowed 2026-01-24 20:32:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-poly-tech-wheel-5x1-38-12-ss-ball>: HTTP status code is not handled or not allowed 2026-01-24 20:32:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-x-18-x-14-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 20:32:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labels-w-do-not-accept-if-damaged-print-5l-x-3w-fluorescent-red-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-24 20:33:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelving-add-on-unit-5-tier-48w-x-21d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 20:33:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/special-screw-model-201planer-6012247>: HTTP status code is not handled or not allowed 2026-01-24 20:33:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-plastex-wheel-6x1-12-34-roller>: HTTP status code is not handled or not allowed 2026-01-24 20:33:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/252809-brochure-hlr-w8-pockets-165w-x-7h-clr-1pack>: HTTP status code is not handled or not allowed 2026-01-24 20:33:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/locks_lockout_devices/lockout_tagout_devices/lockout_padlocks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:33:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips>: HTTP status code is not handled or not allowed 2026-01-24 20:33:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cast-acrylic-sheet-24-l-x-24-w-x-1-8-thick-blue-ps-cact-109>: HTTP status code is not handled or not allowed 2026-01-24 20:33:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labels-with-do-not-stack-print-3l-x-2w-fluorescent-green-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-24 20:33:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/star-washer-m8-powerfeeders-6288764>: HTTP status code is not handled or not allowed 2026-01-24 20:33:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/878-500-internal-containers-for-878-sharps-disposal-yellow-4-case>: HTTP status code is not handled or not allowed 2026-01-24 20:33:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 19575 pages (at 91 pages/min), scraped 12194 items (at 27 items/min) 2026-01-24 20:33:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-qt-e-z-smart-drum-funnel-with-2-threads>: HTTP status code is not handled or not allowed 2026-01-24 20:33:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-silicone-sheet-36-l-x-36-w-x-1-4-thick-blue-detectable-60a>: HTTP status code is not handled or not allowed 2026-01-24 20:33:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-metal-wheel-5x2-12-34-ball>: HTTP status code is not handled or not allowed 2026-01-24 20:33:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/instruction-plate-feed-ro-3330236>: HTTP status code is not handled or not allowed 2026-01-24 20:33:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polystyrene-sheet-96-l-x-48-w-x-1-4-thick-white>: HTTP status code is not handled or not allowed 2026-01-24 20:33:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-x-5-x-24-full-over-lap-container>: HTTP status code is not handled or not allowed 2026-01-24 20:33:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-neo-rubber-on-alum-6x1-12-58-roller>: HTTP status code is not handled or not allowed 2026-01-24 20:33:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-v-groove-wheel-4x2-34-roller>: HTTP status code is not handled or not allowed 2026-01-24 20:33:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cap-screw-m5-x-08p-x-10lg-pfeeders-6288763>: HTTP status code is not handled or not allowed 2026-01-24 20:33:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-x-20-x-14-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 20:33:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700356org-pegboard-powerwing-display-1675w-x-60h-rg-1-pc>: HTTP status code is not handled or not allowed 2026-01-24 20:33:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ring-ret-5100-62-6670008>: HTTP status code is not handled or not allowed 2026-01-24 20:33:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelon-wire-shelving-starter-unit-1-dunnage-shelf-60w-x-18d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 20:33:18 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/locks_lockout_devices/lockout_tagout_devices/lockout_padlocks landed on page that is not a product page. 2026-01-24 20:33:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vision-elite-blx-series-rollerball-pen-05-mm-assorted-5-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:33:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700444org-pinwheel-pegboard-display-wrb-135w-x-44h-rg-1-pc>: HTTP status code is not handled or not allowed 2026-01-24 20:33:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/colored-thermal-transfer-labels-4w-x-6l-3-core-pink-4pack>: HTTP status code is not handled or not allowed 2026-01-24 20:33:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultem-pei-sheet-6-l-x-6-w-x-1-2-thick-amber>: HTTP status code is not handled or not allowed 2026-01-24 20:33:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-flag-16-x16-orange-w-white-x>: HTTP status code is not handled or not allowed 2026-01-24 20:33:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lifting-eye-m20x25px30lg-model-201-6012193>: HTTP status code is not handled or not allowed 2026-01-24 20:33:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700420gre-pinwheel-pegboard-display-16w-x-60h-gn-1-pc>: HTTP status code is not handled or not allowed 2026-01-24 20:33:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700420org-pinwheel-pegboard-display-16w-x-60h-rg-1-pc>: HTTP status code is not handled or not allowed 2026-01-24 20:33:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultem-pei-bar-12-l-x-2-w-x-1-2-thick-amber>: HTTP status code is not handled or not allowed 2026-01-24 20:33:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-flag-16-x16-green-w-yellow-x>: HTTP status code is not handled or not allowed 2026-01-24 20:33:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-epdm-strip-120-l-x-7-w-x-3-16-thick-black>: HTTP status code is not handled or not allowed 2026-01-24 20:33:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interion-174-48-wide-left-handed-return-table-rustic-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:33:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-wall-mounted-shelf-15-inch-120-inch-shelf>: HTTP status code is not handled or not allowed 2026-01-24 20:33:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-retractable-belt-barrier-black-mount-red-belt-7-1-2-ft-l>: HTTP status code is not handled or not allowed 2026-01-24 20:33:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-wall-mounted-shelf-12-inch-48-inch-shelf-with-hooks>: HTTP status code is not handled or not allowed 2026-01-24 20:33:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-moving-box-combo>: HTTP status code is not handled or not allowed 2026-01-24 20:33:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bearing-fafnir-ra012rr-6060056>: HTTP status code is not handled or not allowed 2026-01-24 20:33:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-dia-round-removable-paper-labels-fluorescent-green-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-24 20:33:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/split-ring-imp-s-s-1-2>: HTTP status code is not handled or not allowed 2026-01-24 20:33:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700432pnk-hshaped-pegboard-display-32w-x-60h-pk-1-pc>: HTTP status code is not handled or not allowed 2026-01-24 20:33:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-x-2-universal-clevis-pin-300-series-stainless-steel-clpus-0250-2000>: HTTP status code is not handled or not allowed 2026-01-24 20:33:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bib6-bench-in-a-box-packing-workbench-esd-laminate-t-mold-edge-60-x-30>: HTTP status code is not handled or not allowed 2026-01-24 20:33:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/13-x-18-kra-stayflat-mailers>: HTTP status code is not handled or not allowed 2026-01-24 20:33:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:33:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vinyl-rectangular-desk-pads-17-x-22-clear-pack-of-four>: HTTP status code is not handled or not allowed 2026-01-24 20:33:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bollard-cover-7-dia-60-inch-h-yellow-blue-tape>: HTTP status code is not handled or not allowed 2026-01-24 20:33:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-dia-round-paper-labels-with-do-not-use-print-fluorescent-yellow-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-24 20:33:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bollard-cover-7-dia-52-inch-h-red-white-tape>: HTTP status code is not handled or not allowed 2026-01-24 20:33:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honeywell-barcode-scanner-holder-for-use-with-hyperion-1300g>: HTTP status code is not handled or not allowed 2026-01-24 20:33:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honeywell-85l-usb-cable-for-use-with-hand-held-scanners>: HTTP status code is not handled or not allowed 2026-01-24 20:33:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-x-2-x-2-mailers>: HTTP status code is not handled or not allowed 2026-01-24 20:33:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rectangular-inventory-paper-labels-9l-x-3w-fluorescent-green-roll-of-250>: HTTP status code is not handled or not allowed 2026-01-24 20:33:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-profile-self-dumping-forklift-hopper-1-1-2-cubic-yard-6000-lb-cap>: HTTP status code is not handled or not allowed 2026-01-24 20:33:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-x-4-x-4-mailers>: HTTP status code is not handled or not allowed 2026-01-24 20:33:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/work-height-platform-truck-rnh-3060-6py-with-hardboard-top-30-x-60-fixed-height>: HTTP status code is not handled or not allowed 2026-01-24 20:33:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/13-1-2-x-3-1-2-x-3-1-2-mailers>: HTTP status code is not handled or not allowed 2026-01-24 20:33:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cam-roller-3076230>: HTTP status code is not handled or not allowed 2026-01-24 20:33:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-screw-m8-x-125p-x-16lg-mdl201plnr-6012236>: HTTP status code is not handled or not allowed 2026-01-24 20:33:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets>: HTTP status code is not handled or not allowed 2026-01-24 20:33:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spacer-spdl-1-x-12-27-wd-shaper-3742006>: HTTP status code is not handled or not allowed 2026-01-24 20:33:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-dia-round-paper-labels-with-partial-print-fluorescent-yellow-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-24 20:33:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/work-height-platform-truck-rnb2-2448-6pyah-with-lower-shelf-24-x-48-adj-height>: HTTP status code is not handled or not allowed 2026-01-24 20:33:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ft075c-5-f-t-trap-1-1-4-in>: HTTP status code is not handled or not allowed 2026-01-24 20:33:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-outdoor-in-ground-steel-flat-bench-expanded-metal-red>: HTTP status code is not handled or not allowed 2026-01-24 20:33:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-coverall-knit-cuffs-and-attached-hood-white-red-2x-large-4565-blk-2xl>: HTTP status code is not handled or not allowed 2026-01-24 20:33:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-12-l-x-1-2-w-x-1-4-thick-black-antistatic>: HTTP status code is not handled or not allowed 2026-01-24 20:33:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/office-partition-panel-with-partial-window-and-raceway-48-1-4w-x-100h-gray-gl>: HTTP status code is not handled or not allowed 2026-01-24 20:33:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tsi-jumbo-contractor-bench-seat-center-console-floor-shift-compatible-model-33411-in-black>: HTTP status code is not handled or not allowed 2026-01-24 20:33:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-office-partition-panel-with-partial-window-48-1-4w-x-101-1-2h-blue-gl>: HTTP status code is not handled or not allowed 2026-01-24 20:33:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-socket-3-8-drive-12-point-15-16-short>: HTTP status code is not handled or not allowed 2026-01-24 20:33:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-2-point-discriminator>: HTTP status code is not handled or not allowed 2026-01-24 20:33:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-socket-3-8-drive-12-point-8mm-deep>: HTTP status code is not handled or not allowed 2026-01-24 20:33:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cb12700-600-keys-adjustable-key-box-12w-x-3112d-x-25h-grey-aluminum>: HTTP status code is not handled or not allowed 2026-01-24 20:33:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-1-8-x-9-1-4-x-3-literature-mailers>: HTTP status code is not handled or not allowed 2026-01-24 20:33:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ax13310-keypad-depository-safe-14w-x-14d-x-2714h-072078-cubic-ft>: HTTP status code is not handled or not allowed 2026-01-24 20:34:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/econoline-storage-building-30w-x-15h-x-35l-white-pony-wall>: HTTP status code is not handled or not allowed 2026-01-24 20:34:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/13-x-10-x-4-literature-mailers>: HTTP status code is not handled or not allowed 2026-01-24 20:34:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/af12654-external-battery-pack-for-select-safes>: HTTP status code is not handled or not allowed 2026-01-24 20:34:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-electric-office-partition-panel-with-partial-window-36-1-4w-x-101-1-2h-tan-gl>: HTTP status code is not handled or not allowed 2026-01-24 20:34:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-office-partition-panel-w-partial-window-and-pass-thru-cable-60-1-4wx101-1-2h-tan-gl>: HTTP status code is not handled or not allowed 2026-01-24 20:34:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/econoline-storage-building-20w-x-12h-x-35l-white-pony-wall>: HTTP status code is not handled or not allowed 2026-01-24 20:34:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:34:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-adjustable-wire-shelving-24-w-x-24-d-x-63-h-nexelon>: HTTP status code is not handled or not allowed 2026-01-24 20:34:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lbl2-lp835-2-led-wraparound-mvolt-2000-lumens>: HTTP status code is not handled or not allowed 2026-01-24 20:34:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-office-partition-panel-48-1-4w-x-101-1-2h-tan-gl>: HTTP status code is not handled or not allowed 2026-01-24 20:34:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ax12880-left-opening-biometric-wall-safe-1512w-x-4d-x-3112h-082-cubic-ft>: HTTP status code is not handled or not allowed 2026-01-24 20:34:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-14-1-2x20-self-seal-white-bubble-mailers-25-packs>: HTTP status code is not handled or not allowed 2026-01-24 20:34:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 19673 pages (at 98 pages/min), scraped 12213 items (at 19 items/min) 2026-01-24 20:34:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/msmk404va-designated-smoking-area-sign-10w-x-7h-aluminum>: HTTP status code is not handled or not allowed 2026-01-24 20:34:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-foam-epdm-sheet-12-l-x-12-w-x-3-4-thick-black>: HTTP status code is not handled or not allowed 2026-01-24 20:34:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-60-l-x-1-1-2-w-x-1-2-thick-white>: HTTP status code is not handled or not allowed 2026-01-24 20:34:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/office-partition-panel-with-pass-thru-cable-48-1-4w-x-100h-tan-gl>: HTTP status code is not handled or not allowed 2026-01-24 20:34:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/office-partition-panel-with-pass-thru-cable-60-1-4w-x-100h-tan-gl>: HTTP status code is not handled or not allowed 2026-01-24 20:34:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mabr607vp-caution-sign-open-door-slowly-14w-x-10h-plastic>: HTTP status code is not handled or not allowed 2026-01-24 20:34:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 20:34:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/17-x-21-white-stayflat-mailers>: HTTP status code is not handled or not allowed 2026-01-24 20:34:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/recycled-rubber-tackboard-aluminum-trim-96-1-2w-x-48-1-2h-black>: HTTP status code is not handled or not allowed 2026-01-24 20:34:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/matr806va-notice-sign-no-trespassing-14w-x-10h-aluminum>: HTTP status code is not handled or not allowed 2026-01-24 20:34:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electric-office-partition-panel-with-partial-window-36-1-4w-x-100h-tan-gl>: HTTP status code is not handled or not allowed 2026-01-24 20:34:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flammable-cabinet-55-gallon-self-close-door-4-w-34-d-66-3-8-h>: HTTP status code is not handled or not allowed 2026-01-24 20:34:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-socket-3-8-drive-12-point-1-2-deep>: HTTP status code is not handled or not allowed 2026-01-24 20:34:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labels-with-made-in-usa-print-3l-x-2w-redwhiteblue-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-24 20:34:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-x-30-open-end-static-shielding-bags>: HTTP status code is not handled or not allowed 2026-01-24 20:34:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-socket-1-4-drive-7-32-6-point-short>: HTTP status code is not handled or not allowed 2026-01-24 20:34:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labels-with-this-end-up-print-8l-x-4w-redwhite-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-24 20:34:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-circle-mailing-labels-1-dia-frosty-white-roll-of-5000>: HTTP status code is not handled or not allowed 2026-01-24 20:34:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/daddy-long-legs-side-panel-30l-white>: HTTP status code is not handled or not allowed 2026-01-24 20:34:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/13-x-6-x-15-3-4-kra-shopping-bag>: HTTP status code is not handled or not allowed 2026-01-24 20:34:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-foam-epdm-sheet-36-l-x-36-w-x-1-16-thick-black>: HTTP status code is not handled or not allowed 2026-01-24 20:34:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-24-l-x-1-w-x-1-4-thick-black-antistatic>: HTTP status code is not handled or not allowed 2026-01-24 20:34:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-floor-truck-lock-6-34>: HTTP status code is not handled or not allowed 2026-01-24 20:34:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnetic-wall-mount-barrier-15-red-danger-keep-out-banner>: HTTP status code is not handled or not allowed 2026-01-24 20:34:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnetic-wall-mount-barrier-15-red-do-not-enter-arc-flash-boundary-banner>: HTTP status code is not handled or not allowed 2026-01-24 20:34:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/impact-flex-socket-set-3-8-drive-6-point-short-6-piece>: HTTP status code is not handled or not allowed 2026-01-24 20:34:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plus-barrier-set-x5-red-double-sided-danger>: HTTP status code is not handled or not allowed 2026-01-24 20:34:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-lever-dolly-60-plastex-wheels>: HTTP status code is not handled or not allowed 2026-01-24 20:34:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mrst813vs-notice-sign-wash-hands-before-returning-to-work-14w-x-10h-adhesive-vinyl>: HTTP status code is not handled or not allowed 2026-01-24 20:34:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-composite-carbon-fiber-bar-12-l-x-3-w-x-1-16-thick-black-bulk-cs-cf-19>: HTTP status code is not handled or not allowed 2026-01-24 20:34:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cleaning-pads-for-p-4000-multi-purpose-dual-headed-floor-machine-6>: HTTP status code is not handled or not allowed 2026-01-24 20:34:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tp-2015-n-20-round-floor-machine-1-5-hp-175rpm-120v-50-cord>: HTTP status code is not handled or not allowed 2026-01-24 20:34:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dp-1334-n-13-floor-machine-3-4-hp-175rpm-120v-50-cord>: HTTP status code is not handled or not allowed 2026-01-24 20:34:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s153-zebra-thermal-transfer-ribbons-4w-x-1476l-wax-black-24-rollscase>: HTTP status code is not handled or not allowed 2026-01-24 20:34:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s153-zebra-thermal-transfer-ribbons-3-18w-x-1476l-waxresin-black-24-rollscase>: HTTP status code is not handled or not allowed 2026-01-24 20:34:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/impact-socket-1-drive-1-5-16-6-point-deep>: HTTP status code is not handled or not allowed 2026-01-24 20:34:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-48-l-x-4-w-x-1-2-thick-black-antistatic>: HTTP status code is not handled or not allowed 2026-01-24 20:34:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30x25-liter-floor-base-cabinet-partially-lined-top-tray-31w-x-20d-x-36-5-8h>: HTTP status code is not handled or not allowed 2026-01-24 20:34:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-socket-1-4-drive-deep-6-point-1-2>: HTTP status code is not handled or not allowed 2026-01-24 20:34:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-socket-1-4-drive-deep-6-point-9-16>: HTTP status code is not handled or not allowed 2026-01-24 20:34:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14x36-4-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-24 20:34:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-foam-epdm-sheet-24-l-x-12-w-x-1-16-thick-black>: HTTP status code is not handled or not allowed 2026-01-24 20:34:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mb-4-2545-mobile-service-bench-20-bin-box-and-2-door-locking-cabinet-48-w-x-25-d-x-40-h-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:34:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f6-woven-split-wrappable-sleeve-3-4-dia-50-black>: HTTP status code is not handled or not allowed 2026-01-24 20:34:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/impact-socket-1-2-drive-23mm-6-point-deep>: HTTP status code is not handled or not allowed 2026-01-24 20:34:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pm2013bt-20-bandsaw-with-armorglide-5-hp-1-phase-230v>: HTTP status code is not handled or not allowed 2026-01-24 20:34:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/array-conference-table-with-power-data-grommet-120l-x-48w-x-29h-cherrywhite>: HTTP status code is not handled or not allowed 2026-01-24 20:34:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-strapping-5-8-inches-w-x-0-point-015-inches-x-200-feet>: HTTP status code is not handled or not allowed 2026-01-24 20:34:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-step-grip-st-600-lb-cap-heavy-duty-steel-rolling-ladder-kdhd112242>: HTTP status code is not handled or not allowed 2026-01-24 20:34:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-5-8-tube-od-zinc-plated-anti-vibration-cush-a-clamp>: HTTP status code is not handled or not allowed 2026-01-24 20:34:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mcsp133vs-danger-sign-confined-space-enter-by-permit-only-10w-x-7h-adhesive-vinyl>: HTTP status code is not handled or not allowed 2026-01-24 20:34:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/socket-1-4-drive-t25-internal-torx>: HTTP status code is not handled or not allowed 2026-01-24 20:34:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/socket-1-4-drive-t30-internal-torx>: HTTP status code is not handled or not allowed 2026-01-24 20:34:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polyester-film-300-l-x-36-w-x-0-001-thick-clear>: HTTP status code is not handled or not allowed 2026-01-24 20:34:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8flexo-pet-general-purpose-sleeving-dia-125-green>: HTTP status code is not handled or not allowed 2026-01-24 20:34:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/impact-socket-3-4-drive-2-1-4-6-point-short>: HTTP status code is not handled or not allowed 2026-01-24 20:34:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:34:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/impact-socket-3-4-drive-2-3-8-6-point-short>: HTTP status code is not handled or not allowed 2026-01-24 20:34:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:34:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clamp-on-crossover-3>: HTTP status code is not handled or not allowed 2026-01-24 20:34:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4x6-6-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-24 20:34:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/right-hand-corner-credenza-71w-x-24-36d-x-29-half-h-artisan-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:34:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:34:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mfsd929va-emergency-sign-safety-shower-graphic-14w-x-10h-aluminum>: HTTP status code is not handled or not allowed 2026-01-24 20:34:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cpvc-bar-36-l-x-1-w-x-1-16-thick-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:34:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:34:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mcsp110vs-danger-sign-confined-space-keep-out-14w-x-10h-adhesive-vinyl>: HTTP status code is not handled or not allowed 2026-01-24 20:34:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durawrap-1050-ballistic-nylon-wrap-w-velcro-closure-350-dia-25-black>: HTTP status code is not handled or not allowed 2026-01-24 20:35:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-drawer-hanging-pedestal-in-artisan-gray-executive-modular-furniture>: HTTP status code is not handled or not allowed 2026-01-24 20:35:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-24 20:35:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mfsd913vp-emergency-sign-eye-wash-14w-x-10h-plastic>: HTTP status code is not handled or not allowed 2026-01-24 20:35:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cpvc-sheet-12-l-x-12-w-x-1-4-thick-gray-ps-cpvc-17>: HTTP status code is not handled or not allowed 2026-01-24 20:35:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-ts-1044h-10-10-hybrid-style-table-saw-36-rip-1-34hp-1ph-110v>: HTTP status code is not handled or not allowed 2026-01-24 20:35:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4x5-1-5-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-24 20:35:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 20:35:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/matr901vs-no-trespassing-sign-violators-will-be-prosecuted-10w-x-7h-adhesive-vinyl>: HTTP status code is not handled or not allowed 2026-01-24 20:35:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 20:35:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-1-2x13-1-5-mil-flat-polypropylene-poly-bags>: HTTP status code is not handled or not allowed 2026-01-24 20:35:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cortech-usa-200615j-jaw-grip-mop-handle-hollow-fiberglass-12-case-12-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:35:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/138891-safepace-250-ev-12-radar-feedback-sign-your-speed-white-hip-face-23-x-29>: HTTP status code is not handled or not allowed 2026-01-24 20:35:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 20:35:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insultherm-resin-coated-fiberglass-sleeve-63-dia-200-black>: HTTP status code is not handled or not allowed 2026-01-24 20:35:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 19768 pages (at 95 pages/min), scraped 12230 items (at 17 items/min) 2026-01-24 20:35:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/trash_recycling/trash_recycling_accessories/trash_can_lids_dollies already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:35:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2x10-2-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-24 20:35:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sweep-complete-floor-sweep-w-squeegee-24-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:35:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/msmk919vp-no-smoking-graphic-sign-10w-x-14h-plastic>: HTTP status code is not handled or not allowed 2026-01-24 20:35:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machined-type-needle-roller-bearing-metric-double-sealed-42mm-bore-55mm-od>: HTTP status code is not handled or not allowed 2026-01-24 20:35:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g22-125-ap-30hp-oil-injected-rotary-screw-floor-mount-125psi-3ph-208-230-460v>: HTTP status code is not handled or not allowed 2026-01-24 20:35:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spectrum-color-coded-one-piece-rubber-floor-squeegee-24-white>: HTTP status code is not handled or not allowed 2026-01-24 20:35:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tss103ptp-ok-this-scaffold-erected-to-meet-tag-pf-cardstock-25pack>: HTTP status code is not handled or not allowed 2026-01-24 20:35:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trs243ptp-out-of-service-tag-pf-cardstock-25pack>: HTTP status code is not handled or not allowed 2026-01-24 20:35:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corn-lobby-broom-34>: HTTP status code is not handled or not allowed 2026-01-24 20:35:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/trash_recycling/trash_recycling_accessories/trash_can_lids_dollies landed on page that is not a product page. 2026-01-24 20:35:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coxreels-ez-p-lp-325-hv-3-8inx25ft-300psi-ez-coil-safety-performance-spring-retractable-low-pressure->: HTTP status code is not handled or not allowed 2026-01-24 20:35:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:35:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speco-8-channel-network-video-recorder-with-facial-recognition--smart-analytics-4tb-storage>: HTTP status code is not handled or not allowed 2026-01-24 20:35:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-oz-glass-jars-straight-side-clear-poly-vinyl-liner-case-of-24>: HTTP status code is not handled or not allowed 2026-01-24 20:35:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cpvc-sheet-24-l-x-24-w-x-1-4-thick-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:35:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mabr607vs-caution-sign-open-door-slowly-14w-x-10h-adhesive-vinyl>: HTTP status code is not handled or not allowed 2026-01-24 20:35:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-twist-ties-6-x-532-white-2000-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:35:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acudor-18x18-aluminum-diamond-plate-floor-door-no-hinge>: HTTP status code is not handled or not allowed 2026-01-24 20:35:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-bar-24-l-x-2-w-x-1-4-thick-clear-scratch-and-uv-resistant>: HTTP status code is not handled or not allowed 2026-01-24 20:35:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/easy-pour-coffee-ss-bottom-decanters-64-oz-decaf-2-pk>: HTTP status code is not handled or not allowed 2026-01-24 20:35:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hurricane-360-hoist-and-trolley-combo-standard-1-ton-capacity-30-lift>: HTTP status code is not handled or not allowed 2026-01-24 20:35:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-cable-climbing-system-38-dia-1x7-strands-galvanized-steel-cable>: HTTP status code is not handled or not allowed 2026-01-24 20:35:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-port-hplc-caps-gl45-pp-pack2>: HTTP status code is not handled or not allowed 2026-01-24 20:35:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 20:35:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/booster-pak-5000-es5000>: HTTP status code is not handled or not allowed 2026-01-24 20:35:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-sheet-48-l-x-48-w-x-1-8-thick-clear-antistatic>: HTTP status code is not handled or not allowed 2026-01-24 20:35:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/boost-x-12v-car-charger-16-awg-24l-cable-65w>: HTTP status code is not handled or not allowed 2026-01-24 20:35:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4x6-4-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-24 20:35:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/saf-t-grip-cutting-boards-cbg121812yl>: HTTP status code is not handled or not allowed 2026-01-24 20:35:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cw-stationary-premium-electric-pressure-washer-2000-psi-6-hp-39-gpm-1-phase-230v>: HTTP status code is not handled or not allowed 2026-01-24 20:35:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-a-volumetric-flasks-with-pe-stopper-50ml-case-of-12>: HTTP status code is not handled or not allowed 2026-01-24 20:35:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/38x64-3-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-24 20:35:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/psdtset-pipeline-small-display-table-with-top-28l-x-16w-x-18h>: HTTP status code is not handled or not allowed 2026-01-24 20:35:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2x8-3-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-24 20:35:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jlp-a-series-lever-hoist-1-1-2-ton-10-ft-lift>: HTTP status code is not handled or not allowed 2026-01-24 20:35:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jlh-series-lever-hoist-w-overload-protection-1-ton-5-ft-lift>: HTTP status code is not handled or not allowed 2026-01-24 20:35:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ez-chill-refreezable-ice-packs-b6180>: HTTP status code is not handled or not allowed 2026-01-24 20:35:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/x-treme-guard-edge-protector-12l-x-12w-x-6h-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:35:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3x8-3-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-24 20:35:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/157-1002-series-20pc-hss-bright-screw-machine-drill-and-plug-hand-tap-set>: HTTP status code is not handled or not allowed 2026-01-24 20:35:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/x-treme-guard-edge-protector-39l-x-12w-x-6h-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:35:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/in-counter-napkin-dispensers-h2000sc>: HTTP status code is not handled or not allowed 2026-01-24 20:35:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2712101-4-dual-casters>: HTTP status code is not handled or not allowed 2026-01-24 20:35:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mdb22-3-semi-automatic-adjustable-bottom-belt-drive-case-sealer-with-3-tape-head>: HTTP status code is not handled or not allowed 2026-01-24 20:35:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cw-stationary-premium-electric-pressure-washer-1000-psi-2-hp-25-gpm-1-phase-120v>: HTTP status code is not handled or not allowed 2026-01-24 20:35:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cpvc-bar-12-l-x-3-w-x-1-thick-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:35:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jp-portable-electric-pressure-washer-2000-psi-4-hp-26-gpm-1-phase-230v>: HTTP status code is not handled or not allowed 2026-01-24 20:35:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s1436s-stainless-steel-wire-shelf-36w-x-14d>: HTTP status code is not handled or not allowed 2026-01-24 20:35:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-duty-self-seal-bubble-pouches-6w-x-8l-clear-250pack>: HTTP status code is not handled or not allowed 2026-01-24 20:35:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-freestanding-3-panel-corner-room-divider-24-x-72-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:35:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/euro-ez-fit-c2210c>: HTTP status code is not handled or not allowed 2026-01-24 20:35:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vci-flat-poly-bags-20w-x-20l-4-mil-blue-250pack>: HTTP status code is not handled or not allowed 2026-01-24 20:35:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-5-in-1-t-tubing-connectors-4-x-2-branch-polypropylene-12pk>: HTTP status code is not handled or not allowed 2026-01-24 20:35:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spindrive-orbital-shaker-platform-8-7-8-x-8-7-8-x-2>: HTTP status code is not handled or not allowed 2026-01-24 20:35:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anti-static-bubble-pouches-10w-x-20l-pink-200pack>: HTTP status code is not handled or not allowed 2026-01-24 20:36:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wet-zone-shadow-board-blue-blue-68-x-30-aluminum-sb115al>: HTTP status code is not handled or not allowed 2026-01-24 20:36:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guardair-gun-vac-deluxe-kit>: HTTP status code is not handled or not allowed 2026-01-24 20:36:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-pilaster-with-shoe-7in-w-x-82in-h-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:36:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-pilaster-with-shoe-10in-w-x-82in-h-almond>: HTTP status code is not handled or not allowed 2026-01-24 20:36:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wet-zone-shadow-board-red-white-68-x-30-aluminum-sb109al>: HTTP status code is not handled or not allowed 2026-01-24 20:36:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dry-zone-shadow-board-blue-white-68-x-30-pro-series-acrylic-sb129fg>: HTTP status code is not handled or not allowed 2026-01-24 20:36:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dry-zone-shadow-board-combo-kit-blue-black-68-x-30-alum-composite-panel-sbk132acp>: HTTP status code is not handled or not allowed 2026-01-24 20:36:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lid-storage-container-239921-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:36:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluke-tl238-suregrip-test-lead-set-high-energy-environments>: HTTP status code is not handled or not allowed 2026-01-24 20:36:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-cantilever-bar-rack-4000-lbs-capacity-30-x-60-base-8-phenolic-wheels>: HTTP status code is not handled or not allowed 2026-01-24 20:36:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 19865 pages (at 97 pages/min), scraped 12263 items (at 33 items/min) 2026-01-24 20:36:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelate-wire-shelving-36608ep-60w-x-36d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 20:36:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelate-wire-shelving-add-on-a30728ep-72w-x-30d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 20:36:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-rod-96-l-x-9-16-dia-white>: HTTP status code is not handled or not allowed 2026-01-24 20:36:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-rod-96-l-x-1-2-dia-black>: HTTP status code is not handled or not allowed 2026-01-24 20:36:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bi-metallic-thermometer-50-to-550f-25mm-dial>: HTTP status code is not handled or not allowed 2026-01-24 20:36:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commercial-faucet-elkay-lk811at08l2>: HTTP status code is not handled or not allowed 2026-01-24 20:36:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1500-1999-inventory-tag-2-part-carbonless>: HTTP status code is not handled or not allowed 2026-01-24 20:36:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-frigimat-dry-ice-maker-us-model-14-3-4-x-5-1-2-x-6>: HTTP status code is not handled or not allowed 2026-01-24 20:36:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/needle-nozzle-4in-g901>: HTTP status code is not handled or not allowed 2026-01-24 20:36:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluke-tp40-automotive-back-probe-pin-set-cat-iii-300-v-3-a-rating-ul-listed>: HTTP status code is not handled or not allowed 2026-01-24 20:36:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hon-10500-series-bridge-47w-x-24d-x-29-1-2h-mahogany>: HTTP status code is not handled or not allowed 2026-01-24 20:36:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rosin-kraft-paper-70-lbs-36w-x-520l-red-1-roll>: HTTP status code is not handled or not allowed 2026-01-24 20:36:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-174-chrome-wire-linen-cart-with-nylon-cover-4-shelves-36-l-x-24-w-x-69-h>: HTTP status code is not handled or not allowed 2026-01-24 20:36:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/calibrated-dual-zone-electronic-thermometer-40-70c-40-158f-external>: HTTP status code is not handled or not allowed 2026-01-24 20:36:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-174-chrome-wire-linen-cart-with-nylon-cover-4-shelves-72-l-x-24-w-x-69-h>: HTTP status code is not handled or not allowed 2026-01-24 20:36:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8000-8499-wired-inventory-tag-3-part-carbonless>: HTTP status code is not handled or not allowed 2026-01-24 20:36:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cryogenic-storage-label-sheets-13mm-dots-for-15-2ml-tubes-white-3840-labels>: HTTP status code is not handled or not allowed 2026-01-24 20:36:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-shade-cloth-10-x-10>: HTTP status code is not handled or not allowed 2026-01-24 20:36:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pressurized-26-gallon-used-fluid-evacuation-tank-3624>: HTTP status code is not handled or not allowed 2026-01-24 20:36:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pull-over-cover-for-ultra-275-containment-sump-360-gallon-capacity-models-only>: HTTP status code is not handled or not allowed 2026-01-24 20:36:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/orbis-stakpak-modular-straight-wall-container-24l-x-15w-x-11-12h-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:36:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2500-2999-inventory-tag-2-part-carbonless-stub-style>: HTTP status code is not handled or not allowed 2026-01-24 20:36:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-rod-36-l-x-7-16-dia-black>: HTTP status code is not handled or not allowed 2026-01-24 20:36:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-swivel-1-4in-npt-male-x-1-4in-npt-male-83594>: HTTP status code is not handled or not allowed 2026-01-24 20:36:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:36:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crinkle-paper-10-lbs-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:36:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leviton-csb2-20-20a-120-277v-toggle-double-pole-ac-quiet-switch-commercial-grade-grounding-brown>: HTTP status code is not handled or not allowed 2026-01-24 20:36:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:36:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-3-4-x-1-3-8-1-light-green-strung-tag>: HTTP status code is not handled or not allowed 2026-01-24 20:36:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-max-gtx14-bs-battery-12-amps-12v-b-terminals>: HTTP status code is not handled or not allowed 2026-01-24 20:36:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-epoxy-wire-shelving-14427gz-42w-x-14d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 20:36:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-epoxy-wire-shelving-14247gz-24w-x-14d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 20:36:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h-b-frio-temp-calibrated-elec-verification-lollipop-stem-thermometer-0-70c-32-158f>: HTTP status code is not handled or not allowed 2026-01-24 20:36:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-edge-4w-18l-2-ply-eye-eye-poly-web-sling-flat-eyes>: HTTP status code is not handled or not allowed 2026-01-24 20:36:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-rack-24x36x60-with-64-yellow-drawers>: HTTP status code is not handled or not allowed 2026-01-24 20:36:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-edge-1w-3l-1-ply-endless-poly-flat-web-sling>: HTTP status code is not handled or not allowed 2026-01-24 20:36:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h-b-enviro-safe-liquid-in-glass-pocket-thermometer-0-to-220f-closed-plastic-case>: HTTP status code is not handled or not allowed 2026-01-24 20:36:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/red-plastic-cabinet-drawer-2-3-4w-x-11d-x-2-1-2h>: HTTP status code is not handled or not allowed 2026-01-24 20:36:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock landed on page that is not a product page. 2026-01-24 20:36:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-96-x-30-adj-height-workbench-w-drawer-black-shop-top-square-edge-top>: HTTP status code is not handled or not allowed 2026-01-24 20:36:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 20:36:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-1-4-x-3-1-8-8-orange-fluorescent-tag>: HTTP status code is not handled or not allowed 2026-01-24 20:36:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:36:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-hdpe-bar-24-l-x-3-w-x-1-4-thick-white-marine>: HTTP status code is not handled or not allowed 2026-01-24 20:36:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gray-plastic-cabinet-drawer-5-5-8w-x-11d-x-2-1-2h>: HTTP status code is not handled or not allowed 2026-01-24 20:36:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-ptfe-tube-12-l-x-3-4-id-x-1-od-white>: HTTP status code is not handled or not allowed 2026-01-24 20:36:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h-b-frio-temp-freezer-verification-thermometer-30-to-1c>: HTTP status code is not handled or not allowed 2026-01-24 20:36:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-edge-4w-4l-2-ply-eye-eye-poly-web-sling-twisted-eyes>: HTTP status code is not handled or not allowed 2026-01-24 20:36:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-ptfe-tube-12-l-x-7-8-id-x-1-od-white>: HTTP status code is not handled or not allowed 2026-01-24 20:36:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-epoxy-wire-shelving-14306gz-30w-x-14d-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 20:36:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h-b-enviro-safe-dry-block-incubator-liquid-in-glass-thermometer-24-to-57c-35mm-immersion>: HTTP status code is not handled or not allowed 2026-01-24 20:36:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-rod-60-l-x-7-16-dia-white>: HTTP status code is not handled or not allowed 2026-01-24 20:36:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bench-rack-12x36x21-with-12-yellow-drawers>: HTTP status code is not handled or not allowed 2026-01-24 20:36:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kendall-howard-advanced-classroom-training-table-30-x-36-folkstone>: HTTP status code is not handled or not allowed 2026-01-24 20:36:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/filter-cart-high-viscosity>: HTTP status code is not handled or not allowed 2026-01-24 20:36:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h-b-frio-temp-calibrated-dual-zone-elec-verification-thermometer-50-70c-58-158f-0-50c-32-122f>: HTTP status code is not handled or not allowed 2026-01-24 20:36:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock>: HTTP status code is not handled or not allowed 2026-01-24 20:36:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-sided-rack-12x36x60-with-72-blue-drawers>: HTTP status code is not handled or not allowed 2026-01-24 20:36:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/52-cassius-ceiling-fan-59262-matte-silver>: HTTP status code is not handled or not allowed 2026-01-24 20:36:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h-b-calibrated-electronic-verification-lollipop-stem-thermometer-for-general-applications-0-70c>: HTTP status code is not handled or not allowed 2026-01-24 20:36:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flaskup-polypropylene-flask-holders-for-100ml-round-bottom-flasks-3pk>: HTTP status code is not handled or not allowed 2026-01-24 20:36:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-alloy-latching-j-hook-style-a-1435-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:36:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-8-alloy-latching-j-hook-style-a-1900-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:36:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/non242356w-2-ply-tissue-poly-professional-towels-13-w-x-18-l-white-500-box>: HTTP status code is not handled or not allowed 2026-01-24 20:36:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-ultra-low-headroom-lift-beam-6000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:36:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simplehuman-174-butterfly-step-can-10-gallon-brushed-ss>: HTTP status code is not handled or not allowed 2026-01-24 20:37:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sm-optimus-wall-mounted-eye-face-wash-with-stainless-steel-bowl-se-1050>: HTTP status code is not handled or not allowed 2026-01-24 20:37:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storage-container-with-lid-32-1-2-x-23-1-2-x-20-green>: HTTP status code is not handled or not allowed 2026-01-24 20:37:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/35-square-outdoor-table-gray-polymer-top-white-aluminum-frame>: HTTP status code is not handled or not allowed 2026-01-24 20:37:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buyers-stainless-steel-underbody-box-w-rotary-paddle-white-18x18x48>: HTTP status code is not handled or not allowed 2026-01-24 20:37:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tempco-channel-strip-120v-t1-10-12l-250w>: HTTP status code is not handled or not allowed 2026-01-24 20:37:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-x-5-1-2-red-packing-list-enclosed-full-face>: HTTP status code is not handled or not allowed 2026-01-24 20:37:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buyers-aluminum-topside-w-t-handle-16x13x88>: HTTP status code is not handled or not allowed 2026-01-24 20:37:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buyers-underbody-mounting-brackets-for-48-inch-poly>: HTTP status code is not handled or not allowed 2026-01-24 20:37:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-high-metal-indoor-outdoor-bar-stool-silver>: HTTP status code is not handled or not allowed 2026-01-24 20:37:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:37:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roller-ball-stick-pen-0-7mm-black-ink>: HTTP status code is not handled or not allowed 2026-01-24 20:37:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:37:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/narrow-mouth-250ml-8oz-polyethylene-wash-bottles-blue-polypropylene-cap-28mm-closure-6pk>: HTTP status code is not handled or not allowed 2026-01-24 20:37:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 19956 pages (at 91 pages/min), scraped 12280 items (at 17 items/min) 2026-01-24 20:37:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buyers-steel-trailer-tongue-truck-box-26-inch-w>: HTTP status code is not handled or not allowed 2026-01-24 20:37:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:37:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plywood-seating-cafe-stack-chair-natural>: HTTP status code is not handled or not allowed 2026-01-24 20:37:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mro-industrial-enamel-20-oz-tan-6-cans-case-620-1435>: HTTP status code is not handled or not allowed 2026-01-24 20:37:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spruce-general-use-spray-paint-16-oz-satin-black-12-cans-case-98-24>: HTTP status code is not handled or not allowed 2026-01-24 20:37:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-rod-72-l-x-2-1-4-dia-white>: HTTP status code is not handled or not allowed 2026-01-24 20:37:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_storage_bench_protection/laboratory_bench_racks_stands already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:37:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-top-coat-16-oz-6-cans-case-16-54>: HTTP status code is not handled or not allowed 2026-01-24 20:37:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-hdpe-bar-24-l-x-1-w-x-1-4-thick-black-marine>: HTTP status code is not handled or not allowed 2026-01-24 20:37:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-24 20:37:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n52h2hdw50n-jafline-hd-double-jacket-fire-hose-2-x-50-ft-400-psi-white>: HTTP status code is not handled or not allowed 2026-01-24 20:37:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-wall-mount-wire-shelf-additional-level-24w-x-14d-poly-green>: HTTP status code is not handled or not allowed 2026-01-24 20:37:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-wall-mount-wire-shelving-60w-x-14d-x-54h-3-shelf-starter-poly-green>: HTTP status code is not handled or not allowed 2026-01-24 20:37:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers>: HTTP status code is not handled or not allowed 2026-01-24 20:37:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bsxvl606va19-hvl600-series-armless-stacking-chair-charcoal>: HTTP status code is not handled or not allowed 2026-01-24 20:37:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-wall-mount-wire-shelf-additional-level-54w-x-24d-poly-green>: HTTP status code is not handled or not allowed 2026-01-24 20:37:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-wire-shelves-utility-cart-silver-ep-2-casters-w-brakes-24-w-x-60-l-x-39-h>: HTTP status code is not handled or not allowed 2026-01-24 20:37:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/school-milk-coolers-dual-access-cold-wall-st-49w-st49n>: HTTP status code is not handled or not allowed 2026-01-24 20:37:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-36-l-x-2-1-2-w-x-2-1-2-thick-white>: HTTP status code is not handled or not allowed 2026-01-24 20:37:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-wall-mount-wire-shelving-24w-x-24d-x-34h-2-shelf-add-on-poly-green>: HTTP status code is not handled or not allowed 2026-01-24 20:37:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/draft-barmobile-portable-draft-beer-dispensers-bm-bm23>: HTTP status code is not handled or not allowed 2026-01-24 20:37:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/lab_storage_bench_protection/laboratory_bench_racks_stands>: HTTP status code is not handled or not allowed 2026-01-24 20:37:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-wall-mount-wire-shelving-36w-x-14d-x-63h-4-shelf-starter-poly-green>: HTTP status code is not handled or not allowed 2026-01-24 20:37:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/din319pl-phenolic-ball-knobs-tapped-40mm-diameter-mm-length-m12x175>: HTTP status code is not handled or not allowed 2026-01-24 20:37:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-wire-shelf-stem-caster-truck-black-ep-18-w-x-36-l-x-69-h>: HTTP status code is not handled or not allowed 2026-01-24 20:37:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-36-l-x-1-w-x-3-8-thick-white>: HTTP status code is not handled or not allowed 2026-01-24 20:37:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-solid-plastic-shelving-unit-chrome-posts-24-w-x-48-l-x-74-h>: HTTP status code is not handled or not allowed 2026-01-24 20:37:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers landed on page that is not a product page. 2026-01-24 20:37:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-wall-mount-wire-shelving-60w-x-24d-x-54h-3-shelf-starter-poly-green>: HTTP status code is not handled or not allowed 2026-01-24 20:37:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-x-1100-white-freezer-paper>: HTTP status code is not handled or not allowed 2026-01-24 20:37:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kendall-howard-vline-12u-fixed-wall-rack>: HTTP status code is not handled or not allowed 2026-01-24 20:37:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-wire-shelf-stem-caster-truck-black-ep-24-w-x-48-l-x-69-h>: HTTP status code is not handled or not allowed 2026-01-24 20:37:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-ptfe-sheet-24-l-x-12-w-x-1-thick-off-white-glass-filled>: HTTP status code is not handled or not allowed 2026-01-24 20:37:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reach-in-refrigerator-horizon-solid-half-solid-doors-26w-hrp1-1s>: HTTP status code is not handled or not allowed 2026-01-24 20:37:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kendall-howard-6u-fixed-wall-mount-cabinet>: HTTP status code is not handled or not allowed 2026-01-24 20:37:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-24-l-x-2-w-x-2-thick-black>: HTTP status code is not handled or not allowed 2026-01-24 20:37:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-sheet-48-l-x-24-w-x-1-3-4-thick-black>: HTTP status code is not handled or not allowed 2026-01-24 20:37:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-4lb-paper-bag-30lb-white-5-x-3-1-3-x-9-3-4-500-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:37:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reach-in-refrigerator-horizon-solid-half-solid-doors-35w-hrps1w-1s>: HTTP status code is not handled or not allowed 2026-01-24 20:37:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-25lb-squat-paper-bag-40lb-kraft-brown-8-1-4-x-6-1-8-x-15-7-8-500-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:37:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-20lb-paper-bag-40lb-kraft-brown-8-1-4-x-5-5-16-x-16-1-8-500-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:37:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heat-stress-responder-kit>: HTTP status code is not handled or not allowed 2026-01-24 20:37:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reach-in-refrigerator-horizon-glass-half-glass-doors-52w-hrps2-1g>: HTTP status code is not handled or not allowed 2026-01-24 20:37:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-shelf-wall-mount-wine-rack-single-wide-chrome-14-d-x-48-w-x-34-h>: HTTP status code is not handled or not allowed 2026-01-24 20:37:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-x-24-maple-square-edge-3-drawer-cabinet>: HTTP status code is not handled or not allowed 2026-01-24 20:37:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kendall-howard-12u-corner-wall-mount-cabinet>: HTTP status code is not handled or not allowed 2026-01-24 20:37:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/instant-cold-compress-5-x-9>: HTTP status code is not handled or not allowed 2026-01-24 20:37:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plus-astm-like-liquid-in-glass-thermometer-9f-low-pensky-martens-20-to-230f>: HTTP status code is not handled or not allowed 2026-01-24 20:37:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kendall-howard-6u-hinged-wall-rack>: HTTP status code is not handled or not allowed 2026-01-24 20:37:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aed-3d-vshaped-projecting-wall-sign-metal>: HTTP status code is not handled or not allowed 2026-01-24 20:37:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyethylene-buchner-table-top-funnel-with-coarse-porosity-fixed-plate-1025-id-8h>: HTTP status code is not handled or not allowed 2026-01-24 20:37:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bumpon-resiliant-rollstock-sj5916-9-x-36-yds-black-1-16-thick-70006459864>: HTTP status code is not handled or not allowed 2026-01-24 20:37:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-5-tier-54w-x-21d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 20:37:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-5-tier-42w-x-21d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 20:37:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plug-in-dehumidifier-control-120v-15a-analog-30-90-relative-humidity>: HTTP status code is not handled or not allowed 2026-01-24 20:37:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_dippers_scoops_cups already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:37:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-5-tier-36w-x-36d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 20:37:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gx-100f-spherical-plain-thrust-bearing-metric-high-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:37:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ge-220es-spherical-plain-bearing-metric>: HTTP status code is not handled or not allowed 2026-01-24 20:37:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ge-90te-2rs-spherical-plain-bearing-metric-high-capacity-sealed>: HTTP status code is not handled or not allowed 2026-01-24 20:37:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leviton-5262-sw-15-amp-125v-duplex-receptacle-self-grounding-white>: HTTP status code is not handled or not allowed 2026-01-24 20:37:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commercial-electric-baseboard-convection-heater-48-l-240-208v>: HTTP status code is not handled or not allowed 2026-01-24 20:37:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-5-tier-72w-x-14d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 20:37:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-skid-steer-mount-tractor-attachment>: HTTP status code is not handled or not allowed 2026-01-24 20:37:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-1400-1620-specific-gravity-combined-form-thermo-hydrometer>: HTTP status code is not handled or not allowed 2026-01-24 20:37:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:37:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-glove-box-27-x-13-x-22>: HTTP status code is not handled or not allowed 2026-01-24 20:37:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leviton-5352-w-20-amp-125v-heavy-duty-grade-duplex-receptacle-white>: HTTP status code is not handled or not allowed 2026-01-24 20:38:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leviton-5361-gy-20a-125v-industrial-series-heavy-duty-grade-single-receptacle-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:38:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_dippers_scoops_cups>: HTTP status code is not handled or not allowed 2026-01-24 20:38:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ge-17es-spherical-plain-bearing-metric>: HTTP status code is not handled or not allowed 2026-01-24 20:38:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ge-50te-2rs-spherical-plain-bearing-metric-high-capacity-sealed>: HTTP status code is not handled or not allowed 2026-01-24 20:38:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-wall-receiver-black-qw0210>: HTTP status code is not handled or not allowed 2026-01-24 20:38:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metal-merchandising-strips-12-stations-green>: HTTP status code is not handled or not allowed 2026-01-24 20:38:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-9-21-degree-brix-sugar-scale-combined-form-thermo-hydrometer>: HTTP status code is not handled or not allowed 2026-01-24 20:38:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/display-hook-product-stop-1l-x-1w-clear>: HTTP status code is not handled or not allowed 2026-01-24 20:38:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-5-tier-72w-x-14d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 20:38:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/fire_protection/smoke_carbon_monoxide_detectors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:38:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-24 20:38:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spirit-display-case-black-base-satin-fr-white-back-48w-16d-72h>: HTTP status code is not handled or not allowed 2026-01-24 20:38:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 20046 pages (at 90 pages/min), scraped 12295 items (at 15 items/min) 2026-01-24 20:38:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/108in-wide-complete-3-in-corner-compartments-satin-brushed-ic33660-ss>: HTTP status code is not handled or not allowed 2026-01-24 20:38:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/liberty-letter-box-12-1-4w-x-24-1-8d-x-10-3-4h-white-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:38:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-12-l-x-1-1-2-w-x-1-4-thick-white>: HTTP status code is not handled or not allowed 2026-01-24 20:38:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-abs-bar-24-l-x-1-1-2-w-x-1-2-thick-black>: HTTP status code is not handled or not allowed 2026-01-24 20:38:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12in-x-42in-urinal-screen-stainless-whus12-ss>: HTTP status code is not handled or not allowed 2026-01-24 20:38:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-tray-storage-with-clear-trays>: HTTP status code is not handled or not allowed 2026-01-24 20:38:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stor-drawer-letter-box-14w-x-25-1-2d-x-11-1-2h-white-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:38:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-54w-x-18d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 20:38:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/optional-guard-rail-connection-tube-48-l-pairs>: HTTP status code is not handled or not allowed 2026-01-24 20:38:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/optional-guard-rail-toeboard-96-l>: HTTP status code is not handled or not allowed 2026-01-24 20:38:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-foam-neoprene-roll-720-l-x-12-w-x-1-8-thick-white-food-60a>: HTTP status code is not handled or not allowed 2026-01-24 20:38:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/fire_protection/smoke_carbon_monoxide_detectors>: HTTP status code is not handled or not allowed 2026-01-24 20:38:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/type-3000-medium-outdoor-waterproof-case-w-o-foam-insert-14-1-4-l-x-11-3-4-w-x-6-3-4h-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:38:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/secador-clear-40-horizontal-auto-desiccator-cabinet-230v-19-cu-ft>: HTTP status code is not handled or not allowed 2026-01-24 20:38:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mounting-ring-set-2>: HTTP status code is not handled or not allowed 2026-01-24 20:38:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-48-l-x-2-1-2-w-x-2-1-4-thick-white>: HTTP status code is not handled or not allowed 2026-01-24 20:38:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-48-l-x-2-1-2-w-x-2-thick-white>: HTTP status code is not handled or not allowed 2026-01-24 20:38:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spinbar-teflon-micro-flea-magnetic-stirring-bar-10-x-3mm-white>: HTTP status code is not handled or not allowed 2026-01-24 20:38:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thermodisc-steam-trap-td6526-3-4>: HTTP status code is not handled or not allowed 2026-01-24 20:38:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9534-15-8-inch-axial-ac-metal-com-pax-ial-blower-w-canister-15-ducting>: HTTP status code is not handled or not allowed 2026-01-24 20:38:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-24w-x-18d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 20:38:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mounting-kit-with-18-w-louver-36-w-pegboard-72-w-workbench-tan>: HTTP status code is not handled or not allowed 2026-01-24 20:38:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/removable-steel-bollard-with-removable-rubber-cap-42-x-4-1-2>: HTTP status code is not handled or not allowed 2026-01-24 20:38:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-acetal-bar-12-l-x-2-w-x-3-8-thick-off-white-antistatic>: HTTP status code is not handled or not allowed 2026-01-24 20:38:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-30w-x-18d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 20:38:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spinstar-teflon-magnetic-stirring-bar-70mm-fits-400ml-beakers-white>: HTTP status code is not handled or not allowed 2026-01-24 20:38:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spinbar-teflon-polygon-magnetic-stirring-bar-25-x-8mm-white-without-pivot-ring>: HTTP status code is not handled or not allowed 2026-01-24 20:38:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smooth-bollard-post-sleeve-4-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:38:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12w-x-12d-x-4h-small-basket-white>: HTTP status code is not handled or not allowed 2026-01-24 20:38:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spinbar-teflon-polygon-magnetic-stirring-bar-80-x-10mm-white-without-pivot-ring>: HTTP status code is not handled or not allowed 2026-01-24 20:38:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cejn-series-208-star-tip-20-tube>: HTTP status code is not handled or not allowed 2026-01-24 20:38:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:38:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-60w-x-21d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 20:38:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-abs-sheet-48-l-x-24-w-x-3-4-thick-black>: HTTP status code is not handled or not allowed 2026-01-24 20:38:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9558-15-16-inch-axial-explosion-proof-ex-plastic-blower-w-canister-15-ducting>: HTTP status code is not handled or not allowed 2026-01-24 20:38:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anti-fatigue-rejuvenator-urethane-tile-matting-tile-black-5-8-thick-36x36>: HTTP status code is not handled or not allowed 2026-01-24 20:38:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-ptfe-rod-24-l-x-2-dia-off-white-glass-filled>: HTTP status code is not handled or not allowed 2026-01-24 20:38:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-ptfe-rod-36-l-x-1-dia-off-white-glass-filled>: HTTP status code is not handled or not allowed 2026-01-24 20:38:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hytrol-stationary-floor-support-18w-max-leg-45-5-8h>: HTTP status code is not handled or not allowed 2026-01-24 20:38:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clean-room-chair-vinyl-black>: HTTP status code is not handled or not allowed 2026-01-24 20:38:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-rectangular-amber-low-profile-strobe-light-12v-6-leds-8891910>: HTTP status code is not handled or not allowed 2026-01-24 20:38:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-partition-flexible-five-panel-vinyl-6>: HTTP status code is not handled or not allowed 2026-01-24 20:38:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/splash-shield-mounting-fixture-c-clamp>: HTTP status code is not handled or not allowed 2026-01-24 20:38:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-54w-x-14d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 20:38:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 20:38:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-24-l-x-1-1-2-w-x-1-thick-black>: HTTP status code is not handled or not allowed 2026-01-24 20:38:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:38:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-epoxy-wire-shelving-14485b-48w-x-14d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 20:38:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-mudflaps-1-4-inch-thick-24x36-b36lp>: HTTP status code is not handled or not allowed 2026-01-24 20:38:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_storage_bench_protection/laboratory_trays already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:38:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-lab-drawer-4-compartment-tray-14-x-17-1-2-x-2-1-4>: HTTP status code is not handled or not allowed 2026-01-24 20:38:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-epoxy-wire-shelving-21427b-42w-x-21d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 20:38:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spinbar-teflon-polygon-magnetic-stirring-bar-20-x-8mm-white-without-pivot-ring>: HTTP status code is not handled or not allowed 2026-01-24 20:38:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lkec2037ls-explore-pulldown-bridge-kitchen-faucet-lustrous-steel-double-lever-handle>: HTTP status code is not handled or not allowed 2026-01-24 20:38:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-polypropylene-sterilizing-tray-cover-fits-h16260-0000>: HTTP status code is not handled or not allowed 2026-01-24 20:38:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-abs-bar-24-l-x-2-w-x-1-8-thick-black>: HTTP status code is not handled or not allowed 2026-01-24 20:38:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-tier-handbag-rack-w-8-arms-rectangular-tubing-chrome>: HTTP status code is not handled or not allowed 2026-01-24 20:38:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 20:38:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/competitor-ventilated-three-tier-locker-starter-1-wide-12w-x-18d-x-24h-unassembled-almond>: HTTP status code is not handled or not allowed 2026-01-24 20:38:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-epoxy-wire-shelving-14248b-24w-x-14d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 20:38:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-switch-grid-test-tube-rack-12-places-for-25-30mm-tubes-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:38:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/lab_storage_bench_protection/laboratory_trays>: HTTP status code is not handled or not allowed 2026-01-24 20:38:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/folding-dump-table-semi-gloss-black>: HTTP status code is not handled or not allowed 2026-01-24 20:38:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/recyclockout-lockout-tagout-small-plug-lockout-recycled-plastic-7105>: HTTP status code is not handled or not allowed 2026-01-24 20:38:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-epoxy-wire-shelving-add-on-21728b-72w-x-21d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 20:38:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-square-tubing-outrigger-w-universal-slotting-chrome>: HTTP status code is not handled or not allowed 2026-01-24 20:38:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/200008hf-kp-tongue-interlock-switch-hf-act-3nc-1no-12npt-pe>: HTTP status code is not handled or not allowed 2026-01-24 20:39:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:39:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-1-2w-x-11h-injection-molded-styrene-slatwall-literature-holder-clear>: HTTP status code is not handled or not allowed 2026-01-24 20:39:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-neoprene-roll-240-l-x-36-w-x-1-2-thick-black-high-strength-90a>: HTTP status code is not handled or not allowed 2026-01-24 20:39:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-sterileware-sampling-scoop-83ml-27oz-sterile-plastic-individually-wrapped-50pk>: HTTP status code is not handled or not allowed 2026-01-24 20:39:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cuso-bk-led-outdoor-emergency-light-30k-120-277v-nicad-battery-wet-loc-black>: HTTP status code is not handled or not allowed 2026-01-24 20:39:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/232103-b-esl-sslp-replacement-lid-wshroudled-110v-8l-x-6w-x-4h-ss>: HTTP status code is not handled or not allowed 2026-01-24 20:39:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-36-square-edge-work-bench-shop-top-tan>: HTTP status code is not handled or not allowed 2026-01-24 20:39:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72x30-safety-maple-pedestal-workbench-with-9-drawers>: HTTP status code is not handled or not allowed 2026-01-24 20:39:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sg1-20-pcu-led-low-profile-wall-pack-w-photocontrol-21w-2200l-5000k-dark-bronze-dlc>: HTTP status code is not handled or not allowed 2026-01-24 20:39:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ml-2l3k-1-wh-led-floodlight-w-occ-sensor-26-6w-2100l-3000k-120v-white>: HTTP status code is not handled or not allowed 2026-01-24 20:39:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polyurethane-roll-1200-l-x-30-w-x-3-16-thick-black-low-compression>: HTTP status code is not handled or not allowed 2026-01-24 20:39:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 20138 pages (at 92 pages/min), scraped 12312 items (at 17 items/min) 2026-01-24 20:39:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/142053-gls-rope-pull-switch-2nc-2no-12npt-die-cast>: HTTP status code is not handled or not allowed 2026-01-24 20:39:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eq-dual-speed-electric-chain-hoist-1-ton-10-lift-23-3-8-ft-min-230v>: HTTP status code is not handled or not allowed 2026-01-24 20:39:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls>: HTTP status code is not handled or not allowed 2026-01-24 20:39:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blind-flange-lockout-device-medium-s3923>: HTTP status code is not handled or not allowed 2026-01-24 20:39:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/141030b-glhd-rope-pull-switch-wled-4nc-2no-110120v-die-cast>: HTTP status code is not handled or not allowed 2026-01-24 20:39:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-three-tier-locker-3-wide-12w-x-12d-x-24h-assembled-blue-frost>: HTTP status code is not handled or not allowed 2026-01-24 20:39:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:39:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hinge-w-hardware-fg3975l90000>: HTTP status code is not handled or not allowed 2026-01-24 20:39:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/CTT-4824-48-W-x-24-D-16-ga-Stainless-Steel-Workbench-w-Galvanized-Legs-Shelf>: HTTP status code is not handled or not allowed 2026-01-24 20:39:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/max-attach-polymagnet-rare-earth-ring-magnet-0-75-o-d-x-0-186-i-d-x-0-12-thick>: HTTP status code is not handled or not allowed 2026-01-24 20:39:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/max-attach-polymagnet-rectangular-rare-earth-magnet-0-12-thick-x-1-50-w-x-1-50-l>: HTTP status code is not handled or not allowed 2026-01-24 20:39:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-double-tier-locker-2-wide-12w-x-12d-x-36h-assembled-almond>: HTTP status code is not handled or not allowed 2026-01-24 20:39:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-labels-for-protective-labeling-system-1-x-3-1000pk>: HTTP status code is not handled or not allowed 2026-01-24 20:39:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-silicone-roll-60-l-x-36-w-x-3-8-thick-red-food-70a>: HTTP status code is not handled or not allowed 2026-01-24 20:39:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-series-street-utility-marking-paint-20-oz-orange-fluorescent-20-357-12-case>: HTTP status code is not handled or not allowed 2026-01-24 20:39:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/comp-ez-tee-adapter-3-8-female-comp-x-3-8-od-x-3-8-od>: HTTP status code is not handled or not allowed 2026-01-24 20:39:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-double-tier-locker-2-wide-15w-x-18d-x-36h-assembled-almond>: HTTP status code is not handled or not allowed 2026-01-24 20:39:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cantilever-double-sided-starter-unit-extra-2>: HTTP status code is not handled or not allowed 2026-01-24 20:39:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h-d-cobra-strapping-dispenser-for-3-4and-1-1-4w>: HTTP status code is not handled or not allowed 2026-01-24 20:39:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-level-60x48-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 20:39:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls>: HTTP status code is not handled or not allowed 2026-01-24 20:39:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cast-acrylic-bar-48-l-x-4-w-x-1-8-thick-green>: HTTP status code is not handled or not allowed 2026-01-24 20:39:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cast-acrylic-sheet-48-l-x-12-w-x-1-4-thick-green>: HTTP status code is not handled or not allowed 2026-01-24 20:39:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foam-grip-roper-device-stretch-film-dispenser-ep-890-12-20>: HTTP status code is not handled or not allowed 2026-01-24 20:39:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grab-n-go-high-profile-open-display-merchandiser-50-18w-x-31-12d-x-76-78h>: HTTP status code is not handled or not allowed 2026-01-24 20:39:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cm-1717-96-midnight-countertop-trash-bin-5-inch-w-x-5-inch-d-x-7-inch-h>: HTTP status code is not handled or not allowed 2026-01-24 20:39:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2w-x-8h-grid-panel-semi-gloss-black>: HTTP status code is not handled or not allowed 2026-01-24 20:39:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-buna-n-roll-1440-l-x-36-w-x-3-8-thick-black-high-strength-80a-acrylic-adhesive>: HTTP status code is not handled or not allowed 2026-01-24 20:39:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-rack-netting-one-bay-4>: HTTP status code is not handled or not allowed 2026-01-24 20:39:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/220-ac-dc-tig-welder-amptrol-package-120-230v-5-220a-60hz-green>: HTTP status code is not handled or not allowed 2026-01-24 20:39:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cm-1750-13-mission-square-bowl-display-12-inch-w-x-19-inch-d-x-13-1-2-inch-h-black>: HTTP status code is not handled or not allowed 2026-01-24 20:39:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rotating-slatwall-merchandiser-cube-maple-maple>: HTTP status code is not handled or not allowed 2026-01-24 20:39:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-rack-netting-one-bay-7>: HTTP status code is not handled or not allowed 2026-01-24 20:39:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-125xbrirga24d-125-chameleon-superbright-rga-24v-dc>: HTTP status code is not handled or not allowed 2026-01-24 20:39:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/84-heavyweight-single-slot-w-1-slots-on-2-centers-satin-zinc>: HTTP status code is not handled or not allowed 2026-01-24 20:39:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-buna-n-strip-120-l-x-4-w-x-3-16-thick-black-high-strength-80a>: HTTP status code is not handled or not allowed 2026-01-24 20:39:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fx108n-refrigerant-air-dryer-3-phase-400v-246-cfm-1-1-2-npt>: HTTP status code is not handled or not allowed 2026-01-24 20:39:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:39:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ecp4409t-4-100-hp-1188-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:39:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnetic-bookends-6w-x5d-x7h-black>: HTTP status code is not handled or not allowed 2026-01-24 20:39:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pensacola-chaise-lounge-red>: HTTP status code is not handled or not allowed 2026-01-24 20:39:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lifeguard-chair-62-in-seat-height-white>: HTTP status code is not handled or not allowed 2026-01-24 20:39:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/charleston-adirondack-rocking-chair-white>: HTTP status code is not handled or not allowed 2026-01-24 20:39:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clearwater-adirondack-chair-white>: HTTP status code is not handled or not allowed 2026-01-24 20:39:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/briggs-stratton-104m02-0021-f1-gas-engine-725exi-series-vertical-shaft-7-8-crank-shaft>: HTTP status code is not handled or not allowed 2026-01-24 20:39:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-three-phase-odp-motor-3-4-hp-1725-rpm-230-460v-odp-56-frame>: HTTP status code is not handled or not allowed 2026-01-24 20:39:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/174-j20-53-7425j-153-polypropylene-label-0-5-h-x-2-w-2000-roll-white>: HTTP status code is not handled or not allowed 2026-01-24 20:39:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dust-mold-and-particulate-kit-l-1-each-h05-8921l>: HTTP status code is not handled or not allowed 2026-01-24 20:39:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30649-safety-lever-assembly-set-1-pkg-qty>: HTTP status code is not handled or not allowed 2026-01-24 20:39:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets>: HTTP status code is not handled or not allowed 2026-01-24 20:39:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-125xbrma24d-125xbr-xtra-brite-led-multi-mode-amber-24-vdc-0215-amps>: HTTP status code is not handled or not allowed 2026-01-24 20:39:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cjm3108-1-2-hp-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:39:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cnm3531-25-hp-1140-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:39:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-105xbrmr24d-dual-mode-led-signal-red-24v-dc>: HTTP status code is not handled or not allowed 2026-01-24 20:39:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-hinged-top-basket-truck-10-bu-orange-vinyl-wood-base-all-swivel>: HTTP status code is not handled or not allowed 2026-01-24 20:39:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-hinged-top-basket-truck-8-bu-navy-blue-vinyl-wood-base-all-swivel>: HTTP status code is not handled or not allowed 2026-01-24 20:39:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-hinged-top-basket-truck-12-bu-navy-blue-vinyl-wood-base-all-swivel>: HTTP status code is not handled or not allowed 2026-01-24 20:39:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-125xbrza120a-125xbr-xtra-brite-led-random-flash-pattern-amb-120-vac-0108-amps>: HTTP status code is not handled or not allowed 2026-01-24 20:39:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/urinal-gasket>: HTTP status code is not handled or not allowed 2026-01-24 20:39:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-270sa12240ad-incandescentled-bulb-module-amber-12-240v-acdc>: HTTP status code is not handled or not allowed 2026-01-24 20:39:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-270pzo120240a-70-mm-piezoelectric-sounder-gray-120-240v-ac>: HTTP status code is not handled or not allowed 2026-01-24 20:39:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-hinged-top-basket-truck-12-bu-maroon-vinyl-wood-base-all-swivel>: HTTP status code is not handled or not allowed 2026-01-24 20:39:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/centrifugal-blower-3050-rpm-115v-oao>: HTTP status code is not handled or not allowed 2026-01-24 20:39:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ecp4107t-4-25-hp-3530-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:39:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-248ledmw120a-48-mm-led-stacklight-module-white-120v-ac>: HTTP status code is not handled or not allowed 2026-01-24 20:39:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/die_cut_labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:39:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-1-2-titanium-coated-tradesman-utility-shears>: HTTP status code is not handled or not allowed 2026-01-24 20:39:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90-tooth-flex-head-teardrop-ratchet-with-3-8-drive-tang-11>: HTTP status code is not handled or not allowed 2026-01-24 20:39:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-270ledsg24ad-smd-steady-led-module-and-light-source-green-24v-acdc>: HTTP status code is not handled or not allowed 2026-01-24 20:39:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cwdm3539-5-hp-1140-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:39:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chalk-refill-2-5-lbs-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:40:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-270ledma240a-smd-multi-mode-led-module-and-light-source-amber-240v-ac>: HTTP status code is not handled or not allowed 2026-01-24 20:40:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-single-phase-tefc-motor-2-hp-1740-rpm-115-230v-tefc>: HTTP status code is not handled or not allowed 2026-01-24 20:40:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ctm4103t-25-hp-1760-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:40:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cfdl3516tm-2-hp-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:40:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-inch-straight-jaw-dual-material-tongue-groove-pliers>: HTTP status code is not handled or not allowed 2026-01-24 20:40:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/100-professional-aluminum-chalk-reel>: HTTP status code is not handled or not allowed 2026-01-24 20:40:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/die_cut_labels>: HTTP status code is not handled or not allowed 2026-01-24 20:40:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esd-dissipative-octagonal-turntable-12-1-2l>: HTTP status code is not handled or not allowed 2026-01-24 20:40:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/resettable-drop-n-tell-25g-range-electronic-equipment>: HTTP status code is not handled or not allowed 2026-01-24 20:40:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rapidguard-2-run-post-8-5-4-h>: HTTP status code is not handled or not allowed 2026-01-24 20:40:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-rivet-boltless-shelving-36-inch-w-x-18-inch-d-x-84-inch-h-8-levels-add-on-w-o-decking-tan>: HTTP status code is not handled or not allowed 2026-01-24 20:40:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-cabinet-with-3-drawers-36-w-x-24-d-x-75-h-red>: HTTP status code is not handled or not allowed 2026-01-24 20:40:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/draft-inducer-blower-3000-rpm-115v-oao-1-8-fl-amps>: HTTP status code is not handled or not allowed 2026-01-24 20:40:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/condenser-fans-motor-1-3-hp-825-rpm-208-230v-teao-48-frame>: HTTP status code is not handled or not allowed 2026-01-24 20:40:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fan-and-blower-4-57-hp-1550-rpm-115v-teao>: HTTP status code is not handled or not allowed 2026-01-24 20:40:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 20236 pages (at 98 pages/min), scraped 12328 items (at 16 items/min) 2026-01-24 20:40:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s90-series-manual-chain-hoist-1-ton-10-ft-lift>: HTTP status code is not handled or not allowed 2026-01-24 20:40:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/long-deck-cart-2>: HTTP status code is not handled or not allowed 2026-01-24 20:40:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adirondack-polymer-composite-chair-with-arms-white>: HTTP status code is not handled or not allowed 2026-01-24 20:40:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/direct-drive-motor-1-6-hp-1050-rpm-115v-oao>: HTTP status code is not handled or not allowed 2026-01-24 20:40:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-cabinet-30-w-x-24-d-x-75-h-beige>: HTTP status code is not handled or not allowed 2026-01-24 20:40:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adirondack-polymer-composite-chair-with-arms-barn-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:40:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-shift-industrial-cabinet-36-w-x-24-d-x-75-h-orange>: HTTP status code is not handled or not allowed 2026-01-24 20:40:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd-series-beam-clamp-4400-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-24 20:40:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/car-stop-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:40:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/draft-inducer-blower-3000-rpm-115v-oao-2-63-fl-amps>: HTTP status code is not handled or not allowed 2026-01-24 20:40:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/charging-hose-hav-60-red-hose>: HTTP status code is not handled or not allowed 2026-01-24 20:40:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/recoverxlt-with-tank-overfill-switch>: HTTP status code is not handled or not allowed 2026-01-24 20:40:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multifunction-mesh-back-office-chair>: HTTP status code is not handled or not allowed 2026-01-24 20:40:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cewdm3711t-10-hp-3500-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:40:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-boltless-shelving-w-center-support-48-inch-w-x-30-inch-d-1-level-w-o-decking-black>: HTTP status code is not handled or not allowed 2026-01-24 20:40:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cd6219-1-00-hp-1750-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:40:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/men-indus-wrk-shirt-lng-slv-navy-reg-m-sp14>: HTTP status code is not handled or not allowed 2026-01-24 20:40:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-shift-industrial-cabinet-60-w-x-24-d-x-75-h-orange>: HTTP status code is not handled or not allowed 2026-01-24 20:40:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/men-indus-wrk-shirt-lng-slv-orange-reg-2xl-sp14>: HTTP status code is not handled or not allowed 2026-01-24 20:40:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-boltless-shelving-60-inch-w-x-48-inch-d-1-level-w-wire-decking-black>: HTTP status code is not handled or not allowed 2026-01-24 20:40:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-row-tire-storage-shelving-60-inch-w-x-21-inch-d-x-84-inch-h-3-levels-starter-tan>: HTTP status code is not handled or not allowed 2026-01-24 20:40:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/men-indus-wrk-shirt-lng-slv-dark-blue-reg-l-sp14>: HTTP status code is not handled or not allowed 2026-01-24 20:40:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-row-tire-storage-shelving-60-inch-w-x-21-inch-d-x-120-inch-h-4-levels-starter-tan>: HTTP status code is not handled or not allowed 2026-01-24 20:40:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-5-tier-starter-unit-36-w-x-21-d-x-74-h>: HTTP status code is not handled or not allowed 2026-01-24 20:40:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/men-deluxe-western-style-shirt-reg-xl-sc14-sc14lbrgxl>: HTTP status code is not handled or not allowed 2026-01-24 20:40:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-uniform-cabinet-with-7-drawers-60-w-x-24-d-x-75-h-black>: HTTP status code is not handled or not allowed 2026-01-24 20:40:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-rivet-boltless-shelving-48-inch-w-x-36-inch-d-x-84-inch-h-8-levels-starter-w-o-decking-tan>: HTTP status code is not handled or not allowed 2026-01-24 20:40:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-4-tier-starter-unit-36-w-x-18-d-x-63-h>: HTTP status code is not handled or not allowed 2026-01-24 20:40:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cabinet-with-keyless-entry-lock-36-w-x-24-d-x-75-h-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:40:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oem-replacement-motor-1-hp-1125-rpm-208-230-460v-oao-48-frame>: HTTP status code is not handled or not allowed 2026-01-24 20:40:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-boltless-shelving-96-inch-w-x-24-inch-d-x-84-inch-h-3-levels-add-on-w-o-decking-black>: HTTP status code is not handled or not allowed 2026-01-24 20:40:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-x-42-office-partition-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:40:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-idvsm4314t-60-hp-3600-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:40:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-boltless-shelving-72-inch-w-x-36-inch-d-x-84-inch-h-3-levels-starter-w-wire-decking-black>: HTTP status code is not handled or not allowed 2026-01-24 20:40:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-idvsm4316t-75-hp-3600-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:40:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-x-42-office-partition-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:40:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-boltless-shelving-48-inch-w-x-48-inch-d-x-84-inch-h-3-levels-add-on-w-wire-decking-black>: HTTP status code is not handled or not allowed 2026-01-24 20:40:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-polypropylene-sheet-protector-clear-11-x-8-1-2-25-bx>: HTTP status code is not handled or not allowed 2026-01-24 20:40:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-heavyweight-cleer-adheer-luggage-tag-with-straps-10-pk>: HTTP status code is not handled or not allowed 2026-01-24 20:40:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/definite_purpose_electric_motors/oem_replacement_ac_motors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:40:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-mechanical-sweeper-floor-carpet>: HTTP status code is not handled or not allowed 2026-01-24 20:40:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-swivel-elbow-1-8-tube-x-1-8-swift-fit-universal-thread-2-3-l>: HTTP status code is not handled or not allowed 2026-01-24 20:40:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hvywt-legal-poly-sheet-protector-for-3-or-4-ring-binders-clear-14-x-8-1-2-50-bx>: HTTP status code is not handled or not allowed 2026-01-24 20:40:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/column-protector-8-column-opening-lime>: HTTP status code is not handled or not allowed 2026-01-24 20:40:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oem-replacement-motor-1-4-hp-1075-rpm-115v-oao>: HTTP status code is not handled or not allowed 2026-01-24 20:40:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cabinet-with-electronic-lock-and-card-reader-60-w-x-24-d-x-75-h-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:40:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-high-3-way-two-60-low-panel>: HTTP status code is not handled or not allowed 2026-01-24 20:40:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/112573001-00514271-lumatris-90-led-led-machine-luminaire-36-watts>: HTTP status code is not handled or not allowed 2026-01-24 20:40:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v-grip-wire-shelving-spill-tray-36-l-x-18-w-x-2-1-4-d-5-gallon-capacity-textured-putty>: HTTP status code is not handled or not allowed 2026-01-24 20:40:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-canvas-short-sleeve-work-shirt-light-blue-m-ws20lbssm>: HTTP status code is not handled or not allowed 2026-01-24 20:40:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v-grip-wire-shelving-spill-tray-36-l-x-18-w-x-2-1-4-d-5-gallon-capacity-mirrored-silver>: HTTP status code is not handled or not allowed 2026-01-24 20:40:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-rack-4-shelves-with-12-span-track-flow-units-48-w-x-48-d-x-72-h-mirrored-silver>: HTTP status code is not handled or not allowed 2026-01-24 20:40:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-fiberglass-wastebaskets-14-qt-2>: HTTP status code is not handled or not allowed 2026-01-24 20:40:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-brute-plastic-round-bucket-10-quart-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:40:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-jpl1510t-7-5-hp-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:40:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-48-72-quick-connect-straight-extension-handle>: HTTP status code is not handled or not allowed 2026-01-24 20:40:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/definite_purpose_electric_motors/oem_replacement_ac_motors landed on page that is not a product page. 2026-01-24 20:40:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-l1200-17-hp-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:41:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v-grip-wire-shelving-spill-tray-36-l-x-24-w-x-2-1-4-d-6-5-gallon-capacity-textured-regal-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:41:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-idvsnm3665t-5-hp-6200-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:41:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/filter-multi-pleat-xl10-hc-20-w-x-22-h-x-1-d-12-ctn>: HTTP status code is not handled or not allowed 2026-01-24 20:41:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/propane-heater-infrared-ceramic-26000-btu>: HTTP status code is not handled or not allowed 2026-01-24 20:41:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rivetwell-ez-deck-decking-48-inch-w-x-15-inch-d-x-3-4-inch-h-for-use-w-double-rivets>: HTTP status code is not handled or not allowed 2026-01-24 20:41:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-gdl1610t-10-hp-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:41:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mri-safe-howard-instrument-table-24-l-x-36-w-x-34-h-on-casters-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 20:41:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-l3504-50-5-hp-1425ip44-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:41:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rivetwell-particle-board-decking-60-inch-w-x-18-inch-d-x-5-8-inch>: HTTP status code is not handled or not allowed 2026-01-24 20:41:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bassinet-kit-with-open-shelves-includes-bassinet-basket-w-cardholder-and-mattress-pad>: HTTP status code is not handled or not allowed 2026-01-24 20:41:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/record-storage-rack-starter-letter-legal-48-w-x-42-d-x-120-h-258195n>: HTTP status code is not handled or not allowed 2026-01-24 20:41:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-deck-for-rivetwell-36-inch-w-x-36-inch-d-x-1-4-inch-h-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:41:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 20335 pages (at 99 pages/min), scraped 12357 items (at 29 items/min) 2026-01-24 20:41:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-x-36-non-reflective-roll-up-vinyl-sign-transition-symbol-right>: HTTP status code is not handled or not allowed 2026-01-24 20:41:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/28-grabber-reflective-delineator-orange>: HTTP status code is not handled or not allowed 2026-01-24 20:41:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rivetwell-particle-board-decking-84-inch-w-x-36-inch-d-x-5-8-inch>: HTTP status code is not handled or not allowed 2026-01-24 20:41:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-comp-sink-18x18x12-deep-8-faucet-holes-18r-drainboard-galvanized-legs>: HTTP status code is not handled or not allowed 2026-01-24 20:41:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-floor-pads-red-16>: HTTP status code is not handled or not allowed 2026-01-24 20:41:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-floor-pads-black-12>: HTTP status code is not handled or not allowed 2026-01-24 20:41:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-m1005t-1-5-38-hp-1725-850-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:41:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:41:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-3-pos-ball-valve-closed-center-1-8-nptf-1-1-2-l>: HTTP status code is not handled or not allowed 2026-01-24 20:41:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mini-case-cart-29-5-8-w-x-40-1-2-h-x-29-d-1-s-s-wire-pullout-shelf-1-solid-door>: HTTP status code is not handled or not allowed 2026-01-24 20:41:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:41:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnetic-shop-ticket-holder-9-x-12-15-bx>: HTTP status code is not handled or not allowed 2026-01-24 20:41:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powermatic-8-parallelogram-jointer-2hp-1ph>: HTTP status code is not handled or not allowed 2026-01-24 20:41:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-m1556t-2-5-hp-1725-850-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:41:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-m1557t-3-75-hp-1725-850-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:41:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-m1204t-1-25-hp-1725-850-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:41:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-3-pos-ball-valve-closed-center-3-8-nptf-2-l>: HTTP status code is not handled or not allowed 2026-01-24 20:41:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-flow-control-valve-flow-knob-adjustment-5-32-tube-x-10-32-universal-thread-3-4-l>: HTTP status code is not handled or not allowed 2026-01-24 20:41:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bmp71-series-indoor-outdoor-industrial-vinyl-labels-2-w-yellow-m71c-2000-595-yl>: HTTP status code is not handled or not allowed 2026-01-24 20:41:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits>: HTTP status code is not handled or not allowed 2026-01-24 20:41:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-security-shelving-unit-chrome-14-w-x-48-l-x-66-h>: HTTP status code is not handled or not allowed 2026-01-24 20:41:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/knee-boot-size-10-15-h-composite-toe-chevron-plus-outsole-blue-w-red-sole>: HTTP status code is not handled or not allowed 2026-01-24 20:41:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits>: HTTP status code is not handled or not allowed 2026-01-24 20:41:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-needle-valve-knob-adjustment-3mm-tube-x-m5-swift-fit-universal-thread-3-4-l>: HTTP status code is not handled or not allowed 2026-01-24 20:41:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-needle-valve-screw-adjustment-1-2-tube-x-1-2-swift-fit-universal-thread-1-3-8-l>: HTTP status code is not handled or not allowed 2026-01-24 20:41:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lockout-duffel-bag-only-99162>: HTTP status code is not handled or not allowed 2026-01-24 20:41:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:41:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trim-fit-knee-boot-women-s-size-6-14-h-pvc-plain-toe-cleated-outsole-navy-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:41:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-em4403t-60-hp-1185-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:41:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_lockout_identification_tags already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:41:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-needle-valve-screw-adjustment-8mm-tube-x-1-4-swift-fit-universal-thread-1-l>: HTTP status code is not handled or not allowed 2026-01-24 20:41:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-em3708t-5-hp-1160-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:41:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultradrain-plug-4-diameter>: HTTP status code is not handled or not allowed 2026-01-24 20:41:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-person-first-aid-kit-plastic-case-with-dividers>: HTTP status code is not handled or not allowed 2026-01-24 20:41:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultradrain-guard-oil-sediment-plus-model-10pack>: HTTP status code is not handled or not allowed 2026-01-24 20:41:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultrasidewinder-3-ft-system-with-end-caps-black-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:41:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-shipping-tag-6-1-4-x-3-1-8-green-100-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:41:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_labels>: HTTP status code is not handled or not allowed 2026-01-24 20:41:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/propane-heater-infrared-positive-pressure-straight-tube-75000-btu-20l>: HTTP status code is not handled or not allowed 2026-01-24 20:41:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ejpm2535t-30-hp-1770-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:41:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/consecutively-numbered-repair-tags-4-3-4-x-2-3-8-yellow-100-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:41:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_lockout_identification_tags>: HTTP status code is not handled or not allowed 2026-01-24 20:41:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/consecutively-numbered-repair-tags-pre-wired-4-3-4-x-2-3-8-red-1000-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:41:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/big-d-carpet-deodorant-powder-breeze-50-lb-container>: HTTP status code is not handled or not allowed 2026-01-24 20:41:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyurethane-cleanroom-stool-with-casters-desk-height-black>: HTTP status code is not handled or not allowed 2026-01-24 20:41:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-acrylic-workstation-riser-34in-w-x-8in-d-x-10in-h-clear>: HTTP status code is not handled or not allowed 2026-01-24 20:41:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-gas-heater-infrared-positive-pressure-straight-tube-40000-btu-10l>: HTTP status code is not handled or not allowed 2026-01-24 20:41:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-33in-amber-rectangular-turn-signal-light-kit-with-15-led-5625215>: HTTP status code is not handled or not allowed 2026-01-24 20:41:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-x-10-roof-panel-2>: HTTP status code is not handled or not allowed 2026-01-24 20:41:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-angled-broom-with-vinyl-coated-metal-handle-2>: HTTP status code is not handled or not allowed 2026-01-24 20:41:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-x-27-inch-welded-black-formed-steel-mounting-brackets>: HTTP status code is not handled or not allowed 2026-01-24 20:41:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mini-tank-water-heater-2-5-gallon-120v-plug>: HTTP status code is not handled or not allowed 2026-01-24 20:41:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-em2528t-20-hp-1180-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:41:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-em2531t-25-hp-1760-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:41:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pressure-sensitive-pipe-marker-natural-gas-1>: HTTP status code is not handled or not allowed 2026-01-24 20:41:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precoiled-strap-on-pipe-marker-chilled-water-return-3>: HTTP status code is not handled or not allowed 2026-01-24 20:41:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-water-bucket-square-or-hex-windshield-service-center>: HTTP status code is not handled or not allowed 2026-01-24 20:41:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/task_desk_chairs/task_office_chairs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:41:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stacking-drawer-12-h>: HTTP status code is not handled or not allowed 2026-01-24 20:41:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-5in-clear-thin-mount-rectangular-strobe-light-with-12-led-8892601>: HTTP status code is not handled or not allowed 2026-01-24 20:42:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/door-stop-spring-loaded-h-d-iron-aluminum-finish>: HTTP status code is not handled or not allowed 2026-01-24 20:42:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/surface-mount-wall-cabinet>: HTTP status code is not handled or not allowed 2026-01-24 20:42:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precoiled-strap-on-pipe-marker-carbon-dioxide-1>: HTTP status code is not handled or not allowed 2026-01-24 20:42:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9000-lb-capacity-high-rise-scissor-lift-ateattd-slp9k-fpd>: HTTP status code is not handled or not allowed 2026-01-24 20:42:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precoiled-strap-on-pipe-marker-oxygen-9>: HTTP status code is not handled or not allowed 2026-01-24 20:42:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/400-coverall-front-zipper-serged-seams-open-wrist-ankles-white-md>: HTTP status code is not handled or not allowed 2026-01-24 20:42:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pacon-easel-pad-50-sheet-unruled-27-x-34-50-pad-white-paper>: HTTP status code is not handled or not allowed 2026-01-24 20:42:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pressure-sensitive-pipe-marker-vent-5>: HTTP status code is not handled or not allowed 2026-01-24 20:42:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hemco-emergency-showerdecontamination-booth-40-x-36-x-90>: HTTP status code is not handled or not allowed 2026-01-24 20:42:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precoiled-strap-on-pipe-marker-direction-arrow-23>: HTTP status code is not handled or not allowed 2026-01-24 20:42:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precoiled-strap-on-pipe-marker-sprinkler-fire-5>: HTTP status code is not handled or not allowed 2026-01-24 20:42:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/task_desk_chairs/task_office_chairs landed on page that is not a product page. 2026-01-24 20:42:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precoiled-strap-on-pipe-marker-storm-drain-6>: HTTP status code is not handled or not allowed 2026-01-24 20:42:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/300ww-bridal-hanger-jumbo-heavy-weight-17l-plastic-cl-pkg-qty-100>: HTTP status code is not handled or not allowed 2026-01-24 20:42:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 20429 pages (at 94 pages/min), scraped 12383 items (at 26 items/min) 2026-01-24 20:42:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fellowes-linen-presentation-binding-covers-8-x-11-oversize-letter-black-200-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:42:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precoiled-strap-on-pipe-marker-air-5>: HTTP status code is not handled or not allowed 2026-01-24 20:42:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-174-1525stk-combination-padlock-key-access-with-1-control-key-chart-price-each>: HTTP status code is not handled or not allowed 2026-01-24 20:42:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fellowes-callisto-95-laminator>: HTTP status code is not handled or not allowed 2026-01-24 20:42:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hardware-base-72in-96in-steel-bench-with-back-rest>: HTTP status code is not handled or not allowed 2026-01-24 20:42:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hardware-tan-plastic-picnic-table-top>: HTTP status code is not handled or not allowed 2026-01-24 20:42:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-x-120-21-degree-ring-shank-galvanized-framing-nails-2000-qty>: HTTP status code is not handled or not allowed 2026-01-24 20:42:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-mvm3542d-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:42:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h6fv-optime-95-low-profile-folding-ear-muff-h6fv>: HTTP status code is not handled or not allowed 2026-01-24 20:42:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mlrs15mclk-breast-milk-refrigerator-with-8-lockers-15-cu-ft>: HTTP status code is not handled or not allowed 2026-01-24 20:42:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-gauge-2-l-shape-flooring-cleats-1000-qty>: HTTP status code is not handled or not allowed 2026-01-24 20:42:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-gauge-1-1-2-brad-nail-2500-qty>: HTTP status code is not handled or not allowed 2026-01-24 20:42:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-pick-shelving-5-level-double-straight-straight-78-h-x-50-w-x-40-d-add-on>: HTTP status code is not handled or not allowed 2026-01-24 20:42:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-degree-coil-framing-nails-3-1-4-x-120-smooth-galvanized-2700-qty>: HTTP status code is not handled or not allowed 2026-01-24 20:42:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-x-2-1-2-brass-pipe-nipple-schedule-40>: HTTP status code is not handled or not allowed 2026-01-24 20:42:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24w-round-leather-ottoman-black-hercules-alon-series>: HTTP status code is not handled or not allowed 2026-01-24 20:42:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-2-smooth-shank-15-degree-coil-roofing-nails-7200-qty>: HTTP status code is not handled or not allowed 2026-01-24 20:42:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lab-design-sink-bench-3-3-4-h-x-30-d-x-107-w-model-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:42:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/isbksc003-individual-stop-bleeding-kit-hemostatic-gauze-2-tourniquets>: HTTP status code is not handled or not allowed 2026-01-24 20:42:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/orbis-lid-folding-bulk-shipping-container-32x30>: HTTP status code is not handled or not allowed 2026-01-24 20:42:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/21w-stacking-church-chair-fabric-gray-hercules-series>: HTTP status code is not handled or not allowed 2026-01-24 20:42:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-piece-outdoor-patio-sofa-set-faux-rattan-dark-gray-with-light-gray-cushions>: HTTP status code is not handled or not allowed 2026-01-24 20:42:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-x-3-welded-pipe-nipple-schedule-40-304-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 20:42:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4725-rectangle-plastic-height-adjustable-activity-table-set-with-6-chairs-green>: HTTP status code is not handled or not allowed 2026-01-24 20:42:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-22-high-velocity-fan-dock-truck-mount-115v>: HTTP status code is not handled or not allowed 2026-01-24 20:42:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wood-desk-full-height-pedestals-72-inch-w-x-36-inch-d-x-29-half-inch-h-mahogany>: HTTP status code is not handled or not allowed 2026-01-24 20:42:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-plastic-sign-danger-chemical-storage-area-authorized-personnel-only-2>: HTTP status code is not handled or not allowed 2026-01-24 20:42:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/split-cowhide-drivers-premium-grade-keystone-thumb-gray-l>: HTTP status code is not handled or not allowed 2026-01-24 20:42:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-vem3587t-5-2-hp-1750-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:42:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-1-8-insert-bit-x-1-s2-modified-steel-gunmetal-grey>: HTTP status code is not handled or not allowed 2026-01-24 20:42:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-water-line-hardware-kit-for-bottle-filling-station-761219>: HTTP status code is not handled or not allowed 2026-01-24 20:42:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torq-set-1-4-insert-bit-x-1-1-4-5-16-hex-shank-s2-modified-steel-gunmetal-grey>: HTTP status code is not handled or not allowed 2026-01-24 20:42:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shop-desk-wall-mount-pigeonhole-riser-pegboard-panels-with-shelf-34-1-2-w-x-30-d-x-61-h-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:42:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-vinyl-sign-danger-hot-2>: HTTP status code is not handled or not allowed 2026-01-24 20:42:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torx-174-tamper-10-power-bit-x-6-s2-modified-steel-gunmetal-grey>: HTTP status code is not handled or not allowed 2026-01-24 20:42:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/gloves_sleeves/arc_flash_flame_resistant_gloves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:42:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-vinyl-sign-danger-no-smoking-3>: HTTP status code is not handled or not allowed 2026-01-24 20:42:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-vinyl-sign-caution-confined-space-use-lockout-entry-procedures>: HTTP status code is not handled or not allowed 2026-01-24 20:42:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-container-solid-wall-48x40x31-short-side-runners-yellow-1500-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:42:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4833-es-nantucket-32-tall-planter-espresso>: HTTP status code is not handled or not allowed 2026-01-24 20:42:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carbon-steel-hand-winch-with-brake-1500-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:42:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-plastic-sign-danger-asbestos-cancer-lung-disease-hazard-2>: HTTP status code is not handled or not allowed 2026-01-24 20:42:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/top-grain-cowhide-drivers-straight-thumb-quality-grade-l>: HTTP status code is not handled or not allowed 2026-01-24 20:42:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-partition-base-panel-with-desktop-raceway-power-24-w-x-38-h-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:42:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electrical-rated-gloves-14-l-unlined-smooth-finish-beaded-orange-class-2-size-10>: HTTP status code is not handled or not allowed 2026-01-24 20:42:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-partition-base-panel-with-baseline-raceway-power-30-w-x-38-h-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:42:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torq-set-8-power-bit-x-1-1-4-s2-modified-steel-gunmetal-grey>: HTTP status code is not handled or not allowed 2026-01-24 20:42:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/furniture_parts_accessories/partition_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:42:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-plastic-sign-caution-wheels-must-be-chocked-before-loading-unloading-2>: HTTP status code is not handled or not allowed 2026-01-24 20:42:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bl1500-lifting-magnet-1500-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:42:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-vinyl-sign-caution-floor-load-limit-not-to-exceed>: HTTP status code is not handled or not allowed 2026-01-24 20:42:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/gloves_sleeves/arc_flash_flame_resistant_gloves landed on page that is not a product page. 2026-01-24 20:42:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torx-174-align-25-insert-bit-x-1-s2-modified-steel-gunmetal-grey>: HTTP status code is not handled or not allowed 2026-01-24 20:42:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esp-kit-1-pair-black-esp-glove-1-pair-cow-class-2-size-11>: HTTP status code is not handled or not allowed 2026-01-24 20:42:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/csp-sw38-o-egfrhl-strongwall-ada-compliant-plastic-barricade-orange-38-h-enginer-grade-sheeting>: HTTP status code is not handled or not allowed 2026-01-24 20:42:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-3mm-power-bit-x-3-1-2-s2-modified-steel-gunmetal-grey>: HTTP status code is not handled or not allowed 2026-01-24 20:42:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-vinyl-sign-danger-electrical-hazard>: HTTP status code is not handled or not allowed 2026-01-24 20:42:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-3-tier-wire-shelving-starter-unit-72w-x-14d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 20:42:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/csp-sg-4kit-hip-y-plastic-manhole-guard-nonconductive-high-intensity-prismatice-sheeting>: HTTP status code is not handled or not allowed 2026-01-24 20:42:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-plastic-channelizing-orange-cone-with-handle-4-6-high-intensity-grade-sheeting-bands>: HTTP status code is not handled or not allowed 2026-01-24 20:42:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-vinyl-sign-notice-employees-only-2>: HTTP status code is not handled or not allowed 2026-01-24 20:42:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/furniture_parts_accessories/partition_accessories>: HTTP status code is not handled or not allowed 2026-01-24 20:42:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gloveworks-diamond-textured-industrial-grade-nitrile-gloves-2xl-100-box-10-box-cs>: HTTP status code is not handled or not allowed 2026-01-24 20:42:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-3-fr-long-sleeve-t-shirt-fluorescent-yellow-green-xl>: HTTP status code is not handled or not allowed 2026-01-24 20:42:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-3-tier-wire-shelving-starter-unit-60w-x-24d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 20:42:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/square-2-power-bit-x-3-1-2-gunmetal-bronze-s2-modified-steel>: HTTP status code is not handled or not allowed 2026-01-24 20:42:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/little-giant-mm-lpd-fl-lp-mobile-bench-cabinets-louvered-doors-panel-non-slip-vinyl-top>: HTTP status code is not handled or not allowed 2026-01-24 20:42:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/little-giant-wwc2-3072-heavy-duty-cabinet-workbench-steel-top-center-shelf>: HTTP status code is not handled or not allowed 2026-01-24 20:42:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-10-t-slot-nut-13-16-table-slot-1-3-8-base-width-1-3-8-height-carbon-steel-black-oxide-tn-40>: HTTP status code is not handled or not allowed 2026-01-24 20:43:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cam-follower-sealed-hex-head-5-8-l-stud-0375-w-roller-05-roller-dia>: HTTP status code is not handled or not allowed 2026-01-24 20:43:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-aluminum-sign-danger-construction-area-keep-out>: HTTP status code is not handled or not allowed 2026-01-24 20:43:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cam-follower-sealed-hex-head-1-1-4-l-stud-075-w-roller-125-roller-dia>: HTTP status code is not handled or not allowed 2026-01-24 20:43:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mb3-2d-2448-fl-48-w-mobile-bench-cabinets-center-shelf-2-locking-doors>: HTTP status code is not handled or not allowed 2026-01-24 20:43:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rs-2436-ll-receiving-station-24-x-36>: HTTP status code is not handled or not allowed 2026-01-24 20:43:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-plastic-sign-notice-keep-this-door-closed-2>: HTTP status code is not handled or not allowed 2026-01-24 20:43:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-2-tier-wire-shelving-starter-unit-60w-x-21d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 20:43:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-battery-cable-with-5-16-ring-terminals-2-awg-10-ft-black>: HTTP status code is not handled or not allowed 2026-01-24 20:43:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tuning-fork-without-weight-512-cps-satin-aluminum>: HTTP status code is not handled or not allowed 2026-01-24 20:43:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/operating-scissors-straight-5-1-2-l-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 20:43:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 20523 pages (at 94 pages/min), scraped 12400 items (at 17 items/min) 2026-01-24 20:43:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/watts-big-bubba-industrial-high-flow-housing>: HTTP status code is not handled or not allowed 2026-01-24 20:43:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mayo-dissecting-scissors-5-1-2-l-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 20:43:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-174-automatic-dispenser-foam-hand-soap-sanitizer-white-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:43:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-tier-hanging-chair-coat-truck-holds-84-chairs-70-coats>: HTTP status code is not handled or not allowed 2026-01-24 20:43:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/summit-ada-compliant-built-in-undercounter-refrigerator-complete-ss-wfront-lock>: HTTP status code is not handled or not allowed 2026-01-24 20:43:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelf-truck-esd-21w-x-48l-x-60h-polyurethane-4-swivel-2-brake-casters>: HTTP status code is not handled or not allowed 2026-01-24 20:43:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v6770170-xl-heelstop-anti-slip-heel-traction-aid>: HTTP status code is not handled or not allowed 2026-01-24 20:43:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flexbollard-52h-x-7-14w-w99h-signpost-concrete-yellowred-xl-b528-c-yellowred>: HTTP status code is not handled or not allowed 2026-01-24 20:43:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ice-cream-dipping-cabinet-w-sneeze-guard-cover-60-w>: HTTP status code is not handled or not allowed 2026-01-24 20:43:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelf-truck-esd-18w-x-48l-x-60h-polyurethane-4-swivel-2-brake-casters>: HTTP status code is not handled or not allowed 2026-01-24 20:43:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-step-heavy-duty-mobile-work-platform-handrails-24-x-48-platform>: HTTP status code is not handled or not allowed 2026-01-24 20:43:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/danger-sign-10x7-vinyl-workers-above>: HTTP status code is not handled or not allowed 2026-01-24 20:43:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/danger-sign-10x7-rigid-plastic-p-points-watch-your-hands>: HTTP status code is not handled or not allowed 2026-01-24 20:43:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-4-tier-esd-24w-x-24l-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 20:43:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ch-262008-1-12-muni-ball-plug-18-bypass-22-psi-50-ft>: HTTP status code is not handled or not allowed 2026-01-24 20:43:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/danger-sign-10x7-rigid-plastic-corrosive-avoid-contact>: HTTP status code is not handled or not allowed 2026-01-24 20:43:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hc-30211-all-purpose-leak-detector-with-trigger-spray-16-oz>: HTTP status code is not handled or not allowed 2026-01-24 20:43:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-battery-cable-with-3-8-ring-terminals-2-0-awg-8-ft-black>: HTTP status code is not handled or not allowed 2026-01-24 20:43:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/sinks/wash_fountains already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:43:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/danger-sign-14x10-vinyl-no-smoking-or-open-flame>: HTTP status code is not handled or not allowed 2026-01-24 20:43:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honeywell-dual-standing-pilot-gas-valve-34x34-step-35-wc-09-wc>: HTTP status code is not handled or not allowed 2026-01-24 20:43:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-panel-portable-room-divider-6-ft-h-16-ft-9-inch-fabric-color-navy>: HTTP status code is not handled or not allowed 2026-01-24 20:43:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergency-sign-14x10-aluminum-emergency-shower-keep-area-clear>: HTTP status code is not handled or not allowed 2026-01-24 20:43:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergency-sign-10x7-rigid-plastic-first-aid-station>: HTTP status code is not handled or not allowed 2026-01-24 20:43:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-3-tier-esd-36w-x-36l-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 20:43:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-long-triple-glove-box-dispenser-22-5-8-w-x-4-d-x-10-1-8-h-transparent>: HTTP status code is not handled or not allowed 2026-01-24 20:43:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-hopscotch-playground-stencil-1-8-thick-polytough-plastic-white>: HTTP status code is not handled or not allowed 2026-01-24 20:43:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honeywell-white-electric-heat-thermostat-electric-baseboard-heat>: HTTP status code is not handled or not allowed 2026-01-24 20:43:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-shelf-truck-esd-3-shelf-18w-x-36l-x-40h-polyurethane-4-swivel-2-brake-casters>: HTTP status code is not handled or not allowed 2026-01-24 20:43:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12x12x4-1-2-stackable-bin-boxes>: HTTP status code is not handled or not allowed 2026-01-24 20:43:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40439-satin-disc-wheel-1-8-mandrel-1-1-2-aluminum-oxide-fine>: HTTP status code is not handled or not allowed 2026-01-24 20:43:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10666-surf-cond-belt-ls-fd-48-x-6-aluminum-oxide-very-fine>: HTTP status code is not handled or not allowed 2026-01-24 20:43:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/db-p9225bbg-bath-waste-half-kit-sch-40-uni-lift-stopper-chr-finish-trim-zinc-drain-unit>: HTTP status code is not handled or not allowed 2026-01-24 20:43:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/plumbing/sinks/wash_fountains>: HTTP status code is not handled or not allowed 2026-01-24 20:43:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10665-surf-cond-belt-ls-48-x-6-aluminum-oxide-medium>: HTTP status code is not handled or not allowed 2026-01-24 20:43:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40314-qc-disc-type-s-2-aluminum-oxide-medium>: HTTP status code is not handled or not allowed 2026-01-24 20:43:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ch-271713-3-clean-seal-plug-13-psi-30-ft>: HTTP status code is not handled or not allowed 2026-01-24 20:43:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electric-vehicle-charging-symbol-sm-polytough-plastic-white>: HTTP status code is not handled or not allowed 2026-01-24 20:43:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-2-tier-esd-14w-x-24l-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 20:43:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-w-machine-guard-wire-panel-black>: HTTP status code is not handled or not allowed 2026-01-24 20:43:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-esd-36w-x-60l-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 20:43:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-inch-x-24-inch-x-10-inch-corrugated-boxes>: HTTP status code is not handled or not allowed 2026-01-24 20:43:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-freestanding-office-partition-panel-24-14-w-x-43-12-h-tan>: HTTP status code is not handled or not allowed 2026-01-24 20:43:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-freestanding-office-partition-panel-36-14-w-x-96-h-tan>: HTTP status code is not handled or not allowed 2026-01-24 20:43:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-freestanding-office-partition-panel-60-14-w-x-61-12-h-tan>: HTTP status code is not handled or not allowed 2026-01-24 20:43:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-6-gpf-exposed-water-closet-valve>: HTTP status code is not handled or not allowed 2026-01-24 20:43:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/freestanding-office-partition-panel-with-partial-window-60-14-w-x-42-h-tan>: HTTP status code is not handled or not allowed 2026-01-24 20:43:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-inch-x-16-inch-x-14-inch-doublewall-boxes>: HTTP status code is not handled or not allowed 2026-01-24 20:43:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plant-marking-stencil-20x20-do-not-block>: HTTP status code is not handled or not allowed 2026-01-24 20:43:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mw-30246-ss-beverage-merchandiser-30w-x-24d-x-6h>: HTTP status code is not handled or not allowed 2026-01-24 20:43:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-plastic-garbage-can-with-lid-44-gal-green>: HTTP status code is not handled or not allowed 2026-01-24 20:43:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelf-truck-esd-36w-x-48l-x-60h-polyurethane-4-swivel-2-brake-casters>: HTTP status code is not handled or not allowed 2026-01-24 20:43:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-4-tier-esd-18w-x-54l-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 20:43:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/martin-yale-desktop-letter-opener>: HTTP status code is not handled or not allowed 2026-01-24 20:43:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mw-dun-2460-extra-heavy-duty-dunnage-rack-60w-x-24d-x-12h-3000-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:43:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rust-oleum-v7500-series-450-voc-dtm-alkyd-enamel-high-gloss-white>: HTTP status code is not handled or not allowed 2026-01-24 20:43:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fda-buna-n-sanitary-gasket-for-15-tube>: HTTP status code is not handled or not allowed 2026-01-24 20:43:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/office-partition-panel-with-partial-window-raceway-60-14-w-x-64-h-tan>: HTTP status code is not handled or not allowed 2026-01-24 20:43:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rust-oleum-m1800-sys-watr-bas-prec-line-invert-mark-pnt-aero-fluor-grn>: HTTP status code is not handled or not allowed 2026-01-24 20:43:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10x1-1-2-indented-hex-washer-with-bonded-neo-epdm-washer-f-t-self-piercing-1024PSWN>: HTTP status code is not handled or not allowed 2026-01-24 20:43:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-freestanding-office-partition-panel-wpartial-window-48-14-w-x-73-12-h-tan>: HTTP status code is not handled or not allowed 2026-01-24 20:43:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/individual-character-stencil-24-letter-z>: HTTP status code is not handled or not allowed 2026-01-24 20:44:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-4-tier-esd-24w-x-36l-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 20:44:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-16x3-4-indent-hex-washer-head-thread-cutting-screws-type-f-3712FW>: HTTP status code is not handled or not allowed 2026-01-24 20:44:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-office-partition-panel-with-partial-window-36-14-w-x-61-12-h-tan>: HTTP status code is not handled or not allowed 2026-01-24 20:44:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-office-partition-panel-wpartial-window-pass-thru-cable-36-14wx77-12h-tan>: HTTP status code is not handled or not allowed 2026-01-24 20:44:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rust-oleum-high-perform-v2100-rust-preventive-enam-aero-semi-gloss-blk>: HTTP status code is not handled or not allowed 2026-01-24 20:44:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brown-viton-o-ring-dash-021-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-24 20:44:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-24x1-indented-hex-head-machine-screws-1016MH188>: HTTP status code is not handled or not allowed 2026-01-24 20:44:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metal-detectable-silicone-o-ring-dash-216-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-24 20:44:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-office-partition-panel-48-14-w-x-43-12-h-tan>: HTTP status code is not handled or not allowed 2026-01-24 20:44:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-24x2-flat-head-self-drilling-screws-1232KPFMS4W>: HTTP status code is not handled or not allowed 2026-01-24 20:44:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rust-oleum-as5400-sys-340-voc-anti-slip-one-step-epxy-flr-coat-blk>: HTTP status code is not handled or not allowed 2026-01-24 20:44:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/art1st-sketch-diary-4790-9-x-6-white-70-sheetspad>: HTTP status code is not handled or not allowed 2026-01-24 20:44:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/art1st-sketch-pad-4746-9-x-12-white-50-sheetspad>: HTTP status code is not handled or not allowed 2026-01-24 20:44:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12x1-pan-head-self-tapping-screws-type-a-thread-1216ACP>: HTTP status code is not handled or not allowed 2026-01-24 20:44:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 20617 pages (at 94 pages/min), scraped 12419 items (at 19 items/min) 2026-01-24 20:44:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2x3-8-pan-head-plastite-alternative-48-2-0206LTP>: HTTP status code is not handled or not allowed 2026-01-24 20:44:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-safety-sign-fire-extinguisher-vinyl-4>: HTTP status code is not handled or not allowed 2026-01-24 20:44:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/soft-top-mow-n-vac-93-3-5-inch-l-x-33-1-2-inch-w-x-49-84-100-inch-h-black>: HTTP status code is not handled or not allowed 2026-01-24 20:44:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-8482-8-confined-space-vent-fan-rotomold-plastic-1000-cfm-1-3-hp>: HTTP status code is not handled or not allowed 2026-01-24 20:44:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-sticky-notes-cabinet-pk-65424ssaucp-3-x-3-jewel-pop-70-sheets-24pack>: HTTP status code is not handled or not allowed 2026-01-24 20:44:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/epdm-o-ring-1mm-wide-5mm-id-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-24 20:44:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-safety-sign-fire-extinguisher-inside-vinyl-2>: HTTP status code is not handled or not allowed 2026-01-24 20:44:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/viton-o-ring-2mm-wide-32mm-id-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 20:44:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-panel-shipping-crate-with-lid-pallet-60-x-48-x-48-od>: HTTP status code is not handled or not allowed 2026-01-24 20:44:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8x3-phillips-bugle-head-course-thread-sharp-point-deck-screws-dacrotized-0848DPG>: HTTP status code is not handled or not allowed 2026-01-24 20:44:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rust-oleum-v7400-series-340-voc-dtm-alkyd-enamel-chestnut-brown>: HTTP status code is not handled or not allowed 2026-01-24 20:44:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:44:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rust-oleum-sierra-perf-mtlmx-0-voc-dtm-acryl-enam-sem-gls-wh-past-bas>: HTTP status code is not handled or not allowed 2026-01-24 20:44:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silicone-o-ring-dash-328-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 20:44:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/comfortable-sleep-king-12-foam-pocket-spring-mattress-2-gel-memory-foam-topper>: HTTP status code is not handled or not allowed 2026-01-24 20:44:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chart-tablets-wcursive-cvr-74610-24-x-32-white-25-sheetspack>: HTTP status code is not handled or not allowed 2026-01-24 20:44:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electrical-lockout-center-backboard-rack>: HTTP status code is not handled or not allowed 2026-01-24 20:44:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10x1-1-4-flat-head-self-drilling-screws-1020KPU>: HTTP status code is not handled or not allowed 2026-01-24 20:44:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tingley-174-pilot-g2-knee-boot-composite-safety-toe-15-h-size-4-black>: HTTP status code is not handled or not allowed 2026-01-24 20:44:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-20-serrated-flange-lock-nuts-14NRB>: HTTP status code is not handled or not allowed 2026-01-24 20:44:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/valve-wheel-lockout-fits-10-to-14-diameter>: HTTP status code is not handled or not allowed 2026-01-24 20:44:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metal-platform-bed-frame-14-h-10-pocket-spring-mattress-in-a-box-queen-size>: HTTP status code is not handled or not allowed 2026-01-24 20:44:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-hepa-filter-for-cat-c16v-wet-dry-vacuum-641759>: HTTP status code is not handled or not allowed 2026-01-24 20:44:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards>: HTTP status code is not handled or not allowed 2026-01-24 20:44:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ingersoll-rand-general-purpose-filter-42-cfm>: HTTP status code is not handled or not allowed 2026-01-24 20:44:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tribeca-full-tufted-upholstered-platform-bed-light-gray-memory-foam-mattress>: HTTP status code is not handled or not allowed 2026-01-24 20:44:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/418-2-tubeless-tire-valve>: HTTP status code is not handled or not allowed 2026-01-24 20:44:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/recycled-weekly-appointment-book-black-4-7-8-x-8-2015-aagg21000>: HTTP status code is not handled or not allowed 2026-01-24 20:44:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8x1-bugle-head-drywall-screws-0816YQG>: HTTP status code is not handled or not allowed 2026-01-24 20:44:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:44:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/riverdale-twin-size-tufted-upholstered-platform-bed-in-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:44:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silicone-rubber-cord-0-070-cross-section-100-ft-length>: HTTP status code is not handled or not allowed 2026-01-24 20:44:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-ton-low-height-pin-type-jack-stands-pair-1210>: HTTP status code is not handled or not allowed 2026-01-24 20:44:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/comfortable-sleep-12-memory-foam-and-pocket-spring-mattress-in-a-box-twin-size>: HTTP status code is not handled or not allowed 2026-01-24 20:44:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sustain-landfill-container-23-gal-black>: HTTP status code is not handled or not allowed 2026-01-24 20:44:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-32x1-2-light-series-cold-forged-wing-screws-full-thread-type-a-1108WA>: HTTP status code is not handled or not allowed 2026-01-24 20:44:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-ton-engine-sling-5208a>: HTTP status code is not handled or not allowed 2026-01-24 20:44:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ingersoll-rand-12-install-kit-5-hp>: HTTP status code is not handled or not allowed 2026-01-24 20:44:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buna-n-o-ring-3mm-wide-25mm-id-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-24 20:44:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/dock_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:44:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/printed-barricade-tape-danger-confined-space-do-not-enter-2>: HTTP status code is not handled or not allowed 2026-01-24 20:44:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-20x1-button-head-socket-cap-screws-1416CSB>: HTTP status code is not handled or not allowed 2026-01-24 20:44:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards landed on page that is not a product page. 2026-01-24 20:44:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t5430-stamp-replacement-ink-pad-1-x-1-5-8-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:44:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-15-11112-100-psi-18-cbm-plastic>: HTTP status code is not handled or not allowed 2026-01-24 20:44:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buna-n-o-ring-dash-121-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 20:44:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dp200-36-round-traffic-channelizer-post-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:44:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/heaters/stoves_fireplaces_log_sets_fire_pits/fireplaces_log_sets_fireboxes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:44:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-purpose-storage-chest-3-8-cubic>: HTTP status code is not handled or not allowed 2026-01-24 20:44:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dial-n-stamp-12-phrases-1-1-2-x-1-8>: HTTP status code is not handled or not allowed 2026-01-24 20:44:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/stamp_pads already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:44:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m6-1-0x50-metric-pan-head-machine-screws-M650D7985A>: HTTP status code is not handled or not allowed 2026-01-24 20:44:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-fuel-ventless-gas-stove-23000-btu-t-stat-control>: HTTP status code is not handled or not allowed 2026-01-24 20:44:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/dock_lights>: HTTP status code is not handled or not allowed 2026-01-24 20:44:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-fuel-ventless-gas-fireplace-w-mantel-26000-btu-t-stat-apple-spice-dfs-300t-3as>: HTTP status code is not handled or not allowed 2026-01-24 20:44:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/divider-kit-for-5-h-drawer-of-modular-drawer-cabinet-30-wx27-d-black>: HTTP status code is not handled or not allowed 2026-01-24 20:44:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/domed-seat-straight-back-stacking-chair-chrome-frame-w-blue-vinyl-upholstery>: HTTP status code is not handled or not allowed 2026-01-24 20:44:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:44:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/domed-seat-round-back-stacking-chair-black-frame-w-blue-fabric-upholstery>: HTTP status code is not handled or not allowed 2026-01-24 20:44:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-45-11125ct-100-psi-14-lm-ss>: HTTP status code is not handled or not allowed 2026-01-24 20:44:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/heaters/stoves_fireplaces_log_sets_fire_pits/fireplaces_log_sets_fireboxes>: HTTP status code is not handled or not allowed 2026-01-24 20:44:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14x5-flat-head-wood-screws-1480DPF>: HTTP status code is not handled or not allowed 2026-01-24 20:44:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/basic_supplies/stamp_pads>: HTTP status code is not handled or not allowed 2026-01-24 20:44:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/neoprene-foam-with-acrylic-adhesive-316-thick-x-516w-x-10l>: HTTP status code is not handled or not allowed 2026-01-24 20:44:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-rack-solid-steel-deck-46-w-x-36-d-796893>: HTTP status code is not handled or not allowed 2026-01-24 20:44:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-rack-solid-steel-deck-52-w-x-36-d-796894>: HTTP status code is not handled or not allowed 2026-01-24 20:44:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-rack-solid-steel-deck-58-w-x-42-d-796898>: HTTP status code is not handled or not allowed 2026-01-24 20:44:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extruded-graphite-rod-34-dia-x-12l>: HTTP status code is not handled or not allowed 2026-01-24 20:44:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-25-11110-60-psikpa-14-lm-plastic>: HTTP status code is not handled or not allowed 2026-01-24 20:45:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges>: HTTP status code is not handled or not allowed 2026-01-24 20:45:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triple-view-weekly-monthly-appointment-book-black-8-1-4-x-10-7-8-2015-aag70950v05>: HTTP status code is not handled or not allowed 2026-01-24 20:45:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glow-nyc-sign-down-le>: HTTP status code is not handled or not allowed 2026-01-24 20:45:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silicone-rubber-strip-no-adhesive-40a-18-thick-x-12w-x-10l>: HTTP status code is not handled or not allowed 2026-01-24 20:45:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-brute-174-1779739-trash-container-55-gallon-black>: HTTP status code is not handled or not allowed 2026-01-24 20:45:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-countersunk-finishing-washers-12WC188>: HTTP status code is not handled or not allowed 2026-01-24 20:45:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hirsh-industries-20in-deep-file-file-mobile-pedestal-black>: HTTP status code is not handled or not allowed 2026-01-24 20:45:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fda-silicone-foam-with-high-temp-adhesive-12-thick-x-34w-x-6l>: HTTP status code is not handled or not allowed 2026-01-24 20:45:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hirsh-industries-20in-deep-file-file-mobile-pedestal-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:45:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glow-sign-vinyl-exit-only>: HTTP status code is not handled or not allowed 2026-01-24 20:45:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:45:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/basket-fan-20-black-115v-1-3hp-1ph-s-var-w-brkt-cord-rp>: HTTP status code is not handled or not allowed 2026-01-24 20:45:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/non-illuminated-mushroom-head-pb-metal-bezel-22mm-red-p9m-er4rn>: HTTP status code is not handled or not allowed 2026-01-24 20:45:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-lockers-four-tier-12-x-18-x-18-12-door-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:45:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/easy-peel-laser-mailing-labels-1-1-3-x-4-clear-700-box-5662>: HTTP status code is not handled or not allowed 2026-01-24 20:45:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 20709 pages (at 92 pages/min), scraped 12434 items (at 15 items/min) 2026-01-24 20:45:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/l-90-2-step-300-lb-ladder>: HTTP status code is not handled or not allowed 2026-01-24 20:45:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/easy-peel-inkjet-address-labels-1-1-3-x-4-white-350-pack-8162>: HTTP status code is not handled or not allowed 2026-01-24 20:45:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-main-brush-49-auto-ride-on-sweeper>: HTTP status code is not handled or not allowed 2026-01-24 20:45:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glow-sign-rigid-plastic-exit-salida-bilingual>: HTTP status code is not handled or not allowed 2026-01-24 20:45:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glow-sign-rigid-plastic-5x14-w-door-right-up-arrow>: HTTP status code is not handled or not allowed 2026-01-24 20:45:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/csx3000-power-commercial-series-top-handle-chainsaw-w-battery-holster>: HTTP status code is not handled or not allowed 2026-01-24 20:45:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chisel-tip-permanent-marker-broad-black-60-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:45:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges landed on page that is not a product page. 2026-01-24 20:45:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glow-floor-sign-stop>: HTTP status code is not handled or not allowed 2026-01-24 20:45:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glow-floor-sign-emergency-eye-wash>: HTTP status code is not handled or not allowed 2026-01-24 20:45:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extended-reach-tire-inflator-gauge-10-to-160-psi-2ft-hose-517>: HTTP status code is not handled or not allowed 2026-01-24 20:45:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-six-drawer-narrow-procedure-cart-tall-height-mini-width-key-lock-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:45:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-six-drawer-anesthesia-cart-with-md30-ans3-accessory-package-teal>: HTTP status code is not handled or not allowed 2026-01-24 20:45:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-epoxy-wire-shelving-add-on-a14308b-30w-x-14d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 20:45:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-8-x-12-working-length-j-hook-style-c>: HTTP status code is not handled or not allowed 2026-01-24 20:45:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glow-sign-rigid-plastic-fire-regulations>: HTTP status code is not handled or not allowed 2026-01-24 20:45:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/offshore-life-vest-uscg-type-i-reversible-orange-adult-universal-dx400rtj>: HTTP status code is not handled or not allowed 2026-01-24 20:45:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-easy-peel-labels-2-dia-glossy-white-120-pack-22807>: HTTP status code is not handled or not allowed 2026-01-24 20:45:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/repositionable-address-labels-for-inkjet-printers-1-x-2-5-8-white-750-box-58160>: HTTP status code is not handled or not allowed 2026-01-24 20:45:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-32-k-lock-nut-06NK>: HTTP status code is not handled or not allowed 2026-01-24 20:45:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/odorstop-72-watt-uv-air-treatment-system-with-airflow-sensor-and-16-bulbs>: HTTP status code is not handled or not allowed 2026-01-24 20:45:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-six-drawer-narrow-procedure-cart-tall-height-mini-width-key-lock-hammertone-black>: HTTP status code is not handled or not allowed 2026-01-24 20:45:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-16x5-toggle-anchors-1080TBCR>: HTTP status code is not handled or not allowed 2026-01-24 20:45:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mini-m-series-five-drawer-anesthesia-cart-w-standard-key-lock-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:45:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-12-plastic-anchors-12PAPK>: HTTP status code is not handled or not allowed 2026-01-24 20:45:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/small-plastic-compartment-box-13-compartments>: HTTP status code is not handled or not allowed 2026-01-24 20:45:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-lift-beam-yellow-28-l-x-25-w-x-12-h-225lbs-40000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:45:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forklift-fork-extensions-60-6w-x-60l-pair>: HTTP status code is not handled or not allowed 2026-01-24 20:45:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hivis-crew-neck-sweatshirt-ansi-class-3-polar-fleece-limeblack-3xl>: HTTP status code is not handled or not allowed 2026-01-24 20:45:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6008nr-open-wsnap-ring-40mm-bore-68mm-od>: HTTP status code is not handled or not allowed 2026-01-24 20:45:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-foot-pads-lock-n-stock-ladders-4-pack-pt-hds>: HTTP status code is not handled or not allowed 2026-01-24 20:45:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-series-aluminium-isolation-cart-6-drawers-30-w-x-22-l-x-43-3-4-h-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:45:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-heavy-duty-whiteboard-cleaner-17-oz-bottle-6-pack-pl9868-6>: HTTP status code is not handled or not allowed 2026-01-24 20:45:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6304zznr-double-shielded-wsnap-ring-20mm-bore-52mm-od>: HTTP status code is not handled or not allowed 2026-01-24 20:45:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6202-2rsnr-double-sealed-wsnap-ring-15mm-bore-35mm-od>: HTTP status code is not handled or not allowed 2026-01-24 20:45:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-72-economy-h-beam-adjustable-length-yellow-126-l-x-40-w-x-36-h-765lbs-8000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:45:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/champion-fmps2712rc-fmpro-stainless-steel-27-wx20-dx66-1-8-h-12-drawer-chest-roller-cabinet-combo>: HTTP status code is not handled or not allowed 2026-01-24 20:45:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/floor-cleaner-citrus-scent-1-gallon-bottle-540458>: HTTP status code is not handled or not allowed 2026-01-24 20:45:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-waterproof-reversible-raincoat-ansi-class-3-300d-oxfordpu-coating-limeblack-xl>: HTTP status code is not handled or not allowed 2026-01-24 20:45:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/first-aid-box-polystyrene-9-3-16x2-3-4x6-1-2>: HTTP status code is not handled or not allowed 2026-01-24 20:45:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/champion-fmp5411rc-rd-fmpro-54-w-x-20-d-x-42-1-2-h-11-drawer-red-roller-cabinet>: HTTP status code is not handled or not allowed 2026-01-24 20:45:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/danby-compact-refrigerator-4-4-cu-ft-dcr044b1slm>: HTTP status code is not handled or not allowed 2026-01-24 20:45:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-column-medical-storage-cabinet-key-lock-glass-doors-42-wx29-3-5-dx76-h-forest-green>: HTTP status code is not handled or not allowed 2026-01-24 20:45:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-visibility-reversible-rain-cap-300d-oxfordpu-coating-orangeblack-one-size>: HTTP status code is not handled or not allowed 2026-01-24 20:45:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6305nr-open-wsnap-ring-25mm-bore-62mm-od>: HTTP status code is not handled or not allowed 2026-01-24 20:45:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-8-dia-x-10-working-length-j-hook-style-a>: HTTP status code is not handled or not allowed 2026-01-24 20:45:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-grid-panel-poly-green-24-x-60>: HTTP status code is not handled or not allowed 2026-01-24 20:45:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/champion-fmp2705tc-bk-fmpro-27-w-x-20-d-x-23-5-8-h-5-drawer-black-chest>: HTTP status code is not handled or not allowed 2026-01-24 20:45:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-twin-hoist-lift-beam-yellow-20000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:45:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/suredry-14-scope-drying-cabinet-w-e-lock-36-w-x-24-d-x-93-h-sand>: HTTP status code is not handled or not allowed 2026-01-24 20:45:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6311zz-double-shielded-55mm-bore-120mm-od>: HTTP status code is not handled or not allowed 2026-01-24 20:45:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/champion-fmp2707rc-rd-fmpro-27-w-x-20-d-x-42-1-2-h-7-drawer-red-roller-cabinet>: HTTP status code is not handled or not allowed 2026-01-24 20:45:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dyson-airblade-153-wash-dry-hand-dryer-wall-110-127v-247669-01>: HTTP status code is not handled or not allowed 2026-01-24 20:45:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/axiom-twin-airpot-coffee-brewer-axiom-twin-aps-1-pwr-conn-sf>: HTTP status code is not handled or not allowed 2026-01-24 20:45:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6013zz-double-shielded-65mm-bore-100mm-od>: HTTP status code is not handled or not allowed 2026-01-24 20:45:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dino-lite-software-images-capture-pedal>: HTTP status code is not handled or not allowed 2026-01-24 20:45:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-spread-basket-lifting-beam-yellow-10000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:45:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/weight-driven-grinder-g9wd-rh-120v>: HTTP status code is not handled or not allowed 2026-01-24 20:45:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brewwise-dual-thermofresh-dbc-brewer-120208v>: HTTP status code is not handled or not allowed 2026-01-24 20:45:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-48-economy-h-beam-adjustable-length-yellow-78-l-x-40-w-x-36-h-170lbs-2000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:45:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dot-placard-non-flammable-gas-2-blank-1>: HTTP status code is not handled or not allowed 2026-01-24 20:45:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-data-and-media-safe-dt200e-electronic-lock-29-1-4w-x-27d-x-87-3-4h-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:45:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/depository-safe-rh2715il-electronic-lock-19-1-2w-x-15d-x-27h-black>: HTTP status code is not handled or not allowed 2026-01-24 20:45:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-voltage-370-440-roundrun-capacitor-175-mfd-175dvr>: HTTP status code is not handled or not allowed 2026-01-24 20:45:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-voltage-370-440-roundrun-capacitor-5-mfd-5dvr>: HTTP status code is not handled or not allowed 2026-01-24 20:45:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-ring-label-hold-binder-1capacity-11x8-1-2-black>: HTTP status code is not handled or not allowed 2026-01-24 20:46:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-dia-x-7-working-length-j-hook-style-a>: HTTP status code is not handled or not allowed 2026-01-24 20:46:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dot-placard-slow-moving-vehicle-sign-1>: HTTP status code is not handled or not allowed 2026-01-24 20:46:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/parking-permit-blue-windshield-001-100>: HTTP status code is not handled or not allowed 2026-01-24 20:46:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/330-volt-start-capacitor-189-227-mfd-189d>: HTTP status code is not handled or not allowed 2026-01-24 20:46:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-twin-hoist-lift-beam-yellow-6000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:46:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-feet-x-8-feet-fire-retardant-vinyl-nylon-drain-tarp>: HTTP status code is not handled or not allowed 2026-01-24 20:46:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-feet-x-3-feet-blk-slvr-poly-drain-tarp>: HTTP status code is not handled or not allowed 2026-01-24 20:46:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bronchoscope-drying-cabinet-wall-mount-key-lock-24-w-x-12-3-4-d-x-48-h-teal>: HTTP status code is not handled or not allowed 2026-01-24 20:46:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dot-placard-1863-3-2>: HTTP status code is not handled or not allowed 2026-01-24 20:46:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-tall-isolation-cart-4-drawers-key-lock-18-w-x-18-l-x-40-3-4-h-pink>: HTTP status code is not handled or not allowed 2026-01-24 20:46:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 20809 pages (at 100 pages/min), scraped 12458 items (at 24 items/min) 2026-01-24 20:46:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nestable-tote-20-x-14-x-5-yellow-qty-100-plus>: HTTP status code is not handled or not allowed 2026-01-24 20:46:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-dia-x-72-l-roll-lifting-beam-yellow-18-l-x-40-w-x-78-h-245lbs-6000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:46:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-short-isolation-cart-5-drawers-e-lock-30-w-x-22-l-x-37-1-4-h-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:46:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-short-isolation-cart-5-drawers-e-lock-30-w-x-22-l-x-37-1-4-h-taupe>: HTTP status code is not handled or not allowed 2026-01-24 20:46:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swf20guu-114-id-round-flange-type-linear-bearing-wresin-retainer-seals-steel>: HTTP status code is not handled or not allowed 2026-01-24 20:46:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arsenal-5216-responder-gear-bag-blue-13447>: HTTP status code is not handled or not allowed 2026-01-24 20:46:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dot-placard-spontaneously-combustible-4-2>: HTTP status code is not handled or not allowed 2026-01-24 20:46:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swf8gwuu-12-id-round-flange-doublewide-type-linear-bearing-wresin-retainer-seals>: HTTP status code is not handled or not allowed 2026-01-24 20:46:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swf10wuu-58-id-round-flange-doublewide-type-linear-bearing-wseals-steel>: HTTP status code is not handled or not allowed 2026-01-24 20:46:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-column-medical-storage-cart-glass-door-22-1-2-w-x-27-d-x-75-3-4-h-navy>: HTTP status code is not handled or not allowed 2026-01-24 20:46:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/schoolmate-personal-monostereo-hdph-in-line-vol-controls>: HTTP status code is not handled or not allowed 2026-01-24 20:46:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6811zz-double-shielded-55mm-bore-72mm-od>: HTTP status code is not handled or not allowed 2026-01-24 20:46:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-headroom-hoist-trolley-combo-geared-2000-lb-cap>: HTTP status code is not handled or not allowed 2026-01-24 20:46:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-x-14-working-length-j-hook-style-c>: HTTP status code is not handled or not allowed 2026-01-24 20:46:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6920-open-100mm-bore-140mm-od>: HTTP status code is not handled or not allowed 2026-01-24 20:46:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-96-economy-h-beam-adjustable-length-yellow-102-l-x-40-w-x-36-h-360lbs-4000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:46:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tall-savary-dilator-drying-cart-hepa-filter-23-89-100-wx22-lx51-89-100-h-hammertone-red>: HTTP status code is not handled or not allowed 2026-01-24 20:46:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-twin-hoist-lift-beam-yellow-4000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:46:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-hr-mechanical-timer-125vac-15a-grounded>: HTTP status code is not handled or not allowed 2026-01-24 20:46:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poly-z-brite-2-shelf-container-rack-bk18605z-60w-x-18d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 20:46:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t6s3rss-fan-shroud-type-3r-fits-6in-fan-ss-type-304>: HTTP status code is not handled or not allowed 2026-01-24 20:46:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbl482412g-lbox-lbox-wall-mount-cab-4800x2400x1200>: HTTP status code is not handled or not allowed 2026-01-24 20:46:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6920-2rs-double-sealed-100mm-bore-140mm-od>: HTTP status code is not handled or not allowed 2026-01-24 20:46:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poly-z-brite-2-shelf-container-rack-bk18245z-24w-x-18d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 20:46:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/overpack-label-standard-500-roll>: HTTP status code is not handled or not allowed 2026-01-24 20:46:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poly-z-brite-heavy-duty-wire-shelving-ds24607z-60w-x-24d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 20:46:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-516-38-bullet-piercing-valve-one-quarter-five-sixteenths-three-eighths>: HTTP status code is not handled or not allowed 2026-01-24 20:46:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-3-shelf-container-rack-bk18305n-30w-x-18d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 20:46:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bronchoscope-drying-cabinet-wall-mount-key-lock-24-w-x-12-3-4-d-x-48-h-hammertone-green>: HTTP status code is not handled or not allowed 2026-01-24 20:46:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chuck-guard-assembly-for-jet-power-tools>: HTTP status code is not handled or not allowed 2026-01-24 20:46:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flashing-light-led-120-240vac-hazardous-location-red-191xl-120-240r>: HTTP status code is not handled or not allowed 2026-01-24 20:46:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrosive-placard-un-3266-removable-vinyl-25-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:46:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silver-epoxy-wire-shelf-platform-p1836ep-36w-x-18d>: HTTP status code is not handled or not allowed 2026-01-24 20:46:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilco-rl-f-rail-system-for-48x48-roof-hatch>: HTTP status code is not handled or not allowed 2026-01-24 20:46:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/Global-Wand-Handle-Nozzle-Sprayer-Kit>: HTTP status code is not handled or not allowed 2026-01-24 20:46:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oil-transfer-pump-extension-tube-26-5-with-diameter-of-2-1-8-54mm>: HTTP status code is not handled or not allowed 2026-01-24 20:46:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-tall-procedure-cart-6-drawers-e-lock-36-3-4-w-x-22-l-x-43-3-4-h-beige>: HTTP status code is not handled or not allowed 2026-01-24 20:46:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/q1286pcd-qline-j-box-screw-cover-type-4x-120x80x55mm-polycarb>: HTTP status code is not handled or not allowed 2026-01-24 20:46:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/boss-file-file-mobile-pedestal-16-x-22-x-29h-mahogany>: HTTP status code is not handled or not allowed 2026-01-24 20:46:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rip-fence-5782172>: HTTP status code is not handled or not allowed 2026-01-24 20:46:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-plastic-bar-12-thick-x-3w-x-24l>: HTTP status code is not handled or not allowed 2026-01-24 20:46:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-2-drylin-r-all-polymer-linear-bearing>: HTTP status code is not handled or not allowed 2026-01-24 20:46:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/social_distancing_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:46:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/boss-42-x-24-reversible-return-mahogany>: HTTP status code is not handled or not allowed 2026-01-24 20:46:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-id-double-flanged-sheet-metal-clip-bearing-m250-for-040-075-sheet-metal>: HTTP status code is not handled or not allowed 2026-01-24 20:46:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enpac-hdpe-spill-containment-cover-for-double-ibc-4000i-115l-x-75w-x-95h>: HTTP status code is not handled or not allowed 2026-01-24 20:46:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50ml-suspension-culture-flask-vent-cap-sterile-200-pk>: HTTP status code is not handled or not allowed 2026-01-24 20:46:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cell-scraper-39cm-abs-handle-30mm-scraper-blade-sterile-green-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 20:46:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fda-approved-yellow-x-large-tote-47-1-4l-x-51-1-2w-x-33h>: HTTP status code is not handled or not allowed 2026-01-24 20:46:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triple-column-storage-cabinet-electronic-keypad-lock-60-w-x-28-d-x-81-h-teal>: HTTP status code is not handled or not allowed 2026-01-24 20:46:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-short-emergency-crash-cart-md30-emg1-std-width-4-drawers-breakaway-lock-hammertone-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:46:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1161421-comfort-vest-style-harness-tongue-buckle-quick-connect-m-l>: HTTP status code is not handled or not allowed 2026-01-24 20:46:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1161404-comfort-vest-style-climbing-harness-quick-connect-s>: HTTP status code is not handled or not allowed 2026-01-24 20:46:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1161416-comfort-vest-style-positioning-harness-tongue-buckle-pass-through-2xl>: HTTP status code is not handled or not allowed 2026-01-24 20:46:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60mm-x-15mm-petri-dish-w-grip-ring-sterile-clear-polystyrene-500-pk>: HTTP status code is not handled or not allowed 2026-01-24 20:46:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/social_distancing_signs>: HTTP status code is not handled or not allowed 2026-01-24 20:46:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/515-tank-heavy-duty-shipping-case-tube-without-wheels-17l-x-17w-x-40h-blk>: HTTP status code is not handled or not allowed 2026-01-24 20:46:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/515-tank-heavy-duty-shipping-case-tube-with-wheels-17l-x-17w-x-77h-blk>: HTTP status code is not handled or not allowed 2026-01-24 20:46:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/exam_patient_room_supplies/exam_room_consumables/underpads_disposable_liners already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:46:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1161315-construction-style-positioning-harness-tongue-buckle-quick-connect-s>: HTTP status code is not handled or not allowed 2026-01-24 20:46:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s100x8-female-buttress-x-s60x6-2-male-buttress-thread>: HTTP status code is not handled or not allowed 2026-01-24 20:46:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-16-carbon-steel-insert-for-metal-329-601>: HTTP status code is not handled or not allowed 2026-01-24 20:46:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/250ml-suspension-culture-flask-vent-cap-sterile-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 20:46:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-16-20-carbon-steel-insert-for-metal-329-720>: HTTP status code is not handled or not allowed 2026-01-24 20:46:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/515-tank-heavy-duty-shipping-case-tube-without-wheels-10l-x-10w-x-44h-blk>: HTTP status code is not handled or not allowed 2026-01-24 20:46:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0-5ml-screw-top-micro-tube-cap-self-standing-grip-band-green-grip-cap-sterile-500pk>: HTTP status code is not handled or not allowed 2026-01-24 20:46:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/floor-sign-danger-restricted-area-construction-area>: HTTP status code is not handled or not allowed 2026-01-24 20:46:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-20-thin-wall-insert-for-metal-319-4>: HTTP status code is not handled or not allowed 2026-01-24 20:46:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ps-plasteur-174-pasteur-pipet-5-3-4-l-individually-wrapped-sterile-200-pk>: HTTP status code is not handled or not allowed 2026-01-24 20:47:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rb2034-general-purpose-1-3-hp-1725-rpm-115v-odp>: HTTP status code is not handled or not allowed 2026-01-24 20:47:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-height-triple-column-medical-storage-cabinet-60-2-5-wx27-4-5-dx55-3-5-h-mauve>: HTTP status code is not handled or not allowed 2026-01-24 20:47:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roof-mount-led-warning-light-48-low-profile-design-clear-lens-blue-leds-575b-48>: HTTP status code is not handled or not allowed 2026-01-24 20:47:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labor-law-poster-maryland>: HTTP status code is not handled or not allowed 2026-01-24 20:47:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/exam_patient_room_supplies/exam_room_consumables/underpads_disposable_liners>: HTTP status code is not handled or not allowed 2026-01-24 20:47:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powerstud-wedge-expansion-anchor-sd1-34-x-812>: HTTP status code is not handled or not allowed 2026-01-24 20:47:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-profile-24-light-bar-magnet-or-permanent-mount-clear-lens-blue-leds-7826mp-b>: HTTP status code is not handled or not allowed 2026-01-24 20:47:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/basic-small-desk-24w-x-36d-ash>: HTTP status code is not handled or not allowed 2026-01-24 20:47:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rad-usb-outlet-white>: HTTP status code is not handled or not allowed 2026-01-24 20:47:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ns-soft-splinting-material-18-x-24-x-1-16-non-perforated>: HTTP status code is not handled or not allowed 2026-01-24 20:47:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-56-flanged-press-insert-stainless-260-002-cr>: HTTP status code is not handled or not allowed 2026-01-24 20:47:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hazardous-location-blower-hadp8-12-hp-1-ph-explosion-proof-cw-top-horizontal>: HTTP status code is not handled or not allowed 2026-01-24 20:47:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-vaccine-refrigerator-nsf-certified-23-cf>: HTTP status code is not handled or not allowed 2026-01-24 20:47:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-32-straight-ultrasonic-insert-flanged-fl-132-hi>: HTTP status code is not handled or not allowed 2026-01-24 20:47:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/urethane-cutting-edge>: HTTP status code is not handled or not allowed 2026-01-24 20:47:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 20908 pages (at 99 pages/min), scraped 12475 items (at 17 items/min) 2026-01-24 20:47:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/think-osha-10x14-plastic-be-alert-don-t-get-hurt>: HTTP status code is not handled or not allowed 2026-01-24 20:47:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-ga-double-tier-ventilated-locker-6-compartments-54-w-x-18-d-x-75-h-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:47:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/security-notice-aluminum-this-door-is-alarmed>: HTTP status code is not handled or not allowed 2026-01-24 20:47:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polly-products-econo-mizer-6-ft-backed-bench-cedar-bench-black-frame>: HTTP status code is not handled or not allowed 2026-01-24 20:47:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/emergency_vehicle_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:47:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m4-07-stainless-insert-for-metal-453-4>: HTTP status code is not handled or not allowed 2026-01-24 20:47:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a36h3008sslp3pt-continuous-hinge-w3point-ltch-type-4x-3600x3000x800in-ss-type-304>: HTTP status code is not handled or not allowed 2026-01-24 20:47:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylon-plastic-adjustable-lever-stainless-steel-38-16-126-stud-307l>: HTTP status code is not handled or not allowed 2026-01-24 20:47:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classic-soft-precut-gauntlet-thumb-post-splint-1-12-micro-perforated-medium>: HTTP status code is not handled or not allowed 2026-01-24 20:47:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-3-speed-direct-drive-exhaust-fan-with-pull-chain-246619>: HTTP status code is not handled or not allowed 2026-01-24 20:47:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machine-labels-danger-keep-guards-place>: HTTP status code is not handled or not allowed 2026-01-24 20:47:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbi-sala-lad-saf-35-ft-swaged-stainless-steel-cable-3-8-dia-1-7-strand-6105035>: HTTP status code is not handled or not allowed 2026-01-24 20:47:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cp323512-concepthmi-encl-wblack-trim-type-4-325x350x120mm>: HTTP status code is not handled or not allowed 2026-01-24 20:47:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:47:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/security-stop-sign-stop>: HTTP status code is not handled or not allowed 2026-01-24 20:47:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/security-stop-sign-stop-employees-only>: HTTP status code is not handled or not allowed 2026-01-24 20:47:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/emergency_vehicle_lights>: HTTP status code is not handled or not allowed 2026-01-24 20:47:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flexible-vinyl-corner-guard-2-1-2-inch-wings-3-16-inch-radius-4-foot-h-gray-heavy-duty-flexible-vinyl>: HTTP status code is not handled or not allowed 2026-01-24 20:47:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2000b-table-saw-3hp-1ph-230v-50-rip-w-accu-fence-router-lift>: HTTP status code is not handled or not allowed 2026-01-24 20:47:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/csd30308ss-conceptwall-mt-encl-type-4x-3000x3000x800in-ss-type-304>: HTTP status code is not handled or not allowed 2026-01-24 20:47:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xx-garolite-rod-58-diameter-x-2-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 20:47:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ac4m6st-caster-kit-stationary2-casterskit-3dia>: HTTP status code is not handled or not allowed 2026-01-24 20:47:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/junior-2-in-1-convertible-hand-truck-pneumatic-wheels-987076>: HTTP status code is not handled or not allowed 2026-01-24 20:47:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-in-casters-with-brakes-for-nexel-merchandisers-refrigerators-freezers>: HTTP status code is not handled or not allowed 2026-01-24 20:47:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microwavable-container-combo-black-clear-7in-diameter-32-oz-150-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:47:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-wire-shelf-stem-caster-truck-poly-green-21w-x-60l-x-69h>: HTTP status code is not handled or not allowed 2026-01-24 20:47:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs>: HTTP status code is not handled or not allowed 2026-01-24 20:47:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-54-limit-switch-18-1-gear-ratio-w-2-contact-blocks>: HTTP status code is not handled or not allowed 2026-01-24 20:47:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-wire-shelf-stem-caster-truck-2-with-brakes-poly-green-21w-x-36l-x-69h>: HTTP status code is not handled or not allowed 2026-01-24 20:47:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pictorial-osha-sign-vinyl-caution-slippery-when-wet>: HTTP status code is not handled or not allowed 2026-01-24 20:47:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beam-seating-4-navy-seats>: HTTP status code is not handled or not allowed 2026-01-24 20:47:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/american-lock-solid-aluminum-rectangular-padlock-yellow-no-a1206ylw>: HTTP status code is not handled or not allowed 2026-01-24 20:47:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rtl12061m-bathroom-safety-swivel-seat-shower-stool-14-seat>: HTTP status code is not handled or not allowed 2026-01-24 20:47:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-uhmw-polyethylene-plastic-bar-18-thick-x-3-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 20:47:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shell-style-id-card-holder-vertical-horizontal-with-reel-clear-10-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:47:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/american-lock-solid-steel-blade-cylinder-padlock-keyed-alike-no-ah11ka>: HTTP status code is not handled or not allowed 2026-01-24 20:47:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-position-heavy-duty-channel-w-gussets-b9912>: HTTP status code is not handled or not allowed 2026-01-24 20:47:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-lab-coat-white-polycombed-cotton-tall-l>: HTTP status code is not handled or not allowed 2026-01-24 20:47:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/american-lock-non-rekeyable-solid-steel-padlock-no-a51>: HTTP status code is not handled or not allowed 2026-01-24 20:47:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pipe-grabs-for-steel-pipe-450-lb-capacity-4-pipe-outside-dia>: HTTP status code is not handled or not allowed 2026-01-24 20:47:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-2-ss-split-body-ansi-150-flanged-ball-valve-w-nema-4-115vac>: HTTP status code is not handled or not allowed 2026-01-24 20:47:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/american-lock-non-rekeyable-solid-aluminum-padlock-no-a12>: HTTP status code is not handled or not allowed 2026-01-24 20:47:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-gripper-front-lab-coat-light-blue-polycotton-l>: HTTP status code is not handled or not allowed 2026-01-24 20:47:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2in-tubular-shank-clevis-ball-mount-9-1-4in-long-1803060>: HTTP status code is not handled or not allowed 2026-01-24 20:47:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/restricted-area-plastic-all-visitors-must-get-pass-at-office>: HTTP status code is not handled or not allowed 2026-01-24 20:47:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/restricted-area-aluminum-authorized-personnel-only>: HTTP status code is not handled or not allowed 2026-01-24 20:47:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-flat-face-nipple-3-4-body-size-viton-seals>: HTTP status code is not handled or not allowed 2026-01-24 20:47:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-32-5-16-self-lock-hk-lat-kit>: HTTP status code is not handled or not allowed 2026-01-24 20:47:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-lab-coat-white-polycombed-cotton-regular-s>: HTTP status code is not handled or not allowed 2026-01-24 20:47:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/american-lock-solid-steel-rectangular-padlock-no-a5260>: HTTP status code is not handled or not allowed 2026-01-24 20:47:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gooseneck-hitch-plate-w-2-5-16in-hitch-ball-without-d-rings-g9542>: HTTP status code is not handled or not allowed 2026-01-24 20:47:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-gripper-front-lab-coat-light-blue-polycotton-3xl>: HTTP status code is not handled or not allowed 2026-01-24 20:47:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:47:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-button-front-lab-coat-white-polycombed-cotton-2xl>: HTTP status code is not handled or not allowed 2026-01-24 20:47:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-8-screw-pin-anchor-shackle>: HTTP status code is not handled or not allowed 2026-01-24 20:47:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xx-garolite-sheet-132-thick-x-48-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-24 20:47:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pipe-tongs-2000-lbs-capacity-yellow-steel-adjustable-diameter-15-20-2-28-opening>: HTTP status code is not handled or not allowed 2026-01-24 20:47:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-shelving-60x18x72-2>: HTTP status code is not handled or not allowed 2026-01-24 20:47:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-bar-14-thick-x-2-12-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-24 20:47:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-shelving-60x24x96-2>: HTTP status code is not handled or not allowed 2026-01-24 20:47:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96-shelf-with-electrical-blue-2>: HTTP status code is not handled or not allowed 2026-01-24 20:47:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dripless-caulk-guns-ergo-tech-ets-2000-10-oz-caulk-gun>: HTTP status code is not handled or not allowed 2026-01-24 20:47:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unistrut-1-58-flat-plate-fitting-p1063eg-1-hole-electro-galvanized-38>: HTTP status code is not handled or not allowed 2026-01-24 20:47:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/breakaway-switch-with-48-inch-prewired-cable-loop-5422010>: HTTP status code is not handled or not allowed 2026-01-24 20:47:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-light-8-flush-mount-small-alabaster-mushroom-old-bronze>: HTTP status code is not handled or not allowed 2026-01-24 20:47:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 20:47:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24x36x74-chrome-wire-shelving-with-21-24-d-hopper-bins-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:47:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/odp-w-rigid-base-200-hp-1800-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:47:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24x36x74-chrome-wire-shelving-with-13-24-d-hopper-bins-red>: HTTP status code is not handled or not allowed 2026-01-24 20:47:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shaft-for-mp-mpb-040-ac-pallet-trucks-hy-2302981>: HTTP status code is not handled or not allowed 2026-01-24 20:48:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-industrial-long-sleeve-work-shirt-gray-polycotton-regular-2xl>: HTTP status code is not handled or not allowed 2026-01-24 20:48:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drive-tire-hub-for-mpb-040-e-b827-mpw-045-e-b802-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-24 20:48:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-industrial-long-sleeve-work-shirt-navy-polycotton-regular-2xl>: HTTP status code is not handled or not allowed 2026-01-24 20:48:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8ft-x-4ft-wire-mesh-pallet-rack-guard>: HTTP status code is not handled or not allowed 2026-01-24 20:48:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caster-assembly-for-6hbw30-pallet-trucks-to-00590-00008-71>: HTTP status code is not handled or not allowed 2026-01-24 20:48:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unisex-concealed-snap-front-lab-coat-royal-blue-nomexaramid-3xl>: HTTP status code is not handled or not allowed 2026-01-24 20:48:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-powered-straddle-stacker-2200-lb-cap-62-lift-987104>: HTTP status code is not handled or not allowed 2026-01-24 20:48:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x36x74-chrome-wire-shelving-with-24-6-h-shelf-bins-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:48:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-precision-dial-caliper-534164>: HTTP status code is not handled or not allowed 2026-01-24 20:48:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-industrial-short-sleeve-work-shirt-gray-polycotton-tall-2xl>: HTTP status code is not handled or not allowed 2026-01-24 20:48:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-sheet-4-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 20:48:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14x36x74-chrome-wire-shelving-with-28-giant-stacking-bins-red>: HTTP status code is not handled or not allowed 2026-01-24 20:48:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b8-174-powercrown-153-premium-staples-1-4-6mm-10000-pack-20-case>: HTTP status code is not handled or not allowed 2026-01-24 20:48:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 21005 pages (at 97 pages/min), scraped 12491 items (at 16 items/min) 2026-01-24 20:48:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x36x74-chrome-wire-shelving-with-21-giant-stacking-bins-ivory>: HTTP status code is not handled or not allowed 2026-01-24 20:48:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adhesive-transfer-tape-1-2-wide-x-36yds>: HTTP status code is not handled or not allowed 2026-01-24 20:48:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-spill-tray-p2>: HTTP status code is not handled or not allowed 2026-01-24 20:48:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-drum-truck>: HTTP status code is not handled or not allowed 2026-01-24 20:48:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:48:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-built-in-combo-lock-ada-compliant-lift-handle-purple>: HTTP status code is not handled or not allowed 2026-01-24 20:48:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x36x74-chrome-wire-shelving-with-8-giant-stacking-bins-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:48:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ffar121ss-solid-door-pharmaceutical-refrigerator-stainless-steel-10-1-cu-ft>: HTTP status code is not handled or not allowed 2026-01-24 20:48:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-hard-top-p4-with-drain>: HTTP status code is not handled or not allowed 2026-01-24 20:48:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transparent-tape-3-4-inch-x-1000-inch-1-inch-core-clear-12-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:48:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-containment-berm-collapsible-wall-model-15-x-50>: HTTP status code is not handled or not allowed 2026-01-24 20:48:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-containment-berm-economy-model-6-x-6-x-1>: HTTP status code is not handled or not allowed 2026-01-24 20:48:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-built-in-combination-lock-blue-dial-left-hinged>: HTTP status code is not handled or not allowed 2026-01-24 20:48:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/massage-grip-3-6-diameter-black>: HTTP status code is not handled or not allowed 2026-01-24 20:48:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-containment-tray-with-grating-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:48:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-24 20:48:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-132-thick-x-6-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 20:48:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3m-e-a-r-express-pod-plugs-corded-assorted-colors>: HTTP status code is not handled or not allowed 2026-01-24 20:48:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bowl-fruit-10-9oz-slate-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:48:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tumbler-clrw-9-5oz-24qty-amber>: HTTP status code is not handled or not allowed 2026-01-24 20:48:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pivot-axle-for-ptw-series-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-24 20:48:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bolly-wall-mirror>: HTTP status code is not handled or not allowed 2026-01-24 20:48:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/push-rod-for-mp-mpb-040-ac-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-24 20:48:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microflex-174-supreno-174-ec-sec-375-nitrile-gloves-powder-free-size-xxl-100-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:48:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-assembly-for-b-60-80-z-pallet-trucks-hy-2047714>: HTTP status code is not handled or not allowed 2026-01-24 20:48:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tumbler-dm-24oz-clear>: HTTP status code is not handled or not allowed 2026-01-24 20:48:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pin-for-wp-2000-wp-2300-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-24 20:48:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pivot-axle-for-wp-2000-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-24 20:48:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/koch-filter-merv-8-high-capacity-xl8-pleated-extended-surface-16-inch-w-x-16-inch-h-x-2-inch-d>: HTTP status code is not handled or not allowed 2026-01-24 20:48:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-compartment-tray-white-pt152-10-1-2l-x-5-1-2w-x-2-1-2h>: HTTP status code is not handled or not allowed 2026-01-24 20:48:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mlk-chain-hoist-1100-lb-capacity-10-ft-lift>: HTTP status code is not handled or not allowed 2026-01-24 20:48:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-ed-dual-speed-electric-chain-hoist-250-lbs-10-2610>: HTTP status code is not handled or not allowed 2026-01-24 20:48:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slb-lever-hoists-13200-lb-capacity-15-ft-lift-25-316-head-room>: HTTP status code is not handled or not allowed 2026-01-24 20:48:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-332-thick-x-1-14-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 20:48:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-38-thick-x-38-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 20:48:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storage-tote-ivory-t181-4-16l-x-12w-x-4h>: HTTP status code is not handled or not allowed 2026-01-24 20:48:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shoreline-collection-pellet-underliner-fits-9-1-2-pellet-cranberry>: HTTP status code is not handled or not allowed 2026-01-24 20:48:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/barware-beer-mug-16oz-clear>: HTTP status code is not handled or not allowed 2026-01-24 20:48:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mini-lever-hoists-kx075-10-1650-lb-capacity-10-ft-lift>: HTTP status code is not handled or not allowed 2026-01-24 20:48:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-bank-roller-cabinet-w-14-drawers-42-w-x-22-3-8-d-x-38-1-2-h-tan>: HTTP status code is not handled or not allowed 2026-01-24 20:48:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storage-tote-clear-t183-14l-x-12w-x-4h>: HTTP status code is not handled or not allowed 2026-01-24 20:48:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roller-cabinet-w-6-drawers-34-w-x-25-1-4-d-x-41-h-red>: HTTP status code is not handled or not allowed 2026-01-24 20:48:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/axle-for-mpb045vg-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-24 20:48:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-plastic-bar-w-lse-acrylic-adhesive-12-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 20:48:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tot-mini-stapler-12-sheet-capacity-pink>: HTTP status code is not handled or not allowed 2026-01-24 20:48:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-bank-roller-cabinet-w-7-drawers-27-w-x-22-3-8-d-x-38-1-2-h-red>: HTTP status code is not handled or not allowed 2026-01-24 20:48:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/koch-filter-merv-8-high-capacity-xl8-pleated-extended-surface-18-inch-w-x-24-inch-h-x-4-inch-d>: HTTP status code is not handled or not allowed 2026-01-24 20:48:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-flat-polyurethane-coated-gloves-hi-viz-lime-large-1-pair>: HTTP status code is not handled or not allowed 2026-01-24 20:48:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bracket-for-installation-of-br-500-br-530-and-br-800-series-handrails-eggshell>: HTTP status code is not handled or not allowed 2026-01-24 20:48:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-sheet-2-thick-x-8-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 20:48:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:48:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/toolbox-with-tray-and-2-compartment-boxes-20-1-4-l-x-10-7-8-w-x-9-1-8-h-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:48:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/toolbox-with-tray-21-5-16-l-x-12-1-2-w-x-13-1-2-h-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:48:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/epdm-rubber-roll-no-adhesive-60a-14-thick-x-12-wide-x-10-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 20:48:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/screw-for-wave-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-24 20:48:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-plastic-sheet-w-lse-acrylic-adhesive-14-thick-x-24-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 20:48:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-5200-double-row-angular-contact-bearing-open-10mm-bore-x-30mm-od-x-14-3mm-w>: HTTP status code is not handled or not allowed 2026-01-24 20:48:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-5200zz-double-row-angular-contact-bearing-double-shielded-10mm-bore-x-30mm-od-x-14-3mm-w>: HTTP status code is not handled or not allowed 2026-01-24 20:48:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gripper-front-butcher-coat-wknit-cuffs-pocket-less-spun-polyester-tan-3xl>: HTTP status code is not handled or not allowed 2026-01-24 20:48:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/switch-micro-for-w-40-xl-w-40-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-24 20:48:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/graduated-specimen-container-4-oz-thermometer-strip-red-screwcap-polypropylene-500-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:48:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-button-front-chef-coat-knot-buttons-black-spun-polyester-xs>: HTTP status code is not handled or not allowed 2026-01-24 20:48:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/concrete-dual-drinking-fountain-ada-accessible-gray-limestone-2>: HTTP status code is not handled or not allowed 2026-01-24 20:48:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 20:48:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-5308zz-double-row-angular-contact-bearing-double-shielded-40mm-bore-x-90mm-od-x-36-5mm-w>: HTTP status code is not handled or not allowed 2026-01-24 20:48:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-5206-double-row-angular-contact-bearing-open-30mm-bore-x-62mm-od-x-23-8mm-w>: HTTP status code is not handled or not allowed 2026-01-24 20:49:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hog-heaven-iii-mod-tile-comfort-18-corner-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:49:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12x36x74-chrome-wire-shelving-with-55-4-h-shelf-bins-black>: HTTP status code is not handled or not allowed 2026-01-24 20:49:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cook-pants-white-polyestercotton-32-x-36>: HTTP status code is not handled or not allowed 2026-01-24 20:49:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gripper-front-butcher-coat-wknit-cuffs-pocket-less-spun-polyester-white-5xl>: HTTP status code is not handled or not allowed 2026-01-24 20:49:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gripper-front-butcher-frock-winside-top-pocket-white-polyestercotton-twill-l>: HTTP status code is not handled or not allowed 2026-01-24 20:49:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/bins_totes_containers/storage_totes_tubs/storage_totes_with_lids already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:49:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-34-thick-x-5-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 20:49:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ballymore-10-step-perforated-cantilever-ladder-35-overhang>: HTTP status code is not handled or not allowed 2026-01-24 20:49:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24x36x74-chrome-wire-shelving-with-55-4-h-shelf-bins-ivory>: HTTP status code is not handled or not allowed 2026-01-24 20:49:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rcl-200zw>: HTTP status code is not handled or not allowed 2026-01-24 20:49:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdpe-plastic-bar-1-thick-x-1-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 20:49:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 21099 pages (at 94 pages/min), scraped 12512 items (at 21 items/min) 2026-01-24 20:49:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pin-for-mpb-040-e-b827-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-24 20:49:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-for-mpb030-040-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-24 20:49:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24x36x74-chrome-wire-shelving-with-33-4-h-shelf-bins-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:49:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-w-lse-acrylic-adhesive-14-thick-x-4-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 20:49:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-drawer-wood-top-cast-vertical-storage-dresser-black-dark-gray-fabric-drawers>: HTTP status code is not handled or not allowed 2026-01-24 20:49:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-sheet-132-thick-x-8-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-24 20:49:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/utility-table-60-x-24-black>: HTTP status code is not handled or not allowed 2026-01-24 20:49:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stinger-berm-spill-containment-puncture-resistant-4-l-x-8-w-x-12-h>: HTTP status code is not handled or not allowed 2026-01-24 20:49:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/bins_totes_containers/storage_totes_tubs/storage_totes_with_lids landed on page that is not a product page. 2026-01-24 20:49:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-10-button-front-chef-coat-pearl-buttons-white-polyestercotton-2xl>: HTTP status code is not handled or not allowed 2026-01-24 20:49:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/axle-for-ewp-2-pallet-trucks-br-301766-000>: HTTP status code is not handled or not allowed 2026-01-24 20:49:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-gallon-spill-kit-aggressive>: HTTP status code is not handled or not allowed 2026-01-24 20:49:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-round-breakroom-table-black>: HTTP status code is not handled or not allowed 2026-01-24 20:49:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xx-garolite-sheet-1-thick-x-48-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-24 20:49:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-sheet-2-thick-x-48-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-24 20:49:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/collarless-butcher-wrap-royal-blue-polyestercombed-cotton-l>: HTTP status code is not handled or not allowed 2026-01-24 20:49:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/balancing-damper-radial-galvanized-steel-10-pk2>: HTTP status code is not handled or not allowed 2026-01-24 20:49:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/long-cook-shirt-white-plain-weave-spun-polyester-xl>: HTTP status code is not handled or not allowed 2026-01-24 20:49:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-sheet-1-thick-x-48-wide-x-96-long>: HTTP status code is not handled or not allowed 2026-01-24 20:49:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/collarless-butcher-wrap-wexterior-pockets-white-polyestercombed-cotton-l>: HTTP status code is not handled or not allowed 2026-01-24 20:49:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/desk_accessories/desk_organizers_letter_trays already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:49:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-men-s-35-hip-wader-black-steel-toe-w-cleated-outsole-pvc-size-7>: HTTP status code is not handled or not allowed 2026-01-24 20:49:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alligator-clip-c-clamp-w-5ft-7x19-insulated-stranded-flex-steel-cable>: HTTP status code is not handled or not allowed 2026-01-24 20:49:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-men-s-boot-16-goliath-black-plain-toe-w-power-lug-outsole-pvc-size-15>: HTTP status code is not handled or not allowed 2026-01-24 20:49:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bushing-for-mpb-040-e-b827-mpw-045-e-b802-pallet-trucks-yl-272015100>: HTTP status code is not handled or not allowed 2026-01-24 20:49:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xx-garolite-sheet-316-thick-x-12-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 20:49:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-8482-outdoor-bottle-filling-station-bi-level-fountain-w-water-filter-black>: HTTP status code is not handled or not allowed 2026-01-24 20:49:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stx422-electronic-portable-touchscreen-balance-420g-x-0-01g>: HTTP status code is not handled or not allowed 2026-01-24 20:49:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-magazine-24-brochure-wall-display-medium-oak>: HTTP status code is not handled or not allowed 2026-01-24 20:49:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/countertop-8-pocket-brochure-display-mahogany>: HTTP status code is not handled or not allowed 2026-01-24 20:49:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-kit-for-electric-pallet-jack-truck-gwk-eje120-lw-fits-jungheinrich-model-eje120>: HTTP status code is not handled or not allowed 2026-01-24 20:49:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-men-s-boot-16-goliath-black-steel-toe-w-power-lug-outsole-pvc-size-12>: HTTP status code is not handled or not allowed 2026-01-24 20:49:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-screw-for-7400-reach-pallet-trucks-all-load-wheels-except-6-x-4-closed-toe>: HTTP status code is not handled or not allowed 2026-01-24 20:49:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:49:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/basic_supplies/desk_accessories/desk_organizers_letter_trays>: HTTP status code is not handled or not allowed 2026-01-24 20:49:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/complete-wheel-kit-for-manual-pallet-jack-gwk-hpt28-ck-fits-doosan-model-hpt28>: HTTP status code is not handled or not allowed 2026-01-24 20:49:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-12-thick-x-2-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 20:49:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-kit-for-electric-pallet-jack-truck-gwk-st3000-lw-fits-crown-model-st-3000>: HTTP status code is not handled or not allowed 2026-01-24 20:49:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-332-thick-x-6-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 20:49:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-chemtex-green-bib-overall-plain-front-pvc-on-polyester-size-medium>: HTTP status code is not handled or not allowed 2026-01-24 20:49:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12x48x74-chrome-wire-shelving-with-91-shelf-bins-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:49:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-kit-for-electric-pallet-jack-truck-gwk-7hbw30-lw-xl-fits-toyota-model-7hbw30>: HTTP status code is not handled or not allowed 2026-01-24 20:49:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spun-polyester-long-cook-shirt-black-spun-polyester-xl>: HTTP status code is not handled or not allowed 2026-01-24 20:49:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-2-pvc-socket-x-1-1-2-cast-iron-pvc-coupling>: HTTP status code is not handled or not allowed 2026-01-24 20:49:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-plastic-bar-w-lse-acrylic-adhesive-12-thick-x-2-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 20:49:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-tuftex-yellow-bib-overall-plain-front-pvc-size-large>: HTTP status code is not handled or not allowed 2026-01-24 20:49:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/siesta-air-resin-counter-stool-dove-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:49:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-24 20:49:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-kit-for-electric-pallet-jack-truck-gwk-112-lw-xl-fits-raymond-model-112>: HTTP status code is not handled or not allowed 2026-01-24 20:49:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x60x74-chrome-wire-shelving-with-118-shelf-bins-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:49:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/screw-cap-for-sample-tubes-with-external-threads-s-6030-6059-lilac-1000-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:49:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-stacking-bin-4-1-8x7-3-8x3-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:49:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-stacking-bin-8-1-4x10-3-4x7-black>: HTTP status code is not handled or not allowed 2026-01-24 20:49:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-12-thick-x-6-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 20:49:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sample-tube-5ml-external-threads-pp-white-graduations-marking-area-rd-bottom-1000pk>: HTTP status code is not handled or not allowed 2026-01-24 20:49:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-strength-buna-n-rubber-sheet-no-adhesive-60a-1-thick-x-6-wide-x-6-long>: HTTP status code is not handled or not allowed 2026-01-24 20:49:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/female-upper-plastic-form-on-faux-crystal-base-white>: HTTP status code is not handled or not allowed 2026-01-24 20:49:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-stacking-bin-8-1-4x14-3-4x7-red>: HTTP status code is not handled or not allowed 2026-01-24 20:49:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyurethane-hppe-liner-a2-cut-premium-cut-resistant-gloves-size-large-pkg-qty-12>: HTTP status code is not handled or not allowed 2026-01-24 20:49:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-stacking-bin-16-1-2x14-3-4x7-black>: HTTP status code is not handled or not allowed 2026-01-24 20:49:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48w-x-36h-mobile-reversible-magnetic-whiteboard>: HTTP status code is not handled or not allowed 2026-01-24 20:49:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/msa-isobutylene-cylinder-100-liter-100-ppm-494450>: HTTP status code is not handled or not allowed 2026-01-24 20:49:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milwaukee-circular-saw-blade-6-12-24-carbide-teeth-1-48-40-4108>: HTTP status code is not handled or not allowed 2026-01-24 20:49:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ridgeline-cap-style-hard-hat-matte-black-graphite-pattern-4-point-ratchet-suspension>: HTTP status code is not handled or not allowed 2026-01-24 20:49:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-38-thick-x-38-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-24 20:49:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-strength-neoprene-rubber-strip-with-acrylic-adhesive-40a-14-thick-x-14-wide-x-10-long>: HTTP status code is not handled or not allowed 2026-01-24 20:49:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ridgeline-cap-style-hard-hat-slate-gray-4-point-ratchet-suspension>: HTTP status code is not handled or not allowed 2026-01-24 20:49:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/307cp-p-trap-metal-1-1-2-slip-joint-x-spigot-17-gauge-w-clean-out-chrome>: HTTP status code is not handled or not allowed 2026-01-24 20:49:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/import-diamond-5-12-needle-file-flat-pattern-grit-100-coarse>: HTTP status code is not handled or not allowed 2026-01-24 20:49:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-tuftex-yellow-hood-pvc-size-large>: HTTP status code is not handled or not allowed 2026-01-24 20:50:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brass-bristle-brush-curved-handle-3x1-1-2>: HTTP status code is not handled or not allowed 2026-01-24 20:50:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brass-bristle-brush-flat-2>: HTTP status code is not handled or not allowed 2026-01-24 20:50:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/screw-cap-for-sample-tubes-with-external-threads-s-6030-6059-yellow-1000-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:50:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-plastic-self-dumping-forkli-hopper-1-7-cu-yd-with-caster-base>: HTTP status code is not handled or not allowed 2026-01-24 20:50:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medical-record-drop-box-grey>: HTTP status code is not handled or not allowed 2026-01-24 20:50:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-2-12-thick-x-3-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-24 20:50:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dura-kitchen-interlock-rubber-kitchen-mat-58-thick-3-x-3-black>: HTTP status code is not handled or not allowed 2026-01-24 20:50:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4822-w-yorkshire-window-box-white-vinyl-5-gallon-capacity-24-x-10-x-12>: HTTP status code is not handled or not allowed 2026-01-24 20:50:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-washer-m16-steel-zinc-clear-class-8-8-din-125a-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 20:50:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stretch-wrap-film-5-1000-80-extended-core-handle>: HTTP status code is not handled or not allowed 2026-01-24 20:50:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-head-cap-screw-m8-x-1-25-x-40mm-steel-zinc-yellow-class-10-9-din-931-pkg-of-100-aar08040>: HTTP status code is not handled or not allowed 2026-01-24 20:50:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4221-olympia-climaplus-ceiling-panels-mineral-fiber-white-24-x-24>: HTTP status code is not handled or not allowed 2026-01-24 20:50:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 21194 pages (at 95 pages/min), scraped 12527 items (at 15 items/min) 2026-01-24 20:50:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-nut-m5-x-0-8-steel-zinc-yellow-class-10-9-din-934-pkg-of-100-att05>: HTTP status code is not handled or not allowed 2026-01-24 20:50:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-head-cap-screw-hhcs050094eg-1-2-13-x-1-796982>: HTTP status code is not handled or not allowed 2026-01-24 20:50:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/intubation-kit-case>: HTTP status code is not handled or not allowed 2026-01-24 20:50:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-bin-shelf-cart-w-phenolic-casters-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:50:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-sheet-2-thick-x-8-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-24 20:50:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/76575-eclipse-climaplus-ceiling-panels-mineral-fiber-white-24-x-24>: HTTP status code is not handled or not allowed 2026-01-24 20:50:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/americana-cap-with-accessory-slots-and-4-point-slide-lock-suspension-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:50:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stacking-rack-units-42-w-x-24-d-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:50:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-4-thick-x-4-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-24 20:50:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-plastic-sheet-12-thick-x-36-long-x-48-long>: HTTP status code is not handled or not allowed 2026-01-24 20:50:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-group-lock-box-latch-tight-portable-or-wall-mount-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:50:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-letter-legal-file-tote-34052-18l-x-14w-x-10-3-4h-black>: HTTP status code is not handled or not allowed 2026-01-24 20:50:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fb-pic3188dn-100-portable-color-sewer-drain-camera-100-cable-w-heavy-duty-waterproof-case>: HTTP status code is not handled or not allowed 2026-01-24 20:50:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-plastic-bar-w-lse-acrylic-adhesive-18-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 20:50:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/omega-ii-cap-with-accessory-slots-and-6-point-mega-ratchet-suspension-hi-viz-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:50:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oil-only-medium-weight-sorbent-pads-19-gallon-capacity-15-x-18-100-padsbag>: HTTP status code is not handled or not allowed 2026-01-24 20:50:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-plug-lockout-use-for-110-120v-prongs>: HTTP status code is not handled or not allowed 2026-01-24 20:50:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uhmw-polyethylene-plastic-bar-18-thick-x-12-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 20:50:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/composites/composite_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:50:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-tone-pullover-hooded-sweatshirt-ansi-class-3-lime-black-2x-lbpuhsw-c3-2xl>: HTTP status code is not handled or not allowed 2026-01-24 20:50:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-w-lse-acrylic-adhesive-18-thick-x-12-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 20:50:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-drawer-full-height-36-w-cabinet-black-2>: HTTP status code is not handled or not allowed 2026-01-24 20:50:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/46in-round-metal-mesh-picnic-table-green>: HTTP status code is not handled or not allowed 2026-01-24 20:50:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/omega-ii-full-brim-with-6-point-slide-lock-suspension-orange>: HTTP status code is not handled or not allowed 2026-01-24 20:50:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/diamond-culture-tube-cap-for-18mm-glass-culture-tubes-polypropylene-clear-1000-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:50:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buckets-for-use-with-gcc-mp-swing-rotor-4-each-for-4-x-100ml-tubes-1-tube-per-bucket-4-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:50:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/centrifuge-tube-16-x-100mm-12ml-polypropylene-2000-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:50:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anvil-g1190-1-12-blk-di-floor-flange>: HTTP status code is not handled or not allowed 2026-01-24 20:50:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/centrifuge-tube-15ml-polypropylene-molded-graduations-100-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:50:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/composites/composite_rod_stock>: HTTP status code is not handled or not allowed 2026-01-24 20:50:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-steel-toe-waterproof-boots-16-height-black-size-8-1-pair>: HTTP status code is not handled or not allowed 2026-01-24 20:50:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-drawer-counter-high-24-w-cabinet-black-4>: HTTP status code is not handled or not allowed 2026-01-24 20:50:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-116-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 20:50:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/matric-gurad-post-adjustable-corner-8-h-safety-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:50:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xx-garolite-sheet-12-thick-x-36-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-24 20:50:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-8482-24-down-rod-dc-industrial-ceiling-fan>: HTTP status code is not handled or not allowed 2026-01-24 20:50:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-keypad-see-thru-door-cabinet-56-ld-244-kp-60w-x-24d-x-78h>: HTTP status code is not handled or not allowed 2026-01-24 20:50:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e-1236vs-with-acu-rite-303-dro-with-collet-closer>: HTTP status code is not handled or not allowed 2026-01-24 20:50:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e15-plastic-face-shield-carrier-for-omega-ii-americana-and-liberty-caps-black>: HTTP status code is not handled or not allowed 2026-01-24 20:50:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/100g-3-point-hitch-sprayer-pto-6500c-pump-50-38-hose>: HTTP status code is not handled or not allowed 2026-01-24 20:50:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-drawer-bench-high-24-w-cabinet-light-gray-2>: HTTP status code is not handled or not allowed 2026-01-24 20:50:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/handheld-clamshell-tape-dispenser-2w>: HTTP status code is not handled or not allowed 2026-01-24 20:50:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/digital-pocket-thermometer>: HTTP status code is not handled or not allowed 2026-01-24 20:50:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transfer-pipet-50ml-large-bulb-graduated-to-1ml-145mm-sterile-pp-cellophane-pack-400-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:50:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mesh-task-chair-seat-slider-fabric-black>: HTTP status code is not handled or not allowed 2026-01-24 20:50:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=jtm-1254vs-with-newall-dp700-dro-x-y-powerfeeds-air-power-drawbar already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:50:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/centrifuge-mini-8-place-240v-50hz-blue-lid-7000-rpm-fixed-speed-eu-plug>: HTTP status code is not handled or not allowed 2026-01-24 20:50:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-drawer-full-height-36-w-cabinet-red-2>: HTTP status code is not handled or not allowed 2026-01-24 20:50:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/emergency_eyewash_stations_showers/portable_emergency_eyewash_stations already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:50:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-guard-w-rounded-top-bottom-edges-plastic-clip-retainer-system-7-3-4-h-x-12-l-woodlands>: HTTP status code is not handled or not allowed 2026-01-24 20:50:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-drawer-cabinet-pdc-12bk-12-drawers-6-1-4-w-x-10-1-2-d-x-10-1-4-h>: HTTP status code is not handled or not allowed 2026-01-24 20:50:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polly-products-traditional-4-ft-backed-bench-with-arms-green-bench-brown-frame>: HTTP status code is not handled or not allowed 2026-01-24 20:50:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-drawer-counter-high-30-w-cabinet-black-1>: HTTP status code is not handled or not allowed 2026-01-24 20:50:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drywall-door-prime-coat-with-screwdriver-latch-16-ga-dw12x24pc-sdl>: HTTP status code is not handled or not allowed 2026-01-24 20:50:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jtm-1254vs-with-acu-rite-200s-3x-q-dro-x-y-z-powerfeeds-air-power-drawbar>: HTTP status code is not handled or not allowed 2026-01-24 20:50:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=jtm-1254vs-with-newall-dp700-3x-k-dro-x-powerfeed-air-power-drawbar already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:50:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-drawer-full-height-48-w-cabinet-red-1>: HTTP status code is not handled or not allowed 2026-01-24 20:50:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-modular-drawer-cabinet-3-drawers-w-lock-w-o-dividers-30x27x29-1-2-black>: HTTP status code is not handled or not allowed 2026-01-24 20:50:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=jtm-1254vs-with-newall-dp700-dro-x-y-powerfeeds-air-power-drawbar>: HTTP status code is not handled or not allowed 2026-01-24 20:50:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/back-bar-cooler-2-section-59-78w-23-34d-34-14h-tbb-24gal-60>: HTTP status code is not handled or not allowed 2026-01-24 20:50:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/emergency_eyewash_stations_showers/portable_emergency_eyewash_stations>: HTTP status code is not handled or not allowed 2026-01-24 20:50:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/utility-vbelt-wrapped-fhp-4l250>: HTTP status code is not handled or not allowed 2026-01-24 20:50:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jmd-45gh-geared-head-square-column-mill-drill-with-newall-dp700-2-axis-dro>: HTTP status code is not handled or not allowed 2026-01-24 20:50:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jtm-1254vs-with-newall-dp700-3x-q-dro-x-y-powerfeeds>: HTTP status code is not handled or not allowed 2026-01-24 20:51:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bottle-cooler-flat-top-24-34w-26-58d-33-38h-td-24-7>: HTTP status code is not handled or not allowed 2026-01-24 20:51:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chemical-waterproof-uv-resistent-labels-laser-4-34-x-7-34100box>: HTTP status code is not handled or not allowed 2026-01-24 20:51:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=jtm-1254vs-with-newall-dp700-3x-k-dro-x-powerfeed-air-power-drawbar>: HTTP status code is not handled or not allowed 2026-01-24 20:51:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durable-view-binder-with-slant-rings-1-capacity-coral>: HTTP status code is not handled or not allowed 2026-01-24 20:51:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jtm-1254vs-with-newall-dp700-3x-k-dro-x-y-z-powerfeeds>: HTTP status code is not handled or not allowed 2026-01-24 20:51:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-aquamax-plastic-locker-double-tier-1-wide-12w-x-18d-x-36h-taupe-body-blue-doors>: HTTP status code is not handled or not allowed 2026-01-24 20:51:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-swivel-ring-dsr-u-078>: HTTP status code is not handled or not allowed 2026-01-24 20:51:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-camwear-spoon-white>: HTTP status code is not handled or not allowed 2026-01-24 20:51:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/freezer-merch-3-section-78-18w-29-78d-79-38h-gdm-72f>: HTTP status code is not handled or not allowed 2026-01-24 20:51:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11-camwear-perforated-spoon-beige>: HTTP status code is not handled or not allowed 2026-01-24 20:51:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 21286 pages (at 92 pages/min), scraped 12549 items (at 22 items/min) 2026-01-24 20:51:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-carpet-extraction-cleaner-concentrate-case-of-four-1-gallon-bottles>: HTTP status code is not handled or not allowed 2026-01-24 20:51:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-ice-melt-residue-cleaner-case-of-four-1-gallon-bottles>: HTTP status code is not handled or not allowed 2026-01-24 20:51:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-milk-cooler-8-crts-34w-33-38d-41-12h-tmc-34-s>: HTTP status code is not handled or not allowed 2026-01-24 20:51:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tilt-truck-light-duty-1-cu-yd-rmttld100>: HTTP status code is not handled or not allowed 2026-01-24 20:51:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-507-1405-series-507-screw-to-connect-coupling-1-2-body-size-1-2-female-npt>: HTTP status code is not handled or not allowed 2026-01-24 20:51:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-welded-gear-locker-with-door-foot-locker-legs-24x24x72-red>: HTTP status code is not handled or not allowed 2026-01-24 20:51:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tube-false-bottom-16-x-97mm-5ml-with-threads-polypropylene-natural-1500-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:51:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/etm-949-mill-with-2-axis-acu-rite-203-dro-and-x-axis-jet-powerfeed>: HTTP status code is not handled or not allowed 2026-01-24 20:51:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-8-inch-equal-barbed-y-connector-gray-kynar>: HTTP status code is not handled or not allowed 2026-01-24 20:51:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/advantage-full-brim-hard-hat-vented-4-pt-ratchet-suspension-white>: HTTP status code is not handled or not allowed 2026-01-24 20:51:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-8-inch-to-3-4-inch-barbed-reduction-y-connector-natural-kynar>: HTTP status code is not handled or not allowed 2026-01-24 20:51:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-x-30-mobile-plastic-square-edge-work-bench-adjustable-height-1-5-8-top>: HTTP status code is not handled or not allowed 2026-01-24 20:51:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/telescoping-inner-boxes-6-x-6-x-72>: HTTP status code is not handled or not allowed 2026-01-24 20:51:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slide-storage-cabinet-6-drawers-holds-up-to-4500-slides-metal-green>: HTTP status code is not handled or not allowed 2026-01-24 20:51:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/advantage-front-brim-hard-hat-non-vented-4-pt-ratchet-suspension-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:51:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/summit-counter-height-laboratory-freezer-capable-of-35-c-31-f-operation>: HTTP status code is not handled or not allowed 2026-01-24 20:51:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rack-for-05ml-microcentrifuge-tubes-stackable-polycarbonate-pc-24-place-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:51:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-ratcheting-headgear-face-shield-with-clear-tint-anti-fog-quad500-series>: HTTP status code is not handled or not allowed 2026-01-24 20:51:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-safety-ear-muffs-noise-reducing-dielectric-29db-nrr-black>: HTTP status code is not handled or not allowed 2026-01-24 20:51:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-nesting-dunnage-rack-36x24x12>: HTTP status code is not handled or not allowed 2026-01-24 20:51:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evs-949-mill-with-2-axis-acu-rite-203-dro>: HTTP status code is not handled or not allowed 2026-01-24 20:51:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bowl-pebble-camwear-round-12-pebbled>: HTTP status code is not handled or not allowed 2026-01-24 20:51:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-dot-knit-gloves-double-sided-black-small-1-dozen>: HTTP status code is not handled or not allowed 2026-01-24 20:51:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slide-draining-tray-100-place-for-up-to-200-slides-abs-green-12-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:51:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blockhead-fg-series-ratcheting-headgear>: HTTP status code is not handled or not allowed 2026-01-24 20:51:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stackbin-4000-series-hardboard-over-stainless-72-w-x-30-d-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:51:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/market-tray-18x26-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:51:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stackbin-3512-series-esd-w-t-molding-48-w-x-36-d-black>: HTTP status code is not handled or not allowed 2026-01-24 20:51:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/turquoise-tissue-paper-20-x-30>: HTTP status code is not handled or not allowed 2026-01-24 20:51:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stackbin-3512-series-esd-square-edge-60-w-x-30-d-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:51:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1845-sa-9-1-x-1-4-double-cut-cylindrical-bur-w-o-end-cut>: HTTP status code is not handled or not allowed 2026-01-24 20:51:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-x-36-mobile-phenolic-resin-safety-lab-bench-tan>: HTTP status code is not handled or not allowed 2026-01-24 20:51:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-step-stock-picking-ladder-perforated-tread-spl-10-p>: HTTP status code is not handled or not allowed 2026-01-24 20:51:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-step-stock-picking-ladder-expanded-tread-spl-10-x>: HTTP status code is not handled or not allowed 2026-01-24 20:51:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-ton-shop-press-with-plc-and-portable-power-unit-3-phase-230-volt>: HTTP status code is not handled or not allowed 2026-01-24 20:51:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/55-ton-ironworker-3-phase-230-volt-10-inch-brake>: HTTP status code is not handled or not allowed 2026-01-24 20:51:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mailing-tubes-with-caps-2-x-72>: HTTP status code is not handled or not allowed 2026-01-24 20:51:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-ton-shop-press-and-portable-power-unit-3-phase-460-volt>: HTTP status code is not handled or not allowed 2026-01-24 20:51:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camrack-base-rack-7-1-2-inch-inside-stack-height-teal-nsf>: HTTP status code is not handled or not allowed 2026-01-24 20:51:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/justrite-smokers-cease-fire-cigarette-butt-receptacle-1-gallon-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:51:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-sided-cantilever-144-inch-upright-48-inch-brace-kit-starter>: HTTP status code is not handled or not allowed 2026-01-24 20:51:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-totally-enclosed-high-pressure-exhaust-fan-1-phase-3-4-hp-1>: HTTP status code is not handled or not allowed 2026-01-24 20:51:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wireless-tube-rack-30mm-full-size-24-place-nylon-white>: HTTP status code is not handled or not allowed 2026-01-24 20:51:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/berries-plastic-chair-with-powder-coated-legs-12-ht-camel>: HTTP status code is not handled or not allowed 2026-01-24 20:51:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-explosion-proof-high-pressure-exhaust-fan-3-phase-1-hp>: HTTP status code is not handled or not allowed 2026-01-24 20:51:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/media-flap-parflp007200-for-portacool-hurricane-360>: HTTP status code is not handled or not allowed 2026-01-24 20:51:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securall-75g-sliding-door-vertical-flammable-drum-cabinet-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:51:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2583-22-1-4-hex-impact-driver-kit>: HTTP status code is not handled or not allowed 2026-01-24 20:51:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1847-sc-7-19-05mm-x-25-4mm-cylindrical-ball-nose-bur>: HTTP status code is not handled or not allowed 2026-01-24 20:51:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kraft-literature-mailer-12-1-8-x-9-1-4-x-3>: HTTP status code is not handled or not allowed 2026-01-24 20:51:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securall-extra-shelf-for-cabinets-43-wide-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:51:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wireless-tube-rack-13mm-half-size-36-place-nylon-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:51:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beaker-polypropylene-printed-graduations-100ml-12-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:51:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss11311-round-softclose-seat-colonial-white>: HTTP status code is not handled or not allowed 2026-01-24 20:51:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-tubes-4-3-4-x-60-120>: HTTP status code is not handled or not allowed 2026-01-24 20:51:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-step-aluminum-rolling-ladder-16w-ribbed-tread-14d-top-step>: HTTP status code is not handled or not allowed 2026-01-24 20:51:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1846-sb-1-1-4-x-1-4-standard-cut-cylindrical-bur-w-end-cut>: HTTP status code is not handled or not allowed 2026-01-24 20:51:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poly-clipboard-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:51:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/under-desk-under-counter-push-button>: HTTP status code is not handled or not allowed 2026-01-24 20:51:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-x-40-roll-ultimat-rubber-gray-ur-2440g>: HTTP status code is not handled or not allowed 2026-01-24 20:51:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beaker-pmp-tpx-molded-graduations-250ml>: HTTP status code is not handled or not allowed 2026-01-24 20:51:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-steel-dock-board-with-lock-pins-60in-w-x-60in-l-20-000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:51:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-safety-fan-guard-42-duct-fans>: HTTP status code is not handled or not allowed 2026-01-24 20:51:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pipette-stand-4-place-for-diamond-pipettes-acrylic>: HTTP status code is not handled or not allowed 2026-01-24 20:52:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unsupported-8-step-40-cantilever-ladder-perforated>: HTTP status code is not handled or not allowed 2026-01-24 20:52:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-x-40-roll-ultimat-ii-light-blue-ur-3040xlb>: HTTP status code is not handled or not allowed 2026-01-24 20:52:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/194xl050-standard-timing-belt-xl-1-2-x-19-3-8-t97-trapezoidal>: HTTP status code is not handled or not allowed 2026-01-24 20:52:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cimi-shield-protect-beg-bug-prevention-gallon-bottle-pc-bb20>: HTTP status code is not handled or not allowed 2026-01-24 20:52:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1854-sh-5l6-1-2-x-1-4-standard-cut-flame-shaped-bur>: HTTP status code is not handled or not allowed 2026-01-24 20:52:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-molded-ring-binder-top-open-lilac>: HTTP status code is not handled or not allowed 2026-01-24 20:52:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/furniture_parts_accessories/computer_stand_accessories/medical_computer_cart_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:52:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unex-gravity-flow-roller-rack-with-span-track-starter-48w-x-72d-x-84h-with-4-levels>: HTTP status code is not handled or not allowed 2026-01-24 20:52:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-black-stealthnet-standard-duty-bird-netting-25-x-75-n1-b130>: HTTP status code is not handled or not allowed 2026-01-24 20:52:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jewel-magsnap-magground-coil-cord-adapter>: HTTP status code is not handled or not allowed 2026-01-24 20:52:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machinery-skates-adjustable-8-rollers-nylon>: HTTP status code is not handled or not allowed 2026-01-24 20:52:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beaker-polypropylene-molded-graduations-500ml>: HTTP status code is not handled or not allowed 2026-01-24 20:52:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-vacuum-cleaner-bags-for-powr-flite-bp10p>: HTTP status code is not handled or not allowed 2026-01-24 20:52:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-vacuum-cleaner-bags-for-windsor-chariot-2-atv-ivac-24>: HTTP status code is not handled or not allowed 2026-01-24 20:52:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-tile-grout-stain-remover-case-of-four-1-gallon-bottles>: HTTP status code is not handled or not allowed 2026-01-24 20:52:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 21386 pages (at 100 pages/min), scraped 12571 items (at 22 items/min) 2026-01-24 20:52:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-16-18-x-1-3-4-machine-screw-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 20:52:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-step-cross-over-ladder-103-1-2-l>: HTTP status code is not handled or not allowed 2026-01-24 20:52:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-5kw-240v-flowco-electric-tankless-water-heater>: HTTP status code is not handled or not allowed 2026-01-24 20:52:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pipette-tip-1-200ul-certified-universal-graduated-yellow-54mm-960-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:52:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/furniture_parts_accessories/computer_stand_accessories/medical_computer_cart_accessories>: HTTP status code is not handled or not allowed 2026-01-24 20:52:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/supco-capillary-tubing-028-idx10-ft>: HTTP status code is not handled or not allowed 2026-01-24 20:52:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/little-giant-flip-n-lite-aluminum-platform-step-ladder-36-inch>: HTTP status code is not handled or not allowed 2026-01-24 20:52:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/statfree-z2-dissipative-3-layer-vinyl-roll-black-0125in-x-24in-x-50ft>: HTTP status code is not handled or not allowed 2026-01-24 20:52:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/supco-manual-reset-pressure-switch-410-psi-open>: HTTP status code is not handled or not allowed 2026-01-24 20:52:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-1-4-x-65-1-2-h-deluxe-office-partition-panel-with-pass-thru-cable-blue-277568bl>: HTTP status code is not handled or not allowed 2026-01-24 20:52:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ductless-air-conditioner-inverter-split-system-18k-btu-cool-w-heat-wifi-enabled-230v-19-seer>: HTTP status code is not handled or not allowed 2026-01-24 20:52:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beaker-polypropylene-printed-graduations-500ml-12-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:52:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/mailboxes_drop_boxes/residential_mailboxes_components already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:52:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/supco-therm-o-disc-general-purpose-thermostat-180-160>: HTTP status code is not handled or not allowed 2026-01-24 20:52:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vornado-630b-medium-whole-room-air-circulator>: HTTP status code is not handled or not allowed 2026-01-24 20:52:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ex830nistl-true-rms-acdc-clamp-meter-winfrared-thermometer-nist-certified>: HTTP status code is not handled or not allowed 2026-01-24 20:52:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tork-elevation-bath-tissue-mini-jumbo-roll-twin-dispenser-5555290>: HTTP status code is not handled or not allowed 2026-01-24 20:52:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/structural-plastic-adjustable-vented-shelf-36-x-36>: HTTP status code is not handled or not allowed 2026-01-24 20:52:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mini-cooler-20-c-32-place-4x8-for-15ml-tubes-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:52:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1560h150-standard-timing-belt-h-1-1-2-x-156-t312-trapezoidal>: HTTP status code is not handled or not allowed 2026-01-24 20:52:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2700h100-standard-timing-belt-h-1-x-270-t540-trapezoidal>: HTTP status code is not handled or not allowed 2026-01-24 20:52:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2200h075-standard-timing-belt-h-3-4-x-220-t440-trapezoidal>: HTTP status code is not handled or not allowed 2026-01-24 20:52:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/structural-plastic-adjustable-vented-shelf-96-x-36>: HTTP status code is not handled or not allowed 2026-01-24 20:52:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/mailboxes_drop_boxes/residential_mailboxes_components>: HTTP status code is not handled or not allowed 2026-01-24 20:52:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ht30-heat-stress-wbgt-wet-bulb-globe-temperature-meter-greenorange>: HTTP status code is not handled or not allowed 2026-01-24 20:52:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/lockers/bulk_storage_lockers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:52:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cl200-exstikand174-chlorine-meter-submersed-in-sample-cup-wextab-reagent-tablets>: HTTP status code is not handled or not allowed 2026-01-24 20:52:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ec150-conductivitytdstemperature-meter-ec150-protective-cap-4x-button-cell-batteries>: HTTP status code is not handled or not allowed 2026-01-24 20:52:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roll-stand-60-width-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:52:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-40bs49x118-12-pitch-1-18-finished-bore-49-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:52:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ep-720-24-poly-tubing-dispenser-24>: HTTP status code is not handled or not allowed 2026-01-24 20:52:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oil-only-boom-pillows-10-bale-18-x-8>: HTTP status code is not handled or not allowed 2026-01-24 20:52:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-boltless-shelving-96-w-x-36-d-x-84-h-3-level-starter-no-deck-tan>: HTTP status code is not handled or not allowed 2026-01-24 20:52:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kac-ce-bc-nwpi-aq-sm-med-black>: HTTP status code is not handled or not allowed 2026-01-24 20:52:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kac-iso-nw-blue-2xl>: HTTP status code is not handled or not allowed 2026-01-24 20:52:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gray-fine-fiber-universal-heavy-weight-roll-1-bale-150-x-30>: HTTP status code is not handled or not allowed 2026-01-24 20:52:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-angle-beam-96-w-x-36-d-x-84-h-4-level-add-on-no-deck-tan>: HTTP status code is not handled or not allowed 2026-01-24 20:52:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/lockers/bulk_storage_lockers>: HTTP status code is not handled or not allowed 2026-01-24 20:52:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-tect-174-diamond-grip-shoe-booties-150-pairs>: HTTP status code is not handled or not allowed 2026-01-24 20:52:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-size-w-post-mix-system-bag-box-syrup-water-tank-granite-sand>: HTTP status code is not handled or not allowed 2026-01-24 20:52:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securall-312x96x100-hazmat-chemical-building>: HTTP status code is not handled or not allowed 2026-01-24 20:52:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60bs24hx114-34-pitch-1-14-finished-bore-24-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:52:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carboy-round-with-handles-ldpe-white-polypropylene-screwcap-20-liter-molded-graduations>: HTTP status code is not handled or not allowed 2026-01-24 20:52:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-motor-planetary-gear-reversible-255-rpm-3-hp>: HTTP status code is not handled or not allowed 2026-01-24 20:52:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ice-caddies-beige-125-lbs-cap-tall>: HTTP status code is not handled or not allowed 2026-01-24 20:52:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cassette-tissue-embedding-with-attached-lid-30-writing-area-aqua-1000-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:52:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80bs16hx112-1-pitch-1-12-finished-bore-16-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:52:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-16bs16hx35-metric-1-pitch-35mm-finished-bore-16-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:52:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-user-mechanical-lock-3630-for-right-hinged-doors-and-spring-locking>: HTTP status code is not handled or not allowed 2026-01-24 20:52:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hydrofarm-cktrayhd-heavy-duty-flat-cut-kit-tray-21-13l-x-110w-x-28h>: HTTP status code is not handled or not allowed 2026-01-24 20:52:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ts200-020-electric-chain-hoist-w-trolley-4-button-pendulum-2-ton-230v>: HTTP status code is not handled or not allowed 2026-01-24 20:52:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-o-smith-direct-drive-blower-motor-1075-rpm-115-volts-5>: HTTP status code is not handled or not allowed 2026-01-24 20:52:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/chemical_storage_buildings already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:52:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securall-explosion-proof-heater-6143-btu>: HTTP status code is not handled or not allowed 2026-01-24 20:52:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dynabrade-3-vacuum-ready-backing-pad-1-4-20-thread>: HTTP status code is not handled or not allowed 2026-01-24 20:52:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3420-bulk-squids-174-swivel-glove-clip-holder-dual-clips-100-pack-red>: HTTP status code is not handled or not allowed 2026-01-24 20:52:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securall-primer-coat-saltwater-agb2400>: HTTP status code is not handled or not allowed 2026-01-24 20:52:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securall-explosion-proof-receptacle-interior-only-not-weatherproof>: HTTP status code is not handled or not allowed 2026-01-24 20:52:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60bs22hx114-34-pitch-1-14-finished-bore-22-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:52:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8366-glowear-174-type-r-class-3-lightweight-hi-vis-rain-jacket-lime-s>: HTTP status code is not handled or not allowed 2026-01-24 20:52:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-16a22-metric-a-plate-1-pitch-24mm-bore-22-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:52:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drawer-stack-2-6-h-overall-12-h-x-18-d-x-12-w>: HTTP status code is not handled or not allowed 2026-01-24 20:52:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-vis-lined-bib-overall-class-e-m-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:52:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/chemical_storage_buildings>: HTTP status code is not handled or not allowed 2026-01-24 20:53:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trio-twist-in-earplugs-reusable-corded-nrr-27db-50-pairs>: HTTP status code is not handled or not allowed 2026-01-24 20:53:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/structural-plastic-vented-shelf-36-x-24>: HTTP status code is not handled or not allowed 2026-01-24 20:53:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/structural-plastic-vented-shelf-48-x-24>: HTTP status code is not handled or not allowed 2026-01-24 20:53:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-8-horizontal-steel-slat-outdoor-park-bench-with-back-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:53:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-12bs20hx40-metric-34-pitch-40mm-finished-bore-20-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:53:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-o-smith-hot-water-circulator-motor-1725-rpm-115-volts-1-8hp>: HTTP status code is not handled or not allowed 2026-01-24 20:53:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-o-smith-nesbitt-replacement-motor-850-rpm-115-volts-1-6hp>: HTTP status code is not handled or not allowed 2026-01-24 20:53:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50bs32hx112-58-pitch-1-12-finished-bore-32-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:53:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-s60x6-female-buttress-x-3-4-male-garden-hose-pipe-thread-adapter>: HTTP status code is not handled or not allowed 2026-01-24 20:53:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leviton-16362-w-20-amp-125-volt-decora-plus-duplex-receptacle-straight-blade-white>: HTTP status code is not handled or not allowed 2026-01-24 20:53:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dz-7-hospital-grade-detergent-disinfectant>: HTTP status code is not handled or not allowed 2026-01-24 20:53:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-pallet-covers-51-x-49-x-73-2-mil-50-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:53:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 21482 pages (at 96 pages/min), scraped 12594 items (at 23 items/min) 2026-01-24 20:53:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/armor-poly-vci-bags-4-mil-70-x-44-x-100-blue-25-roll>: HTTP status code is not handled or not allowed 2026-01-24 20:53:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/armor-poly-vci-sheeting-4-mil-48-x-500-blue-1-roll-pkg>: HTTP status code is not handled or not allowed 2026-01-24 20:53:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bottle-with-screwcap-wide-mouth-polypropylene-graduated-100ml-10-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:53:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35bs18hx58-38-pitch-58-finished-bore-18-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:53:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/384-well-pcr-plate-a24-p24-two-notch-design-roche-style-clear-10-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:53:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-06bs16hx18-metric-38-pitch-18mm-finished-bore-16-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:53:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-o-smith-ge-21-29-frame-replacement-motor-115-208-230-volts-1050-rpm-2>: HTTP status code is not handled or not allowed 2026-01-24 20:53:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mixed-pallet-please-break-3-5-fluorescent-green-black>: HTTP status code is not handled or not allowed 2026-01-24 20:53:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hazardous-waste-6-6-yellow-red-black>: HTTP status code is not handled or not allowed 2026-01-24 20:53:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50-amp-portable-smart-charger>: HTTP status code is not handled or not allowed 2026-01-24 20:53:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/screw-cap-for-microtube-with-o-ring-blue-1000-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:53:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quicksnap-02ml-8-strip-tubes-with-individually-attached-flat-caps-clear-120-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:53:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/number-1-adjustable-height-complete-packing-workbench-laminate-square-edge-96-x-36-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:53:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/number-1-adjustable-height-complete-packing-workbench-laminate-safety-edge-with-t-mold-96x30-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:53:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25-ton-pin-style-jack-stand-single-unit>: HTTP status code is not handled or not allowed 2026-01-24 20:53:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hazard-class-2-inhalation-hazard-4-4-white-black>: HTTP status code is not handled or not allowed 2026-01-24 20:53:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/01ml-8-strip-tubes-low-profile-with-separate-8-strip-clear-flat-caps-white-125-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:53:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-foot-6-gauge-heavy-buty-jumper-booster-cables>: HTTP status code is not handled or not allowed 2026-01-24 20:53:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/test-tube-16-x-75mm-8ml-polystyrene-no-rim-1000-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:53:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jun-2-3-fluorescent-red-black>: HTTP status code is not handled or not allowed 2026-01-24 20:53:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-o-smith-5-shaded-pole-motor-1050-rpm-115-volts-2>: HTTP status code is not handled or not allowed 2026-01-24 20:53:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-o-smith-5-shaded-pole-motor-1050-rpm-115-volts-3>: HTTP status code is not handled or not allowed 2026-01-24 20:53:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paper-twist-ties-4-x-5-32-red-candy-stripe-2000-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:53:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/syphon-spray-system-20-gallon-drum>: HTTP status code is not handled or not allowed 2026-01-24 20:53:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pto-generator-kit-31-000-watts-120-240v-125a-black>: HTTP status code is not handled or not allowed 2026-01-24 20:53:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/attn-static-sens-5-8-2-yellow-black>: HTTP status code is not handled or not allowed 2026-01-24 20:53:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50bs24hx1-58-pitch-1-finished-bore-24-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:53:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/this-side-up-fragile-4-6-fluorescent-red-black>: HTTP status code is not handled or not allowed 2026-01-24 20:53:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-tier-1-door-premium-steel-locker-12-w-x-15-d-x-72-h-black-assembled>: HTTP status code is not handled or not allowed 2026-01-24 20:53:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/top-load-3-5-fluorescent-red-black>: HTTP status code is not handled or not allowed 2026-01-24 20:53:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/curbside-truck-silver-ep-21w-x-36l-x-69h-4-wire-shelves-polyurethane-2-rigid-casters>: HTTP status code is not handled or not allowed 2026-01-24 20:53:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-3-3-shaded-pole-totally-enclosed-motor-115-208-230-volts-1550-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:53:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-3-4-4-bright-yellow-rectangle>: HTTP status code is not handled or not allowed 2026-01-24 20:53:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-o-smith-5-split-capacitor-motor-208-230-volts-1075-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:53:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/curbside-cart-chrome-24w-x-36l-x-40h-2-wire-1-solid-shelf-polyurethane-2-rigid-casters>: HTTP status code is not handled or not allowed 2026-01-24 20:53:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-o-smith-4-5-16-shaded-pole-motor-850-rpm-230-volts>: HTTP status code is not handled or not allowed 2026-01-24 20:53:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acoustic-water-leak-detection-system>: HTTP status code is not handled or not allowed 2026-01-24 20:53:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qc-passed-2-dia-blue-white>: HTTP status code is not handled or not allowed 2026-01-24 20:53:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/circuit-board-shipper-w-pink-foam-6-3-16-x-6-x-1-1-2-in>: HTTP status code is not handled or not allowed 2026-01-24 20:53:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-reflective-plastic-barrier-chain-2-x-25-ft-magenta>: HTTP status code is not handled or not allowed 2026-01-24 20:53:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/to-go-rack-chrome-24w-x-48l-x-34h-3-wire-shelves-dividers-ledges>: HTTP status code is not handled or not allowed 2026-01-24 20:53:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-small-connecting-link-fits-1-1-1-2-chain-pack-of-10-traffic-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:53:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keep-away-from-heat-3-4-white-red-black>: HTTP status code is not handled or not allowed 2026-01-24 20:53:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keep-dry-fragile-up-3-4>: HTTP status code is not handled or not allowed 2026-01-24 20:53:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/curbside-cart-silver-ep-24w-x-72l-x-40h-2-wire-1-solid-shelf-polyurethane-2-rigid-casters>: HTTP status code is not handled or not allowed 2026-01-24 20:53:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-aluminum-camlock-fitting-dust-cap-thread>: HTTP status code is not handled or not allowed 2026-01-24 20:53:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-pallet-dolly-with-tilt-rollers-36-x-36-6000-lb-cap>: HTTP status code is not handled or not allowed 2026-01-24 20:53:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-41bs40x114-12-pitch-1-14-finished-bore-40-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:53:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/curbside-truck-silver-ep-24w-x-30l-x-69h-4-wire-shelves-polyurethane-2-rigid-casters>: HTTP status code is not handled or not allowed 2026-01-24 20:53:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/curbside-truck-chrome-21w-x-30l-x-69h-4-wire-shelves-polyurethane-2-rigid-casters>: HTTP status code is not handled or not allowed 2026-01-24 20:53:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60bs19hx118-34-pitch-1-18-finished-bore-19-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:53:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onyx-surveyors-safety-vest-black-2xl>: HTTP status code is not handled or not allowed 2026-01-24 20:53:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/colorstar-impressions-custom-logo-mat--3--x-5--48-mil-universal-cleated-sbr>: HTTP status code is not handled or not allowed 2026-01-24 20:53:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/colorstar-impressions-custom-logo-mat--6--x-12--48-mil-universal-cleated-sbr>: HTTP status code is not handled or not allowed 2026-01-24 20:53:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-e-premium-two-tone-poly-filled-winter-insulated-bibs-w-multi-pockets-2xl-3xl>: HTTP status code is not handled or not allowed 2026-01-24 20:53:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-40sk72-qd-bushed-12-pitch-72-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:53:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-3-3-shaded-pole-dra-inducer-blower-115-volts-3000-rpm-25>: HTTP status code is not handled or not allowed 2026-01-24 20:53:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-100btl18h-taper-bushed-1-14-pitch-18-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:53:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-16a25-metric-a-plate-1-pitch-24mm-bore-25-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:53:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fragile-delicate-instruments-3-4-red-white>: HTTP status code is not handled or not allowed 2026-01-24 20:53:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2hp-motor-tpi-fixed-24-30-fans>: HTTP status code is not handled or not allowed 2026-01-24 20:53:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/curbside-cart-silver-ep-18w-x-36l-x-43h-2-wire-1-solid-shelf-pneumatic-2-rigid-casters>: HTTP status code is not handled or not allowed 2026-01-24 20:53:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fri-2-dia-fluorescent-red-black>: HTTP status code is not handled or not allowed 2026-01-24 20:53:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-bulk-storage-locker-double-tier-starter-36x48x45>: HTTP status code is not handled or not allowed 2026-01-24 20:53:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/curbside-truck-silver-ep-21w-x-54l-x-72h-4-wire-shelves-pneumatic-2-rigid-casters>: HTTP status code is not handled or not allowed 2026-01-24 20:53:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-slotted-pallet-rack-upright-42-inches-d-x-72-inches-h>: HTTP status code is not handled or not allowed 2026-01-24 20:54:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/administrator-key-for-no-3681-blue-tooth-built-in-locker-lock>: HTTP status code is not handled or not allowed 2026-01-24 20:54:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-slotted-pallet-rack-add-on-no-deck-120-inches-w-x-42-inches-d-x-192-inches-h>: HTTP status code is not handled or not allowed 2026-01-24 20:54:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crown-verity-portable-hand-sink-cold-water>: HTTP status code is not handled or not allowed 2026-01-24 20:54:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1671-10-10-1-4-id-1-4-mnpt-polyurethane-rekoil-hose>: HTTP status code is not handled or not allowed 2026-01-24 20:54:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/19-piece-12-point-standard-sae-mechanics-tool-set-with-12-drive-tang>: HTTP status code is not handled or not allowed 2026-01-24 20:54:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mantis-4x4-extra-large-fly-light-white>: HTTP status code is not handled or not allowed 2026-01-24 20:54:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/653-4-1-2-fnpt-x-3-8-mnpt-hose-fitting-adapter>: HTTP status code is not handled or not allowed 2026-01-24 20:54:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s-682-tank-conversion-kit-with-4-hose-assembly-kit>: HTTP status code is not handled or not allowed 2026-01-24 20:54:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-demi-diamond-blue-plastic-station-50-box>: HTTP status code is not handled or not allowed 2026-01-24 20:54:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/615w-1-4-mnpt-reusable-hose-end-fitting>: HTTP status code is not handled or not allowed 2026-01-24 20:54:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 21578 pages (at 96 pages/min), scraped 12614 items (at 20 items/min) 2026-01-24 20:54:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60bs60x11116-34-pitch-1116-finished-bore-60-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:54:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/artisan-old-english-style-traditional-rug-8-ft-x-10-ft-red>: HTTP status code is not handled or not allowed 2026-01-24 20:54:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jelly-belly-50-flavors-jelly-beans-assortment-3-lb-gr>: HTTP status code is not handled or not allowed 2026-01-24 20:54:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-drawer-mobile-cabinet-24-wx21-dx43-1-4-h-black>: HTTP status code is not handled or not allowed 2026-01-24 20:54:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-ec2002b-2-hp-1-phase-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:54:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-drawer-mobile-cabinet-24-wx21-dx35-1-4-h-everest-blue-1>: HTTP status code is not handled or not allowed 2026-01-24 20:54:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-9036-1-3-hp-1-phase-1075-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:54:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/starbucks-by-nespresso-pods-variety-pack-10-count-6-pack-gr>: HTTP status code is not handled or not allowed 2026-01-24 20:54:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/54-centered-mobile-cart-hsk-63-32-wx27-dx59-1-4-h-black>: HTTP status code is not handled or not allowed 2026-01-24 20:54:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/newman-s-own-organics-special-blend-coffee-k-cups-100-count-gr>: HTTP status code is not handled or not allowed 2026-01-24 20:54:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-ec1004b-1-hp-1-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:54:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-step-16-wx46-d-stainless-steel-rolling-safety-ladder-perforated-tread>: HTTP status code is not handled or not allowed 2026-01-24 20:54:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/narrow-collapsible-hamper-steel-orange-vinyl-bag>: HTTP status code is not handled or not allowed 2026-01-24 20:54:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pom-wonderful-100-pomegranate-juice-12-oz-6-count-gr>: HTTP status code is not handled or not allowed 2026-01-24 20:54:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dum-dums-bulk-lollipops-30-lb-gr>: HTTP status code is not handled or not allowed 2026-01-24 20:54:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/antimicrobial-swivel-barstool-black>: HTTP status code is not handled or not allowed 2026-01-24 20:54:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-6794-home-exercise-package-pro-set>: HTTP status code is not handled or not allowed 2026-01-24 20:54:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/three-panel-stationary-medical-privacy-screen-81-wx69-h-mauve-vinyl-panels>: HTTP status code is not handled or not allowed 2026-01-24 20:54:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-4-liter-capacity-sanitizing-cleaning-all-purpose-pump-sprayer>: HTTP status code is not handled or not allowed 2026-01-24 20:54:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-2867-1-5-hp-1-phase-1050-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:54:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nature-valley-oats-n-honey-granola-bars-49-count-gr>: HTTP status code is not handled or not allowed 2026-01-24 20:54:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/abt-hc-ucbi-0404-ada-premier-built-in-ada-compliant-undercounter-refrigerator-right-hinged-door-4-6-cu-ft>: HTTP status code is not handled or not allowed 2026-01-24 20:54:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vertical-31-4-drawer-legal-fire-and-water-resistant-file-cabinet-putty-lgl4w31p>: HTTP status code is not handled or not allowed 2026-01-24 20:54:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-inch-brushless-paddle-switch-small-angle-grinder-with-kickback-brake-no-lock-on-dwe43214n>: HTTP status code is not handled or not allowed 2026-01-24 20:54:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-t1c2jcr-1-hp-1-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:54:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-t1c2p18z-1-hp-1-phase-1740-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:54:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-step-16-w-stainless-steel-ladder-with-rails-perforated-tread>: HTTP status code is not handled or not allowed 2026-01-24 20:54:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-step-24-w-stainless-steel-ladder-with-rails-serrated-grating>: HTTP status code is not handled or not allowed 2026-01-24 20:54:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lateral-31-4-drawer-fire-and-water-resistant-file-cabinet-black-lat4w31b>: HTTP status code is not handled or not allowed 2026-01-24 20:54:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-t32ca2jcr-1-1-2-hp-1-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:54:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-deep-mobile-pedestal-box-file-with-black-seat-cushion-black>: HTTP status code is not handled or not allowed 2026-01-24 20:54:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hog-heaven-anti-fatigue-mat-7-8-thick-48-w-yellow-border-from-3-up-to-60>: HTTP status code is not handled or not allowed 2026-01-24 20:54:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-wd12c1jcr-1-2-hp-1-phase-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:54:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lx-adj-dw-mobile-height-adjustable-lectern-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:54:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-motorized-shut-off-damper-24v>: HTTP status code is not handled or not allowed 2026-01-24 20:54:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/idlewood-wall-mount-infection-control-station-cherry-id-3d-chmain>: HTTP status code is not handled or not allowed 2026-01-24 20:54:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-u2v2bc-2-hp-3-phase-1735-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:54:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pipeline-4-way-adjustable-gloss-white-garment-rack>: HTTP status code is not handled or not allowed 2026-01-24 20:54:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pipeline-16-in-l-gloss-white-add-on-arm-for-garment-racks>: HTTP status code is not handled or not allowed 2026-01-24 20:54:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/infinity-174-locker-single-tier-18x18x72-1-door-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:54:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/datum-large-mobile-cpu-locker-black>: HTTP status code is not handled or not allowed 2026-01-24 20:54:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-d3v2b-3-hp-3-phase-1770-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:54:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/infinity-174-locker-single-tier-18x18x72-3-door-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:54:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ego-cs1804-power-56v-18-cordless-chain-saw-kit-w-5-0ah-battery-charger>: HTTP status code is not handled or not allowed 2026-01-24 20:54:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/closed-shelving-drawer-unit-36x18x39-36-drawers-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:54:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wwe-khn87-40-1-h-184tc-cast-iron-helical-bevel-speed-reducer-184tc-input-flange-40-1-foot-mt>: HTTP status code is not handled or not allowed 2026-01-24 20:54:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fast-lift-service-jack-3-5-ton>: HTTP status code is not handled or not allowed 2026-01-24 20:54:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-u34s3ac-3-4-hp-3-phase-1140-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:54:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uberlight-26-flex-led-task-light-with-round-base-white>: HTTP status code is not handled or not allowed 2026-01-24 20:54:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flask-erlenmeyer-globe-glass-narrow-mouth-dual-graduations-astm-e1404-2000ml-4-box>: HTTP status code is not handled or not allowed 2026-01-24 20:54:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wwe-khn47-7-5-1-h-143-5tc-cast-iron-helical-bevel-speed-reducer-143-5tc-input-flange-7-5-1-ft-mt>: HTTP status code is not handled or not allowed 2026-01-24 20:54:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cylinder-graduated-globe-glass-class-a-to-deliver-td-dual-grads-astm-e1272-1000ml>: HTTP status code is not handled or not allowed 2026-01-24 20:54:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wwe-khn77-10-1-h-213-5tc-cast-iron-helical-bevel-speed-reducer-213-5tc-input-flange-10-1-foot-mt>: HTTP status code is not handled or not allowed 2026-01-24 20:54:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-assembled-electronic-access-locker-double-tier-1-wide-12x18x78>: HTTP status code is not handled or not allowed 2026-01-24 20:54:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-knock-down-corrosion-resistant-locker-single-tier-1-wide-12x15x78>: HTTP status code is not handled or not allowed 2026-01-24 20:54:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-uj3c2p18m-3-hp-1-phase-1740-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:54:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/floor-supported-portable-personal-safety-partition-2w-x-5h-clear>: HTTP status code is not handled or not allowed 2026-01-24 20:54:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-gas-infrared-patio-heater-two-stage-35000-btu-glass-series>: HTTP status code is not handled or not allowed 2026-01-24 20:54:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-u1p2d-1-hp-3-phase-1755-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:54:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-assembled-corrosion-resistant-locker-single-tier-3-wide-12x15x78>: HTTP status code is not handled or not allowed 2026-01-24 20:54:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-u32p2g-1-5-hp-3-phase-1755-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:54:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-t1p1a-1-hp-3-phase-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:54:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-ply-disposable-face-mask-youth-size-individually-packed-50-box>: HTTP status code is not handled or not allowed 2026-01-24 20:54:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-rivet-steel-shelving-with-wood-deck-84-w-x-18-d-x-96-h-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:54:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-xs12sa2d-1-2-hp-3-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:54:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-x3p2b-3-hp-3-phase-1765-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:54:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cone-chain-connector-kit-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:54:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-galvanized-steel-blower-wheel-4-1-4-diameter-1-4-bore-2>: HTTP status code is not handled or not allowed 2026-01-24 20:54:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lau-4-blade-condenser-propeller-12-diameter-cw-rotation-1>: HTTP status code is not handled or not allowed 2026-01-24 20:55:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-d3p1d-3-hp-3-phase-3490-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:55:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-chain-1-1-2-inch-links-500-feet-trade-size-6-black>: HTTP status code is not handled or not allowed 2026-01-24 20:55:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-d32p2h-1-5-hp-3-phase-1750-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:55:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-bread-bags-10x15-1-25-mili>: HTTP status code is not handled or not allowed 2026-01-24 20:55:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/retaining-ring-external-stainless-steel-assortment-100-pieces>: HTTP status code is not handled or not allowed 2026-01-24 20:55:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-1693-1-3-1-4-hp-1-phase-1625-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:55:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-x10e2b-10-hp-3-phase-1760-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:55:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camshelving-stationary-starter-4-vented-shelves-24x60x64>: HTTP status code is not handled or not allowed 2026-01-24 20:55:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-8675-1-3-hp-1-phase-825-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:55:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-9993-1-3-hp-1-phase-1050-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:55:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-3183-1-2-hp-1-phase-825-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:55:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-5460-1-2-hp-1-phase-1075-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:55:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 21678 pages (at 100 pages/min), scraped 12634 items (at 20 items/min) 2026-01-24 20:55:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/small-pegboard-hanging-bin-system>: HTTP status code is not handled or not allowed 2026-01-24 20:55:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spring-loaded-aluminum-dockplate-30-w-x-48-l-1670-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:55:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wwe-kan67-10-1-h-145tc-cast-iron-helical-bevel-speed-reducer-145tc-input-flange-10-1>: HTTP status code is not handled or not allowed 2026-01-24 20:55:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kan-series-helical-bevel-speed-reducer-b5-output-flg-kit-size-97-b5-output-flg-kit-hardw-incl>: HTTP status code is not handled or not allowed 2026-01-24 20:55:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kan-series-helical-bevel-speed-reducer-b5-output-flg-kit-size-47-b5-output-flg-kit-hardw-incl>: HTTP status code is not handled or not allowed 2026-01-24 20:55:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=us-motor-1693-1-3-1-4-hp-1-phase-1625-rpm>: HTTP status code is not handled or not allowed 2026-01-24 20:55:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kan-series-helical-bevel-speed-reducer-b5-output-flg-kit-size-37-b5-output-flg-kit-hardw-incl>: HTTP status code is not handled or not allowed 2026-01-24 20:55:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wwe-kan97-20-1-h-213-5tc-cast-iron-helical-bevel-speed-reducer-213-5tc-input-flange-20-1>: HTTP status code is not handled or not allowed 2026-01-24 20:55:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-qt-crock-with-lid>: HTTP status code is not handled or not allowed 2026-01-24 20:55:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interion-174-mobile-office-partition-panel-with-partial-window-36-1-4-w-x-96-h-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:55:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/memory-foam-lumbar-support-backrest-qty-5>: HTTP status code is not handled or not allowed 2026-01-24 20:55:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-roll-file-24-compartment>: HTTP status code is not handled or not allowed 2026-01-24 20:55:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/versa-food-bars-serving-buffet-cold-food-60-x-29-low-6-heavy-duty-swivel-casters-with-brakes-black>: HTTP status code is not handled or not allowed 2026-01-24 20:55:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-wall-mount-hand-sink-w-strainer-14-x-10-x-5-deep>: HTTP status code is not handled or not allowed 2026-01-24 20:55:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thermobarrier-21-1-4x13x1-1-2-removable-insulated-shelf-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:55:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-core-cable-with-bulb-auger-5-16-x-25-for-670462>: HTTP status code is not handled or not allowed 2026-01-24 20:55:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hoover-paper-vacuum-bags-for-hoover-type-n>: HTTP status code is not handled or not allowed 2026-01-24 20:55:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/optional-high-base-steel-flat-files-white>: HTTP status code is not handled or not allowed 2026-01-24 20:55:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/end-panel-for-flat-top-locker-12-x-72-beige>: HTTP status code is not handled or not allowed 2026-01-24 20:55:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camtherm-hot-tall-profile-granite-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:55:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stackable-acid-corrosive-cabinet-with-manual-doors-11-gallon>: HTTP status code is not handled or not allowed 2026-01-24 20:55:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hot-cold-cart-tall-profile-granite-green-celsius-10-rear-casters>: HTTP status code is not handled or not allowed 2026-01-24 20:55:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hot-cold-cart-tall-profile-granite-green>: HTTP status code is not handled or not allowed 2026-01-24 20:55:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gray-counter-blue-panel-u-shaped-reception-station-1>: HTTP status code is not handled or not allowed 2026-01-24 20:55:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camshelving-elements-starter-unit-24-w-x-60-l-x-72-h-4-shelf-brushed-graphite>: HTTP status code is not handled or not allowed 2026-01-24 20:55:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-camtainer-beverage-carrier-insulated-plastic-10-1-2-gallon-capacity-granite-sand>: HTTP status code is not handled or not allowed 2026-01-24 20:55:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-flat-top-cabinet-hinged-doors-galv-24-x-48>: HTTP status code is not handled or not allowed 2026-01-24 20:55:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/union-y-12mm-x-2-8mm-tube>: HTTP status code is not handled or not allowed 2026-01-24 20:55:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camcarrier-ultra-pancarrier-front-loading-approximately-cap-60-qt-foam-insulation-stackable-black>: HTTP status code is not handled or not allowed 2026-01-24 20:55:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/surface-mounted-end-stall-sanitary-napkin-disposal>: HTTP status code is not handled or not allowed 2026-01-24 20:55:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-angle-frame-mirror-24-wx30-h>: HTTP status code is not handled or not allowed 2026-01-24 20:55:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/45degree-plug-in-elbow-5-32-tube-x-5-32-stud>: HTTP status code is not handled or not allowed 2026-01-24 20:55:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reducer-1-2-male-stud-x-1-4-tube>: HTTP status code is not handled or not allowed 2026-01-24 20:55:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-ceiling-bracket-20kw-30kw-unit-heaters>: HTTP status code is not handled or not allowed 2026-01-24 20:55:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fan-forced-horizontal-discharge-unit-heater-5kw-240v-1ph>: HTTP status code is not handled or not allowed 2026-01-24 20:55:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camshelving-elements-mobile-starter-unit-18-w-x-36-l-x-70-h-4-shelf-brushed-graphite-4-premium-casters>: HTTP status code is not handled or not allowed 2026-01-24 20:55:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-workbench-2-3-4-backsplash-ss-undershelf-48-w-x-30-d>: HTTP status code is not handled or not allowed 2026-01-24 20:55:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reducing-manifold-8mm-tube-x-4mm-tube-x-1-4-swift-fit>: HTTP status code is not handled or not allowed 2026-01-24 20:55:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-union-10mm>: HTTP status code is not handled or not allowed 2026-01-24 20:55:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cast-iron-jaw-coupling-hub-style-l110-5-8-inch-bore-diameter-3-16-x-3-32-keyway>: HTTP status code is not handled or not allowed 2026-01-24 20:55:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cast-iron-jaw-coupling-hub-style-l150-1-3-16-inch-bore-diameter-1-4-x-1-8-keyway>: HTTP status code is not handled or not allowed 2026-01-24 20:55:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-male-metal-release-collet-6mm-tube-x-m5-bspp>: HTTP status code is not handled or not allowed 2026-01-24 20:55:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triple-pack-ball-l-wr-sets-10999-10937-gorillagrip-fold-up-set-12632>: HTTP status code is not handled or not allowed 2026-01-24 20:55:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-convection-incubator-programmable-161-liters-115-volt>: HTTP status code is not handled or not allowed 2026-01-24 20:55:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compact-electronic-indicator-flat-back-0-2-inch-0-5mm-range>: HTTP status code is not handled or not allowed 2026-01-24 20:55:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-digital-caliper-0-12-inch-0-300mm-range>: HTTP status code is not handled or not allowed 2026-01-24 20:55:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pin-gage-handle-w-1pc-handle-and-6-pairs-of-collecting-bushes-228-281-inch-range>: HTTP status code is not handled or not allowed 2026-01-24 20:55:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shield-octagonal-w-o-handle-36-inch-reach-w-interlock-3-arms-28-inch-l-x-10-inch-w-30-lb>: HTTP status code is not handled or not allowed 2026-01-24 20:55:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shield-flat-direct-mount-w-interlock-led-16-inch-w-x-14-inch-h>: HTTP status code is not handled or not allowed 2026-01-24 20:55:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/bar_restaurant_equipment/dispensers_organizers/cup_lid_organizers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:55:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ez-fit-hot-lid-dispenser-black-large-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-24 20:55:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ecp-automatic-slide-cutter-24-inch-l>: HTTP status code is not handled or not allowed 2026-01-24 20:55:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/birdslide-10-5-ft-sections-50w-beige>: HTTP status code is not handled or not allowed 2026-01-24 20:55:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tower-guard-14-zip-ties-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 20:55:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbw-ups-fedexable-antimicrobial-vinyl-basket-truck-18-bushel-capacity-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:55:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbw-ups-fedexable-antimicrobial-vinyl-basket-truck-18-bushel-capacity-red>: HTTP status code is not handled or not allowed 2026-01-24 20:55:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aquavantage-av-exposed-manual-diaphragm-flush-valve-with-3-5-gpf-in-chrome>: HTTP status code is not handled or not allowed 2026-01-24 20:55:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/weldon-4-backless-picnic-bench-black>: HTTP status code is not handled or not allowed 2026-01-24 20:55:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exeter-6-garden-bench-whitewash>: HTTP status code is not handled or not allowed 2026-01-24 20:55:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/springville-lounge-chair-black>: HTTP status code is not handled or not allowed 2026-01-24 20:55:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/outdoor_structures_storage/gazebos_pavilions_pergolas already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:55:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/bar_restaurant_equipment/dispensers_organizers/cup_lid_organizers>: HTTP status code is not handled or not allowed 2026-01-24 20:55:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/muskoka-adirondack-5-piece-bistro-dining-set-36h-table-weathered-acorn>: HTTP status code is not handled or not allowed 2026-01-24 20:55:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_chair_table_sets/patio_bar_dining_sets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:55:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/orlando-silver-lines-slab-door-wood-36w-x-80h-whiskey-oak>: HTTP status code is not handled or not allowed 2026-01-24 20:55:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/key-largo-gold-lines-slab-door-wood-36w-x-80h-ice-maple>: HTTP status code is not handled or not allowed 2026-01-24 20:55:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:55:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/largo-silver-lines-slab-door-wood-30w-x-80h-whiskey-oak>: HTTP status code is not handled or not allowed 2026-01-24 20:55:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ocala-silver-lines-slab-door-wood-32w-x-80h-white>: HTTP status code is not handled or not allowed 2026-01-24 20:55:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:55:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/key-west-gold-lines-slab-door-wood-28w-x-80h-white>: HTTP status code is not handled or not allowed 2026-01-24 20:56:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/naples-gold-lines-slab-door-wood-30w-x-80h-whiskey-oak>: HTTP status code is not handled or not allowed 2026-01-24 20:56:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hog-heavenanti-fatigue-mat-78-thick-50l-x-2w-black-striped-border>: HTTP status code is not handled or not allowed 2026-01-24 20:56:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hog-heavenanti-fatigue-mat-58-thick-10l-x-4w-black-striped-border>: HTTP status code is not handled or not allowed 2026-01-24 20:56:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/outdoor_structures_storage/gazebos_pavilions_pergolas>: HTTP status code is not handled or not allowed 2026-01-24 20:56:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/digital-long-stem-thermometer-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:56:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_chair_table_sets/patio_bar_dining_sets>: HTTP status code is not handled or not allowed 2026-01-24 20:56:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extreme-hiker-boots-size-10-black>: HTTP status code is not handled or not allowed 2026-01-24 20:56:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors>: HTTP status code is not handled or not allowed 2026-01-24 20:56:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chillshield-mens-insulated-pants-xl-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:56:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors>: HTTP status code is not handled or not allowed 2026-01-24 20:56:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 21773 pages (at 95 pages/min), scraped 12653 items (at 19 items/min) 2026-01-24 20:56:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chillshield-mens-insulated-jacket-medium-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:56:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/placesetter-satin-non-laminated-foam-plate-7-dia-white-pack-of-900>: HTTP status code is not handled or not allowed 2026-01-24 20:56:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reflections-heavyweight-spoon-silver-pack-of-320>: HTTP status code is not handled or not allowed 2026-01-24 20:56:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commercial-coffee-filters-w-flat-bottom-6-gal-pack-of-252>: HTTP status code is not handled or not allowed 2026-01-24 20:56:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coffee-tea-brewer-filters-w-flat-bottom-pack-of-3000>: HTTP status code is not handled or not allowed 2026-01-24 20:56:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flexstyle-food-combo-pack-32-oz-4-5-8-dia-x-5-5-16-h-symphony-pack-of-500>: HTTP status code is not handled or not allowed 2026-01-24 20:56:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/world-art-food-container-4-1-16-dia-x-2-1-2-h-pack-of-500>: HTTP status code is not handled or not allowed 2026-01-24 20:56:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ecocraft-grease-resistant-paper-wraps-and-liners-14-l-x-14-w-natural-pack-of-4000>: HTTP status code is not handled or not allowed 2026-01-24 20:56:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cake-container-9-shallow-9-dia-x-3-3-8-h-clear-black-pack-of-90>: HTTP status code is not handled or not allowed 2026-01-24 20:56:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/world-art-lids-for-12-oz-16-oz-32-oz-soup-container-pack-of-500>: HTTP status code is not handled or not allowed 2026-01-24 20:56:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deerfield-hot-drink-cup-lids-for-10-oz-to-20-oz-cups-white-pack-of-1000>: HTTP status code is not handled or not allowed 2026-01-24 20:56:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polycoated-paper-hot-drink-cups-4-oz-white-pack-of-1000>: HTTP status code is not handled or not allowed 2026-01-24 20:56:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bare-eco-forward-treated-paper-cold-drink-cups-5-oz-green-white-pack-of-3000>: HTTP status code is not handled or not allowed 2026-01-24 20:56:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paper-hot-coffee-cups-12-oz-pack-of-1000>: HTTP status code is not handled or not allowed 2026-01-24 20:56:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lifting-synthetic-shackle-5-16-dia-x-10-l-2-5-8-ton-wll>: HTTP status code is not handled or not allowed 2026-01-24 20:56:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lithium-ion-battery-charger-starter-kit>: HTTP status code is not handled or not allowed 2026-01-24 20:56:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drywall-cut-out-tool-24v>: HTTP status code is not handled or not allowed 2026-01-24 20:56:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/colormaxx-paint-primer-11-oz-metallic-gold-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-24 20:56:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/templog-premier-chromatography-refrigerator-47-cuft-capacity-glass-door>: HTTP status code is not handled or not allowed 2026-01-24 20:56:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glass-tube-patio-heater-40000-btu-propane-black>: HTTP status code is not handled or not allowed 2026-01-24 20:56:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hot-melt-adhesive-11-lb-capacity-transparent>: HTTP status code is not handled or not allowed 2026-01-24 20:56:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4200fc-marine-adhesive-sealant-295-ml-capacity-white>: HTTP status code is not handled or not allowed 2026-01-24 20:56:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scotch-weld-epoxy-potting-compound-adhesive-duo-pak-485-ml-capacity-black>: HTTP status code is not handled or not allowed 2026-01-24 20:56:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-powered-computer-workstation-40ah-battery-black-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 20:56:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bar-height-outdoor-dining-set-36-in-square-x-42-in-h-table-4-chairs-black>: HTTP status code is not handled or not allowed 2026-01-24 20:56:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-2-x-4-led-light-panel-1000-k-with-wiring>: HTTP status code is not handled or not allowed 2026-01-24 20:56:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/instant-hot-packs-6l-x-6w-pack-of-24>: HTTP status code is not handled or not allowed 2026-01-24 20:56:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/k1-basic-wheelchair-w-swing-back-desk-length-arms-elevating-leg-rests-nylon-20w-seat>: HTTP status code is not handled or not allowed 2026-01-24 20:56:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/water-tumbler-pitcher-set-32-oz-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-24 20:56:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-patient-gowns-w-opening-belt-30-x-42-mauve-50-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:56:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultralight-aluminium-transport-wheelchair-19w-seat-red-frame>: HTTP status code is not handled or not allowed 2026-01-24 20:56:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guardian-k3-wheelchair-w-full-length-arms-swing-away-footrests-16w-seat>: HTTP status code is not handled or not allowed 2026-01-24 20:56:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ring-cutter-round-handle-6-12-l>: HTTP status code is not handled or not allowed 2026-01-24 20:56:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/povidone-iodine-prep-solution-1-quart-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-24 20:56:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-pharmacy-upright-refrigerator-26-cu-ft-capacity-solid-door>: HTTP status code is not handled or not allowed 2026-01-24 20:56:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-sided-solid-box-truck-steel-1400-lb-capacity-42-1-4l-x-24-1-4w-x-28-3-16h>: HTTP status code is not handled or not allowed 2026-01-24 20:56:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cartridge-for-moen-single-handle-faucets-brass-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-24 20:56:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deck-mount-sink-faucet-w-8-12-gooseneck-spout>: HTTP status code is not handled or not allowed 2026-01-24 20:56:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-handle-basin-faucet-12-gpm-chrome>: HTTP status code is not handled or not allowed 2026-01-24 20:56:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-turn-angle-stop-chrome-plated-12-fip-x-38-od-comp-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 20:56:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vl500-wet-dry-vacuum-cleaner-14-gallon-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:56:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rope-hanger-14l-x-2-12w-x-10-12h-black>: HTTP status code is not handled or not allowed 2026-01-24 20:56:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/business-furniture-l-shaped-table-desk-w-3-drawers-71w-x-77-3-8d-x-29-7-8h-white>: HTTP status code is not handled or not allowed 2026-01-24 20:56:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/business-furniture-coventry-l-shaped-sectional-couch-111w-x-111d-x-35-3-4h-beige>: HTTP status code is not handled or not allowed 2026-01-24 20:56:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/business-furniture-coventry-u-shaped-sectional-couch-113w-x-87d-x-35-3-4h-beige>: HTTP status code is not handled or not allowed 2026-01-24 20:56:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/apartment-sized-refrigeratorfreezer-combo-unit-101-cuft-capacity-silver>: HTTP status code is not handled or not allowed 2026-01-24 20:56:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/apartment-size-refrigerator-11-cuft-capacity-white>: HTTP status code is not handled or not allowed 2026-01-24 20:56:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dh016a1w-t-undercounter-laboratory-refrigerator-solid-door-16-cuft-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:56:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shop-wet-and-dry-vacuum-3-hp-55-gallon>: HTTP status code is not handled or not allowed 2026-01-24 20:56:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-tier-6-door-wood-locker-36-w-x-15-d-x-72-h-cherry-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 20:56:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-tier-6-door-digital-wood-locker-36-w-x-15-d-x-72-h-gray-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 20:56:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/organizer-cabinet-w-fixed-shelves-assembled-48w-x-18d-x-78h-walnut>: HTTP status code is not handled or not allowed 2026-01-24 20:56:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/counter-height-cabinet-w-drawer--laminate-top-30w-x-18d-x-36h-pure-green>: HTTP status code is not handled or not allowed 2026-01-24 20:56:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sliding-door-cabinet-assembled-36w-x-21d-x-72h-white>: HTTP status code is not handled or not allowed 2026-01-24 20:56:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wardrobe-cabinet-assembled-42w-x-24d-x-72h-navy-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:56:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cabinet-w-drawer--shelves-assembled-30w-x-18d-x-72h-black>: HTTP status code is not handled or not allowed 2026-01-24 20:56:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bookcase-assembled-36w-x-13d-x-60h-walnut>: HTTP status code is not handled or not allowed 2026-01-24 20:56:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-wardrobe-cabinet-assembled-36w-x-24d-x-72h-denim-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:56:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bookcase-assembled-36w-x-18d-x-55h-leaf-green>: HTTP status code is not handled or not allowed 2026-01-24 20:56:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/planzboard-mobile-work-station-50-14l-x-45w-x-20h-white>: HTTP status code is not handled or not allowed 2026-01-24 20:56:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/testculture-tube-polystyrene-non-sterile-12-x-60-mm-3-ml-capacity-pack-of-5000>: HTTP status code is not handled or not allowed 2026-01-24 20:56:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/teardrop-pallet-rack-starter-108-in-w-x-24-in-d-x-120-in-h>: HTTP status code is not handled or not allowed 2026-01-24 20:56:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-premium-heavy-duty-vest-w-multi-pockets-sm-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:56:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onyx-enhanced-visibility-full-zip-hoodie-polyester-fleece-black-4xl>: HTTP status code is not handled or not allowed 2026-01-24 20:57:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-premium-heavy-duty-vest-w-multi-pockets-lxl-red>: HTTP status code is not handled or not allowed 2026-01-24 20:57:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-frame-shelf-truck-w-lower-shelf-3600-lb-capacity-40l-x-36w-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:57:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-frame-sheet--panel-truck-w-swivel-casters-3600-lb-cap-48l-x-24w-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:57:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-platform-truck-w-end--side-handle-3600-lb-cap-48l-x-30w-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:57:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/platform-truck-w-end--side-racks-handle-3600-lb-capacity-60l-x-24w-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:57:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-class-2-hi-vis-safety-vest-2-pockets-mesh-orange-bk-l-xl>: HTTP status code is not handled or not allowed 2026-01-24 20:57:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-class-2-hi-vis-safety-vest-2-reflective-strips-poly-mesh-lime-l-xl>: HTTP status code is not handled or not allowed 2026-01-24 20:57:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chill-its-6667-wet-evaporative-cooling-vest-pva-zipper-4xl-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:57:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8287-hi-vis-winter-jacket--vest-w-detachable-sleeves-2xl-lime>: HTTP status code is not handled or not allowed 2026-01-24 20:57:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-818wp-thermal-waterproof-winter-work-gloves-s-black>: HTTP status code is not handled or not allowed 2026-01-24 20:57:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8280bk-hi-vis-performance-t-shirt-black-bottom-class-2-5xl-lime>: HTTP status code is not handled or not allowed 2026-01-24 20:57:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 21870 pages (at 97 pages/min), scraped 12675 items (at 22 items/min) 2026-01-24 20:57:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-922cr-cut-resistant-gloves-nitrile-coated-ansi-a3-m-gray-96-pairs>: HTTP status code is not handled or not allowed 2026-01-24 20:57:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-922cr-cut-resistant-gloves-nitrile-coated-ansi-a3-l-gray-96-pairs>: HTTP status code is not handled or not allowed 2026-01-24 20:57:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8291bk-hi-vis-long-sleeve-t-shirt-type-r-class-2-2xl-lime>: HTTP status code is not handled or not allowed 2026-01-24 20:57:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-7044-cut-resistant-gloves-polyurethane-coated-ansi-a4-l-gray-12-pairs>: HTTP status code is not handled or not allowed 2026-01-24 20:57:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-7024-cut-resistant-gloves-polyurethane-coated-ansi-a2-2xl-blue-12-pairs>: HTTP status code is not handled or not allowed 2026-01-24 20:57:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-7024-cut-resistant-gloves-polyurethane-coated-ansi-a2-m-gray-1-pair>: HTTP status code is not handled or not allowed 2026-01-24 20:57:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-step-hybrid-rolling-safety-ladder-450-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:57:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quint-face-mask-dispenser-8w-x-5-1316d-x-23-18h-quartz-beige>: HTTP status code is not handled or not allowed 2026-01-24 20:57:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-air-impact-wrench-1350-ft-lbs-torque-maintenance-duty-pistol-grip-titanium-hammercase-orange>: HTTP status code is not handled or not allowed 2026-01-24 20:57:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-12-air-impact-wrench-20000-ft-lbs-torque-t-handle>: HTTP status code is not handled or not allowed 2026-01-24 20:57:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-drill-reversible-roll-throttle-350-rpm-175hp-no-2-morse-taper>: HTTP status code is not handled or not allowed 2026-01-24 20:57:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adhesive-remover-air-tool-4-pad-3700-rpm-includes-1-eraser-pad>: HTTP status code is not handled or not allowed 2026-01-24 20:57:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-tier-1-door-digital-locker-12in-w-x-18in-d-x-78in-h-tan-assembled>: HTTP status code is not handled or not allowed 2026-01-24 20:57:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-tier-3-door-digital-locker-12in-w-x-15in-d-x-78in-h-tan-assembled>: HTTP status code is not handled or not allowed 2026-01-24 20:57:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-tier-9-door-digital-locker-36in-w-x-15in-d-x-78in-h-tan-assembled>: HTTP status code is not handled or not allowed 2026-01-24 20:57:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-bench-diamond-seat-surface-mount-8l-black>: HTTP status code is not handled or not allowed 2026-01-24 20:57:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/protective-hat-fas-trac-iii-suspension-white>: HTTP status code is not handled or not allowed 2026-01-24 20:57:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/built-in-all-freezer-ada-compliant-32-cu-ft-capacity-white>: HTTP status code is not handled or not allowed 2026-01-24 20:57:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silicone-rubber-for-ks-fs455>: HTTP status code is not handled or not allowed 2026-01-24 20:57:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silicone-rubber-sponge-for-wn-500h-wn-500hc>: HTTP status code is not handled or not allowed 2026-01-24 20:57:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cs-6050-up-conveyor-belt>: HTTP status code is not handled or not allowed 2026-01-24 20:57:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-carbon-brush-for-cbs-880fr-770hl-m810frm-810frm-1120cfrm-1010>: HTTP status code is not handled or not allowed 2026-01-24 20:57:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-convex-mirror-w-galvanised-metal-backer-plastic-outdoor-26-dia>: HTTP status code is not handled or not allowed 2026-01-24 20:57:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/colt-electronic-variable-speed-palm-router-1-hp>: HTTP status code is not handled or not allowed 2026-01-24 20:57:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tough-coat-adv-spray-paint-w-rust-barrier-tech-20-oz-cap-galvanizing-gray-primer>: HTTP status code is not handled or not allowed 2026-01-24 20:57:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/par38-led-spotlight-bulb-medium-base-15w-1200-lumens-3000k-warm-white>: HTTP status code is not handled or not allowed 2026-01-24 20:57:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/radius-cone-shaped-bur-single-cut-2-1316l-x-14-shank-dia>: HTTP status code is not handled or not allowed 2026-01-24 20:57:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cylindrical-no-endcut-bur-double-cut-6-34l-x-14-shank-dia-38-cut-dia>: HTTP status code is not handled or not allowed 2026-01-24 20:57:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ball-shaped-bur-double-cut-3l-x-18-shank-dia-18-cut-dia>: HTTP status code is not handled or not allowed 2026-01-24 20:57:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pointed-tree-shaped-bur-double-cut-2-12l-x-14-shank-dia-516-cut-dia>: HTTP status code is not handled or not allowed 2026-01-24 20:57:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pointed-cone-bur-double-cut-2-18l-x-18-shank-dia>: HTTP status code is not handled or not allowed 2026-01-24 20:57:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-shaped-bur-for-stainless-steel-2-12l-x-14-shank-dia>: HTTP status code is not handled or not allowed 2026-01-24 20:57:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bur-box-diamond-cut-sb5-sc3-sc5-sd5-sf3-sf5-sg3-sl4-set-of-8>: HTTP status code is not handled or not allowed 2026-01-24 20:57:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/radius-tree-shaped-bur-single-cut-3l-x-14-shank-dia>: HTTP status code is not handled or not allowed 2026-01-24 20:57:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/radius-cylindrical-bur-double-cut-2-34l-x-14-shank-dia-716-cut-dia>: HTTP status code is not handled or not allowed 2026-01-24 20:57:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ball-shaped-bur-double-cut-6-12l-x-14-shank-dia>: HTTP status code is not handled or not allowed 2026-01-24 20:57:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pl-led-light-bulb-4-pin-g24q-base-16w-1850-lumens-4000k-cool-white>: HTTP status code is not handled or not allowed 2026-01-24 20:57:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-rectangular-up-down-wall-sconce-22w-300-lumens-80-cri-12-size-black>: HTTP status code is not handled or not allowed 2026-01-24 20:57:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-fluorescent-replacement-bulb-120-277v-23w-2800-lumens-4-pin-base-warm-white>: HTTP status code is not handled or not allowed 2026-01-24 20:57:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ptfe-15mm-steel-core-grinding-balls>: HTTP status code is not handled or not allowed 2026-01-24 20:57:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-wall-sconce-light-fixture-15w-850-lumens-80-cri-13-12-size-warm-white>: HTTP status code is not handled or not allowed 2026-01-24 20:57:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drychill-ice-free-cooling-block-6x50ml-tube-capacity>: HTTP status code is not handled or not allowed 2026-01-24 20:57:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compact-balance-300g-capacity-01g-reaability-230v>: HTTP status code is not handled or not allowed 2026-01-24 20:57:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-king-sk2000-step-beam-w-seismic-weld-96w-x-4-34h-6080-pair-capacity-poppy-orange>: HTTP status code is not handled or not allowed 2026-01-24 20:57:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-round-picnic-table-surface-mount-steel-black-36l>: HTTP status code is not handled or not allowed 2026-01-24 20:57:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultracoat-round-picnic-table-span-leg-steel-black-46l>: HTTP status code is not handled or not allowed 2026-01-24 20:57:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xtract-film-disc-360l-p400-grit-5-dia-pack-of-50000>: HTTP status code is not handled or not allowed 2026-01-24 20:57:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clean-sanding-soft-interface-disc-pad-20278-5-dia-x-12-thick-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 20:57:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-add-on-48l-x-40w-x-72h-black>: HTTP status code is not handled or not allowed 2026-01-24 20:57:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-sided-starter-36l-x-16w-x-48h-black>: HTTP status code is not handled or not allowed 2026-01-24 20:57:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-sided-starter-36l-x-16w-x-54h-black>: HTTP status code is not handled or not allowed 2026-01-24 20:57:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/end-unit-w-t-joiner-support-clip-36l-x-20w-x-96h-white>: HTTP status code is not handled or not allowed 2026-01-24 20:57:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-starter-36l-x-32w-x-60h-black>: HTTP status code is not handled or not allowed 2026-01-24 20:57:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-starter-48l-x-40w-x-72h-black>: HTTP status code is not handled or not allowed 2026-01-24 20:57:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/end-unit-w-t-joiner-support-clip-48l-x-14w-x-48h-antique-white>: HTTP status code is not handled or not allowed 2026-01-24 20:57:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-air-conditioner-10000-btu-1120w-115v-white>: HTTP status code is not handled or not allowed 2026-01-24 20:57:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8251hdz-bk-two-tone-hi-vis-safety-vest-ansi-class-2-2xl3xl-lime>: HTTP status code is not handled or not allowed 2026-01-24 20:57:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mmb-ytx20-bs-12v-18ah-270cca-battery>: HTTP status code is not handled or not allowed 2026-01-24 20:57:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/skullerz-osmin-afas-half-frame-safety-glasses-indooroutdoor-lens-matte-black-frame>: HTTP status code is not handled or not allowed 2026-01-24 20:57:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/skullerz-osmin-half-frame-safety-glasses-smoke-lens-matte-black-frame>: HTTP status code is not handled or not allowed 2026-01-24 20:57:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spout-cap-tan-small-34408>: HTTP status code is not handled or not allowed 2026-01-24 20:57:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h1-2-bar-aluminum-ladder-rack-system-for-chevy-express-1996-on-black>: HTTP status code is not handled or not allowed 2026-01-24 20:58:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/first-aid-supply-hip-pack-w-guard-logo-mesh-drain-clear-49-pieces>: HTTP status code is not handled or not allowed 2026-01-24 20:58:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-mega-swivel-anchor-for-steel-applications>: HTTP status code is not handled or not allowed 2026-01-24 20:58:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/protect-liquid-cooled-standby-generator-130kw-120-208v-3-phase-lp>: HTTP status code is not handled or not allowed 2026-01-24 20:58:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-16bs21x35-metric-1-pitch-35mm-finished-bore-21-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:58:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=protect-liquid-cooled-standby-generator-150kw-120-240v-3-phase-ng already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:58:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ceramic-30-inch--digital-space-saving-pedestal-heater-w-digital-remote-1500w-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:58:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dryconn-waterproof-wire-connectors-small-aqua-orange-6-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:58:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pediatric-head-immobilizer-royal-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:58:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-20a45-metric-a-plate-1-14-pitch-32mm-bore-45-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:58:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-24bs11hx35-metric-1-12-pitch-35mm-finished-bore-11-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:58:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compact-controlled-room-temperature-cabinet-1-cuft-capacity-solid-door>: HTTP status code is not handled or not allowed 2026-01-24 20:58:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 21963 pages (at 93 pages/min), scraped 12696 items (at 21 items/min) 2026-01-24 20:58:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80bs34x158-1-pitch-1-58-finished-bore-34-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:58:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microfiber-cleaning-cloths-16-inch-x-16-inch-pink-24-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:58:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trailgrip-work-athletic-hiker-boots-alloy-toe-size-13w-4-inch-h-coyote-black>: HTTP status code is not handled or not allowed 2026-01-24 20:58:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trailgrip-work-athletic-hiker-boots-alloy-toe-size-7w-4-inch-h-coyote-black>: HTTP status code is not handled or not allowed 2026-01-24 20:58:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60bs28hx1-34-pitch-1-finished-bore-28-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:58:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foodservice-towels-1-ply-12-1-4-inch-x-21-inch-white-red-stripe-200-carton>: HTTP status code is not handled or not allowed 2026-01-24 20:58:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=protect-liquid-cooled-standby-generator-150kw-120-240v-3-phase-ng>: HTTP status code is not handled or not allowed 2026-01-24 20:58:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strong-tall-kitchen-and-trash-bags-13-gal-0-9-mil-white-40-box>: HTTP status code is not handled or not allowed 2026-01-24 20:58:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-120e19h-qd-bushed-1-12-pitch-19-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:58:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/select-flat-box-facial-tissue-2-ply-white-100-sheets-box-30-boxes-carton>: HTTP status code is not handled or not allowed 2026-01-24 20:58:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:58:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-140btl16-1-34-pitch-taper-bushed-16-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:58:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-24bs14hx32-metric-1-12-pitch-32mm-finished-bore-14-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:58:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-24bs16hx42-metric-1-12-pitch-42mm-finished-bore-16-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:58:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bike-fixation-outdoor-public-bike-pump-with-gauge-hose-floor-mount>: HTTP status code is not handled or not allowed 2026-01-24 20:58:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dock-shelter-guard-post-steel-42-h-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:58:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-safety-crafted-waterproof-logger-work-boots-steel-toe-size-6-5w-8-inch-h-dark-brown>: HTTP status code is not handled or not allowed 2026-01-24 20:58:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-safety-crafted-waterproof-logger-work-boots-steel-toe-size-10w-8-inch-h-dark-brown>: HTTP status code is not handled or not allowed 2026-01-24 20:58:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-100bs24hx114-1-14-pitch-1-14-finished-bore-24-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:58:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/red-dot-ecg-monitoring-electrodes-0-9-inch-x-0-9-inch-neonatal-radiolucent-3-bg-100-bg-cs>: HTTP status code is not handled or not allowed 2026-01-24 20:58:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steri-drape-adhesive-split-sheet-1094-89-inch-x-125-inch-tube-and-cord-organizer-20-cs>: HTTP status code is not handled or not allowed 2026-01-24 20:58:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steri-drape-laparoscopy-drape-1032-95-inch-4-tube-organizer-8-bx-2-bx-cs>: HTTP status code is not handled or not allowed 2026-01-24 20:58:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-safety-crafted-waterproof-work-boots-steel-toe-size-11-5w-6-inch-h-dark-brown>: HTTP status code is not handled or not allowed 2026-01-24 20:58:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots>: HTTP status code is not handled or not allowed 2026-01-24 20:58:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steri-drape-adhesive-split-sheet-9045-102-inch-x-89-inch-15-each-carton-2-carton-case>: HTTP status code is not handled or not allowed 2026-01-24 20:58:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:58:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zig-elusion-heritage-work-high-top-sneaker-composite-toe-size-9m-black-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:58:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steri-drape-long-u-drape-1019-47-inch-x-70-inch-10-each-carton-4-carton-case>: HTTP status code is not handled or not allowed 2026-01-24 20:58:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steri-drape-orthopaedic-surgical-pack-9051-72-inch-x-124-inch-15-each-carton-2-carton-case>: HTTP status code is not handled or not allowed 2026-01-24 20:58:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bair-hugger-plus-warming-gown-kit-84202-x-large-20-case>: HTTP status code is not handled or not allowed 2026-01-24 20:58:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/loban-2-antimicrobial-incise-drape-6650ez-23-inch-x-17-inch-10-each-carton-4-cartons-case>: HTTP status code is not handled or not allowed 2026-01-24 20:58:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bair-hugger-warming-gown-81101-small-44-inch-l-20-case>: HTTP status code is not handled or not allowed 2026-01-24 20:58:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zig-elusion-heritage-work-high-top-sneaker-composite-toe-size-10m-black-blue>: HTTP status code is not handled or not allowed 2026-01-24 20:58:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-100bs20hx1716-1-14-pitch-1-716-finished-bore-20-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:58:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-flowtubes-with-standard-cap-sterile-5-ml-500-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:58:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-propette-le-12-channel-pipette-30-300ul>: HTTP status code is not handled or not allowed 2026-01-24 20:58:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-24 20:58:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-square-shape-weigh-boat-80-x-80-mm-100-ml-polystyrene-black-500-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:58:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-propette-reach-long-neck-pipette-controller-with-quick-stand-115v>: HTTP status code is not handled or not allowed 2026-01-24 20:58:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vitals-skate-inspired-work-shoes-composite-toe-size-7-5w-wintermoss-whitecap-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:58:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zig-elusion-heritage-work-women-s-low-cut-sneaker-composite-toe-size-11-5m-black>: HTTP status code is not handled or not allowed 2026-01-24 20:58:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-12a12h-metric-a-plate-34-pitch-14mm-bore-12-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:58:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:58:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-screw-cap-macrotube-sterile-bags-of-100-5-ml-500-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:58:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35bs34hx1-38-pitch-1-finished-bore-34-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:58:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-storage-box-with-hinged-lid-for-1-5-ml-tubes-100-place-blue-5-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:58:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-carousel-for-6-halo-pipettes>: HTTP status code is not handled or not allowed 2026-01-24 20:58:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:58:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-duoclick-culture-tubes-sterile-14-ml-10-foam-racks-of-50-tubes>: HTTP status code is not handled or not allowed 2026-01-24 20:58:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-reservoir-for-multi-channel-pipettes-sterile-25-ml-200-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:58:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/true-skate-inspired-work-shoes-composite-toe-size-13w-black-gum>: HTTP status code is not handled or not allowed 2026-01-24 20:58:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-small-bulb-transfer-pipette-with-extended-tip-sterile-3-ml-500-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:58:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-reservoir-for-multi-channel-pipettes-100-ml-sterile-100-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:58:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60a14hx34-a-plate-34-pitch-34-bore-14-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:58:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-aspir-8-reservoir-for-8-channel-pipettes-25-ml-sterile-100-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:58:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-24 20:58:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/true-skate-inspired-work-shoes-composite-toe-size-5-5m-black>: HTTP status code is not handled or not allowed 2026-01-24 20:58:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-mx-z-440-lc-437cc-snowmobile-replacement-battery-12v-i>: HTTP status code is not handled or not allowed 2026-01-24 20:58:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/true-skate-inspired-work-shoes-composite-toe-size-9-5w-black>: HTTP status code is not handled or not allowed 2026-01-24 20:58:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-summit-xrs-793cc-snowmobile-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 20:58:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35bs25hx58-38-pitch-58-finished-bore-25-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:58:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 20:58:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/truflex-fly-blucher-sneaker-composite-toe-size-10-5w-black-gold>: HTTP status code is not handled or not allowed 2026-01-24 20:58:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pgo-m2-50-50cc-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 20:58:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:58:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-mach-z-774cc-snowmobile-replacement-battery-12v-i>: HTTP status code is not handled or not allowed 2026-01-24 20:59:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-crafted-western-work-boots-w-steel-toe-and-internal-met-guard-size-10w-brown>: HTTP status code is not handled or not allowed 2026-01-24 20:59:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-crafted-western-work-boots-w-steel-toe-and-internal-met-guard-size-6w-brown>: HTTP status code is not handled or not allowed 2026-01-24 20:59:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-crafted-western-work-boots-w-steel-toe-and-internal-met-guard-size-10m-brown>: HTTP status code is not handled or not allowed 2026-01-24 20:59:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60bs12hx1-34-pitch-1-finished-bore-12-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:59:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-crafted-western-work-boots-steel-toe-size-15m-brown-burnt-orange>: HTTP status code is not handled or not allowed 2026-01-24 20:59:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35bs28hx1316-38-pitch-1-316-finished-bore-28-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:59:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35a45x58-a-plate-38-pitch-58-bore-45-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:59:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sealed-lead-acid---agm---vrla-battery-3-2ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 20:59:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-10bs20hx35-metric-58-pitch-35mm-finished-bore-20-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:59:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stone-skate-inspired-work-shoes-composite-toe-size-4-5m-blue-navy>: HTTP status code is not handled or not allowed 2026-01-24 20:59:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60a20hx34-a-plate-34-pitch-34-bore-20-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:59:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stone-skate-inspired-work-shoes-composite-toe-size-11m-gray-black>: HTTP status code is not handled or not allowed 2026-01-24 20:59:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-08a38-metric-a-plate-12-pitch-15mm-bore-38-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:59:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots>: HTTP status code is not handled or not allowed 2026-01-24 20:59:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 22056 pages (at 93 pages/min), scraped 12713 items (at 17 items/min) 2026-01-24 20:59:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/marquette-32132-medical-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 20:59:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-bags-8-x-10-2-mil-black-1000-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:59:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pastel-colors-tissue-paper-480-sheet-assortment-pack-20-x-30>: HTTP status code is not handled or not allowed 2026-01-24 20:59:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nellcor-pb2801-medical-replacement-battery-4-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 20:59:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/philips-a212-28g6-medical-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 20:59:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premier-work-casual-oxford-shoes-steel-toe-size-7d-navy>: HTTP status code is not handled or not allowed 2026-01-24 20:59:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/philips-str112053-medical-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 20:59:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50bs19hx138-58-pitch-1-38-finished-bore-19-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:59:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nanoflex-tr-athletic-work-shoes-composite-toe-size-5-5w-black-seafoam-green-white>: HTTP status code is not handled or not allowed 2026-01-24 20:59:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-epp-alarm-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 20:59:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nanoflex-tr-athletic-work-shoes-composite-toe-size-14w-navy-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:59:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caterpillar-112-industrial-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 20:59:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stone-op-art-skate-inspired-work-shoes-composite-toe-size-13w-dark-gray-charcoal>: HTTP status code is not handled or not allowed 2026-01-24 20:59:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nanoflex-tr-athletic-work-shoes-composite-toe-size-6w-black-seafoam-green-white>: HTTP status code is not handled or not allowed 2026-01-24 20:59:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stone-op-art-skate-inspired-work-shoes-composite-toe-size-10-5m-dark-brown-khaki>: HTTP status code is not handled or not allowed 2026-01-24 20:59:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-air-bubble-rolls-16-x-375-x-5-16-3-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:59:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stone-op-art-skate-inspired-work-shoes-composite-toe-size-7w-dark-gray-charcoal>: HTTP status code is not handled or not allowed 2026-01-24 20:59:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-12bs14hx20-metric-34-pitch-20mm-finished-bore-14-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:59:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 20:59:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-12bs14hx32-metric-34-pitch-32mm-finished-bore-14-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:59:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adi-9092w-alarm-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 20:59:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-40bs24hx1716-12-pitch-1-716-finished-bore-24-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:59:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-twist-ties-5-x-5-32-green-2000-pack>: HTTP status code is not handled or not allowed 2026-01-24 20:59:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/construction-harness-tongue-buckle-legs-small>: HTTP status code is not handled or not allowed 2026-01-24 20:59:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rb3315-nanoflex-tr-athletic-work-shoes-composite-toe-size-3-5m-black>: HTTP status code is not handled or not allowed 2026-01-24 20:59:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/severance-chatter-free-stop-countersink-cutter-82-5-8-7-32-pilot-hole>: HTTP status code is not handled or not allowed 2026-01-24 20:59:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rb3315-nanoflex-tr-athletic-work-shoes-composite-toe-size-11-5m-black>: HTTP status code is not handled or not allowed 2026-01-24 20:59:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/napco-gema1000e4lb-alarm-replacement-battery-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 20:59:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nanoflex-tr-athletic-work-shoes-composite-toe-size-8w-black-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:59:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nanoflex-tr-athletic-work-shoes-composite-toe-size-8-5w-black-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:59:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nanoflex-tr-athletic-work-shoes-composite-toe-size-8-5m-black-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:59:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nano-x1-adventure-athletic-work-shoes-composite-toe-size-7-5w-black>: HTTP status code is not handled or not allowed 2026-01-24 20:59:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-10bs23hx32-metric-58-pitch-32mm-finished-bore-23-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:59:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 20:59:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nanoflex-tr-athletic-work-shoes-composite-toe-size-13m-black-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:59:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nanoflex-tr-athletic-work-shoes-composite-toe-size-12m-black-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:59:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/altronix-al600ulpd8-alarm-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 20:59:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nano-x1-adventure-athletic-work-shoes-composite-toe-size-11-5w-black>: HTTP status code is not handled or not allowed 2026-01-24 20:59:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-41a48x34-a-plate-12-pitch-34-bore-48-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:59:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80btl14h-bushed-tapered-locks-1-pitch-14-teeth-14-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:59:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/591-portable-blower-power-fishing-system>: HTTP status code is not handled or not allowed 2026-01-24 20:59:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adi-pwps12260f-alarm-replacement-battery-26ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 20:59:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96-w-x-30-d-pedestal-workbench-w-4-drawers-cabinet-plastic-laminate-square-edge-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:59:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35a32hx12-a-plate-38-pitch-12-bore-32-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:59:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60bs09hx58-34-pitch-58-finished-bore-9-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:59:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultratech-ut12180-alarm-replacement-battery-9ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 20:59:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/altronix-maxim33-alarm-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 20:59:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-shelves-48-w-storage-cabinet-tan>: HTTP status code is not handled or not allowed 2026-01-24 20:59:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60bs34x1316-34-pitch-1-316-finished-bore-34-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:59:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paper-adhesive-tape-1-x-10-yds-white-12pack>: HTTP status code is not handled or not allowed 2026-01-24 20:59:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chloride-power-d2mf50-alarm-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 20:59:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nano-x1-adventure-athletic-work-mid-cut-shoes-composite-toe-size-12m-black>: HTTP status code is not handled or not allowed 2026-01-24 20:59:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/invacare-atm-wheelchair-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 20:59:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/damaco-d99-wheelchair-replacement-battery-18ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 20:59:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merits-mpii-a-wheelchair-replacement-battery-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 20:59:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acme-rb4-home-alarm-alarm-replacement-battery-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 20:59:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merits-sp23-wheelchair-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 20:59:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merits-p320-wheelchair-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 20:59:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hook-n-loop-tape-fasteners-green-3-8-inch-x-75-feet>: HTTP status code is not handled or not allowed 2026-01-24 20:59:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quickie-p-220-wheelchair-replacement-battery-75ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 20:59:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lavante-trail-2-running-work-shoes-composite-toe-size-11-5m-gray-black>: HTTP status code is not handled or not allowed 2026-01-24 20:59:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lavante-trail-2-running-work-shoes-composite-toe-size-8m-army-green-black-yellow>: HTTP status code is not handled or not allowed 2026-01-24 20:59:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-hook-with-acrylic-adhesive-half-inch-x-75-feet>: HTTP status code is not handled or not allowed 2026-01-24 20:59:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/91173-food-storage-container-wlid-hdpe-422-qt-stackable-transparent>: HTTP status code is not handled or not allowed 2026-01-24 20:59:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/toro-te12-lawn-and-garden-replacement-battery-9ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 20:59:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sears-25780-lawn-and-garden-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 20:59:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35bs17hx1-38-pitch-1-finished-bore-17-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:59:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35bs17hx12-38-pitch-12-finished-bore-17-teeth>: HTTP status code is not handled or not allowed 2026-01-24 20:59:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/41500-food-transport-container-polyethylene-507-qt-insulated-gray>: HTTP status code is not handled or not allowed 2026-01-24 20:59:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/09834-food-pan-polycarbonate-119-qt-stackable-clear>: HTTP status code is not handled or not allowed 2026-01-24 21:00:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-40bs10hx34-12-pitch-34-finished-bore-10-teeth>: HTTP status code is not handled or not allowed 2026-01-24 21:00:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:00:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tork-661-emergency-light-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:00:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elan-st2-emergency-light-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:00:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-40bs12hx78-12-pitch-78-finished-bore-12-teeth>: HTTP status code is not handled or not allowed 2026-01-24 21:00:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merits-pioneer-wheelchair-replacement-battery-75ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:00:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adtech-bulk-hot-melt-glue-tan-950-viscosity-135f-57c>: HTTP status code is not handled or not allowed 2026-01-24 21:00:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hivis-insulated-softshell-high-bib-black-lime-10f-comfort-rating-l>: HTTP status code is not handled or not allowed 2026-01-24 21:00:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hybrid-skate-inspired-work-shoes-composite-toe-size-7w-navy-celestial-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:00:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rotary-screw-air-compressor-wdryer-460v-115-psi-185-cfm-5hp-3ph>: HTTP status code is not handled or not allowed 2026-01-24 21:00:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extreme-softshell-high-bib-black-60f-comfort-rating-m>: HTTP status code is not handled or not allowed 2026-01-24 21:00:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extreme-softshell-high-bib-black-60f-comfort-rating-xl>: HTTP status code is not handled or not allowed 2026-01-24 21:00:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rotary-screw-air-compressor-wdryer-460v-140-psi-50-cfm-15hp-3ph>: HTTP status code is not handled or not allowed 2026-01-24 21:00:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 22153 pages (at 97 pages/min), scraped 12730 items (at 17 items/min) 2026-01-24 21:00:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gs-gt6b-3-powersports-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:00:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evolve-skate-inspired-work-shoes-composite-toe-size-14w-olive-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:00:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers>: HTTP status code is not handled or not allowed 2026-01-24 21:00:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mule-gc690-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:00:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-quest-opt-500cc-atv-replacement-battery-12v-i>: HTTP status code is not handled or not allowed 2026-01-24 21:00:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hinge-zinc-die-cast-157-157-inches>: HTTP status code is not handled or not allowed 2026-01-24 21:00:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evolve-skate-inspired-work-shoes-composite-toe-size-8m-olive-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:00:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evolve-skate-inspired-high-top-work-shoes-composite-toe-size-10w-triple-black>: HTTP status code is not handled or not allowed 2026-01-24 21:00:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mojo-hawk-decoy-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:00:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-43505-bl-3505-series-5-adjustable-frame-41w-x-27d-x-32-37h-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:00:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honeywell-motorized-zone-valve-v8043e1004-24v-35-cv-12-sweat>: HTTP status code is not handled or not allowed 2026-01-24 21:00:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pressboard-report-cover-prong-clip-letter-3-capacity-dark-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:00:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/side-opening-pressguard-report-cover-prong-fastener-letter-black>: HTTP status code is not handled or not allowed 2026-01-24 21:00:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sym-hd-200-motorcycle-replacement-battery-2013-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:00:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honeywell-steam-vaporstat-controller-wgrounding-screw-l408j1017-14-female-15-psi-max>: HTTP status code is not handled or not allowed 2026-01-24 21:00:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evolve-skate-inspired-work-shoes-composite-toe-size-9-5w-navy>: HTTP status code is not handled or not allowed 2026-01-24 21:00:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evolve-skate-inspired-work-shoes-composite-toe-size-13w-navy>: HTTP status code is not handled or not allowed 2026-01-24 21:00:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mbk-stunt-50cc-motorcycle-replacement-battery-2001-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:00:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interstate-faytz14s-powersports-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:00:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/daystart-slip-on-work-shoes-steel-toe-size-7w-navy>: HTTP status code is not handled or not allowed 2026-01-24 21:00:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sigmastek-stx20hl-bs-powersports-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:00:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bb4500-high-top-work-sneaker-composite-toe-size-7w-black-white>: HTTP status code is not handled or not allowed 2026-01-24 21:00:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sh1230-steel-shelf-12-d-x-30-w-with-2-brackets>: HTTP status code is not handled or not allowed 2026-01-24 21:00:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/daystart-slip-on-work-shoes-steel-toe-size-10-5w-navy>: HTTP status code is not handled or not allowed 2026-01-24 21:00:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-frame-motor-wwht40-18-324tc-f2-gp-tefc-rigid-3-ph-f2-324tc-485-fla>: HTTP status code is not handled or not allowed 2026-01-24 21:00:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/btlslssh101p-98-bottleless-stainless-steel-commercial-hot-cold-water-cooler-w-filtration>: HTTP status code is not handled or not allowed 2026-01-24 21:00:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/comfortie-casual-work-shoes-steel-toe-size-9m-black-white>: HTTP status code is not handled or not allowed 2026-01-24 21:00:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:00:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/d-flange-kit-w280td-284t-286t-8-pole>: HTTP status code is not handled or not allowed 2026-01-24 21:00:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pc101b-54-commercial-room-temperature-cold-water-cooler-dispenser-silver>: HTTP status code is not handled or not allowed 2026-01-24 21:00:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keyspan-4-port-high-speed-serial-to-usb-adapter-hub-6-usb-cable>: HTTP status code is not handled or not allowed 2026-01-24 21:00:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/z-grip-retractable-ballpoint-pen-blue-ink-medium-24-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:00:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enviro-world-freegarden-rain-rain-barrel-system-ewc-10>: HTTP status code is not handled or not allowed 2026-01-24 21:00:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-lcd-control-panel-for-48-in-portable-evaporative-cooler-604152>: HTTP status code is not handled or not allowed 2026-01-24 21:00:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mbk-ovetto-50cc-motorcycle-replacement-battery-2001-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:00:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/permobil-400ps-mobility-scooter-replacement-battery-75ah-12v-it>: HTTP status code is not handled or not allowed 2026-01-24 21:00:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-frame-motor-wwht20-12-286tc-gp-tefc-rigid-c-3-ph-286tc-263-fla>: HTTP status code is not handled or not allowed 2026-01-24 21:00:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-gtx-600-ho-sdi-593cc-snowmobile-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:00:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-instrument-table-with-lower-shelf-36x20x34-987207>: HTTP status code is not handled or not allowed 2026-01-24 21:00:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 21:00:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yamaha-vmax-700-xtcp-700cc-snowmobile-replacement-battery-12v-i>: HTTP status code is not handled or not allowed 2026-01-24 21:00:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-gtx-le600-595cc-snowmobile-replacement-battery-2009-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:00:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hydraulic-lift-furniture-mover-3950-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:00:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdmi-over-dual-cat5-cat6-extender-wall-plate-kit-f-f>: HTTP status code is not handled or not allowed 2026-01-24 21:00:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carton-sealing-tape-stop-if-seal-is-broken-3-x-110-yds-redtan-pack-of-24>: HTTP status code is not handled or not allowed 2026-01-24 21:00:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dunkin-french-vanilla-coffee-regular-medium-ground-k-cup-pods-037-oz-capacity-pack-of-22>: HTTP status code is not handled or not allowed 2026-01-24 21:00:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mark-bracket-for-wall-mounting-of-fire-extinguisher-for-model-galaxy-5-12-mercury-5-cosmic-5>: HTTP status code is not handled or not allowed 2026-01-24 21:00:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/greenline-636-hand-truck-w-dual-handle-solid-rubber-wheels-600-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:00:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/greenline-636-hand-truck-w-dual-handle-mold-on-rubber-wheels-600-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:00:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/greenline-636-hand-truck-w-dual-handle-semi-pneumatic-wheels-600-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:00:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbi-sala-nano-lok-xl-self-retracting-lifeline-w-quick-connect-aluminum-carabiner-11l>: HTTP status code is not handled or not allowed 2026-01-24 21:00:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-hand-truck-w-curved-continuous-handle-mold-on-rubber-wheels-900-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:00:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electronic-time-clock-blue-white-tcx-85-a189>: HTTP status code is not handled or not allowed 2026-01-24 21:00:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/greenline-626d-hand-truck-w-single-loop-handle-solid-rubber-wheels-600-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:00:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/900-economy-carton-sealing-tape-2-x-55-yds-tan-pack-of-36>: HTTP status code is not handled or not allowed 2026-01-24 21:00:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rotary-screw-air-compressor-w-dryer-200v-115psi-185cfm-5hp-3ph-120-gal-tank>: HTTP status code is not handled or not allowed 2026-01-24 21:00:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acme-medical-system-7300-medical-replacement-battery-1-3ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:00:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/amstar-ams2000led-jump-starter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:00:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ajc-teledyne-118-0013-6v-12ah-emergency-light-battery>: HTTP status code is not handled or not allowed 2026-01-24 21:00:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/parks-medical-00-mini-lab-medical-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:00:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/digitech-safety-view-plus-4-tier-12-doors-box-locker-36w-x-18d-x-78h-tan-assembled>: HTTP status code is not handled or not allowed 2026-01-24 21:00:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/digitech-safety-view-plus-4-tier-4-doors-box-locker-12w-x-18d-x-78h-tan-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 21:00:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/positector-200-b-standard-ultrasonic-coating-thickness-gage-non-metal-substrates>: HTTP status code is not handled or not allowed 2026-01-24 21:00:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/digitech-safety-view-plus-4-tier-12-doors-wardrobe-locker-36w-x-12d-x-78h-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 21:00:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/desk-cabinet-corner-mover-dollies-set-of-4-1320-cap-lbs-set-987220>: HTTP status code is not handled or not allowed 2026-01-24 21:00:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/epdm-commercial-grade-60a-rubber-sheet-3-16-thick-x-6-x-6-length-black-5-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:00:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-2-tier-6-doors-locker-54w-x-21d-x-78h-tan-assembled>: HTTP status code is not handled or not allowed 2026-01-24 21:00:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/marquette-1500-respond-medical-replacement-battery-1-3ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:00:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bigboy-2000-folding-saw-360mm-extra-large-teeth>: HTTP status code is not handled or not allowed 2026-01-24 21:00:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scheduled-maintenance-kit-for-8kw-standby-generator-2013-or-later>: HTTP status code is not handled or not allowed 2026-01-24 21:00:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yellow-post-safety-barrier-15-ft-yellow-black-caution-do-not-enter>: HTTP status code is not handled or not allowed 2026-01-24 21:00:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/health-o-meter-professional-connectivity-kit-for-welch-allyn-lxi-monitors>: HTTP status code is not handled or not allowed 2026-01-24 21:00:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/health-o-meter-120v-ac-adapter-for-professional-digital-scales>: HTTP status code is not handled or not allowed 2026-01-24 21:00:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-waist-high-digital-scale-600lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:01:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sbr-red-65a-sheet-3-16-thick-x-3-x-10-length>: HTTP status code is not handled or not allowed 2026-01-24 21:01:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bb-bpl33-12-sealed-lead-acid-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:01:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aleco-anchorbar-chairmat-rectangle-46w-x-60l-premium-thick-beveled-edge>: HTTP status code is not handled or not allowed 2026-01-24 21:01:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/see-thru-sensory-table-toddler>: HTTP status code is not handled or not allowed 2026-01-24 21:01:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/calm75-ds-double-output-shaft-for-calm-series-75mm-aluminum-worm-gear-reducer>: HTTP status code is not handled or not allowed 2026-01-24 21:01:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kydz-ladderback-chair-12-height>: HTTP status code is not handled or not allowed 2026-01-24 21:01:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/calm90-251-56c-aluminum-worm-gear-reducer-90mm-251-56c-nema-frame>: HTTP status code is not handled or not allowed 2026-01-24 21:01:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gator-grip-power-lift-drive-ergonomic-drum-handler-987237>: HTTP status code is not handled or not allowed 2026-01-24 21:01:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-resin-patio-fence-3-panel-section-brown-238684bn>: HTTP status code is not handled or not allowed 2026-01-24 21:01:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/calm30-751-56c-aluminum-worm-gear-reducer-30mm-751-56c-nema-frame>: HTTP status code is not handled or not allowed 2026-01-24 21:01:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 22252 pages (at 99 pages/min), scraped 12746 items (at 16 items/min) 2026-01-24 21:01:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leoch-lp12-30-sealed-lead-acid-replacement-battery-26ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:01:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/976s02-catwalk-72-high-unit-walkway-24-x-48>: HTTP status code is not handled or not allowed 2026-01-24 21:01:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-step-heavy-duty-double-entry-work-platform-36w-steps-dep7-3648>: HTTP status code is not handled or not allowed 2026-01-24 21:01:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exell-eb12180-sealed-lead-acid-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:01:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jerry-can-w-spout-adapter-green-10-liter-3014>: HTTP status code is not handled or not allowed 2026-01-24 21:01:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jerry-can-w-spout-adapter-red-5-liter-3015>: HTTP status code is not handled or not allowed 2026-01-24 21:01:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tempest-tr712-sealed-lead-acid-replacement-battery-9ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 21:01:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aprilaire-70-pint-per-day-whole-home-hehumidifer-1830>: HTTP status code is not handled or not allowed 2026-01-24 21:01:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/casil-ca12100-sealed-lead-acid-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 21:01:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/best-sla12180-sealed-lead-acid-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:01:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/outdoor-dock-storage-box-82119569-small-36-l-x-21-w-x-16-h-white>: HTTP status code is not handled or not allowed 2026-01-24 21:01:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-in-high-galvanized-fixed-height-adjustable-width-louver>: HTTP status code is not handled or not allowed 2026-01-24 21:01:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yuasa-npx-80-sealed-lead-acid-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:01:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sigmastek-sp6-6-sealed-lead-acid-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:01:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shockstrap-tie-downs-ratchet-strap-3000-lb-capacity-7l-x-1-12w-black>: HTTP status code is not handled or not allowed 2026-01-24 21:01:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mighty-products-mn-tm-b0620-6-x-20-tennis-court-wind-screen>: HTTP status code is not handled or not allowed 2026-01-24 21:01:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yuasa-np1812b-sealed-lead-acid-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:01:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cove-base-vinyl-4x18x120-coil-pewter>: HTTP status code is not handled or not allowed 2026-01-24 21:01:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-tile-low-profile-circular-design-50cm-black>: HTTP status code is not handled or not allowed 2026-01-24 21:01:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cove-base-700-series-tpr-4x18x48-steel-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:01:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cove-base-pinnacle-rubber-4x18x120-coil-blackbrown>: HTTP status code is not handled or not allowed 2026-01-24 21:01:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-house-650net-12v-45ah-ups-battery>: HTTP status code is not handled or not allowed 2026-01-24 21:01:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/29438e-spherical-roller-thrust-bearing-extra-capacity-bronze-cage>: HTTP status code is not handled or not allowed 2026-01-24 21:01:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enclosed-shop-desk-psx-2436-3rbkdr-24-x-36-with-locking-storage-drawer>: HTTP status code is not handled or not allowed 2026-01-24 21:01:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/maintenance-free-quiet-2-tier-2-doors-locker-18w-x-24d-x-78h-dark-gray-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 21:01:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergonomic-adj-height-pallet-stand-pdse4848-solid-deck-48-x-48-with-floor-locks>: HTTP status code is not handled or not allowed 2026-01-24 21:01:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fashion-seal-unisex-knot-button-lab-coat-39-12l-100-cotton-size-s-white>: HTTP status code is not handled or not allowed 2026-01-24 21:01:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manual-chain-hoist-w-overload-protection-10-ton-capacity-30-lift>: HTTP status code is not handled or not allowed 2026-01-24 21:01:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/upsonic-is-1000-sealed-lead-acid-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:01:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/altronix-al600ulxpd16cb-alarm-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 21:01:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n202436-2-shelving-unit-2-shelf-20w-x-24h-x-36l-aluminum>: HTTP status code is not handled or not allowed 2026-01-24 21:01:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-in-close-black-steel-pipe-nipple-150-psi-lead-free>: HTTP status code is not handled or not allowed 2026-01-24 21:01:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n184860-3-equipment-stands-3-shelf-shelving-unit-18w-x-48h-x-60l-aluminum>: HTTP status code is not handled or not allowed 2026-01-24 21:01:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/volt153-portable-electric-heater-w-thermostat-240v-1-phase-3000w>: HTTP status code is not handled or not allowed 2026-01-24 21:01:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-in-galvanized-malleable-cap-150-psi-lead-free>: HTTP status code is not handled or not allowed 2026-01-24 21:01:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/altronix-al600ulacmcbj-alarm-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 21:01:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-in-galvanized-malleable-tee-150-psi-lead-free>: HTTP status code is not handled or not allowed 2026-01-24 21:01:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ge-caddx-networx-nx-6-alarm-replacement-battery-4-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:01:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gravity-fed-eyewash-insulated-jacket-for-10-gallon-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:01:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-piece-peg-hook-set-large-hooks>: HTTP status code is not handled or not allowed 2026-01-24 21:01:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ibmotrin-pain-reliever-50-bx>: HTTP status code is not handled or not allowed 2026-01-24 21:01:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/98898c-kit-ez-hardware-for-panels>: HTTP status code is not handled or not allowed 2026-01-24 21:01:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dyno-lever-hoist-9-ton-capacity-15-lift>: HTTP status code is not handled or not allowed 2026-01-24 21:01:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sunnyway-sw1223w-sealed-lead-acid-replacement-battery-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:01:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/instrument-storage-tray-polypropylene-small-white-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-24 21:01:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-ribbon-for-mjr-7000-time-clock>: HTTP status code is not handled or not allowed 2026-01-24 21:01:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reagent-bottle-wide-mouth-pp-1000ml-capacity-clear-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-24 21:01:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guardian-trek-3-wheelchair-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 21:01:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/62202rs-deep-groove-ball-bearing-double-sealed-100mm-bore-180mm-od>: HTTP status code is not handled or not allowed 2026-01-24 21:01:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/32207-tapered-roller-bearing-metric-35mm-bore-72mm-od>: HTTP status code is not handled or not allowed 2026-01-24 21:01:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/siphon-pump-transfer-most-liquids-pumps-air-for-inflatables-siphons-gas-oil-etc>: HTTP status code is not handled or not allowed 2026-01-24 21:01:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10000-lbs-capacity-tow-strap-1-3-4-inch-x-25-ft-forged-hooks>: HTTP status code is not handled or not allowed 2026-01-24 21:01:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/63092rsnr-deep-groove-ball-bearing-double-sealed-snap-ring-45mm-bore-100mm-od>: HTTP status code is not handled or not allowed 2026-01-24 21:01:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60112rs-deep-groove-ball-bearing-double-sealed-55mm-bore-90mm-od>: HTTP status code is not handled or not allowed 2026-01-24 21:01:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/permobil-x850-corpus-wheelchair-replacement-battery-75ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:01:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-tier-test-tube-rack-plastic-18-places-white-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-24 21:01:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/calibration-weight-hooked-1000g>: HTTP status code is not handled or not allowed 2026-01-24 21:01:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/62002rs-p53-deep-groove-ball-bearing-double-sealed-abec-5-10mm-bore-30mm-od>: HTTP status code is not handled or not allowed 2026-01-24 21:01:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wooden-test-tube-rack-12-places-tan>: HTTP status code is not handled or not allowed 2026-01-24 21:01:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/production-booster-463t72bk-72w-x-36h-3-shelves-black>: HTTP status code is not handled or not allowed 2026-01-24 21:01:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/teardrop-bolted-pallet-rack-starter-108w-x-42d-x-96h>: HTTP status code is not handled or not allowed 2026-01-24 21:01:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fortress-1600acv-u1-wheelchair-replacement-battery-18ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:01:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-aerator-aluminum-shaft-with-silicon-lip-micro>: HTTP status code is not handled or not allowed 2026-01-24 21:01:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-tier-12-door-digital-wood-locker-36-w-x-15-d-x-72-h-cherry-assembled>: HTTP status code is not handled or not allowed 2026-01-24 21:01:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-tier-1-door-digital-wood-locker-12-w-x-15-d-x-72-h-cherry-assembled>: HTTP status code is not handled or not allowed 2026-01-24 21:01:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fortress-1600acv-u1-wheelchair-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:02:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6305-deep-groove-ball-bearing-open-25mm-bore-62mm-od>: HTTP status code is not handled or not allowed 2026-01-24 21:02:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-universal-adjustable-ergonomic-forklift-seat-with-suspension-black>: HTTP status code is not handled or not allowed 2026-01-24 21:02:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-cart-black-epoxy-72l-x-14w-x-40h-polyurethane-brake-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:02:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelf-truck-24l-x-21w-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:02:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ariens-st-1336-lawn-and-garden-replacement-battery-8ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 21:02:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6305zz-deep-groove-ball-bearing-double-shielded-25mm-bore-62mm-od>: HTTP status code is not handled or not allowed 2026-01-24 21:02:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ticonderoga-triwrite-hb-2-pencil-with-latexfree-eraser-yellow-dozen>: HTTP status code is not handled or not allowed 2026-01-24 21:02:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quicker-clicker-mechanical-pencil-07mm-transparent-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:02:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-w-x-30-d-standing-height-workbench-plastic-laminate-square-edge-black>: HTTP status code is not handled or not allowed 2026-01-24 21:02:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sears-25710-lawn-and-garden-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:02:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelf-truck-72l-x-24w-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:02:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/twisterase-click-mechanical-pencil-refillable-05mm-black>: HTTP status code is not handled or not allowed 2026-01-24 21:02:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:02:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelf-truck-54l-x-24w-x-80h-polyurethane-brake-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:02:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 22354 pages (at 102 pages/min), scraped 12768 items (at 22 items/min) 2026-01-24 21:02:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/61500072873-belt-4w-x-132l-vfn-grit>: HTTP status code is not handled or not allowed 2026-01-24 21:02:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/61500125705-belt-1-2w-x-12l-vfn-grit>: HTTP status code is not handled or not allowed 2026-01-24 21:02:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/retractable-gel-pen-comfort-grip-07mm-blue-ink-dozen>: HTTP status code is not handled or not allowed 2026-01-24 21:02:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultratech-ut-1270-lawn-and-garden-replacement-battery-9ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 21:02:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cylindrical-bearing-double-row-bore-55mm-0005-to-0015-radial-clearance-nn3011m2kc9nap4>: HTTP status code is not handled or not allowed 2026-01-24 21:02:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cylindrical-bearing-double-row-bore-110mm-0040-to-0060-radial-clearance-nn3022m2kc1nap4>: HTTP status code is not handled or not allowed 2026-01-24 21:02:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bisley-x-back-airflow153-womens-long-sleeve-shirt-class-3-xl-hi-vis-orange>: HTTP status code is not handled or not allowed 2026-01-24 21:02:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:02:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bisley-womens-long-sleeve-shirt-l-hi-vis-orange>: HTTP status code is not handled or not allowed 2026-01-24 21:02:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8020-20-series-1316-x-1316-four-t-slotted-extrusion-profile-118-18l-stock-bar>: HTTP status code is not handled or not allowed 2026-01-24 21:02:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bisley-airflow153-womens-long-sleeve-shirt-class-2-xl-hi-vis-orangenavy>: HTTP status code is not handled or not allowed 2026-01-24 21:02:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bisley-womens-long-sleeve-shirt-class-3-2xl-hi-vis-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:02:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers>: HTTP status code is not handled or not allowed 2026-01-24 21:02:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/retractable-permanent-marker-fine-point-black-ink-1-each>: HTTP status code is not handled or not allowed 2026-01-24 21:02:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ryobi-971255100-lawn-and-garden-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:02:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/king-size-permanent-marker-chisel-red-ink-dozen>: HTTP status code is not handled or not allowed 2026-01-24 21:02:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/highlighters_markers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:02:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-153-plastic-jumbo-bathroom-tissue-dispenser-one-9-roll>: HTTP status code is not handled or not allowed 2026-01-24 21:02:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9682061-bench-grinder-6-wheel-dia-12hp-115230v>: HTTP status code is not handled or not allowed 2026-01-24 21:02:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers>: HTTP status code is not handled or not allowed 2026-01-24 21:02:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zgrip-ballpoint-retractable-pen-10mm-red-ink-dozen>: HTTP status code is not handled or not allowed 2026-01-24 21:02:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mule-6gc028l-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:02:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zgrip-ballpoint-retractable-pen-10mm-black-ink-dozen>: HTTP status code is not handled or not allowed 2026-01-24 21:02:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medium-duty-no-bump-dump-hopper-steel-2-cu-yd-4000-lb-capacity-soda-red>: HTTP status code is not handled or not allowed 2026-01-24 21:02:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hubbell-he615-emergency-light-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:02:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/accent-tank-highlighter-smear-guard-chisel-tip-yellow-ink-dozen>: HTTP status code is not handled or not allowed 2026-01-24 21:02:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/siltron-em50-emergency-light-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:02:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/highlighters_markers>: HTTP status code is not handled or not allowed 2026-01-24 21:02:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-mesh-divider-dmb539c-for-qmb539c-each>: HTTP status code is not handled or not allowed 2026-01-24 21:02:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hyperlite-extension-ladder-w-cable-hooks-ratchet-leveler-32-type-iaa-375-lb-cap>: HTTP status code is not handled or not allowed 2026-01-24 21:02:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hyperlite-extension-ladder-w-v-rung-ratchet-levelers-20-type-iaa-375-lb-cap>: HTTP status code is not handled or not allowed 2026-01-24 21:02:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fsrzi-a-remote-zone-indicator>: HTTP status code is not handled or not allowed 2026-01-24 21:02:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-cage-platform-ladder-w-wheels-8-type-iaa-8-step-375-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:02:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electrostatic-air-filter-merv-8-16w-x-1d-x-25h>: HTTP status code is not handled or not allowed 2026-01-24 21:02:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6x3-8-dralon-woven-mini-roller-w-14-frame-12-case-614-wv038>: HTTP status code is not handled or not allowed 2026-01-24 21:02:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hyperlite-extension-ladder-w-cable-hooks-auto-leveler-28-type-iaa-375-lb-cap>: HTTP status code is not handled or not allowed 2026-01-24 21:02:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electrostatic-air-filter-merv-8-18w-x-1d-x-18h>: HTTP status code is not handled or not allowed 2026-01-24 21:02:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pam1-encapsulated-spdt-relay>: HTTP status code is not handled or not allowed 2026-01-24 21:02:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergi-lite-6-emergency-light-replacement-battery-18ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:02:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sure-lites-r6c-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:02:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f-dact-digital-communicatormodemlcd-module>: HTTP status code is not handled or not allowed 2026-01-24 21:02:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sonnenschein-6-emergency-light-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:02:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/atlite-241002-6v-45ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-24 21:02:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bb-bp76-t1-6v-7ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-24 21:02:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hubbell-702941-emergency-light-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:02:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bb-bp2612-t2-12v-26ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-24 21:02:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-esc26-emergency-light-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:02:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hyperlite-extension-ladder-w-cable-hooks-claw-v-bar-28-type-ia-300-lb-cap>: HTTP status code is not handled or not allowed 2026-01-24 21:02:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/professional-66-led-dual-function-rechargeable-work-light-slr2166-yellow-2pk>: HTTP status code is not handled or not allowed 2026-01-24 21:02:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-head-pro-series-sl200-yellow-6pk>: HTTP status code is not handled or not allowed 2026-01-24 21:02:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powercell-pc670-sealed-lead-acid-agm-vrla-battery>: HTTP status code is not handled or not allowed 2026-01-24 21:02:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powersonic-ps640f1-6v-5ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-24 21:02:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clip-finished-end-panel-24-inch-d-x-87-inch-h-jet-black>: HTTP status code is not handled or not allowed 2026-01-24 21:02:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-side-rubber-for-scrubber-641840-641841-641842>: HTTP status code is not handled or not allowed 2026-01-24 21:02:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/big-beam-2se6s8-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:02:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/panasonic-lcr1212p1-12v-12ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-24 21:02:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qp-two-stage-splash-lubricated-air-compressor-175-psi-14-hp-30-gallon-capacity-horizontal>: HTTP status code is not handled or not allowed 2026-01-24 21:02:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slide-door-4w-x-8h>: HTTP status code is not handled or not allowed 2026-01-24 21:02:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/big-beam-2tc6s50-emergency-light-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:02:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leoch-djw670-sealed-lead-acid-agm-vrla-battery>: HTTP status code is not handled or not allowed 2026-01-24 21:02:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qt-max-two-stage-air-compressor-vertical-75-hp-80-gallon-capacity-3-phase-200v>: HTTP status code is not handled or not allowed 2026-01-24 21:02:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/panasonic-6v-7ah-6v-7ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-24 21:02:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enersys-np2412bfr-12v-26ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-24 21:03:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/laminate-collection-pedestal-tables-cube-table-29wx29dx15-1-2h-maple>: HTTP status code is not handled or not allowed 2026-01-24 21:03:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-stage-electric-air-compressor-5-hp-80-gal-cap-1-phase-230v-457-lb-wt>: HTTP status code is not handled or not allowed 2026-01-24 21:03:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-cvt3gbi-emergency-light-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:03:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/astral-octagon-umbrella-w-centerpost-sunbrella-awning-196w-x-196d-x-110h-parchment>: HTTP status code is not handled or not allowed 2026-01-24 21:03:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-rack-placard-label-holder-4-x-100-ft-rollblue-797071>: HTTP status code is not handled or not allowed 2026-01-24 21:03:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-cvt3gwi-emergency-light-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:03:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/astralite-dca-400-emergency-light-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:03:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/diamec-yj69a-12v-12ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-24 21:03:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1503242-112-x-112-tslotted-profile-242-stock-bar>: HTTP status code is not handled or not allowed 2026-01-24 21:03:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durable-binder-with-slant-rings-vinyl-11-x-8-1-2-3inch-red>: HTTP status code is not handled or not allowed 2026-01-24 21:03:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sure-lites-sl2646-emergency-light-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:03:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergi-lite-12m1-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:03:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 22448 pages (at 94 pages/min), scraped 12790 items (at 22 items/min) 2026-01-24 21:03:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nerp020ld-fg-10-ner-food-grade-hoist-with-push-trolley-2-ton-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:03:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tner005l-60-tner-theatrical-chain-hoist-1-2-ton-capacity-60-lift>: HTTP status code is not handled or not allowed 2026-01-24 21:03:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lithonia-elb0612a-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:03:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/omnis-controller-w-enclosure-30-amp>: HTTP status code is not handled or not allowed 2026-01-24 21:03:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/13037-selfaligning-rollin-tnut-wball-spring>: HTTP status code is not handled or not allowed 2026-01-24 21:03:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tissue-box-holder-petg-small-clear>: HTTP status code is not handled or not allowed 2026-01-24 21:03:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kimwipe-holder-large-white>: HTTP status code is not handled or not allowed 2026-01-24 21:03:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esd-3-4-length-jacket-snap-cuff-black-x-small>: HTTP status code is not handled or not allowed 2026-01-24 21:03:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dyna-ray-dr74714s-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:03:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/phlebotomy-pro-cart-32-38l-x-20-14w-x-43-18h-white>: HTTP status code is not handled or not allowed 2026-01-24 21:03:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-cvec50i-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:03:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2111-standard-tslot-cover>: HTTP status code is not handled or not allowed 2026-01-24 21:03:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-cvt3rw5-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:03:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4000-watt-pure-sine-inverter-charger-picoglf40w12v120v>: HTTP status code is not handled or not allowed 2026-01-24 21:03:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-120225-emergency-light-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:03:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-grip-tilt-fork-mount-drum-handler-1500-lb-weight-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:03:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-paper-towel-dispenser-clear>: HTTP status code is not handled or not allowed 2026-01-24 21:03:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergi-lite-10-emergency-light-replacement-battery-4-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:03:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnetic-base-indicator-holder-set>: HTTP status code is not handled or not allowed 2026-01-24 21:03:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/protection-one-bt0002n-12v-45ah-alarm-battery>: HTTP status code is not handled or not allowed 2026-01-24 21:03:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adi-465654-6v-5ah-alarm-battery>: HTTP status code is not handled or not allowed 2026-01-24 21:03:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/notice-drivers-restricted-to-truck-loading--dock-only--sign-plastic-10-x-14-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:03:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/peg-perego-igor0050-ride-on-toys-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:03:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mototec-24v-electric-mini-quad-12v-12ah-scooter-battery>: HTTP status code is not handled or not allowed 2026-01-24 21:03:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/physician-parking-only-safety-sign-egp-aluminum-18-x-12-greenwhite>: HTTP status code is not handled or not allowed 2026-01-24 21:03:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ups-pick-up-delivery-location-sign-adhesive-vinyl-10-x-14-blackwhite>: HTTP status code is not handled or not allowed 2026-01-24 21:03:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6725-double-flange-linear-bearing>: HTTP status code is not handled or not allowed 2026-01-24 21:03:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/no-riders-except-operator-sign-adhesive-vinyl-3-x-7-blackwhite-pack-of-5>: HTTP status code is not handled or not allowed 2026-01-24 21:03:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nerp015sd-15-ner-electric-hoist-push-trolley-1-12-ton-15-lift-183-ftmin-230v>: HTTP status code is not handled or not allowed 2026-01-24 21:03:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caution-wide-turn-sign-graphic-adhesive-reflective-sheet-10-x-14-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:03:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sonnenschein-1103-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:03:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oversize-load-transportation-sign-mesh-vinyl-18-x-84-blackyellow>: HTTP status code is not handled or not allowed 2026-01-24 21:03:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ups-pickupups-no-pickup-sign-double-sided-aluminum-10-x-14-brownred>: HTTP status code is not handled or not allowed 2026-01-24 21:03:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2194-deluxe-leveling-foot>: HTTP status code is not handled or not allowed 2026-01-24 21:03:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ner003hd-15-ner-dual-speed-electric-chain-hoist-14-ton-15-lift-539-ftmin-230v>: HTTP status code is not handled or not allowed 2026-01-24 21:03:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-ahead-or-down-arrow-safety-sign-egp-aluminum-15-x-21-whiteblue>: HTTP status code is not handled or not allowed 2026-01-24 21:03:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/parking_traffic_signs/parking_traffic_signs_1 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:03:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3379-end-fastener-double-wing-clip>: HTTP status code is not handled or not allowed 2026-01-24 21:03:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kymco-mxu375-irs-375cc-atv-replacement-battery-2010-2011-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:03:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/moto-morini-camel-500cc-motorcycle-replacement-battery-1984-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:03:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/schwinn-mini-e-electric-bicycle-replacement-battery-7ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 21:03:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wrong-way-bicycle-traffic-safety-sign-hip-aluminum-18-x-12-whitered>: HTTP status code is not handled or not allowed 2026-01-24 21:03:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/atk-604-cc-604cc-motorcycle-replacement-battery-1991-1992-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:03:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sym-hd200-200cc-motorcycle-replacement-battery-2004-2012-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:03:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mbk-sorriso-50cc-motorcycle-replacement-battery-1984-2008-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:03:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7217bmuc3-angular-contact-ball-bearing-flush-ground-85mm-bore-x-150mm-od-x-28mm-w>: HTTP status code is not handled or not allowed 2026-01-24 21:03:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-7043-cut-resistant-gloves-nitrile-coated-ansi-a4-l-gray-1-pair>: HTTP status code is not handled or not allowed 2026-01-24 21:03:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-contoured-reversible-puffer-jacket-ansi-l-hi-vis-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:03:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/parking_traffic_signs/parking_traffic_signs_1>: HTTP status code is not handled or not allowed 2026-01-24 21:03:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/indexing-plunger-lock-out-type-stainless-steel-38-16-thread>: HTTP status code is not handled or not allowed 2026-01-24 21:03:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nerp020s-15-ner-electric-chain-hoist-push-trolley-2-ton-15-lift-28-ftmin-230v>: HTTP status code is not handled or not allowed 2026-01-24 21:03:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/short-press-fit-delrin-ball-plunger-stainless-steel-236-diameter>: HTTP status code is not handled or not allowed 2026-01-24 21:03:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ft-one153-full-body-3d-belted-harness-quick-connect-chest--tongue-buckle-legs-s>: HTTP status code is not handled or not allowed 2026-01-24 21:03:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9800-system-340-voc-dtm-urethane-mastic-white-9892383>: HTTP status code is not handled or not allowed 2026-01-24 21:03:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/al100-hand-chain-hoist-aluminum-1-ton-capacity-15-ft-lift>: HTTP status code is not handled or not allowed 2026-01-24 21:03:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nerm020s-l-10-ner-electric-hoist-motor-trolley-2-ton-10-lift-28-ftmin-208v>: HTTP status code is not handled or not allowed 2026-01-24 21:03:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/snerp030c-10-sner-electric-hoist-push-trolley-3-ton-10-lift-35-ftmin-230v>: HTTP status code is not handled or not allowed 2026-01-24 21:03:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/al100-hand-chain-hoist-aluminum-5-ton-capacity-15-ft-lift>: HTTP status code is not handled or not allowed 2026-01-24 21:03:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/recycle-sign-label-aluminum-5-x-7>: HTTP status code is not handled or not allowed 2026-01-24 21:03:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fortress-1700fs-mobility-scooter-replacement-battery-18ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:03:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nerm010l-l-20-ner-electric-hoist-motor-trolley-1-ton-20-lift-14-ftmin-208v>: HTTP status code is not handled or not allowed 2026-01-24 21:03:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/snerm005s-l-20-sner-electric-hoist-motor-trolley-12-ton-20-lift-15-ftmin-230v>: HTTP status code is not handled or not allowed 2026-01-24 21:03:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cardboard-only-label-w-recycle-sign-adhesive-vinyl-7-x-10>: HTTP status code is not handled or not allowed 2026-01-24 21:03:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/snerp003s-15-sner-electric-hoist-push-trolley-14-ton-15-lift-14-ftmin-230v>: HTTP status code is not handled or not allowed 2026-01-24 21:03:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/snerm003s-l-15-sner-electric-hoist-motor-trolley-14-ton-15-lift-14-ftmin-230v>: HTTP status code is not handled or not allowed 2026-01-24 21:03:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-straight-floor-squeegee-w-handle-6-pack-1038>: HTTP status code is not handled or not allowed 2026-01-24 21:03:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ac-charging-cable-global-industrial-153-portable-power-station-812541>: HTTP status code is not handled or not allowed 2026-01-24 21:03:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-head-platform-pad-18900043-mx-s-vortex-mixers-requires-universal-adapter>: HTTP status code is not handled or not allowed 2026-01-24 21:03:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60020010520-roll-661x-1w-x-50l-x-m15>: HTTP status code is not handled or not allowed 2026-01-24 21:04:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bravo-jet-star-electric-scooter-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:04:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60410006963-roll-372l-8w-x-150l-x-m9>: HTTP status code is not handled or not allowed 2026-01-24 21:04:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kb-unit-heater-with-bracket-thermostat-10-kw-480v-1-phase>: HTTP status code is not handled or not allowed 2026-01-24 21:04:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kb-unit-heater-with-bracket-thermostat-125-kw-240208v-1-phase>: HTTP status code is not handled or not allowed 2026-01-24 21:04:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manual-pipette-filler-74010005-use-w-1-100ml-pipets-pink>: HTTP status code is not handled or not allowed 2026-01-24 21:04:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anti-scratch-semi-rimless-safety-glasses-clear-lens-black-frame-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-24 21:04:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ceramic-clip-it-magnet-rb50bncc-w-attached-black-clip-35-lbs-pull-chrome-plating>: HTTP status code is not handled or not allowed 2026-01-24 21:04:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g-tek-polykor-seamless-knit-blended-cr-gloves-nitrile-coated-ansi-a4-s-blue-1-pair>: HTTP status code is not handled or not allowed 2026-01-24 21:04:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-x-48-slatwall-panel-brushed-aluminum-hpl-without-inserts-pkg-qty-4>: HTTP status code is not handled or not allowed 2026-01-24 21:04:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/loop-tape-individual-dots-with-adhesive-78l-x-78w-black-pack-of-900>: HTTP status code is not handled or not allowed 2026-01-24 21:04:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ezip-ezip-750-electric-scooter-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 21:04:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-ski-doo-mx-z-800-ho-renegade-and-renegade-x-800cc-snowmobile-battery-2004-2013-12v>: HTTP status code is not handled or not allowed 2026-01-24 21:04:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bombardier-mx-z-600-ho-sdi-renegade-renegade-x-600cc-snowmobile-battery-2004-2013-12v>: HTTP status code is not handled or not allowed 2026-01-24 21:04:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 22544 pages (at 96 pages/min), scraped 12805 items (at 15 items/min) 2026-01-24 21:04:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bombardier-summit-x-e-tec-800r-146-800cc-snowmobile-replacement-battery-2015-12v>: HTTP status code is not handled or not allowed 2026-01-24 21:04:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inkjet-post-card-4-inch-x-6-inch-matte-white-100-cards-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:04:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bruno-vpl-3153b-vertical-platform-stairlift-replacement-battery-18ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:04:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gold-fibre-jr-legal-pads-5-inch-x-8-inch-16-lb-white-50-sheets-pad-4-pads-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:04:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ferno-ille-125-128-129-lift-chair-medical-replacement-battery-18ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:04:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durapro-knee-boots-safety-loc-outsole-plain-toe-size-15-16h-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:04:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/engineering-calculation-pad-8-1-2-inch-x-11-inch-quad-ruled-green-200-sheets-pad>: HTTP status code is not handled or not allowed 2026-01-24 21:04:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/Classic-Chair-Sensor-Pad-7-x-15-1-Year-10112>: HTTP status code is not handled or not allowed 2026-01-24 21:04:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/Advanced-Bed-Alarm-W-Nurse-Call-10260>: HTTP status code is not handled or not allowed 2026-01-24 21:04:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pharmacia-deltec-9000-profusion-system-medical-replacement-battery-26ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:04:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hook-tape-individual-dots-with-adhesive-12l-x-12w-white-pack-of-1440>: HTTP status code is not handled or not allowed 2026-01-24 21:04:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/abbott-laboratories-75-life-care-breeze-medical-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:04:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/index-maker-clear-label-divider-with-tabs-blank-85x11-8-tabs-white-white>: HTTP status code is not handled or not allowed 2026-01-24 21:04:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ibt-technologies-bt12-12-sealed-lead-acid-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 21:04:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-height-aluminum-gantry-crane-6000-lb-capacity-15w-x-13-316h>: HTTP status code is not handled or not allowed 2026-01-24 21:04:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nilfisk-advance-micromatic-17b-industrial-replacement-battery-18ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:04:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dc-powered-jib-crane-2000-lb-capacity-87-14h-span-106-12h-under-beam>: HTTP status code is not handled or not allowed 2026-01-24 21:04:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fold-down-forklift-work-maintenance-platform-600-lb-capacity-37l-x-37w-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:04:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-collated-legal-exhibit-divider-printed-51-to-75-85x11-26-tabs-white-white>: HTTP status code is not handled or not allowed 2026-01-24 21:04:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-tab-divider-blank-85x11-8-tabs-multicolor-multicolor>: HTTP status code is not handled or not allowed 2026-01-24 21:04:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/worksaver-big-tab-insertable-tab-divider-blank-85x11-8-tabs-white-multicolor>: HTTP status code is not handled or not allowed 2026-01-24 21:04:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ready-index-toc-reference-divider-1-to-8-85x11-8-tabs-white-multi>: HTTP status code is not handled or not allowed 2026-01-24 21:04:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-decker-electromate-400-jump-starter-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:04:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-day-deluxe-dry-bag-2-person-backpack-80-pieces>: HTTP status code is not handled or not allowed 2026-01-24 21:04:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ratchet-trailer-stabilizing-jack-hard-rubber-wheels-40000-lb-lifting-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:04:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/para-systems-ps-12180-f2-sealed-lead-acid-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:04:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nanuk-320-0011-320-series-nano-case-clear>: HTTP status code is not handled or not allowed 2026-01-24 21:04:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-drawer-procedure-cart-front-locking-20l-x-31w-x-48h-global-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:04:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=jvm-836-3-mill3-axis-acu-rite-200s-dro-x-y-axis-powerfeeds already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:04:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lb-style-threaded-conduit-body-w-cover-and-gasket-1-14-size>: HTTP status code is not handled or not allowed 2026-01-24 21:04:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/phillips-and-square-flat-head-machine-screw-10-32-x-1-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:04:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conduit-hanger-w-hex-nut-and-bolt-carbon-steel-1-14-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:04:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bubble-sof-tred-12-thick-safety-anti-fatigue-floor-mat-3x6-black>: HTTP status code is not handled or not allowed 2026-01-24 21:04:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/diamond-tuff-1-thick-antifatigueantislip-classic-fl-mat-3x75-blackyl>: HTTP status code is not handled or not allowed 2026-01-24 21:04:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/910-0000-910-case-143l-x-1111w-x-47h-tan>: HTTP status code is not handled or not allowed 2026-01-24 21:04:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/950-0007-950-case-228l-x-183w-x-117h-graphite>: HTTP status code is not handled or not allowed 2026-01-24 21:04:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-nestable-shelf-bin-6-5-8w-x-11-5-8d-x-6h>: HTTP status code is not handled or not allowed 2026-01-24 21:04:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-nestable-shelf-bin-8-3-8w-x-11-5-8d-x-4h>: HTTP status code is not handled or not allowed 2026-01-24 21:04:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shoprider-streamer-sport-888wsb-wheelchair-replacement-battery-8ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 21:04:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-super-size-12-3-8w-x-20d-x-8h>: HTTP status code is not handled or not allowed 2026-01-24 21:04:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=jvm-836-3-mill3-axis-acu-rite-200s-dro-x-y-axis-powerfeeds>: HTTP status code is not handled or not allowed 2026-01-24 21:04:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/950-2004-950-case-w-padded-divider-228l-x-183w-x-117h-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:04:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pride-mobility-quantum-6000-xl-wheelchair-replacement-battery-75ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:04:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bright-way-group-bw-12220-nb-sealed-lead-acid-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:04:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/imc-heartway-rumba-sf-p-4f-u1-wheelchair-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:04:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-pump-jack-24-ft-pole-pj-24p>: HTTP status code is not handled or not allowed 2026-01-24 21:04:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bright-way-group-bw-12750-it-sealed-lead-acid-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:04:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-nut-grade-2-carbon-steel-38-16-size-tuff-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:04:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lr-threaded-conduit-body-w-cover-and-gasket-1-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:04:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-ft-type-1aa-aluminum-round-rung-straight-ladder-524-1>: HTTP status code is not handled or not allowed 2026-01-24 21:04:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmx-iiia-flame-resistant-premium-coverall-cnb2-navy-45-oz-size-52-long>: HTTP status code is not handled or not allowed 2026-01-24 21:04:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-recycling-container-with-waste-disposal-opening28-gallon-cap-charcoal-240627CH>: HTTP status code is not handled or not allowed 2026-01-24 21:04:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ef-flame-resistant-relaxed-fit-denim-jeans-pej2-dark-denim-125-oz-size-38-x-37u>: HTTP status code is not handled or not allowed 2026-01-24 21:04:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-recycling-container-with-combo-opening36-gallon-capacity-blue-261625BL>: HTTP status code is not handled or not allowed 2026-01-24 21:04:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ef-ct-fr-dress-uniform-shirt-slu2-silver-gray-size-m-long>: HTTP status code is not handled or not allowed 2026-01-24 21:04:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nsf-37-certified-air-curtain-144-inch-w-door-208v-3-4hp-3-ph-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:04:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-nut-18-8-stainless-steel-12-13-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:04:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insulated-ground-bushing-w-lay-in-lug-zincaluminum-1-14-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:04:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merits-s246-pioneer-2-twin-scooter-wheelchair-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:04:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smrtvrmt-handheld-ir-remote-for-smrtv60240-control>: HTTP status code is not handled or not allowed 2026-01-24 21:04:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s15120w-infrared-heater-s15120w-15kw-120v-white>: HTTP status code is not handled or not allowed 2026-01-24 21:05:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/friendly-robotics-robomower-rm200-lawn-and-garden-replacement-battery-22ah-12v>: HTTP status code is not handled or not allowed 2026-01-24 21:05:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kpex-78-03-125-electronic-super-knips-comfort-grip-5-oal-box-joint-design>: HTTP status code is not handled or not allowed 2026-01-24 21:05:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slotted-flat-head-machine-screw-14-20-x-3-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:05:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/supreme-replacement-impeller-for-mag-drive-35>: HTTP status code is not handled or not allowed 2026-01-24 21:05:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1004-standard-class-2-mesh-hook-loop-safety-vest-orange-xl>: HTTP status code is not handled or not allowed 2026-01-24 21:05:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobcat-by-textron-zt-227-efi-lawn-and-garden-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:05:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trio-lightning-tl930219-emergency-light-replacement-battery-1-3ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:05:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-fishing-piston-durable-foam-3-size>: HTTP status code is not handled or not allowed 2026-01-24 21:05:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-decker-sweeper-cs100-lawn-and-garden-replacement-battery-4-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:05:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sonnenschein-a412-8-5-sr-emergency-light-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 21:05:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibration-mount-1-tapped-hole-1-threaded-stud-31-inch-dia-8mm-h-m3-x-6-thread>: HTTP status code is not handled or not allowed 2026-01-24 21:05:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 22638 pages (at 94 pages/min), scraped 12827 items (at 22 items/min) 2026-01-24 21:05:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pull-rope-polypropylene-316-x-3600-size>: HTTP status code is not handled or not allowed 2026-01-24 21:05:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-fishing-piston-durable-foam-1-12-size>: HTTP status code is not handled or not allowed 2026-01-24 21:05:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nail-strap-carbon-steel-12-emt-pack-of-1000>: HTTP status code is not handled or not allowed 2026-01-24 21:05:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slotted-flat-head-machine-screw-8-32-x-3-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:05:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slotted-flat-head-machine-screw-38-16-x-4-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:05:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oval-head-wall-plate-screw-slotted-brown-6-32-x-516-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:05:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-mate-pm1240-pm1245-pm1250-emergency-light-replacement-battery-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:05:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/69000-series-return-shell-48-inchw-x-24-inchd-x-29-1-2-inchh-cherry>: HTTP status code is not handled or not allowed 2026-01-24 21:05:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vexilar-edge2-fishinglocator-fish-finder-replacement-battery-8ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 21:05:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/69000-series-return-shell-35-inchw-x-24-inchd-x-29-1-2-inchh-mahogany>: HTTP status code is not handled or not allowed 2026-01-24 21:05:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sonnenschein-nga5060010hsosa-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:05:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/69000-series-return-shell-35-inchw-x-24-inchd-x-29-1-2-inchh-cherry>: HTTP status code is not handled or not allowed 2026-01-24 21:05:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chloride-power-100-001-137-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:05:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wallpro-400-black-wall-mount-retracting-barrier-15-blue-belt>: HTTP status code is not handled or not allowed 2026-01-24 21:05:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sheet-metal-screw-combo-pan-head-carbon-steel-12-x-2-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:05:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/teledyne-big-beam-2sc6g8p2-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:05:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reducing-bushing-zinc-alloy-2-x-12-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:05:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microfiber-towels-assorted-2lb-bulk-bag-2503-ac-bg>: HTTP status code is not handled or not allowed 2026-01-24 21:05:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96h-x-48w-slatwall-panelblack-laminate-without-inserts-968340B>: HTTP status code is not handled or not allowed 2026-01-24 21:05:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tear-tab-lid-with-plastic-spout-for-3-12-5-6-gallon-open-head-pailsnatural-987420>: HTTP status code is not handled or not allowed 2026-01-24 21:05:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hsst312ryg-3inone-manual-control-signal-tower-kit-red-ambergreen-light-dc24v>: HTTP status code is not handled or not allowed 2026-01-24 21:05:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-stapler-20-sheet-capacity-black>: HTTP status code is not handled or not allowed 2026-01-24 21:05:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/concrete-screw-anchor-slotted-hex-washer-head-316-x-3-14-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:05:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chamberlain-3-4-hps-myq-belt-drive-garage-door-replacement-battery-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:05:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:05:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/humminbird-piranhamax-150-portable-fish-finder-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:05:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honda-trx420fpa-rancher-at-4x4-w-power-steering-420cc-atv-battery-2009-2014-12v>: HTTP status code is not handled or not allowed 2026-01-24 21:05:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/loose-leaf-book-rings-1-1-2-diameter-silver-100-box>: HTTP status code is not handled or not allowed 2026-01-24 21:05:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/peg-perego-falcon-quad-green-ed033-ride-on-toys-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:05:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrs206201l-cast-iron-right-angle-worm-gear-reducer-201-ratio>: HTTP status code is not handled or not allowed 2026-01-24 21:05:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-head-wood-screw-phillips-carbon-steel-10-x-2-12-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:05:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honda-gl1200a-gold-wing-aspencade-1200cc-motorcycle-battery-1984-1987-12v>: HTTP status code is not handled or not allowed 2026-01-24 21:05:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrf175601h56c-cast-iron-right-angle-worm-gear-reducer-601-ratio-56c-frame>: HTTP status code is not handled or not allowed 2026-01-24 21:05:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-bands-size-33-3-1-2-x-1-8-natural-1-lb-box>: HTTP status code is not handled or not allowed 2026-01-24 21:05:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-bands-size-32-3x-1-8-natural-1-lb-box>: HTTP status code is not handled or not allowed 2026-01-24 21:05:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-24 21:05:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrs175401l-cast-iron-right-angle-worm-gear-reducer-401-ratio>: HTTP status code is not handled or not allowed 2026-01-24 21:05:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/moto-morini-excalibur-501cc-motorcycle-replacement-battery-1945-1987-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:05:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sheet-metal-screw-phillipsslotted-hex-washer-head-8-x-12-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:05:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fender-washer-low-carbon-steel-316-x-1-14-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:05:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fender-washer-low-carbon-steel-38-x-1-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:05:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/piaggio-vespa-mp3-500-500cc-motorcycle-replacement-battery-2008-2014-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:05:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrs325601l-cast-iron-right-angle-worm-gear-reducer-601-ratio>: HTTP status code is not handled or not allowed 2026-01-24 21:05:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/suzuki-gsx750f-katana-750cc-motorcycle-replacement-battery-1998-2006-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:05:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/wire_markers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:05:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-punch-2-4-holes-adjustable-centers-a4-compatible-40-sheets-black>: HTTP status code is not handled or not allowed 2026-01-24 21:05:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-bands-size-19-3-1-2x-1-16-natural-1-lb-box>: HTTP status code is not handled or not allowed 2026-01-24 21:05:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrf325301l56c-cast-iron-right-angle-worm-gear-reducer-301-ratio-56c-frame>: HTTP status code is not handled or not allowed 2026-01-24 21:05:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrf325301l1824tc-cast-iron-right-angle-worm-gear-reducer-301-ratio-1824t-frame>: HTTP status code is not handled or not allowed 2026-01-24 21:05:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-liquid-tight-connector-zinc-die-cast-2-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:05:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aprilia-pegaso-strada-factory-trail-650cc-motorcycle-battery-2005-2012-12v>: HTTP status code is not handled or not allowed 2026-01-24 21:05:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bourget-bike-works-shredder-1900cc-motorcycle-replacement-battery-2010-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:05:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harley-davidson-cvo-breakout-fxsbse-1800cc-motorcycle-battery-2013-2014-12v>: HTTP status code is not handled or not allowed 2026-01-24 21:05:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grip-compact-stapler-25-sheet-capacity-silver>: HTTP status code is not handled or not allowed 2026-01-24 21:05:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/small-flags-3-16-x-1-assorted-240-flags-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:05:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/wire_markers>: HTTP status code is not handled or not allowed 2026-01-24 21:05:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-e-fr-overall-snap-fly-front-fluorescent-orangered-s>: HTTP status code is not handled or not allowed 2026-01-24 21:05:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kawasaki-zx636-c-ninja-zx-6r-636cc-motorcycle-replacement-battery-2003-2017-12v>: HTTP status code is not handled or not allowed 2026-01-24 21:05:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-container-placard-label-holder-istp2-wplace-label-here-4-12x6-12-white>: HTTP status code is not handled or not allowed 2026-01-24 21:05:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/closed-starter-csb85-2448-7x-48-inch-w-x-24-inch-d-x-85-inch-h-7-shelves-18-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:05:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-head-tap-bolt-low-carbon-steel-38-16-x-3-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:05:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bx1642x-42-inch-w-x-16-inch-d-18-gauge-800-lb-capacity-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:05:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/excelsior-henderson-super-x-1386cc-motorcycle-replacement-battery-1999-2000-12v>: HTTP status code is not handled or not allowed 2026-01-24 21:05:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bx1548-48-inch-w-x-15-inch-d-20-gauge-600-lb-capacity-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:05:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harley-davidson-road-king-flhri-motorcycle-replacement-battery-1996-2006-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:05:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bx1248x-48-inch-w-x-12-inch-d-18-gauge-750-lb-capacity-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:06:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f71ey10000000003-victor-wall-mount-control-station-yellow-1-hole-momentary-estop>: HTTP status code is not handled or not allowed 2026-01-24 21:06:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/excelsior-henderson-hcx-1386cc-motorcycle-replacement-battery-1999-2001-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:06:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5000-replacement-roller-kit-3-rollers-3-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:06:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/closed-add-on-cab85-1248-6x-48-inch-w-x-12-inch-d-x-85-inch-h-6-shelves-18-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:06:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-head-lag-screw-carbon-steel-12-x-4-12-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:06:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-head-tap-bolt-stainless-steel-516-18-x-1-12-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:06:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modern-equipment-meco-54p-4-wheel-warehouse-hand-truck-polyolefin-1000-lb>: HTTP status code is not handled or not allowed 2026-01-24 21:06:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eco-funnel-system-ef-4-fs70-sys-4-5-gal-cap-size-70mm-red-lid>: HTTP status code is not handled or not allowed 2026-01-24 21:06:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honda-gl1800hpna-gold-wing-audio-comfort-navi-abs-1800cc-motorcycle-battery-2006-2012-12v>: HTTP status code is not handled or not allowed 2026-01-24 21:06:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/contempo-pop-popcorn-machine-4-oz-popper-92-one-oz-servings-per-hour>: HTTP status code is not handled or not allowed 2026-01-24 21:06:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 22734 pages (at 96 pages/min), scraped 12850 items (at 23 items/min) 2026-01-24 21:06:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/professional-series-popcorn-machine-12-oz-popper-240-one-oz-servings-per-hour>: HTTP status code is not handled or not allowed 2026-01-24 21:06:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sheila-shine-stainless-steel-cleaner-polish-gallon-can-1-case-ssi4ea>: HTTP status code is not handled or not allowed 2026-01-24 21:06:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kawasaki-jet-ski-jt750-stx-750cc-personal-watercraft-battery-1996-1998-12v>: HTTP status code is not handled or not allowed 2026-01-24 21:06:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-ski-doo-rxt-1500cc-personal-watercraft-replacement-battery-2005-2017-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:06:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-in-general-purpose-npt-valve-body-7-5-cv-vt2427>: HTTP status code is not handled or not allowed 2026-01-24 21:06:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-ucp211-35-pillow-block-bearing-set-screw-locking-bore-55-5625-mm>: HTTP status code is not handled or not allowed 2026-01-24 21:06:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kawasaki-jet-ski-jh750-b3-750cc-personal-watercraft-battery-1993-1995-12v>: HTTP status code is not handled or not allowed 2026-01-24 21:06:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-ucfl212-39-flange-mount-bearing-2-bolt-flange-set-screw-locking-bore-61-9125-mm>: HTTP status code is not handled or not allowed 2026-01-24 21:06:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drive-medical-ventura-4-mobility-scooter-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:06:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-row-spherical-roller-bearing-22212exw33kc3-60mm-bore-110mm-od-tapered-bore>: HTTP status code is not handled or not allowed 2026-01-24 21:06:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-ucp208-24-pillow-block-bearing-set-screw-locking-bore-38-1-mm>: HTTP status code is not handled or not allowed 2026-01-24 21:06:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mid-calf-knee-boots-chevron-plus-outsole-composite-safety-toe-size-7-10h-bluered>: HTTP status code is not handled or not allowed 2026-01-24 21:06:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metatarsal-knee-boots-chevron-outsole-steel-toe-size-11-16h-brick-redcream>: HTTP status code is not handled or not allowed 2026-01-24 21:06:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-row-spherical-roller-bearing-23140ew33c3-200mm-bore-340mm-od>: HTTP status code is not handled or not allowed 2026-01-24 21:06:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-sblf205-16g-flange-mount-bearing-2-bolt-flange-set-screw-locking-bore-25-4-mm>: HTTP status code is not handled or not allowed 2026-01-24 21:06:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ev200-evaporative-whole-room-humidifier-12-pints-output-per-day-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:06:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-row-spherical-roller-bearing-22308exw33c3-40mm-bore-90mm-od>: HTTP status code is not handled or not allowed 2026-01-24 21:06:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-ucfl209-26-flange-mount-bearing-2-bolt-flange-set-screw-locking-bore-41-275-mm>: HTTP status code is not handled or not allowed 2026-01-24 21:06:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-ucfl210-50mm-flange-mount-bearing-2-bolt-flange-set-screw-locking-bore-50-mm>: HTTP status code is not handled or not allowed 2026-01-24 21:06:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-row-spherical-roller-bearing-22317exw33c3-85mm-bore-180mm-od>: HTTP status code is not handled or not allowed 2026-01-24 21:06:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cease-fire-round-waste-receptacle-with-aluminum-head-30-gallon-capacity-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:06:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dalton-3-wheel-sc-s235bl-electric-scooter-replacement-battery-18ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:06:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drive-medical-titanx23-mobility-scooter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:06:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/piaggio-vespa-gts300-super-300cc-scooter-and-moped-battery-2009-2016-12v>: HTTP status code is not handled or not allowed 2026-01-24 21:06:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-row-spherical-roller-bearing-23960-ew33-300mm-bore-420mm-od>: HTTP status code is not handled or not allowed 2026-01-24 21:06:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3400-grabber-red>: HTTP status code is not handled or not allowed 2026-01-24 21:06:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/loki-pz-safety-glassesgoggles-polarized-smoke-lens-black-frame>: HTTP status code is not handled or not allowed 2026-01-24 21:06:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pride-mobility-jazzy-614hd-mobility-scooter-replacement-battery-55ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:06:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crib-saver-cover-for-stowaway-folding-compact-cribs-42-34l-x-6-34w-x-32h-tan>: HTTP status code is not handled or not allowed 2026-01-24 21:06:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-row-cylindrical-roller-bearing-nu216myc3-80mm-bore-140mm-od>: HTTP status code is not handled or not allowed 2026-01-24 21:06:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/everest-jennings-sabreltd-mobility-scooter-replacement-battery-55ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:06:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3216-microfiber-cleaning-cloth-black>: HTTP status code is not handled or not allowed 2026-01-24 21:06:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-row-cylindrical-roller-bearing-nj311myc3-55mm-bore-120mm-od>: HTTP status code is not handled or not allowed 2026-01-24 21:06:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8950-bump-cap-navy-long-brim-one-size>: HTTP status code is not handled or not allowed 2026-01-24 21:06:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quantum-partition-store-single-bay-cart-with-wire-baskets-25l-x-29-12w-x-58-12h-white>: HTTP status code is not handled or not allowed 2026-01-24 21:06:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6710fr-evaporative-fr-cooling-triangle-hat-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:06:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-row-cylindrical-roller-bearing-nu311c3-55mm-bore-120mm-od>: HTTP status code is not handled or not allowed 2026-01-24 21:06:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e-scooter-24-volt-100-watt-electric-scooter-replacement-battery-5ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 21:06:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gt-gt-200-electricscooter-electric-scooter-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 21:06:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8089-noncertified-tshirt-orange-s>: HTTP status code is not handled or not allowed 2026-01-24 21:06:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/avigo-surge-24v-dirt-rider-electric-scooter-replacement-battery-7ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 21:06:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stretch-wrap-film-18-x-1500-x-70-gauge-clear-hand-dispenser-skid-lot>: HTTP status code is not handled or not allowed 2026-01-24 21:06:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/razor-ground-forcedrifter-electric-scooter-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:06:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ball-transfer-conveyor-drop-in-insert-plate-bti363-with-3-center-75-lb-cap>: HTTP status code is not handled or not allowed 2026-01-24 21:06:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-ball-transfer-conveyor-table-btit100203-with-3-center-75-lb-cap>: HTTP status code is not handled or not allowed 2026-01-24 21:06:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merits-travel-ease-commuter-heavy-duty-p181-mobility-scooter-battery-55ah-12v>: HTTP status code is not handled or not allowed 2026-01-24 21:06:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keyboard-mouse-tray-w-value-swivel-arm>: HTTP status code is not handled or not allowed 2026-01-24 21:06:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/klip-it-open-add-on-oak85-1836-8-36-inch-w-x-18-inch-d-x-85-inch-h-8-shelves-20-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:06:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/frank-mobility-e-fix-e-20-standard-mobility-scooter-replacement-battery-35ah-12v>: HTTP status code is not handled or not allowed 2026-01-24 21:06:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-2-in-lead-free-brass-tee-fnpt-125-psi-import>: HTTP status code is not handled or not allowed 2026-01-24 21:06:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pillar-technology-all-othermodels-mobility-scooter-replacement-battery-35ah-12v>: HTTP status code is not handled or not allowed 2026-01-24 21:06:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/craftmatic-comfort-coach-6scooter-mobility-scooter-replacement-battery-35ah-12v>: HTTP status code is not handled or not allowed 2026-01-24 21:06:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-bolt-closed-add-on-cat75-1248-5x-48-inch-w-x-12-inch-d-x-75-inch-h-5-shelves-18-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:06:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/cart_truck_accessories/vehicle_tugger_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:06:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-shelf-chrome-esd-wire-shelving-unit-starter-72w-x-12d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 21:06:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-bolt-open-starter-ost75-1236-6-36-inch-w-x-12-inch-d-x-75-inch-h-6-shelves-20-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:06:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pride-mobility-jazzy-selectelite-mobility-scooter-replacement-battery-35ah-12v>: HTTP status code is not handled or not allowed 2026-01-24 21:06:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-bolt-open-starter-ost75-1842-5-42-inch-w-x-18-inch-d-x-75-inch-h-5-shelves-20-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:06:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-bolt-open-add-on-oat75-2436-6-36-inch-w-x-24-inch-d-x-75-inch-h-6-shelves-20-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:06:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-bolt-open-starter-ost75-3648-5-48-inch-w-x-36-inch-d-x-75-inch-h-5-shelves-20-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:06:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bravo-b2-lightning-electricscooter-electric-scooter-replacement-battery-5ah-12v>: HTTP status code is not handled or not allowed 2026-01-24 21:06:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-bolt-open-add-on-oat99-3048-8-48-inch-w-x-30-inch-d-x-99-inch-h-8-shelves-20-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:06:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-shelf-stainless-steel-wire-shelving-unit-starter-42w-x-21d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 21:06:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-shelf-stainless-steel-wire-shelving-unit-starter-42w-x-24d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 21:06:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-in-x-5-in-stainless-steel-pipe-nipple-16168-psi-sch-40-domestic>: HTTP status code is not handled or not allowed 2026-01-24 21:06:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-bolt-closed-starter-cst87-1248-8x-48-inch-w-x-12-inch-d-x-87-inch-h-8-shelves-18-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:06:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-2-in-304-stainless-steel-90-degree-street-elbow-mnpt-x-fnpt-125-psi-import>: HTTP status code is not handled or not allowed 2026-01-24 21:06:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-in-x-5-1-2-in-lead-free-seamless-red-brass-pipe-nipple-140-psi-sch-40-import>: HTTP status code is not handled or not allowed 2026-01-24 21:06:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-chrome-wire-shelving-unit-add-on-72w-x-12d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 21:06:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/carts-trucks/cart_truck_accessories/vehicle_tugger_accessories>: HTTP status code is not handled or not allowed 2026-01-24 21:06:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-in-lead-free-brass-coupling-fnpt-125-psi-import>: HTTP status code is not handled or not allowed 2026-01-24 21:06:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-stainless-steel-wire-shelving-unit-starter-48w-x-18d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 21:07:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bombardier-renegade-adrenaline-ace-900-900cc-snowmobile-battery-2014-12v>: HTTP status code is not handled or not allowed 2026-01-24 21:07:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-stainless-steel-wire-shelving-unit-add-on-54w-x-18d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 21:07:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss62022rs-deep-groove-ball-bearing-stainless-steel-double-sealed-od-35mm-bore-15mmmetric>: HTTP status code is not handled or not allowed 2026-01-24 21:07:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yamaha-rs90ltgt-rs-vector-ltx-gt-1049cc-snowmobile-battery-2010-2011-12v>: HTTP status code is not handled or not allowed 2026-01-24 21:07:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-bolt-open-add-on-oat87-3048-6x-48-inch-w-x-30-inch-d-x-87-inch-h-6-shelves-18-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:07:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/68032rs-deep-groove-ball-bearing-double-sealed-od-26mm-bore-17mm-metric>: HTTP status code is not handled or not allowed 2026-01-24 21:07:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-stainless-steel-wire-shelving-unit-add-on-36w-x-30d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 21:07:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-stainless-steel-wire-shelving-unit-add-on-54w-x-24d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 21:07:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/68062rs-deep-groove-ball-bearing-double-sealed-od-42mm-bore-30mm-metric>: HTTP status code is not handled or not allowed 2026-01-24 21:07:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-shelf-stainless-steel-wire-shelving-unit-add-on-72w-x-18d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 21:07:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-grand-touring-600-593cc-snowmobile-replacement-battery-2003-2012-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:07:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c748f-snow-safety-sign-caution-proceed-slowly-watch-for-traffic-32-x-24-yellow-black>: HTTP status code is not handled or not allowed 2026-01-24 21:07:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 22832 pages (at 98 pages/min), scraped 12864 items (at 14 items/min) 2026-01-24 21:07:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-chrome-wire-shelving-unit-add-on-72w-x-12d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 21:07:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-shelf-stainless-steel-wire-shelving-unit-add-on-30w-x-24d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 21:07:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/print-or-write-notarial-certificate-seals-2-diameter-gold-44-labels-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:07:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-975xl2-1-2-in-fnpt-x-fnpt-reduced-pressure-principle-assembly-175-psi-cast-bronze>: HTTP status code is not handled or not allowed 2026-01-24 21:07:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rapid-memo-book-2-part-carbonless-8-1-2-x-8-1-4-50-sets-book>: HTTP status code is not handled or not allowed 2026-01-24 21:07:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-ski-doo-gtx-380f-377cc-snowmobile-replacement-battery-2005-2006-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:07:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-poly-bags-36-x-48-8-mil-clear-50-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:07:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-shelf-stainless-steel-wire-shelving-unit-add-on-60w-x-36d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 21:07:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-ski-doo-mx-z-800-800cc-snowmobile-replacement-battery-2004-2013-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:07:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-chrome-wire-shelving-unit-starter-48w-x-14d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 21:07:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-summit-x-800r-800cc-snowmobile-replacement-battery-2008-2009-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:07:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-ski-doo-mx-z-380f-368cc-snowmobile-replacement-battery-2001-2003-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:07:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-chrome-wire-shelving-unit-add-on-36w-x-30d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 21:07:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/job-work-order-book-2-part-carbonless-5-1-2-x-8-1-2-50-sets-book>: HTTP status code is not handled or not allowed 2026-01-24 21:07:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-two-pocket-insertable-dividers-assorted-5-tabs-set>: HTTP status code is not handled or not allowed 2026-01-24 21:07:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bombardier-renegade-adrenaline-800r-800cc-snowmobile-battery-2014-2016-12v>: HTTP status code is not handled or not allowed 2026-01-24 21:07:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yamaha-rs90nr-nytro-er-973cc-snowmobile-replacement-battery-2006-2007-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:07:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pre-inked-message-stamp-faxed-1-2-x-1-5-8-red>: HTTP status code is not handled or not allowed 2026-01-24 21:07:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/expansion-poly-mailers-13-x-16-x-4-2-5-mil-white-100-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:07:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-13-x-18-8-mil-clear-250-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:07:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-grand-touring-700-700cc-snowmobile-replacement-battery-1998-2000-12v-i>: HTTP status code is not handled or not allowed 2026-01-24 21:07:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-mx-z-600-renegade-x-snowmobile-replacement-battery-2004-2009-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:07:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/employee-record-master-file-jackets-1-expansion-25-3-4-x-9-1-2-manila-15-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:07:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-chrome-esd-wire-shelving-unit-starter-72w-x-12d-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 21:07:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-poly-green-wire-shelving-unit-add-on-36w-x-18d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 21:07:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-poly-green-wire-shelving-unit-add-on-60w-x-21d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 21:07:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-poly-green-wire-shelving-unit-starter-48w-x-12d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 21:07:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-poly-green-wire-shelving-unit-starter-72w-x-12d-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 21:07:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-4-gas-appliance-regulator-210d-1-1-4-up-to-6500000-btu>: HTTP status code is not handled or not allowed 2026-01-24 21:07:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-poly-bags-18-x-24-8-mil-clear-100-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:07:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-boxes-12-x-5-x-4-200lb-test-ect-32-25-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:07:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honeywell-wiring-subbase-q7800b1011-burner-wall-mount-3-knockouts-each-end>: HTTP status code is not handled or not allowed 2026-01-24 21:07:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-poly-green-wire-shelving-unit-starter-60w-x-14d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 21:07:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modern-equipment-meco-bct96-4-bar-cradle-truck-96-l-4000-lb-cap>: HTTP status code is not handled or not allowed 2026-01-24 21:07:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kwik-spin-w-maxcore-cable-25-feet-l-x-1-4-inch-w>: HTTP status code is not handled or not allowed 2026-01-24 21:07:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inner-core-cable-75-feet-l-x-3-8-inch-for-use-w-k-3800-drum-machine>: HTTP status code is not handled or not allowed 2026-01-24 21:07:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-tundra-sport-ace-600-600cc-snowmobile-replacement-battery-2012-2018-12v>: HTTP status code is not handled or not allowed 2026-01-24 21:07:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-ski-doo-gsx-le-600-ho-594cc-snowmobile-replacement-battery-2014-2015-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:07:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/permanent-roof-anchor-ra41>: HTTP status code is not handled or not allowed 2026-01-24 21:07:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-pedestal-post-for-585280-292231>: HTTP status code is not handled or not allowed 2026-01-24 21:07:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-summit-sp-600-ho-600cc-snowmobile-replacement-battery-2014-2016-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:07:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-8714-containment-berm-rapid-rise-12-x-60-copolymer-2000>: HTTP status code is not handled or not allowed 2026-01-24 21:07:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-8482-8-doors-tablet-cell-phone-locker-with-keyed-lock-hasp-master-door-open-black>: HTTP status code is not handled or not allowed 2026-01-24 21:07:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-gtx-sport-600-ho-sdi-594cc-snowmobile-replacement-battery-2007-2008-12v>: HTTP status code is not handled or not allowed 2026-01-24 21:07:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-plastic-locker-11-961217820-six-tier-12-x-18-x-12-1-wide-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:07:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-plastic-locker-11-951215600-five-tier-12-x-15-x-12-1-wide-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:07:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/digital-caliper-absolute-origin-0-12-in-300-mm-ip54-factory-calibrated>: HTTP status code is not handled or not allowed 2026-01-24 21:07:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-plastic-locker-11-911818720-single-tier-18-x-18-x-72-1-wide-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:07:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-ski-doo-renegade-backcountry-800r-800cc-snowmobile-battery-2014-2016-12v>: HTTP status code is not handled or not allowed 2026-01-24 21:07:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-plastic-locker-11-961217820-six-tier-12-x-18-x-12-1-wide-burgundy>: HTTP status code is not handled or not allowed 2026-01-24 21:07:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-stainless-steel-toggle-switch-in-enclosure-30-amp-600v>: HTTP status code is not handled or not allowed 2026-01-24 21:07:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/al3346g-gfci-surge-receptacle-cover-6l>: HTTP status code is not handled or not allowed 2026-01-24 21:07:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-plastic-locker-11-921215720-double-tier-12-x-15-x-36-1-wide-charcoal>: HTTP status code is not handled or not allowed 2026-01-24 21:07:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/courier-3000-shipping-scale-70-lb-cap-x-005-lb-readability-13-x-12-58-platform>: HTTP status code is not handled or not allowed 2026-01-24 21:07:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-plus-phenolic-locker-11-8z1515721-z-style-15-x-15-x-72-silver-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:07:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-ski-doo-mx-z-600-ho-adrenaline-593cc-snowmobile-replacement-battery-2006-12v>: HTTP status code is not handled or not allowed 2026-01-24 21:07:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-725-2-piece-large-port-ball-valve-carbon-steel-34-threaded>: HTTP status code is not handled or not allowed 2026-01-24 21:07:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yamaha-rst90gt-rs-venture-gt-1049cc-snowmobile-replacement-battery-2009-2015-12v>: HTTP status code is not handled or not allowed 2026-01-24 21:07:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/removable-self-adhesive-display-protector-8-1-2w-x-11h-clear-10-pk>: HTTP status code is not handled or not allowed 2026-01-24 21:07:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-ski-doo-mx-z-sport-ace-600-600cc-snowmobile-replacement-battery-2014-2015-12v>: HTTP status code is not handled or not allowed 2026-01-24 21:07:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-522-307-short-block-with-cast-iron-bore-for-engine-models-ch740>: HTTP status code is not handled or not allowed 2026-01-24 21:07:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3rd-generation-smart-tsa-travel-padlock-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:07:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ch740lpng-e09-gas-engine-horizontal-shaft-25-hp>: HTTP status code is not handled or not allowed 2026-01-24 21:07:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kawasaki-kaf400-mule-sx-4x4-xc-se-utv-replacement-battery-2018-2019-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:08:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sealer-sales-kf-series-20-hand-sealer-w-5mm-seal>: HTTP status code is not handled or not allowed 2026-01-24 21:08:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inverter-duty-motor-y996-215thfw7726-10hp-230-460v-1800rpm-3ph-215tc-tefc>: HTTP status code is not handled or not allowed 2026-01-24 21:08:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4930-double-sided-vhb-acrylic-foam-tape-3-4-x-5-yds-25-mil-white>: HTTP status code is not handled or not allowed 2026-01-24 21:08:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-514pnl-lf-brass-sillcock-valve-34-f1807-pex>: HTTP status code is not handled or not allowed 2026-01-24 21:08:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/r-673nl-lf-brass-14-turn-sillcock-valve-12-sweat>: HTTP status code is not handled or not allowed 2026-01-24 21:08:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/small-room-fan-forced-wall-heater-sra2027dsf-2000-1500w-277-240v>: HTTP status code is not handled or not allowed 2026-01-24 21:08:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inverter-duty-motor-y364-56h17t2021-1hp-230-460v-1800rpm-3ph-56c-tefc>: HTTP status code is not handled or not allowed 2026-01-24 21:08:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/professional-aluminum-pizza-peel-12>: HTTP status code is not handled or not allowed 2026-01-24 21:08:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/explosion-proof-motor-g650-056c11g5306-1-4hp-115-208-230v-1200rpm-1ph-epfc>: HTTP status code is not handled or not allowed 2026-01-24 21:08:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-337di-g-butterfly-valve-with-buna-seals--gear-operator-10-wafer>: HTTP status code is not handled or not allowed 2026-01-24 21:08:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ferno-ille-3000-shower-trolley-medical-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:08:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 22927 pages (at 95 pages/min), scraped 12885 items (at 21 items/min) 2026-01-24 21:08:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-311-cast-iron-swing-check-valve-2-flanged>: HTTP status code is not handled or not allowed 2026-01-24 21:08:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/siemens-lem-portable-x-ray-medical-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 21:08:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/parks-medical-1al-doppler-medical-replacement-battery-1-3ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:08:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/phs-west-ergo-express-3000-cart-medical-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:08:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harmar-sl350ac-indoor-straight-stairlift-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:08:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-335di-butterfly-valve-with-epdm-seals--10-position-handle-5-wafer>: HTTP status code is not handled or not allowed 2026-01-24 21:08:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-337di-g-butterfly-valve-with-buna-seals--gear-operator-3-wafer>: HTTP status code is not handled or not allowed 2026-01-24 21:08:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/solar-jumper850-j850-jump-starter-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:08:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shuriken-sk-bt20-sealed-lead-acid-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:08:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/apex-battery-apx1270f1-sealed-lead-acid-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:08:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-pump-motor-u536-200hp-460v-3600rpm-3ph-444ts-fr-dp>: HTTP status code is not handled or not allowed 2026-01-24 21:08:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/118608-dock-watcher-forklift-sensor-1-box-1-sensor-2-remote-sensors-15-cord>: HTTP status code is not handled or not allowed 2026-01-24 21:08:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u-channel-sign-post-7l-112-lbsft-galvanized-post-holes-half-way-down-post>: HTTP status code is not handled or not allowed 2026-01-24 21:08:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pharmacia-deltec-delphin-700-medical-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:08:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_alarms_signals/forklift_alert_sensors_systems already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:08:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/118602-overhead-forklift-door-monitor-2-boxes-2-sensors-2-lights-15-cord>: HTTP status code is not handled or not allowed 2026-01-24 21:08:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/solar-booster-pac-es5000-jump-starter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:08:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/apex-battery-apx12100s-sealed-lead-acid-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:08:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s036-056c17d2067-3-4hp-1800rpm-115-230v-1ph-56-fr-dp>: HTTP status code is not handled or not allowed 2026-01-24 21:08:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e264-5kcr46mn0055y-1-2hp-1725-1425rpm-100-120-200-240v-1ph-56c-fr-tefc>: HTTP status code is not handled or not allowed 2026-01-24 21:08:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/springer-contr-merz-w151-3-i2-da-20a-3-pole-encl-reversing-switch-maintained-lockable-handle>: HTTP status code is not handled or not allowed 2026-01-24 21:08:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-312-ductile-iron-check-valve-with-epdm-seats-stainless-steel-disc-4-wafer>: HTTP status code is not handled or not allowed 2026-01-24 21:08:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mighty-max-ml8-12-sealed-lead-acid-replacement-battery-8ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:08:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-312-ductile-iron-check-valve-with-epdm-seats-aluminum-bronze-disc-6-wafer>: HTTP status code is not handled or not allowed 2026-01-24 21:08:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-e-premium-flame-resistant-rain-bib-pants-hrc2-yellow-5xl>: HTTP status code is not handled or not allowed 2026-01-24 21:08:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-e-premium-flame-resistant-rain-bib-pants-hrc2-yellow-6x>: HTTP status code is not handled or not allowed 2026-01-24 21:08:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cooling-tower-duty-motor-y390-60-15-hp-1800-900-rpm-460v-3ph-365t-fr-tefc>: HTTP status code is not handled or not allowed 2026-01-24 21:08:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_alarms_signals/forklift_alert_sensors_systems>: HTTP status code is not handled or not allowed 2026-01-24 21:08:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/refrigeration-duty-motor-x507-1hp-3-4hp-900rpm-200-230-460v-3ph-56y-opao>: HTTP status code is not handled or not allowed 2026-01-24 21:08:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-head-solar-street-parking-lot-led-area-light-150w-31500-lm-black>: HTTP status code is not handled or not allowed 2026-01-24 21:08:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hvac-motor-hg706-5kh37pna569t-1-3-1-8hp-1725-1140rpm-115v-split-ph-56z-fr>: HTTP status code is not handled or not allowed 2026-01-24 21:08:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hvac-motor-g142-056t17d5347-1hp-1800rpm-208-230-460v-3-ph-56-fr>: HTTP status code is not handled or not allowed 2026-01-24 21:08:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-pipe-fitting-90-degree-two-socket-tee-1-1-4-dia>: HTTP status code is not handled or not allowed 2026-01-24 21:08:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oil-burner-motor-o011-48c34d2028-1-2hp-3600rpm-115-208-230v-1ph-48nz-dp>: HTTP status code is not handled or not allowed 2026-01-24 21:08:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fan-blower-motor-x019-48a11o1351-1-3hp-1075rpm-277v-1ph-48y-fr-opao>: HTTP status code is not handled or not allowed 2026-01-24 21:08:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-nexelon-blue-wire-shelving-unit-add-on-42w-x-21d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 21:08:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/orange-traffic-cone-mount-retracting-belt-barrier-10-orangeblack-diagonal-stripe-belt>: HTTP status code is not handled or not allowed 2026-01-24 21:08:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/orange-traffic-cone-mount-retracting-belt-barrier-10-yellowblack-stripe-belt>: HTTP status code is not handled or not allowed 2026-01-24 21:08:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-nexelon-blue-wire-shelving-unit-add-on-42w-x-14d-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 21:08:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-nexelon-blue-wire-shelving-unit-add-on-60w-x-21d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 21:08:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/batterymart-sla-18-sealed-lead-acid-replacement-battery-18ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:08:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gruber-power-gps12-6-sealed-lead-acid-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:08:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wtc-2d-3048-ll-48w-x-30d-work-center-cabinet-with-butcher-block-top>: HTTP status code is not handled or not allowed 2026-01-24 21:08:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-nexelon-blue-wire-shelving-unit-starter-36w-x-24d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 21:08:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-shelf-nexelon-blue-wire-shelving-unit-starter-54w-x-21d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 21:08:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-tee-23200-use-for-1516w-grid-white>: HTTP status code is not handled or not allowed 2026-01-24 21:08:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/duracell-durhr12-370-sealed-lead-acid-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:08:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chewy-chocolate-caramels-individually-wrapped-187-lbs-6carton>: HTTP status code is not handled or not allowed 2026-01-24 21:08:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ironguard-steel-faced-dock-bumper-22-1100-13-w-x-4-d-x-20-h>: HTTP status code is not handled or not allowed 2026-01-24 21:08:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thirst-quencher-mix-pouch-fruit-punch-21-oz>: HTTP status code is not handled or not allowed 2026-01-24 21:08:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/batterymart-sla-12-f2-sealed-lead-acid-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:08:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thirst-quencher-energy-drink-fruit-punch-24-oz-24carton>: HTTP status code is not handled or not allowed 2026-01-24 21:08:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-bag-candy-assorted-flavors-5-lbs-8bag>: HTTP status code is not handled or not allowed 2026-01-24 21:08:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bubble-gum-300-pieces-8carton>: HTTP status code is not handled or not allowed 2026-01-24 21:08:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ironguard-laminated-dock-bumper-24-1053-26-w-x-4-1-2-d-x-10-h>: HTTP status code is not handled or not allowed 2026-01-24 21:08:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pp-150-gal-conical-bottom-tank-w-cover-32dia-x-57h-1-4wall-off-white>: HTTP status code is not handled or not allowed 2026-01-24 21:08:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-shelf-nexelate-silver-epoxy-wire-shelving-unit-add-on-60w-x-14d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 21:08:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vertical-5-10-cylinder-aluminum-storage-cabinet-30w-x-32d-x-65h>: HTTP status code is not handled or not allowed 2026-01-24 21:08:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/business-card-size-thermal-laminating-pouches-5-mil-3-3-4-x-2-3-8-20-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:08:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smartcut-easyblade-plus-trimmer-replacement-cartridge>: HTTP status code is not handled or not allowed 2026-01-24 21:08:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-latching-55-gallon-drum-lid-with-vent-and-gasket>: HTTP status code is not handled or not allowed 2026-01-24 21:08:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/invacare-atm-take-along-wheelchair-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:08:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quasar-500-manual-comb-binding-machine-18-1-8-x-15-3-8-x-5-1-8-metallic-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:08:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96-gallon-2-door-self-close-paint-ink-cabinet-34w-x-34d-x-65h-red>: HTTP status code is not handled or not allowed 2026-01-24 21:08:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-nexelate-silver-epoxy-wire-shelving-unit-add-on-72w-x-14d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 21:08:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merits-p170-p171w-mp11-wheelchair-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:09:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-55-gal-drum-manual-flammable-cabinet-incl-drum-support-34w-x-34d-x-65h-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:09:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powerpack-2-6-portable-power-system>: HTTP status code is not handled or not allowed 2026-01-24 21:09:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-30-gal-drum-self-close-flammable-cabinet-incl-drum-rollers-43w-x-34d-x-65hgray>: HTTP status code is not handled or not allowed 2026-01-24 21:09:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adapter-39005075-38-male-pipe-x-38-female-pipe-swivel>: HTTP status code is not handled or not allowed 2026-01-24 21:09:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fallte-7035xl-journeyman-3-d-full-body-harness-3-d-rings-size-x-large>: HTTP status code is not handled or not allowed 2026-01-24 21:09:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/amigo-mobility-mcx-690000b-wheelchair-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:09:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adapter-39005275-34-male-pipe-x-34-female-pipe-swivel>: HTTP status code is not handled or not allowed 2026-01-24 21:09:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/apache-dust-cap-39041852-12-body-2-ea-for-use-on-coupler-female-half-of-connection>: HTTP status code is not handled or not allowed 2026-01-24 21:09:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fallte-82706sb1-duratech-6-compact-web-srd-with-steel-carabiner-and-snap-hook>: HTTP status code is not handled or not allowed 2026-01-24 21:09:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fallte-8209-4-to-6-restraint-lanyard-adjustable-single-leg-with-2-snap-hooks>: HTTP status code is not handled or not allowed 2026-01-24 21:09:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machine-enclosure-panel-10w-x-5-6h>: HTTP status code is not handled or not allowed 2026-01-24 21:09:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/everest-jennings-sprint-wheelchair-replacement-battery-55ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:09:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wheelcare-sporster-stretch-wheelchair-replacement-battery-75ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:09:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-inlet-blower-wheel-6-1-4-inch-dia-cw-2000-rpm-1-2-inch-bore-4-inch-w-galvanized>: HTTP status code is not handled or not allowed 2026-01-24 21:09:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 23021 pages (at 94 pages/min), scraped 12899 items (at 14 items/min) 2026-01-24 21:09:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-poly-z-brite-wire-shelving-unit-add-on-48w-x-14d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 21:09:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-shelf-poly-z-brite-wire-shelving-unit-add-on-72w-x-21d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 21:09:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-oil-bearings-with-insulator>: HTTP status code is not handled or not allowed 2026-01-24 21:09:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-inlet-blower-wheel-9-1-2-inch-dia-cw-or-ccw-9-1-2-inch-w-3-4-inch-bore>: HTTP status code is not handled or not allowed 2026-01-24 21:09:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/w-01353-1-1-2-in-x-3-4-in-wrot-copper-reducer-coupling-street-x-copper>: HTTP status code is not handled or not allowed 2026-01-24 21:09:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-inlet-blower-wheel-10-5-8-inch-dia-cw-or-ccw-10-5-8-inch-w-5-8-inch-bore>: HTTP status code is not handled or not allowed 2026-01-24 21:09:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-inlet-blower-wheel-6-5-16-inch-dia-cw-2000-rpm-1-2-inch-bore-3-7-16-inch-w-galvanized>: HTTP status code is not handled or not allowed 2026-01-24 21:09:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-inlet-blower-wheel-6-5-16-inch-dia-ccw-2000-rpm-1-2-inch-bore-2-1-2-inch-w-galvanized>: HTTP status code is not handled or not allowed 2026-01-24 21:09:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-poly-z-brite-wire-shelving-unit-starter-54w-x-24d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 21:09:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-poly-z-brite-wire-shelving-unit-starter-24w-x-14d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 21:09:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acl-f-36x54-combination-louver-damper-with-flange-36wx54h>: HTTP status code is not handled or not allowed 2026-01-24 21:09:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/everest-jennings-xcaliber-wheelchair-replacement-battery-55ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:09:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dfl-f-48x24-drainable-blade-fixed-louver-with-flange-48wx24h>: HTTP status code is not handled or not allowed 2026-01-24 21:09:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hv-34-high-velocity-fan-34-230460v-3-ph>: HTTP status code is not handled or not allowed 2026-01-24 21:09:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-inlet-blower-wheel-4-27-32-inch-dia-cw-3600-rpm-1-2-inch-bore-7-inch-w-galvanized>: HTTP status code is not handled or not allowed 2026-01-24 21:09:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-inlet-blower-wheel-7-1-2-inch-dia-cw-1800-rpm-1-4-inch-bore-6-11-32-inch-w-aluminium>: HTTP status code is not handled or not allowed 2026-01-24 21:09:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pp2-bl-58p-next-generation-rolling-blue-sign-base-w-58-pole>: HTTP status code is not handled or not allowed 2026-01-24 21:09:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arctic-insulated-bin-lid-966c-50197-48l-x-43w-x-3h-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:09:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portalac-gs-pe4512-emergency-light-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:09:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelite-3-shelf-vented-plastic-mat-shelving-unit-starter-54w-x-21d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 21:09:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelite-3-shelf-vented-plastic-mat-shelving-unit-starter-54w-x-21d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 21:09:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/900-112-hobby-knife-small>: HTTP status code is not handled or not allowed 2026-01-24 21:09:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelite-3-shelf-vented-plastic-mat-shelving-unit-starter-54w-x-18d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 21:09:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-inlet-blower-wheel-5-1-4-inch-dia-ccw-3450-rpm-1-2-inch-bore-2-1-2-inch-w-galvanized>: HTTP status code is not handled or not allowed 2026-01-24 21:09:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/teledyne-2cl12s15-emergency-light-replacement-battery-18ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:09:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/professional-series-depth-adjustable-monitor-arm>: HTTP status code is not handled or not allowed 2026-01-24 21:09:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/300-015-crimper-d-sub-closed-barrel-contacts-4-way-indent-awg-26-20>: HTTP status code is not handled or not allowed 2026-01-24 21:09:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/big-beam-2tc12s20-emergency-light-replacement-battery-26ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:09:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-bucket-5-gallon-capacity-987555>: HTTP status code is not handled or not allowed 2026-01-24 21:09:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/die-cut-shipping-labels-24-x-39-white-300-roll>: HTTP status code is not handled or not allowed 2026-01-24 21:09:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/p-touch-tze-labeling-tape-1w-white-on-black>: HTTP status code is not handled or not allowed 2026-01-24 21:09:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/service-vehicle-compressor-w11hgb30t-11hp-honda-electric-start-175-psi-30-gal-tank>: HTTP status code is not handled or not allowed 2026-01-24 21:09:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/p-touch-tc-tape-cartridges-1-2w-black-on-white-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:09:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/p-touch-tze-labeling-tape-3-4w-red-on-white>: HTTP status code is not handled or not allowed 2026-01-24 21:09:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-ring-business-card-binder>: HTTP status code is not handled or not allowed 2026-01-24 21:09:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/filter-ntbk-privacy-156-bk>: HTTP status code is not handled or not allowed 2026-01-24 21:09:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-high-cap-boltless-shelving-add-on-72-in-w-x-24-in-d-x-96-in-h-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:09:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/label_tapes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:09:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/filter-lcd-privacy-185-bk>: HTTP status code is not handled or not allowed 2026-01-24 21:09:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rolled-formed-post-1-1-4-inch-x-1-1-4-inch-x-3ft3-inch-h-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:09:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergi-lite-80-8000-emergency-light-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:09:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnetic-red-strips-2-x-7-8-25-per-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:09:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rc4ctcbk-poke-thru-flange-slide-holder-assembly-black>: HTTP status code is not handled or not allowed 2026-01-24 21:09:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chloride-power-cmf25ts2-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:09:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u-design-rooftop-platforms-40-inch-h-support-legs-urtsl40>: HTTP status code is not handled or not allowed 2026-01-24 21:09:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-wide-boltless-shelving-add-on-72-in-w-x-15-in-d-x-84-in-h>: HTTP status code is not handled or not allowed 2026-01-24 21:09:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:09:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hl10000-corner-l-steel-desk-left-hand-return-66-x-72-charcoal-mahogany>: HTTP status code is not handled or not allowed 2026-01-24 21:09:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-wide-boltless-shelving-add-on-96in-w-x-24in-d-x-60in-h-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:09:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/label_tapes>: HTTP status code is not handled or not allowed 2026-01-24 21:09:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pgs-3122-1-1-drop-in-stainless-steel-1-hole-large-single-bowl-sink>: HTTP status code is not handled or not allowed 2026-01-24 21:09:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wm-cascade-7-pocket-magazine-rack-light-oak>: HTTP status code is not handled or not allowed 2026-01-24 21:09:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wm-valley-bariatric-guest-chair-with-sled-base-solid-vinyl-black-light-oak>: HTTP status code is not handled or not allowed 2026-01-24 21:09:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ballistic-blanket-level-iii-a-standard-48l-x-24w-black>: HTTP status code is not handled or not allowed 2026-01-24 21:09:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lithonia-lithonia-elsq-emergency-light-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 21:09:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wirecraftersrackback-wire-mesh-pallet-rack-enclosure-hinged-door-108x120-797134>: HTTP status code is not handled or not allowed 2026-01-24 21:09:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wirecraftersrackback-wire-mesh-pallet-rack-enclosure-hinged-door-144x96-797139>: HTTP status code is not handled or not allowed 2026-01-24 21:09:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/value-line-storage-cabinet-all-welded-30w-x-18d-x-72h-textured-black>: HTTP status code is not handled or not allowed 2026-01-24 21:09:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spill-pallet-p4-flexible-model-with-drain>: HTTP status code is not handled or not allowed 2026-01-24 21:09:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elite-bookcase-assembled-36w-x-12d-x-72h-putty>: HTTP status code is not handled or not allowed 2026-01-24 21:09:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks>: HTTP status code is not handled or not allowed 2026-01-24 21:09:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/made-in-usa-rng-32-c-2-carbide-insert>: HTTP status code is not handled or not allowed 2026-01-24 21:09:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ps-88-f-sand-privacy-screen>: HTTP status code is not handled or not allowed 2026-01-24 21:09:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/female-ballistic-bullet-resistant-vest-level-iii-a-foam-xl-black>: HTTP status code is not handled or not allowed 2026-01-24 21:09:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/five-shelf-extra-hd-boltless-shelving-add-on-36inw-x-24ind-x-96inh-laminate-deck-usa>: HTTP status code is not handled or not allowed 2026-01-24 21:09:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/four-shelf-extra-hd-boltless-shelving-add-on-36inw-x-12ind-x-60inh-wood-deck-usa>: HTTP status code is not handled or not allowed 2026-01-24 21:09:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/55-gal-drum-gauge-stainless-bushing-rods-float-2-fitting-33-1-2-depth>: HTTP status code is not handled or not allowed 2026-01-24 21:09:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vexilar-underwater-fish-scout-fsdv01dt-fish-finder-battery-7ah-12v>: HTTP status code is not handled or not allowed 2026-01-24 21:09:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/awsw-partition-70x7-f-navy-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:10:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/material-handling/drum_handling_equipment/drum_liquid_level_alarms_gauges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:10:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regenerative-blower-ab-1002-3-phase-2-stage-115-hp>: HTTP status code is not handled or not allowed 2026-01-24 21:10:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/five-shelf-extra-hd-boltless-shelving-add-on-48inw-x-18ind-x-60inh-wood-deck-usa>: HTTP status code is not handled or not allowed 2026-01-24 21:10:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smoking-shelter-s52fdkb-4sided-wleft-open-front-15l-x-5w-flat-roof-dk-bronze>: HTTP status code is not handled or not allowed 2026-01-24 21:10:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smoking-shelter-s64vrdkb-3sided-left-open-front-15l-x-10w-vented-standing-seam-roof-dk-brz>: HTTP status code is not handled or not allowed 2026-01-24 21:10:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/five-shelf-heavy-duty-boltless-shelving-add-on-36in-w-x-18in-d-x-84in-h-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:10:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/suzuki-quadsport-lt-z50-50cc-atv-replacement-battery-2006-2011-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:10:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mongoose-cbal24v450-electric-bicycle-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 21:10:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/five-shelf-heavy-duty-boltless-shelving-add-on-48in-w-x-12in-d-x-60in-h-wire-deck-usa>: HTTP status code is not handled or not allowed 2026-01-24 21:10:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd18-one-18-skate>: HTTP status code is not handled or not allowed 2026-01-24 21:10:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd24-one-24-skate>: HTTP status code is not handled or not allowed 2026-01-24 21:10:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 23117 pages (at 96 pages/min), scraped 12919 items (at 20 items/min) 2026-01-24 21:10:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/material-handling/drum_handling_equipment/drum_liquid_level_alarms_gauges>: HTTP status code is not handled or not allowed 2026-01-24 21:10:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/933xl-high-yield-yellow-original-ink-cartridge>: HTTP status code is not handled or not allowed 2026-01-24 21:10:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qvtrk-192-individual-v-track-length-192>: HTTP status code is not handled or not allowed 2026-01-24 21:10:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/model-1-1-ton-imported-single-leverage-arbor-press>: HTTP status code is not handled or not allowed 2026-01-24 21:10:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polaris-all-425cc-models-425cc-atv-replacement-battery-1995-2002-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:10:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qvtrk-3-216est-double-unit-track-assembly-kit-3-tracks-216>: HTTP status code is not handled or not allowed 2026-01-24 21:10:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mt182418-2k195-machine-table-with-one-shelf-24w-x-18d-x-18h>: HTTP status code is not handled or not allowed 2026-01-24 21:10:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yamaha-yzf750r-750cc-motorcycle-replacement-battery-1994-1998-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:10:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mt364836-3k295-machine-table-with-2-shelves-48w-x-36d-x-36h>: HTTP status code is not handled or not allowed 2026-01-24 21:10:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-cabinet-workbench-with-2-shelves--floor-lock-48w-x-30d-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:10:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polaris-dc-classic-1507cc-motorcycle-replacement-battery-1998-2004-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:10:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-cabinet-workbench-with-3-shelves--wheel-brakes-48w-x-24d-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:10:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbi-sala-nano-lok-twin-leg-arc-flash-self-retracting-lifeline-web--aluminum-rebar-hook-8l>: HTTP status code is not handled or not allowed 2026-01-24 21:10:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/920-case-16-11-16l-x-13-3-8w-x-6-13-16h-orange>: HTTP status code is not handled or not allowed 2026-01-24 21:10:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/k2002-carbon-eyebolt-plain-pattern-5-16-18-1-1-8-shank>: HTTP status code is not handled or not allowed 2026-01-24 21:10:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-ceiling-hanging-bracket-mounting-hardware-salamander-heater>: HTTP status code is not handled or not allowed 2026-01-24 21:10:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/victory-gunner-1731cc-motorcycle-replacement-battery-2014-2017-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:10:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/940-case-21-11-16l-x-16-7-8w-x-8-1-2h-black>: HTTP status code is not handled or not allowed 2026-01-24 21:10:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-utility-cart-het-3672-2-5k-95-4000-lb-cap-2-shelves-8-wheels>: HTTP status code is not handled or not allowed 2026-01-24 21:10:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heat-shield-blanket-36x30-3630-1>: HTTP status code is not handled or not allowed 2026-01-24 21:10:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/woolen-fire-blanket-21-610>: HTTP status code is not handled or not allowed 2026-01-24 21:10:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/footwear_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:10:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harley-davidson-cvo-softail-flstse3-1803cc-motorcycle-battery-2012-12v>: HTTP status code is not handled or not allowed 2026-01-24 21:10:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-welded-workbench-48-x-30-open-base-steel-square-edge>: HTTP status code is not handled or not allowed 2026-01-24 21:10:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/toner-ivr-dell-b3460-bk>: HTTP status code is not handled or not allowed 2026-01-24 21:10:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electric-baseboard-heater-2k2405a-500w-240v-27l-almond>: HTTP status code is not handled or not allowed 2026-01-24 21:10:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lug-gray-fda-plastic-tub-for-lug-carts-25-l-x-15-1-2-w-x-8-3-4-h-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:10:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electric-baseboard-heater-3k2407a-750w-240v-36l-almond>: HTTP status code is not handled or not allowed 2026-01-24 21:10:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd-welded-workbench-72-x-36-lower-shelf--pegboard-panel-steel-square-edge>: HTTP status code is not handled or not allowed 2026-01-24 21:10:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bsm-inc-elite-ada-ss-sink-1-station-with-metering-faucet-wall-mount-24l-x-21w-x-5d-sink>: HTTP status code is not handled or not allowed 2026-01-24 21:10:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/286960000-graduated-cylinder-tpx-polymethylpentene-1000ml>: HTTP status code is not handled or not allowed 2026-01-24 21:10:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yamaha-fxi-700-700cc-personal-watercraft-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:10:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kawasaki-kz1000-j-1000cc-motorcycle-replacement-battery-1981-1983-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:10:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securio-autofeed-300c-300-sheet-stack-5-7-sheet-micro-cut-9-gal-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:10:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-cabinet-workbench-with-steel-square-edge-48w-x-30d-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:10:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/footwear_accessories>: HTTP status code is not handled or not allowed 2026-01-24 21:10:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/bin_cabinets/stationary_bin_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:10:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gssafe-1702-class-2-heavy-duty-safety-vest-orange-4xl>: HTTP status code is not handled or not allowed 2026-01-24 21:10:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gssafe-5113-class-3-microfiber-birdseye-short-sleeve-t-shrit-w-black-bottom-lime-l>: HTTP status code is not handled or not allowed 2026-01-24 21:10:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/si-1400-incubator-genie-shaking-incubator-120v>: HTTP status code is not handled or not allowed 2026-01-24 21:10:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/voyager-gt-overboots-cleated-outsole-s-11h-black>: HTTP status code is not handled or not allowed 2026-01-24 21:10:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/imported-3-64-3-4-inch-key-type-light-medium-duty-drill-chuck-3jt>: HTTP status code is not handled or not allowed 2026-01-24 21:10:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-stp-stl-all-direct-safet-roll-lad-wt-actu-16-perf-stp-wa-ad-082414p>: HTTP status code is not handled or not allowed 2026-01-24 21:10:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bollard-cover-cl1385qq64-4-1-2dia-x-64h-black-w-no-tape>: HTTP status code is not handled or not allowed 2026-01-24 21:10:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelate-42w-x-18d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:10:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-stp-stl-safet-roll-lad-wt-actu-stp-16-serr-stp-wa072414g>: HTTP status code is not handled or not allowed 2026-01-24 21:10:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f2-19m-str-2-sludge-dispersant-de-icer-1-gal-tip-pour>: HTTP status code is not handled or not allowed 2026-01-24 21:10:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/cabinets/bin_cabinets/stationary_bin_cabinets>: HTTP status code is not handled or not allowed 2026-01-24 21:10:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/264-91m-spin-on-oil-filter-replacement-element-only-less-top>: HTTP status code is not handled or not allowed 2026-01-24 21:10:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/daytona-3-gt-s35006gt-electric-scooter-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:10:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelate-48w-x-21d-x-80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 21:10:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kn-rp47c-rattle-paddle-47-red>: HTTP status code is not handled or not allowed 2026-01-24 21:10:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelate-36w-x-30d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:10:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-security-pass-thru-drawer-w-internal-speaker-ul-approved-bullet-proof>: HTTP status code is not handled or not allowed 2026-01-24 21:10:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thru-wall-drawer-medium-transfer-area-ul-approved-bullet-resistant-faceplate-insert>: HTTP status code is not handled or not allowed 2026-01-24 21:10:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelate-48w-x-24d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:10:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/invacare-panthermx-4-mobility-scooter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 21:10:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-oal-straight-galvanized-steel-roller-conveyor-1-3-8-roller-dia-16-bf-3-axle-center>: HTTP status code is not handled or not allowed 2026-01-24 21:10:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ccsm-small-cylinder-cart>: HTTP status code is not handled or not allowed 2026-01-24 21:10:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/du1682-ns-cantilever-rack-double-sided-upright-82w-x-16h>: HTTP status code is not handled or not allowed 2026-01-24 21:10:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelate-30w-x-18d-x-80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 21:10:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bladez-xtr-comp-4-800w-electric-scooter-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 21:10:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnetic-adapter>: HTTP status code is not handled or not allowed 2026-01-24 21:10:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clip-strip-12-station-wsuction-cup-32-12l-white>: HTTP status code is not handled or not allowed 2026-01-24 21:10:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelate-30w-x-24d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:11:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grip-tite-wire-flag-sign-holder-250dia>: HTTP status code is not handled or not allowed 2026-01-24 21:11:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelate-54w-x-18d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:11:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arctic-cat-mountain-cat-800-le-144-snowmobile-replacement-battery-12v-i>: HTTP status code is not handled or not allowed 2026-01-24 21:11:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metal-merchandising-strip-double-sided-12-stations-15-12l>: HTTP status code is not handled or not allowed 2026-01-24 21:11:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u1274-ns-cantilever-rack-single-sided-upright-74w-x-12h>: HTTP status code is not handled or not allowed 2026-01-24 21:11:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yamaha-nytro-1000cc-snowmobile-replacement-battery-2006-2010-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:11:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-drum-spillnest-utility-tray-t8106-without-grate-57-12-x-108-12-x-3-60-gallon-cap>: HTTP status code is not handled or not allowed 2026-01-24 21:11:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-nexelate-60w-x-21d-x-80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 21:11:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-gsx-se-800r-799cc-snowmobile-replacement-battery-2014-2015-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:11:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelate-48w-x-24d-x-60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 21:11:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 23214 pages (at 97 pages/min), scraped 12943 items (at 24 items/min) 2026-01-24 21:11:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelate-60w-x-36d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:11:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/complete-wheel-kit-for-manual-pallet-jack-gwk-bf-ck-fits-mighty-lift-model-ml55>: HTTP status code is not handled or not allowed 2026-01-24 21:11:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/import-carbon-steel-hex-re-threading-die-1-3-8-8-carbon-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:11:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-formula-si-497cc-snowmobile-replacement-battery-1997-1998-12v-i>: HTTP status code is not handled or not allowed 2026-01-24 21:11:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unassembled-350w-direct-drive-mixer-w-stand>: HTTP status code is not handled or not allowed 2026-01-24 21:11:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelate-24w-x-14d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:11:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-fan-blades-for-global-36-blower-fan-model-258320-600596>: HTTP status code is not handled or not allowed 2026-01-24 21:11:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-ski-doo-gtx-fan-553cc-snowmobile-replacement-battery-2009-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:11:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-multi-purpose-cleaner-degreaser-gallon-bottle4-bottlescase-640407>: HTTP status code is not handled or not allowed 2026-01-24 21:11:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelate-54w-x-18d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:11:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelate-60w-x-18d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:11:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelate-36w-x-14d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:11:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cubby-cabinet-cc-52-18-welded-34-1-2w-x-13-1-2d-x-52h-bright-red>: HTTP status code is not handled or not allowed 2026-01-24 21:11:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vttr5-2424-24-x-24-18-ga-stainless-workbench-with-5-riser-galvanized-base>: HTTP status code is not handled or not allowed 2026-01-24 21:11:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-counter-height-storage-cabinet-4218dlx-53-welded-36w-x-18d-x-42h-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:11:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vttr5-6024-60-x-24-18-ga-stainless-workbench-with-5-riser-galvanized-base>: HTTP status code is not handled or not allowed 2026-01-24 21:11:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yamaha-rx1000r-1000cc-snowmobile-replacement-battery-2003-2005-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:11:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelate-72w-x-18d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:11:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/schaefer-stacking-transport-container-14-6-2-f-pl-18-l-x-12-w-x-3-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:11:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd-rolling-door-lock-keyed-alike-hdrdl-ka>: HTTP status code is not handled or not allowed 2026-01-24 21:11:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inkcart-pg-245xl-bk>: HTTP status code is not handled or not allowed 2026-01-24 21:11:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-elite-1500-2nd-edition-1500cc-snowmobile-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:11:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/attachment-lock-short-keyed-differently-ats-kd>: HTTP status code is not handled or not allowed 2026-01-24 21:11:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arctic-cat-m1100-1100cc-snowmobile-replacement-battery-2001-2013-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 21:11:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ba-fans-basic-6-ceiling-fan-16-ft-208v-3ph>: HTTP status code is not handled or not allowed 2026-01-24 21:11:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carbide-tip-engraving-cutter-for-computer-engravers-010k>: HTTP status code is not handled or not allowed 2026-01-24 21:11:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-workstation-3-shelf-w-pegboard-48w-x-24d>: HTTP status code is not handled or not allowed 2026-01-24 21:11:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-3l800-v-belt-3-8-x-80>: HTTP status code is not handled or not allowed 2026-01-24 21:11:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vestil-steel-wheel-chock-912-l>: HTTP status code is not handled or not allowed 2026-01-24 21:11:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-black-epoxy-72w-x-18d-x-80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 21:11:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-black-epoxy-48w-x-24d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:11:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vestil-pallet-truck-109316l-x-27w-4000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:11:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-black-epoxy-42w-x-24d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:11:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-a139-v-belt-1-2-x-141>: HTTP status code is not handled or not allowed 2026-01-24 21:11:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-b40-5l430-v-belt-5-8-x-43>: HTTP status code is not handled or not allowed 2026-01-24 21:11:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chill-its-6489-2-layer-cooling-multi-band-performance-knit-s-m-hi-vis-orange>: HTTP status code is not handled or not allowed 2026-01-24 21:11:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-black-epoxy-24w-x-14d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:11:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-b198-v-belt-5-8-x-201>: HTTP status code is not handled or not allowed 2026-01-24 21:11:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/door-closer-adjustable-size-1-to-6-grade-1-stop-arm-with-hold-open>: HTTP status code is not handled or not allowed 2026-01-24 21:11:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elevated-stands-w-support-for-30-gallon-conical-bottom-tank>: HTTP status code is not handled or not allowed 2026-01-24 21:11:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8249z-s-hi-vis-surveyors-vest-class-2-economy-single-size-2xl-orange>: HTTP status code is not handled or not allowed 2026-01-24 21:11:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mth1-1630-ah-heavy-duty-machine-table-adjustable-height>: HTTP status code is not handled or not allowed 2026-01-24 21:11:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ww3036-8phfl-heavy-duty-7-gauge-mobile-workbench-8-phenolic-casters-floor-lock>: HTTP status code is not handled or not allowed 2026-01-24 21:11:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/long-john-us-flag-safety-air-gun-75lj012aaus-12-ext-34-scfm>: HTTP status code is not handled or not allowed 2026-01-24 21:11:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8246z-s-two-tone-mesh-hi-vis-safety-vest-class-2-single-size-2xl-orange>: HTTP status code is not handled or not allowed 2026-01-24 21:11:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/skullerz-8974v-class-e-safety-helmet-with-visor-kit-anti-fog-clear-lens-orange>: HTTP status code is not handled or not allowed 2026-01-24 21:11:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/freestanding-office-partitions-blue-48-14w-x-60h-238637FBL>: HTTP status code is not handled or not allowed 2026-01-24 21:11:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelite-48w-x-18d-x-80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 21:11:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-lockout-padlock-keyed-alike-1-1-2-plastic-steel-orange-6-pk>: HTTP status code is not handled or not allowed 2026-01-24 21:11:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-b225-v-belt-5-8-x-228>: HTTP status code is not handled or not allowed 2026-01-24 21:11:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-lockout-padlock-keyed-alike-1-1-2-plastic-nylon-red-12-pk>: HTTP status code is not handled or not allowed 2026-01-24 21:11:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-lockout-padlock-keyed-alike-1-1-2-aluminum-steel-red-12-pk>: HTTP status code is not handled or not allowed 2026-01-24 21:11:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-black-epoxy-36w-x-14d-x-69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 21:11:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rackguard-3-w-x-3-d-x-23-1-2-h-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:11:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelite-42w-x-21d-x-80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 21:11:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/as-iqcrew-40x-1000x-dual-illumination-microscope-with-slide-prep-kit-book-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:11:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/as-iqcrew-40x-1000x-dual-illumination-microscope-digital-eyepiece-slide-prep-kit-book>: HTTP status code is not handled or not allowed 2026-01-24 21:11:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8456-glass-citation-goblet-10-oz-24-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:11:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelite-30w-x-24d-x-60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 21:12:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-black-epoxy-48w-x-18d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:12:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hoover-174-hushtone-153-13-ch54113-upright-vacuum>: HTTP status code is not handled or not allowed 2026-01-24 21:12:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vital-id-worker-emergency-id-tag-w-data-window-3-x-2-1-2-fits-to-hard-hat-reflective-25-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:12:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelite-36w-x-24d-x-69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 21:12:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-linea-71l-x-30d-table-with-metal-legs-bark-grey>: HTTP status code is not handled or not allowed 2026-01-24 21:12:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-phase-motor-em2555t-5-100-hp-1780-rpm-404t-frame-foot-mount-opsb-575-volts>: HTTP status code is not handled or not allowed 2026-01-24 21:12:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-phase-motor-em3556t-5-1155-hp-1-rpm-145t-frame-foot-mount-tefc-575-volts>: HTTP status code is not handled or not allowed 2026-01-24 21:12:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelite-36w-x-18d-x-60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 21:12:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-black-epoxy-36w-x-21d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:12:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ken-forging-tb110-j-s-hdg-turnbuckle-assembly-stub-stub-1-8-x-30-galvanized>: HTTP status code is not handled or not allowed 2026-01-24 21:12:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-black-epoxy-54w-x-21d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:12:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 23307 pages (at 93 pages/min), scraped 12966 items (at 23 items/min) 2026-01-24 21:12:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelate-48w-x-18d-x-69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 21:12:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dynamic-rocky-industrial-climbing-helmet-polycarbonate-abs-shell-ratchet-adjustment-white>: HTTP status code is not handled or not allowed 2026-01-24 21:12:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-motor-208-230-460-v-3-hp-1160-rpm-3-ph-213tc-tefc>: HTTP status code is not handled or not allowed 2026-01-24 21:12:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washdown-motor-cesswdm3559t-3-ph-3-hp-3470-rpm-208-230-460-v-tefc-145tc-fr>: HTTP status code is not handled or not allowed 2026-01-24 21:12:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evolution-deluxe-6161-full-brim-hard-hat-hdpe-shell-6-pt-suspension-ratchet-adjustment-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:12:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kilimanjaro-full-brim-hard-hat-hdpe-shell-4-pt-nylon-webbing-cradle-ratchet-adjustment-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:12:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50-hertz-motor-vm3534-57-3-ph-033-hp-1425-ip44-rpm-230-400-volts-tefc-56c-frame>: HTTP status code is not handled or not allowed 2026-01-24 21:12:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-poly-green-42w-x-14d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:12:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dynamic-dom-cap-style-dome-hard-hat-hdpe-shell-6-pt-suspension-rachet-adjustment-white>: HTTP status code is not handled or not allowed 2026-01-24 21:12:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-poly-green-54w-x-21d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:12:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stromboli-cap-style-dome-hard-hat-abs-polycarbonate-shell-4-pt-suspension-rachet-adjustment-red>: HTTP status code is not handled or not allowed 2026-01-24 21:12:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-poly-green-42wx24dx80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 21:12:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2100m-rotating-literature-display-w-16-oversized-wire-pockets-round-base-black>: HTTP status code is not handled or not allowed 2026-01-24 21:12:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelon-54w-x-21d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:12:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1003654-00u0019-economy-retail-gondola-single-sided-36w-x-14d-x-54h-almond>: HTTP status code is not handled or not allowed 2026-01-24 21:12:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unit-handling-motor-vebm3558t-5d-3-ph-2-hp-575-v-1755-rpm-tefc-145tc-frame>: HTTP status code is not handled or not allowed 2026-01-24 21:12:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-poly-green-24w-x-18d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:12:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/prefilled-homogenizer-tubes-2-0ml-silica-glass-beads-0-1mm-acid-washed-50-pk>: HTTP status code is not handled or not allowed 2026-01-24 21:12:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-resettable-3-dial-combination-padlock-16040-c-12>: HTTP status code is not handled or not allowed 2026-01-24 21:12:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-poly-green-54wx14dx60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 21:12:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inkcart-cli-251-gy>: HTTP status code is not handled or not allowed 2026-01-24 21:12:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bare-ground-calcium-chloride-ice-melter-liquid-275-gallon-tote-bgb-275tc>: HTTP status code is not handled or not allowed 2026-01-24 21:12:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/digital-hotplate-magnetic-stirrer-7x7>: HTTP status code is not handled or not allowed 2026-01-24 21:12:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v303-gate-valve-lockout-1-25-diameter-red-00362>: HTTP status code is not handled or not allowed 2026-01-24 21:12:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s59-4016x-point-of-use-thermostatic-mixing-valve-11-gpm>: HTTP status code is not handled or not allowed 2026-01-24 21:12:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5e-ssmach-rod-end-machined-304-stainless-1-2-13-x-6>: HTTP status code is not handled or not allowed 2026-01-24 21:12:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hvac-motor-ehm2543t-8-3-ph-50-hp-200-v-1775-rpm-opsb-326t-frame>: HTTP status code is not handled or not allowed 2026-01-24 21:12:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/severe-duty-motor-ecp83580t-4-3-ph-1-hp-460-v-3450-rpm-tefc-143t-frame>: HTTP status code is not handled or not allowed 2026-01-24 21:12:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/open-head-plastic-drum-16-gallon-capacity-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:12:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extension-tubes-attatchment-for-gvc-models-390-1100-1250-1502>: HTTP status code is not handled or not allowed 2026-01-24 21:12:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rectangular-pedestal-table-36-x-72-with-walnut-top-black-x-base>: HTTP status code is not handled or not allowed 2026-01-24 21:12:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-poly-green-72wx14dx80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 21:12:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-w-x-30-d-maple-butcher-block-square-edge-workbench-adjustable-height-black>: HTTP status code is not handled or not allowed 2026-01-24 21:12:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelon-54w-x-21d-x-80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 21:12:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/intergral-tach-dc-motor-cdpt3440-075-hp-1750-rpm-tefc-56c-frame>: HTTP status code is not handled or not allowed 2026-01-24 21:12:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unit-handling-motor-vebm3554-d-3-ph-15-hp-208-230-460-v-1760-rpm-tefc-56c-frame>: HTTP status code is not handled or not allowed 2026-01-24 21:12:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7200sb-blind-stitch-sewing-machine-w-direct-drive>: HTTP status code is not handled or not allowed 2026-01-24 21:12:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-step-steel-roll-ladd-rear-exit-gate-24-30-plat-serr-rwsr107242-xr>: HTTP status code is not handled or not allowed 2026-01-24 21:12:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-step-steel-roll-ladd-rear-exit-gate-24-30-plat-serr-rwsr110242-xr>: HTTP status code is not handled or not allowed 2026-01-24 21:12:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-poly-green-30wx21dx69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 21:12:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rigid-lock-quickberm-20l-x-12w-x-12h-1795-gallons-28526>: HTTP status code is not handled or not allowed 2026-01-24 21:12:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machined-chrome-steel-needle-roller-bearing-w-inner-ring-25-x-42-x-23mm>: HTTP status code is not handled or not allowed 2026-01-24 21:12:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-poly-green-36wx18dx80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 21:12:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-shelf-steel-stock-cart-36-x-24-800-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:12:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-steel-stock-cart-36-x-24-800-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:12:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/welder-box-4-cu-ft-steel-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:12:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wesco-non-telescoping-manual-high-lift-pallet-truck-2200-lbs-27-forks-987669>: HTTP status code is not handled or not allowed 2026-01-24 21:12:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vl-san-giorgio-vsmk10am-vsg-life-jacket-solas-med-orange-adult-oversize>: HTTP status code is not handled or not allowed 2026-01-24 21:12:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evap-cooling-triangle-hat-w-cooling-towel-stars-stripes-12581>: HTTP status code is not handled or not allowed 2026-01-24 21:12:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evap-cooling-triangle-hat-w-cooling-towel-navy-western-12584>: HTTP status code is not handled or not allowed 2026-01-24 21:12:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dc-power-hydraulic-double-scissor-li-cart-1100-lb-cap>: HTTP status code is not handled or not allowed 2026-01-24 21:12:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-poly-green-24wx14dx69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 21:12:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-pegboard-a-frame-afpb1s2448-tl60-60-tall-1-sided>: HTTP status code is not handled or not allowed 2026-01-24 21:12:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fireproof-storage-cabinet-cf4436-daw-1-hour-fire-rating-36-x-19-14-x-44-arctic-white>: HTTP status code is not handled or not allowed 2026-01-24 21:12:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dc-control-accessory-bc144-forward-brake-reverse-switch-kit-for-bc140>: HTTP status code is not handled or not allowed 2026-01-24 21:12:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/square-hole-pegboard-strip-18Wx45H-white-epoxy>: HTTP status code is not handled or not allowed 2026-01-24 21:12:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-hydraulic-hose-3-8-id-10-l-3-8-18-nptf>: HTTP status code is not handled or not allowed 2026-01-24 21:12:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manifold-block-with-2-needle-valves-6-inch>: HTTP status code is not handled or not allowed 2026-01-24 21:12:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rpm-iii-dc-motor-tachometer-kit-417708-126-dpg-dpfv-tenv-frames-dc180-c180-dc2112>: HTTP status code is not handled or not allowed 2026-01-24 21:12:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-nexelon-36w-x-21d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:12:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speedclean-starter-kit-wnk01>: HTTP status code is not handled or not allowed 2026-01-24 21:12:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fox-and-hound-hotwire-live-wire-tone-and-probe-kit-w-led-headlights>: HTTP status code is not handled or not allowed 2026-01-24 21:12:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-ac-motor-c-face-kit-field-conversion-37-1402-odp-37-254-6uc-tc-frame>: HTTP status code is not handled or not allowed 2026-01-24 21:13:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conduit-box-kit-standard-size-34cb5001a01sp-48-nema-frame>: HTTP status code is not handled or not allowed 2026-01-24 21:13:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-in-1-environmental-meter-with-air-flow-red>: HTTP status code is not handled or not allowed 2026-01-24 21:13:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/constant-velocity-blower-cooling-conversion-kit-blwl12-l-1-ph-115v-324tc-326tc-nema-frame>: HTTP status code is not handled or not allowed 2026-01-24 21:13:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelon-24w-x-21d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:13:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelon-54w-x-14d-x-69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 21:13:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waring-wdm120-milk-shake-mixer-single-spindle-3-speed-120v>: HTTP status code is not handled or not allowed 2026-01-24 21:13:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waring-wfp14s2-batch-bowl-sealed-with-liquid-lock>: HTTP status code is not handled or not allowed 2026-01-24 21:13:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/celltreat-scientific-products-250l-pipette-tips-mla-stacked-dense-rack-nonsterile-1000-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:13:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelon-36w-x-14d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:13:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qsms2402-mini-split-stand-24h-dual-fan-stand>: HTTP status code is not handled or not allowed 2026-01-24 21:13:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/celltreat-scientific-products-90l-pipette-mla-dtipper-silver>: HTTP status code is not handled or not allowed 2026-01-24 21:13:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 23406 pages (at 99 pages/min), scraped 12990 items (at 24 items/min) 2026-01-24 21:13:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/celltreat-scientific-products-3l-pipette-mla-micro-dtipper-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:13:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelon-72w-x-21d-x-69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 21:13:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-bowls-12-oz-reusable-disposable-125-pack-white>: HTTP status code is not handled or not allowed 2026-01-24 21:13:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90007-salad-spinner-inner-basket>: HTTP status code is not handled or not allowed 2026-01-24 21:13:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90009-meat-tenderizer-red-6-count>: HTTP status code is not handled or not allowed 2026-01-24 21:13:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lids-for-styrofoam-cups-lift-n-lock-fold-tab-16-oz-1000-carton-white>: HTTP status code is not handled or not allowed 2026-01-24 21:13:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/celltreat-scientific-products-1000l-pipette-ovation-f1-fixed-volume-one-stroke>: HTTP status code is not handled or not allowed 2026-01-24 21:13:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/we-mc400-14375-vhs-alternate-coupling-bore-size-14375-frame-404tp-or-405tp>: HTTP status code is not handled or not allowed 2026-01-24 21:13:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/celltreat-scientific-products-100ml-reagent-reservoir-bulk-nonsterile-100-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:13:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/we-mc280-1-vhs-alternate-coupling-bore-size-1-frame-284tp-or-286tp>: HTTP status code is not handled or not allowed 2026-01-24 21:13:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/knives-medium-weight-polypropylene-1000-carton-white>: HTTP status code is not handled or not allowed 2026-01-24 21:13:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/celltreat-scientific-products-0220l-pipette-ovation-m-mechanical>: HTTP status code is not handled or not allowed 2026-01-24 21:13:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/r036c-6-air-sander-vacuum-ready-332-orbit-dia-13000-rpm>: HTTP status code is not handled or not allowed 2026-01-24 21:13:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/omnicart-cab-60-cap-w-locking-panel-woodgrain-49-3-4w-x-17d-x-70-1-8h>: HTTP status code is not handled or not allowed 2026-01-24 21:13:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stirrers-5-1-2l-plastic-1000-box-white-red>: HTTP status code is not handled or not allowed 2026-01-24 21:13:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/celltreat-scientific-products-20200l-pipette-ovation-quickset-qs-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:13:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n-ferno-6477-cold-storage-thermal-insulated-bib-overalls-2xl-navy>: HTTP status code is not handled or not allowed 2026-01-24 21:13:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/touch-n-tone-spray-paint-10-oz-aerosol-can-gloss-leather-brown-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-24 21:13:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cordless-compact-pistol-grip-drill-driver-kit-12-keyless-chuck-1800-rpm>: HTTP status code is not handled or not allowed 2026-01-24 21:13:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/r036c-6-air-sander-self-generating-vacuum-332-orbit-dia-13000-rpm>: HTTP status code is not handled or not allowed 2026-01-24 21:13:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rcl-307-top-push-roller>: HTTP status code is not handled or not allowed 2026-01-24 21:13:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lzws-lrpbm8k-filtered-bottle-filling-station-w-integral-refrig-fountain>: HTTP status code is not handled or not allowed 2026-01-24 21:13:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-cal-diamond-plate-metallic-pvc-flooring-silver-2-5mm-x-4-x-20>: HTTP status code is not handled or not allowed 2026-01-24 21:13:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/topoge-series-180-handhole-gasket-3-x-4-x-58-black-rubber-elliptical-25-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:13:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/laundry_detergent already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:13:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/topoge-series-180-manway-gasket-1158-x-1558-x-1114-black-rubber-obround-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:13:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-single-face-shelving-base-37-w-x-12-1-4-d-x-71-1-8-h-oiled-cherry>: HTTP status code is not handled or not allowed 2026-01-24 21:13:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bcs-electronic-crane-scale-with-remote-10000-lb-capacity-x-5-lb-readability>: HTTP status code is not handled or not allowed 2026-01-24 21:13:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/selectivend-ws4000-snack-machine-32-selections-holds-up-to-474-items-6-adjustable-trays>: HTTP status code is not handled or not allowed 2026-01-24 21:13:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/confined-space-plenum-rated-heater-h3h5705t-5kw-240-208v-3-ph>: HTTP status code is not handled or not allowed 2026-01-24 21:13:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/infrared-electric-heater-fss-9520-3-heavy-duty-950-kw-208v>: HTTP status code is not handled or not allowed 2026-01-24 21:13:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/digital-acrylic-door-incubator-with-forced-air-12-140ae-295w>: HTTP status code is not handled or not allowed 2026-01-24 21:13:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/topoge-series-180-handhole-gasket-134-x-334-x-38-black-rubber-elliptical-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:13:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-message-mat-194-safety-starts-here-3x5-black>: HTTP status code is not handled or not allowed 2026-01-24 21:13:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jurni-multi-purpose-table-with-post-legs--casters-72l-x-24w-x-29h-columbian-walnut>: HTTP status code is not handled or not allowed 2026-01-24 21:13:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cl-focus-ii-boost-28-walk-behind-midsize-scrubber-boost-05375a>: HTTP status code is not handled or not allowed 2026-01-24 21:13:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jurni-multi-purpose-table-with-t-legs--glides-72l-x-24w-x-29h-columbian-walnut>: HTTP status code is not handled or not allowed 2026-01-24 21:13:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/hydraulics/hydraulic_motor_parts/hydraulic_motor_clamps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:13:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/laundry_detergent>: HTTP status code is not handled or not allowed 2026-01-24 21:13:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/topoge-series-180-handhole-gasket-6-x-10-x-58-black-rubber-elliptical-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:13:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hospitality-base-cabinet-with-1-drawer--2-doors-36w-x-25d-x-39h-mahogany>: HTTP status code is not handled or not allowed 2026-01-24 21:13:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-hex-union-h5306x6-3-8-inch-tube-od>: HTTP status code is not handled or not allowed 2026-01-24 21:13:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jurni-round-bistro-table-36dia-fusion-maple-topsilver-base>: HTTP status code is not handled or not allowed 2026-01-24 21:13:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mechanix-wear-original174-work-gloves-synthetic-leather-wtrekdry153-cooling-black-large>: HTTP status code is not handled or not allowed 2026-01-24 21:13:41 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-24 21:13:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cep-6210pdc30-2-100-amp-3-ph-cart-w-30kva-transformer>: HTTP status code is not handled or not allowed 2026-01-24 21:13:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90-female-elbow-h5455x16-1-inch-tube-od-1-inch-female-pipe-thread>: HTTP status code is not handled or not allowed 2026-01-24 21:13:42 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:13 2026-01-24 21:13:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tx4000-esd-cleanroom-apparel-coverall-5xl-white>: HTTP status code is not handled or not allowed 2026-01-24 21:13:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simpson-aaa173-triplex-pump-kit-4000-psi-35-gpm>: HTTP status code is not handled or not allowed 2026-01-24 21:13:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simpson-universal-second-story-nozzles-cold-water-use-up-to-5000-psi>: HTTP status code is not handled or not allowed 2026-01-24 21:13:45 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/hydraulics/hydraulic_motor_parts/hydraulic_motor_clamps landed on page that is not a product page. 2026-01-24 21:13:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-series-clamp-for-pipe-hdsscp100-1-inch-id>: HTTP status code is not handled or not allowed 2026-01-24 21:13:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ma519-super-wide-field-10x-focusing-eyepiece-single-field-no-21>: HTTP status code is not handled or not allowed 2026-01-24 21:13:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/achim-sterling-self-adhesive-vinyl-floor-planks-6-x-36-walnut-10-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:13:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/achim-interlocking-foam-anti-fatigue-floor-tiles-24-x-24-walnut-4-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:13:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/achim-nexus-self-adhesive-vinyl-floor-tile-12-x-12-mosaic-20-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:13:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hydraulic-motor-cm002p-2-bolt-317-cipr-969-max-rpm-28-displacement>: HTTP status code is not handled or not allowed 2026-01-24 21:13:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/achim-nexus-self-adhesive-vinyl-floor-planks-6-x-36-light-grey-oak-10-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:13:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smr915rbkit-rebuild-kit-for-an-original-style-shaft-mount-reducer-smr9-151>: HTTP status code is not handled or not allowed 2026-01-24 21:13:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/achim-sterling-self-adhesive-vinyl-floor-tile-12-x-12-medium-oak-plank-20-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:13:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wsmr4-151-shaft-mount-reducer-size-4-151-ratio-2-716-tapered-bore>: HTTP status code is not handled or not allowed 2026-01-24 21:13:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/union-tee-h5705x6-3-8-inch-tube-od>: HTTP status code is not handled or not allowed 2026-01-24 21:13:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ediusa-ballistic-body-shield-tested-to-level-iii-ballistic-resistance-black>: HTTP status code is not handled or not allowed 2026-01-24 21:13:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ediusa-ballistic-vest-tested-to-level-iiia-ballistic-resistance-large>: HTTP status code is not handled or not allowed 2026-01-24 21:13:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7wtbk-3316-tapered-bushing-kit-3-316-fits-reducer-styles-smr7wsmr7>: HTTP status code is not handled or not allowed 2026-01-24 21:13:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ediusa-highcut-ballistic-helmet-with-side-rails-front-mount-medium-green>: HTTP status code is not handled or not allowed 2026-01-24 21:13:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stamped-welding-flange-fa150-1-1-2-inch-aluminum-325-inch-od-055-inch-thick>: HTTP status code is not handled or not allowed 2026-01-24 21:13:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/upright-hydraulic-reservoir-ur70s-675-gal-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:13:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/techstar-mini-starcarts-standard-gray-19wx33dx28h>: HTTP status code is not handled or not allowed 2026-01-24 21:13:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/techstar-sos-storage-bin-5-1-2-cu-ft-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:13:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/meter-b40b45-10-80v-dc-hour-meter>: HTTP status code is not handled or not allowed 2026-01-24 21:13:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sa-mirella-conference-table-standing-height-8-rectangle-whiteash>: HTTP status code is not handled or not allowed 2026-01-24 21:14:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/orbis-solid-color-flipak-tote-20-3-5-x13-1-2-x6-1-2>: HTTP status code is not handled or not allowed 2026-01-24 21:14:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sa-whiffle-typical-15-double-60-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:14:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/first_aid_wound_treatment/bandages_dressings already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:14:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-bolt-flange-adapter-kit-b433232u2-inch-adapter-kit>: HTTP status code is not handled or not allowed 2026-01-24 21:14:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/orbis-translucent-flipak-tote-15-1-5-x10-9-10-x9-7-10>: HTTP status code is not handled or not allowed 2026-01-24 21:14:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/45-elbow-h3559x12-3-4-inch-x-3-4-inch-female-to-female>: HTTP status code is not handled or not allowed 2026-01-24 21:14:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tee-h3709x20-1-1-4-inch-female-npt-to-female-npt>: HTTP status code is not handled or not allowed 2026-01-24 21:14:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sa-safco-lounge-chair-cognac>: HTTP status code is not handled or not allowed 2026-01-24 21:14:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medical-transporter-infinite-recline-5-casters-push-bar-side-table-300-lbs-max-aquamarine>: HTTP status code is not handled or not allowed 2026-01-24 21:14:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polycarbonate-goggles-indirect-vent>: HTTP status code is not handled or not allowed 2026-01-24 21:14:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sa-skinny-pedestal-tall-graysteel>: HTTP status code is not handled or not allowed 2026-01-24 21:14:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_cabinets_pedestals already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:14:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sa-mirella-low-wall-cabinet-wood-doors-stonegray>: HTTP status code is not handled or not allowed 2026-01-24 21:14:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/husky-frame-protector-for-pallet-racks-24h>: HTTP status code is not handled or not allowed 2026-01-24 21:14:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 23503 pages (at 97 pages/min), scraped 13007 items (at 17 items/min) 2026-01-24 21:14:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sa-mirella-glass-door-display-cabinet-southerntobacco>: HTTP status code is not handled or not allowed 2026-01-24 21:14:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/first_aid_wound_treatment/bandages_dressings>: HTTP status code is not handled or not allowed 2026-01-24 21:14:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lxny-w-3-r-m4-led-steel-exit-sign-2w-standard-non-emergency-white>: HTTP status code is not handled or not allowed 2026-01-24 21:14:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lbw-portable-kerosene-heater-tradesman-k170-170k-btu-1-or-2-fuel-oil>: HTTP status code is not handled or not allowed 2026-01-24 21:14:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lbw-portable-gas-heater-tradesman-400-250k-400k-btu-lpg>: HTTP status code is not handled or not allowed 2026-01-24 21:14:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/worldwide-electric-winl37-40-1-56c-helical-inline-speed-reducer-56c-input-flange-40-1-ratio>: HTTP status code is not handled or not allowed 2026-01-24 21:14:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-long-sleeve-button-down-work-shirt-xl-green>: HTTP status code is not handled or not allowed 2026-01-24 21:14:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-control-valve-f600s-3-8-inch-steel-flow-control-valve-npt>: HTTP status code is not handled or not allowed 2026-01-24 21:14:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/full-flow-control-ball-valve-hbv050-1-2-inch-valve-600-wog-non-shock>: HTTP status code is not handled or not allowed 2026-01-24 21:14:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/snap-button-down-long-sleve-work-shirt-4xl-tall-black>: HTTP status code is not handled or not allowed 2026-01-24 21:14:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-control-valve-f400sae-steel-flow-control-valve-4-sae>: HTTP status code is not handled or not allowed 2026-01-24 21:14:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/snap-button-down-long-sleve-work-shirt-l-tall-black>: HTTP status code is not handled or not allowed 2026-01-24 21:14:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_cabinets_pedestals>: HTTP status code is not handled or not allowed 2026-01-24 21:14:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-30-maple-square-edge-mobile-pedestal-workbench-blue-318651BL>: HTTP status code is not handled or not allowed 2026-01-24 21:14:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/worldwide-electric-winl87-2-1-213-5tc-helical-inline-speed-reducer-213-5tc-input-flange-2-1-ratio>: HTTP status code is not handled or not allowed 2026-01-24 21:14:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wind-pro-full-zip-hooded-sweater-reflective-flame-resistant-m-orange>: HTTP status code is not handled or not allowed 2026-01-24 21:14:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:14:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/long-sleeve-henley-interlock-knit-flame-resistant-5xl-orange>: HTTP status code is not handled or not allowed 2026-01-24 21:14:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ds-176270b-shrink-wrap-17w-x-270l-6mil-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:14:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:14:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wildeck-44-high-post-column-for-triple-rail-wct44>: HTTP status code is not handled or not allowed 2026-01-24 21:14:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5424200-36x24-shelf-zinc-finish>: HTTP status code is not handled or not allowed 2026-01-24 21:14:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hooded-fleece-sweater-14-25-oz-flame-resistant-5xl-tall-navy>: HTTP status code is not handled or not allowed 2026-01-24 21:14:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gear-locker-24w-x-18d-x-72h-w-top-shelf-foot-locker-grand-slam-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 21:14:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/monitor-riser-black>: HTTP status code is not handled or not allowed 2026-01-24 21:14:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-welded-stainless-steel-cabinet-36w-x-18dx-41h>: HTTP status code is not handled or not allowed 2026-01-24 21:14:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-under-desk-keyboard-tray-platinum>: HTTP status code is not handled or not allowed 2026-01-24 21:14:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-locker-accessory-slope-top-end-closure-18d-x-6h-725-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:14:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:14:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-30w-x-33l-navy>: HTTP status code is not handled or not allowed 2026-01-24 21:14:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pork-chop-truck-bx-black-alum-driver-side-full-size-3-4-cu-ft-172-5-01>: HTTP status code is not handled or not allowed 2026-01-24 21:14:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/truck_boxes_utility_boxes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:14:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts>: HTTP status code is not handled or not allowed 2026-01-24 21:14:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-34w-x-28l-navy>: HTTP status code is not handled or not allowed 2026-01-24 21:14:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts>: HTTP status code is not handled or not allowed 2026-01-24 21:14:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vented-plastic-shelving-60x21x54-nexelon-finish-331480N>: HTTP status code is not handled or not allowed 2026-01-24 21:14:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chair-truck-68-l-x-31-w-black>: HTTP status code is not handled or not allowed 2026-01-24 21:14:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5456800-48x18x74-shelving>: HTTP status code is not handled or not allowed 2026-01-24 21:14:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-31w-x-33l-black>: HTTP status code is not handled or not allowed 2026-01-24 21:14:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ezi-action-60-15-drum-pump-with-m63r-fitting-food-grade>: HTTP status code is not handled or not allowed 2026-01-24 21:14:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:14:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts>: HTTP status code is not handled or not allowed 2026-01-24 21:14:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700-50r-50-industrial-reel-plated-steel-cable-le-21c-copper-alligator-clip>: HTTP status code is not handled or not allowed 2026-01-24 21:14:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:14:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/truck_boxes_utility_boxes>: HTTP status code is not handled or not allowed 2026-01-24 21:14:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sealed-sheave-bearing-e5014xnnts1-without-snap-rings-and-coating-70mm-bore-110mm-od>: HTTP status code is not handled or not allowed 2026-01-24 21:14:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-50w-x-34l-brown>: HTTP status code is not handled or not allowed 2026-01-24 21:14:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-40w-x-29l-tan>: HTTP status code is not handled or not allowed 2026-01-24 21:14:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:14:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lewiston-post-with-support-brace-decorative-fluted-base-ball-finial-no-mailbox-black>: HTTP status code is not handled or not allowed 2026-01-24 21:14:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-33w-x-34l-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:14:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-30w-x-33l-tan>: HTTP status code is not handled or not allowed 2026-01-24 21:14:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-42w-x-33l-tan>: HTTP status code is not handled or not allowed 2026-01-24 21:14:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:14:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 21:14:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-inch-drive-socket-3-4-inch-6-point-1-1-2-inch-long>: HTTP status code is not handled or not allowed 2026-01-24 21:14:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:14:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 21:14:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-inch-drive-deep-impact-socket-33-mm-6-point-3-1-2-inch-long>: HTTP status code is not handled or not allowed 2026-01-24 21:14:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/sockets_bits/impact_sockets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:14:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/450hs-50-inch-top-chest-8-drawer-blue-50-inch-l-x-27-inch-h-x-25-inchd>: HTTP status code is not handled or not allowed 2026-01-24 21:14:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/141-piece-railroad-electrician-foots-set>: HTTP status code is not handled or not allowed 2026-01-24 21:14:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:14:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 21:14:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/telescopic-circular-inspection-mirror-2-1-4-inch-diameter>: HTTP status code is not handled or not allowed 2026-01-24 21:14:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-50w-x-34l-navy>: HTTP status code is not handled or not allowed 2026-01-24 21:14:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b2416-oh-needle-bearing-drawn-cup-full-complement-oil-hole-bore-381mm>: HTTP status code is not handled or not allowed 2026-01-24 21:14:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 21:14:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-38w-x-31l-green>: HTTP status code is not handled or not allowed 2026-01-24 21:14:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b168-oh-needle-bearing-drawn-cup-full-complement-oil-hole-bore-254mm>: HTTP status code is not handled or not allowed 2026-01-24 21:14:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 21:14:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-46w-x-28l-navy>: HTTP status code is not handled or not allowed 2026-01-24 21:14:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-32w-x-31l-black>: HTTP status code is not handled or not allowed 2026-01-24 21:15:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/sockets_bits/impact_sockets>: HTTP status code is not handled or not allowed 2026-01-24 21:15:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dry-erase-activity-pocket-chart-14-rows-13-inch-x-34-inch-purple>: HTTP status code is not handled or not allowed 2026-01-24 21:15:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/creativity-street-economy-grade-craft-sticks-4-1-2-inch-x-3-8-inch-natural-1000-box>: HTTP status code is not handled or not allowed 2026-01-24 21:15:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 21:15:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-46w-x-32l-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:15:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/el11306-75hp-56-frame-3600rpm-115-230v-odp-rigid-premium-efficiency>: HTTP status code is not handled or not allowed 2026-01-24 21:15:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-coin-wrapper-sprtcw05-2-nickels-capacity-1000-pk>: HTTP status code is not handled or not allowed 2026-01-24 21:15:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/el11317-2hp-56h-frame-3600rpm-115-230v-odp-rigid-premium-efficiency>: HTTP status code is not handled or not allowed 2026-01-24 21:15:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/praise-words-stinky-stickers-variety-pack-435-stickers-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:15:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:15:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-46w-x-33l-navy>: HTTP status code is not handled or not allowed 2026-01-24 21:15:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/em30004-25hp-1200rpm-48-frame-3ph-230-460v-odp-rigid-premium-efficiency>: HTTP status code is not handled or not allowed 2026-01-24 21:15:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/saltdogg-tailgate-salt-spreader-4-cu-capacity-tgs02>: HTTP status code is not handled or not allowed 2026-01-24 21:15:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/artista-ii-washable-tempera-paint-nontoxic-16-oz-red>: HTTP status code is not handled or not allowed 2026-01-24 21:15:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/synonyms-bingo-game-age-10-and-up-3-to-36-players-1-box>: HTTP status code is not handled or not allowed 2026-01-24 21:15:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 23587 pages (at 84 pages/min), scraped 13015 items (at 8 items/min) 2026-01-24 21:15:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-34w-x-34l-black>: HTTP status code is not handled or not allowed 2026-01-24 21:15:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nitro-euro-style-walker-rollator-rtl10266bk-black>: HTTP status code is not handled or not allowed 2026-01-24 21:15:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-56w-x-31l-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:15:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transparent-tape-600-3-4-x-1296-boxed-1-core-1-roll>: HTTP status code is not handled or not allowed 2026-01-24 21:15:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stock-tank-w-155b-w-series-155-gal>: HTTP status code is not handled or not allowed 2026-01-24 21:15:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-50w-x-33l-green>: HTTP status code is not handled or not allowed 2026-01-24 21:15:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trailer-rv-entry-step-ms-14ag-14>: HTTP status code is not handled or not allowed 2026-01-24 21:15:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-40w-x-34l-black>: HTTP status code is not handled or not allowed 2026-01-24 21:15:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 21:15:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/water-tank-brass-rod-zm-6052-10>: HTTP status code is not handled or not allowed 2026-01-24 21:15:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-fence-feeder-ff-24b>: HTTP status code is not handled or not allowed 2026-01-24 21:15:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-12-gauge-combination-cabinet-w-drawers-shelves-48w-x-24d-x-78h>: HTTP status code is not handled or not allowed 2026-01-24 21:15:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-52w-x-33l-tan>: HTTP status code is not handled or not allowed 2026-01-24 21:15:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-30w-x-unhemmed-black>: HTTP status code is not handled or not allowed 2026-01-24 21:15:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ctl-bandsaw-blade-11-foot-1-inch-long-x-3-4-inch-wide-18-tpi-x-0035-thick>: HTTP status code is not handled or not allowed 2026-01-24 21:15:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ctl-bandsaw-blade-7-foot-9-inch-long-x-3-4-inch-wide-4-6-tpi-x-0035-thick>: HTTP status code is not handled or not allowed 2026-01-24 21:15:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-ghs2032al2-corrosive-ghs-label-paper-2-x-2-250-roll>: HTTP status code is not handled or not allowed 2026-01-24 21:15:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-ghs2044al2-compressed-air-ghs-label-4-x-4-250-roll>: HTTP status code is not handled or not allowed 2026-01-24 21:15:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-ghs2062al2-compressed-air-ghs-label-paper-2-x-2-250-roll>: HTTP status code is not handled or not allowed 2026-01-24 21:15:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qvc-quick-valve-change-tool-51025>: HTTP status code is not handled or not allowed 2026-01-24 21:15:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ctl-bandsaw-blade-13-foot-2-inch-long-x-1-inch-wide-10-14-tpi-x-0035-thick>: HTTP status code is not handled or not allowed 2026-01-24 21:15:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-slotted-pallet-rackrow-spacer-12-798023>: HTTP status code is not handled or not allowed 2026-01-24 21:15:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-42w-x-34l-dark-navy>: HTTP status code is not handled or not allowed 2026-01-24 21:15:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20153-cutlery-set-7-pc-white-handle>: HTTP status code is not handled or not allowed 2026-01-24 21:15:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/17443-bakers-spatula-high-carbon-steel-white-handle-8-l>: HTTP status code is not handled or not allowed 2026-01-24 21:15:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-34w-x-unhemmed-navy>: HTTP status code is not handled or not allowed 2026-01-24 21:15:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inf-whisper-ultrasonic-leak-detector-711-202-g1-detects-refrigerant-and-nitrogen-leaks>: HTTP status code is not handled or not allowed 2026-01-24 21:15:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/protective-eyewear-clear-lens-black-frame-10-each>: HTTP status code is not handled or not allowed 2026-01-24 21:15:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/weld-curtain-w-hardware-6w-x-9h>: HTTP status code is not handled or not allowed 2026-01-24 21:15:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/09453-vegetable-produce-knife-high-carbon-steel-stamped-white-handle-5-l>: HTTP status code is not handled or not allowed 2026-01-24 21:15:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-double-tier-locker-3-wide-12w-x-12d-x-30h-assembled-black>: HTTP status code is not handled or not allowed 2026-01-24 21:15:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-three-tier-locker-1-wide-12w-x-18d-x-24h-assembled-black>: HTTP status code is not handled or not allowed 2026-01-24 21:15:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-three-tier-locker-2-wide-12w-x-12d-x-24h-assembled-black>: HTTP status code is not handled or not allowed 2026-01-24 21:15:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/r75i-125-2303-rotary-screw-air-compressor-3-phase-230-volts-10hp-120-gal>: HTTP status code is not handled or not allowed 2026-01-24 21:15:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/r4i-tas-115-2003-rotary-screw-air-compressor-3-phase-200-volts-5hp-120-gal>: HTTP status code is not handled or not allowed 2026-01-24 21:15:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xlerator-174-hand-dryer-stainless-steel-110-120v-xl-sb-110-120>: HTTP status code is not handled or not allowed 2026-01-24 21:15:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wesco-30x18-open-deck-hardwood-dolly-3-casters-900-lb-cap-987838>: HTTP status code is not handled or not allowed 2026-01-24 21:15:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/threadlock-red-27536-36ml-bottle>: HTTP status code is not handled or not allowed 2026-01-24 21:15:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-work-pants-reflective-material-flame-resistant-32w-x-32l-red>: HTTP status code is not handled or not allowed 2026-01-24 21:15:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edgr-1-g-el-m4-led-edge-lit-exit-sign-green>: HTTP status code is not handled or not allowed 2026-01-24 21:15:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rescue-series-anesthesiology-medical-cart-white-frame-with-5-blue-drawers>: HTTP status code is not handled or not allowed 2026-01-24 21:15:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:15:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96-performance-200-series-lan-station-african-mahogany>: HTTP status code is not handled or not allowed 2026-01-24 21:15:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oxygen-tank-holder-with-accessory-rail-for-whisper-er-loaded-carts>: HTTP status code is not handled or not allowed 2026-01-24 21:15:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-storage-bin-with-accessory-rail-for-whisper-medical-carts>: HTTP status code is not handled or not allowed 2026-01-24 21:15:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-visibility-ripstop-cargo-pants-34w-x-30l-navy>: HTTP status code is not handled or not allowed 2026-01-24 21:15:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:15:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-westex-34w-x-unhemmed-navy>: HTTP status code is not handled or not allowed 2026-01-24 21:15:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b77k-v-belt-kevlar-5-8-x-80>: HTTP status code is not handled or not allowed 2026-01-24 21:15:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b42k-v-belt-kevlar-5-8-x-45>: HTTP status code is not handled or not allowed 2026-01-24 21:15:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cmss-2-cup-mount-700-max-load-lbs-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:15:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:15:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nvs-4-70-leveling-compression-shear-mount-1-14x8-15000-lbs>: HTTP status code is not handled or not allowed 2026-01-24 21:15:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-jeans-indura-denim-14-oz-36w-x-32l-flame-resistant-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:15:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 21:15:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-angle-equal-leg-58263-72-inchl-x-1-inchw-x-1-inchh-x-1-8-inchd-silver-d0234>: HTTP status code is not handled or not allowed 2026-01-24 21:15:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cargo-pants-with-double-reinforced-knees-flame-resistant-30w-x-34l-black>: HTTP status code is not handled or not allowed 2026-01-24 21:15:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dpu-16-universal-drip-pan-105-l-x-105-w-x-3-h-16-per-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:15:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-work-pants-reflective-material-flame-resistant-38w-x-unhemmed-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:15:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 21:15:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cargo-pants-with-double-reinforced-knees-flame-resistant-42w-x-unhemmed-black>: HTTP status code is not handled or not allowed 2026-01-24 21:15:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-wire-shelving-add-on-48-inch-w-x-14-inch-d-x-54-inch-h>: HTTP status code is not handled or not allowed 2026-01-24 21:15:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-westex-44w-x-34l-tan>: HTTP status code is not handled or not allowed 2026-01-24 21:15:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-westex-48w-x-30l-navy>: HTTP status code is not handled or not allowed 2026-01-24 21:15:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rd812nh-rotary-hinged-vane-pump-portable-8-gpm-multiple-mounting-options>: HTTP status code is not handled or not allowed 2026-01-24 21:15:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 21:15:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-wire-shelving-72-inch-w-x-21-inch-d-x-86-inch-h>: HTTP status code is not handled or not allowed 2026-01-24 21:15:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mh7008-7-magnetic-wall-mount-barrier-danger-keep-out-banner>: HTTP status code is not handled or not allowed 2026-01-24 21:15:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-westex-48w-x-30l-black>: HTTP status code is not handled or not allowed 2026-01-24 21:15:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-westex-50w-x-34l-navy>: HTTP status code is not handled or not allowed 2026-01-24 21:15:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-wire-shelving-add-on-42-inch-w-x-18-inch-d-x-86-inch-h>: HTTP status code is not handled or not allowed 2026-01-24 21:15:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-wire-shelving-54-inch-w-x-21-inch-d-x-74-inch-h>: HTTP status code is not handled or not allowed 2026-01-24 21:15:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-wire-shelving-add-on-48-inch-w-x-14-inch-d-x-63-inch-h>: HTTP status code is not handled or not allowed 2026-01-24 21:16:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cesm-21cb-4242-safety-switch-mat-42-x-42-black>: HTTP status code is not handled or not allowed 2026-01-24 21:16:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cesm-21cb-4848-safety-switch-mat-48-x-48-black>: HTTP status code is not handled or not allowed 2026-01-24 21:16:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cargo-pants-with-double-reinforced-knees-flame-resistant-44w-x-32l-navy>: HTTP status code is not handled or not allowed 2026-01-24 21:16:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/utility-jeans-basket-weave-flame-resistant-8-oz-40w-x-30l-navy>: HTTP status code is not handled or not allowed 2026-01-24 21:16:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:16:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/painters-select-urethane-fortified-satin-porch-floor-coating-white-gallon-106649>: HTTP status code is not handled or not allowed 2026-01-24 21:16:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/maintenance-one-concentrated-all-purpose-cleaner-5-gallon-pail-1-case-513025>: HTTP status code is not handled or not allowed 2026-01-24 21:16:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/x-o-rust-12-oz-aerosol-rust-preventative-paint-primer-in-one-reed-gloss-125842>: HTTP status code is not handled or not allowed 2026-01-24 21:16:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-30w-x-14d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 21:16:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-9a-3ph-direct-online-start-stop-250-500v-5-7-7-6a>: HTTP status code is not handled or not allowed 2026-01-24 21:16:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-9a-3ph-remote-start-start-stop-100-250v-5-7-7-6a>: HTTP status code is not handled or not allowed 2026-01-24 21:16:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/x-o-rust-oil-base-dtm-enamel-gloss-finish-blue-gallon-802082>: HTTP status code is not handled or not allowed 2026-01-24 21:16:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/start-right-interior-exterior-stain-blocking-primer-sealer-gallon-133281>: HTTP status code is not handled or not allowed 2026-01-24 21:16:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 23681 pages (at 94 pages/min), scraped 13027 items (at 12 items/min) 2026-01-24 21:16:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/painters-select-latex-block-filler-flat-finish-white-5-gallon-151730>: HTTP status code is not handled or not allowed 2026-01-24 21:16:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-9a-1ph-direct-online-reset-button-100-250v-2-3-3-1a>: HTTP status code is not handled or not allowed 2026-01-24 21:16:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 21:16:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bp2200-automatic-blood-pressure-wrist-monitor>: HTTP status code is not handled or not allowed 2026-01-24 21:16:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manitowoc-ice-iy-0606a-indigo-series-ice-maker-cube-style-air-cooled-self-contained-condenser>: HTTP status code is not handled or not allowed 2026-01-24 21:16:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-12a-3ph-start-stop-24-60v-4-2-5-7a>: HTTP status code is not handled or not allowed 2026-01-24 21:16:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-12a-3ph-remote-start-terminals-start-stop-250-500v-10-13a>: HTTP status code is not handled or not allowed 2026-01-24 21:16:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-12a-3ph-direct-online-start-stop-100-250v-5-7-7-6a>: HTTP status code is not handled or not allowed 2026-01-24 21:16:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eco-premier-carpet-tile-22157114000-geometric-18l-x-18w-x-14h-indigo-12pk>: HTTP status code is not handled or not allowed 2026-01-24 21:16:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lightweight-shelf-back-panel-13-1-2-in-52-in-9605-3-01>: HTTP status code is not handled or not allowed 2026-01-24 21:16:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vestil-side-mount-steel-floor-lock-fl-lk-smr-l-le-side-mount>: HTTP status code is not handled or not allowed 2026-01-24 21:16:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-12a-3ph-remote-start-reset-button-250-500v-2-3-3-1a>: HTTP status code is not handled or not allowed 2026-01-24 21:16:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-12a-1ph-remote-start-reset-button-24-60v-3-1-4-2a>: HTTP status code is not handled or not allowed 2026-01-24 21:16:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wesco-174-plastic-box-truck-12-bushel-blue-272537-8-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:16:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-12a-1ph-remote-start-terminals-start-stop-100-250v-5-7-7-6a>: HTTP status code is not handled or not allowed 2026-01-24 21:16:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-12a-1ph-remote-start-start-stop-100-250v-7-6-10a>: HTTP status code is not handled or not allowed 2026-01-24 21:16:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-16a-3ph-start-stop-24-60v-4-2-5-7a>: HTTP status code is not handled or not allowed 2026-01-24 21:16:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-16a-3ph-remote-start-terminals-start-stop-250-500v-4-2-5-7a>: HTTP status code is not handled or not allowed 2026-01-24 21:16:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b11214-jib-crane>: HTTP status code is not handled or not allowed 2026-01-24 21:16:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-8482-outdoor-metal-slatted-receptacle-w-access-door-flat-lid-36-gallon-green>: HTTP status code is not handled or not allowed 2026-01-24 21:16:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-16a-1ph-separate-coil-voltage-start-stop-100-250v-7-6-10a>: HTTP status code is not handled or not allowed 2026-01-24 21:16:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-26a-3ph-direct-online-reset-button-100-250v-13-16a>: HTTP status code is not handled or not allowed 2026-01-24 21:16:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-26a-3ph-separate-coil-voltage-start-stop-250-500v-16-20a>: HTTP status code is not handled or not allowed 2026-01-24 21:16:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-26a-1ph-direct-online-start-stop-100-250v-16-20a>: HTTP status code is not handled or not allowed 2026-01-24 21:16:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-26a-3ph-remote-start-terminals-reset-button-250-500v-16-20a>: HTTP status code is not handled or not allowed 2026-01-24 21:16:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-38a-3ph-direct-online-start-stop-250-500v-24-29a>: HTTP status code is not handled or not allowed 2026-01-24 21:16:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/332035-hex-machine-screw-nut-4-40-low-carbon-steel-plain-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:16:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/d667rb-osha-sign-danger-overhead-power-lines-10-x-14-whiteredblack>: HTTP status code is not handled or not allowed 2026-01-24 21:16:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dcl106-nfpa-label-symbol-use-no-water-7-12-x-7-12-whiteblack-5pk>: HTTP status code is not handled or not allowed 2026-01-24 21:16:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inl2-inspection-label-calibrated-1-x-2-14-blackwhite>: HTTP status code is not handled or not allowed 2026-01-24 21:16:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m58ab-security-sign-no-trespassing-10-x-14-whitered>: HTTP status code is not handled or not allowed 2026-01-24 21:16:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m726lp-flammablecombustible-sign-mixed-two-cycle-fuel-2-x-5-whiteblack>: HTTP status code is not handled or not allowed 2026-01-24 21:16:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-38a-1ph-remote-start-terminals-start-stop-100-250v-24-29a>: HTTP status code is not handled or not allowed 2026-01-24 21:16:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-38a-1ph-direct-online-start-stop-100-250v-16-20a>: HTTP status code is not handled or not allowed 2026-01-24 21:16:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-38a-1ph-separate-coil-voltage-start-stop-100-250v-20-24a>: HTTP status code is not handled or not allowed 2026-01-24 21:16:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/d18ab-osha-sign-danger-diesel-no-smoking-10-x-14-whiteredblack>: HTTP status code is not handled or not allowed 2026-01-24 21:16:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-52a-3ph-direct-online-start-stop-100-250v-25-33a>: HTTP status code is not handled or not allowed 2026-01-24 21:16:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-52a-3ph-separate-coil-voltage-reset-button-250-500v-30-40a>: HTTP status code is not handled or not allowed 2026-01-24 21:16:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precision-compression-spring-024-od-x-0022-x-0375-music-wire-pkg-of-5>: HTTP status code is not handled or not allowed 2026-01-24 21:16:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/d11rb-osha-sign-danger-safety-glasses-required-in-this-area-10-x-14-whiteredblack>: HTTP status code is not handled or not allowed 2026-01-24 21:16:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-52a-1ph-remote-start-reset-button-24-60v-44-53a>: HTTP status code is not handled or not allowed 2026-01-24 21:16:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-52a-1ph-remote-start-reset-button-100-250v-25-33a>: HTTP status code is not handled or not allowed 2026-01-24 21:16:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c525pb-osha-sign-caution-hot-surface-do-not-touch-10-x-14-yellowblack>: HTTP status code is not handled or not allowed 2026-01-24 21:16:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-65a-3ph-reset-button-24-60v-44-53a>: HTTP status code is not handled or not allowed 2026-01-24 21:16:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tm145j-traffic-sign-handicapped-parking-only-18-x-12-whitebluegreen>: HTTP status code is not handled or not allowed 2026-01-24 21:16:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-65a-3ph-reset-button-24-60v-50-60a>: HTTP status code is not handled or not allowed 2026-01-24 21:16:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-80a-3ph-remote-start-terminals-start-stop-100-250v-48-60a>: HTTP status code is not handled or not allowed 2026-01-24 21:16:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-80a-3ph-separate-coil-voltage-start-stop-100-250v-48-60a>: HTTP status code is not handled or not allowed 2026-01-24 21:16:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rpt172-tags-danger-barricade-tag-6-x-3-whiteredblack-25pk>: HTTP status code is not handled or not allowed 2026-01-24 21:16:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-80a-3ph-separate-coil-voltage-reset-button-250-500v-57-68a>: HTTP status code is not handled or not allowed 2026-01-24 21:16:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vsm-abrasive-belt-106932-zirconia-alumina-1-x-42-120-grit>: HTTP status code is not handled or not allowed 2026-01-24 21:16:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vsm-abrasive-belt-115114-aluminum-oxide-3-x-18-60-grit>: HTTP status code is not handled or not allowed 2026-01-24 21:16:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6ft-4ft-sago-palm-double-potted-silk-tree>: HTTP status code is not handled or not allowed 2026-01-24 21:16:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6ft-curved-bamboo-silk-tree>: HTTP status code is not handled or not allowed 2026-01-24 21:16:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vsm-abrasive-belt-119448-zirconia-alumina-4-x-36-120-grit>: HTTP status code is not handled or not allowed 2026-01-24 21:16:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6ft-banana-silk-tree>: HTTP status code is not handled or not allowed 2026-01-24 21:16:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16in-cedar-bonsai-silk-plant>: HTTP status code is not handled or not allowed 2026-01-24 21:16:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-80a-3ph-remote-start-terminals-reset-button-100-250v-48-60a>: HTTP status code is not handled or not allowed 2026-01-24 21:16:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mini-bamboo-palm-with-decorative-vase>: HTTP status code is not handled or not allowed 2026-01-24 21:16:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dancing-lady-silk-flower-arrangement-purple>: HTTP status code is not handled or not allowed 2026-01-24 21:16:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-80a-1ph-start-stop-24-60v-65-78a>: HTTP status code is not handled or not allowed 2026-01-24 21:16:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rb-products-single-garment-rack-36-steel-chrome-finish>: HTTP status code is not handled or not allowed 2026-01-24 21:16:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-biometric-rifle-safe-gun-cabinet-13-3-4w-x-13-3-4d-x-57h>: HTTP status code is not handled or not allowed 2026-01-24 21:16:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vsm-abrasive-belt-73580-silicon-carbide-6-x-48-150-grit>: HTTP status code is not handled or not allowed 2026-01-24 21:16:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/k-flex-for-left-ear-extra-large>: HTTP status code is not handled or not allowed 2026-01-24 21:16:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vsm-abrasive-belt-80542-zirconia-alumina-37-x-60-80-grit>: HTTP status code is not handled or not allowed 2026-01-24 21:16:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/patriot-2-wire-surveillance-earpiece-kenwood-hyt-relm-radios>: HTTP status code is not handled or not allowed 2026-01-24 21:16:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/workbench-riser-72-w-x-10-1-2-d-x-12-h-black>: HTTP status code is not handled or not allowed 2026-01-24 21:16:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-step-800-lb-capacity-work-platform-with-36-w-x-36-l-platform>: HTTP status code is not handled or not allowed 2026-01-24 21:16:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-impact-hex-drive-master-set-2637-20-piece-alloy-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:16:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pioneer-mod-3mo160-mb-single-lever-bathroom-faucet-with-pop-up-matte-black>: HTTP status code is not handled or not allowed 2026-01-24 21:17:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blackbox-zone-usb-programming-cable-with-software-for-digital-and-analog-2-way-radios-zone-usb>: HTTP status code is not handled or not allowed 2026-01-24 21:17:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/olympia-i2-l-6050-mb-single-lever-bathroom-faucet-with-pop-up-matte-black>: HTTP status code is not handled or not allowed 2026-01-24 21:17:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-step-configurable-forward-descent-roll-ladder-perforated-ukdec109246>: HTTP status code is not handled or not allowed 2026-01-24 21:17:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-step-configurable-forward-descent-roll-ladder-grip-strut-ukdec108242>: HTTP status code is not handled or not allowed 2026-01-24 21:17:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/power_tools/power_saws_blades/band_saws/band_saw_blades already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:17:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b-8160-two-handle-bar-faucet-polished-chrome>: HTTP status code is not handled or not allowed 2026-01-24 21:17:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-wood-slab-transport-rack-w-casters-97-inch-x-64-inch-4000-lb-capacity-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:17:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-ibc-stairway-stair36i-7-48>: HTTP status code is not handled or not allowed 2026-01-24 21:17:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-rack-guard-left-side-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:17:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fao-3-shelf-first-aid-cabinet-100-person-ansi-compliant-class-a>: HTTP status code is not handled or not allowed 2026-01-24 21:17:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/62-574-21-piece-multi-bit-ratcheting-screwdriver-set>: HTTP status code is not handled or not allowed 2026-01-24 21:17:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/45-300-aluminum-carpenters-square-16-inch-x-24-inch>: HTTP status code is not handled or not allowed 2026-01-24 21:17:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/84-114-3-piece-basic-plier-set>: HTTP status code is not handled or not allowed 2026-01-24 21:17:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 23781 pages (at 100 pages/min), scraped 13044 items (at 17 items/min) 2026-01-24 21:17:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fao-smartcompliance-large-metal-first-aid-cabinet-w-medications-ansi-b>: HTTP status code is not handled or not allowed 2026-01-24 21:17:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/120-ibc-stairway-stair36i-18-120>: HTTP status code is not handled or not allowed 2026-01-24 21:17:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/power_tools/power_saws_blades/band_saws/band_saw_blades>: HTTP status code is not handled or not allowed 2026-01-24 21:17:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/116-osha-stairway-stair36o-13-116>: HTTP status code is not handled or not allowed 2026-01-24 21:17:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-48-inch-w-x-18-inch-d-x-86-inch-h-2000-lbs-cap-4-open-and-1-solid-grid-shelves>: HTTP status code is not handled or not allowed 2026-01-24 21:17:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-48-inch-w-x-21-inch-d-x-62-inch-h-2000-lbs-cap-3-open-and-1-solid-grid-shelves>: HTTP status code is not handled or not allowed 2026-01-24 21:17:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/110-osha-stairway-stair36o-13-110>: HTTP status code is not handled or not allowed 2026-01-24 21:17:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/138-osha-stairway-stair36o-16-138>: HTTP status code is not handled or not allowed 2026-01-24 21:17:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-30-inch-w-x-24-inch-d-x-50-inch-h-2000-lbs-cap-5-open-grid-shelves>: HTTP status code is not handled or not allowed 2026-01-24 21:17:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-30-inch-w-x-24-inch-d-x-62-inch-h-2000-lbs-cap-5-open-grid-shelves>: HTTP status code is not handled or not allowed 2026-01-24 21:17:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-60-inch-w-x-24-inch-d-x-50-inch-h-2000-lbs-cap-4-open-grid-shelves-dunnage-stand>: HTTP status code is not handled or not allowed 2026-01-24 21:17:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-48-inch-w-x-24-inch-d-x-62-inch-h-2000-lbs-cap-4-open-and-1-solid-grid-shelves>: HTTP status code is not handled or not allowed 2026-01-24 21:17:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-keyless-rigid-coupling-c600m100-c600-series-steel-100mm-shaft>: HTTP status code is not handled or not allowed 2026-01-24 21:17:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-54-inch-w-x-21-inch-d-x-62-inch-h-2000-lbs-cap-3-open-1-solid-shelf-dunnage-stand>: HTTP status code is not handled or not allowed 2026-01-24 21:17:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-42-inch-w-x-18-inch-d-x-74-inch-h-2000-lbs-cap-5-solid-shelves>: HTTP status code is not handled or not allowed 2026-01-24 21:17:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pillow-block-uhmwpe-bearing-pbuhuh062-solidrigid-housing-58-bore>: HTTP status code is not handled or not allowed 2026-01-24 21:17:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-72-inch-w-x-21-inch-d-x-62-inch-h-2000-lbs-cap-4-open-grid-shelves-dunnage-stand>: HTTP status code is not handled or not allowed 2026-01-24 21:17:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-54-inch-w-x-24-inch-d-x-62-inch-h-2000-lbs-cap-4-open-1-solid-shelf-dunnage-stand>: HTTP status code is not handled or not allowed 2026-01-24 21:17:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-frame-folding-tank-alf-3500-171l-x-171w-x-29h-3500-gallon-cap-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:17:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/label-holder-235-inch-l-clear-for-use-w-30-inch-d-shelves>: HTTP status code is not handled or not allowed 2026-01-24 21:17:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-frame-folding-tank-alf-500-72l-x-72w-x-29h-500-gallon-cap-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:17:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/125-dia-locking-assembly-c192-series-c192e125-steel-m6-x-20>: HTTP status code is not handled or not allowed 2026-01-24 21:17:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-sided-self-supporting-tank-hs-500-81-dia-x-40h-500-gallon-cap-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:17:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dressing-stick-rtl2032-with-24-wood-handle-s-and-c-hook>: HTTP status code is not handled or not allowed 2026-01-24 21:17:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-frame-pvc-decontamination-pool-alfdp-48-48l-x-12w-x-55h-230-gallon-cap-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:17:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-24-inch-w-x-18-inch-d-x-62-inch-h-2000-lbs-cap-4-open-grid-shelves>: HTTP status code is not handled or not allowed 2026-01-24 21:17:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/as-interface-strip-cable-stripper-15-20mm-as-interface-cables>: HTTP status code is not handled or not allowed 2026-01-24 21:17:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drain-pump-system-131-0001-with-m98-pump-1-2-hp>: HTTP status code is not handled or not allowed 2026-01-24 21:17:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quick-adjust-wire-shelving-36-x-18-x-63-poly-z-brite>: HTTP status code is not handled or not allowed 2026-01-24 21:17:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/performance-corner-accessory-bar>: HTTP status code is not handled or not allowed 2026-01-24 21:17:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hotel-residential-electronic-security-black-keyed-alike-18w-x-15d-x-18h>: HTTP status code is not handled or not allowed 2026-01-24 21:17:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kni-high-leverage-lineman-s-plier-w-fish-tape-puller-crimper>: HTTP status code is not handled or not allowed 2026-01-24 21:17:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kni-pliers-wrench-set-in-tool-roll-5-pc>: HTTP status code is not handled or not allowed 2026-01-24 21:17:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b-rate-depository-safe-front-loading-combo-lock-30-3-4w-x-21d-x-27-1-4h>: HTTP status code is not handled or not allowed 2026-01-24 21:17:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tower-fan-w-remote-control-3-speed-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:17:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3x5-rejuvenator-end-female>: HTTP status code is not handled or not allowed 2026-01-24 21:17:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jettrac-satellite-post-39h-65l-florescent-yellow-retractable-belt-standard-hook>: HTTP status code is not handled or not allowed 2026-01-24 21:17:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jib-crane-10x12-1000lb>: HTTP status code is not handled or not allowed 2026-01-24 21:17:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-modular-frl-wo-end-ports-d33352-gauge-poly-bowl-manual-fog-lubricator>: HTTP status code is not handled or not allowed 2026-01-24 21:17:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medina-right-curved-desk-extension-47w-x-28d-x-29-1-2h-mocha>: HTTP status code is not handled or not allowed 2026-01-24 21:17:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/34-mod-frl-wend-ports-vc33356w-ga-metal-bowl-wsight-manual-ultra-fog-lubr-osha-valve>: HTTP status code is not handled or not allowed 2026-01-24 21:17:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stethoscope-single-head-22-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:17:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pump-tf2111-75-hp-243-cfm-175-max-psi>: HTTP status code is not handled or not allowed 2026-01-24 21:17:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bacitracin-zinc-ointment-4-oz-tube-pack-of-72>: HTTP status code is not handled or not allowed 2026-01-24 21:17:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/recoil-air-hose-mp287400av-50l-x-14-id>: HTTP status code is not handled or not allowed 2026-01-24 21:17:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ag-silver-calcium-alginate-dressing-12l-x-14w-50-pcs>: HTTP status code is not handled or not allowed 2026-01-24 21:17:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sensi-wrap-self-adherent-bandage-rolls-6w-x-5-yards-tan-12-pcs>: HTTP status code is not handled or not allowed 2026-01-24 21:17:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-basket-truck-with-vinyl-liner-18-bushel-black>: HTTP status code is not handled or not allowed 2026-01-24 21:17:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torx-insert-bit-23024-1-4-drive-t25-tip>: HTTP status code is not handled or not allowed 2026-01-24 21:17:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kit-festooning-13-16>: HTTP status code is not handled or not allowed 2026-01-24 21:17:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/curbless-folding-ramp-traverse-sfel08-31-x-96-3-16-1000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:17:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zinc-oxide-ointment-15-oz-jar-12-pcs>: HTTP status code is not handled or not allowed 2026-01-24 21:17:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sensi-wrap-self-adherent-bandage-rolls-3w-x-5-yards-assorted-24-pcs>: HTTP status code is not handled or not allowed 2026-01-24 21:17:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rb1067-10-m-beamer-athletic-work-hiker-plain-toe-men-s-size-10>: HTTP status code is not handled or not allowed 2026-01-24 21:17:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-quiet-oil-free-air-compressor-with-auto-drain-valve-20-gal-horizontal>: HTTP status code is not handled or not allowed 2026-01-24 21:17:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deep-throat-forged-body-c-clamp-408-0-8-opening-4-1-2-throat-depth-5900-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:17:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deep-sae-socket-5032h-3-8-drive-1-socket-2-1-2-long-6-pt>: HTTP status code is not handled or not allowed 2026-01-24 21:17:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u-sling-wo-head-support-medium>: HTTP status code is not handled or not allowed 2026-01-24 21:17:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-position-car-seat-headrest-mount>: HTTP status code is not handled or not allowed 2026-01-24 21:17:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u-sling-w-head-support-large>: HTTP status code is not handled or not allowed 2026-01-24 21:17:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-cubbie-triple-mobile-fold-n-lock-w-clear-trays96x15x29-1-2birch-plywood>: HTTP status code is not handled or not allowed 2026-01-24 21:17:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-storage-island-twin-w-clear-trays-48w-x-29d-x-29-1-2h-birch-plywood>: HTTP status code is not handled or not allowed 2026-01-24 21:18:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medi-cut-disposable-sterile-scalpels-size-12-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 21:18:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kid-bench-locker-5-wide-camel-cushion-48w-x-15d-x-16h-birch-plywood>: HTTP status code is not handled or not allowed 2026-01-24 21:18:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-tub-mobile-cubbie-w-tubs-60w-x-15d-x-29-1-2h-gray-laminate-black-edge>: HTTP status code is not handled or not allowed 2026-01-24 21:18:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tms312j-traffic-sign-reserved-parking-permit-florida-18-x-12-white>: HTTP status code is not handled or not allowed 2026-01-24 21:18:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cubbie-lid-13-1-2l-x-8-5-8w-x-1-4h-orange>: HTTP status code is not handled or not allowed 2026-01-24 21:18:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tms328j-traffic-sign-reserved-parking-north-carolina-24-x-12-white>: HTTP status code is not handled or not allowed 2026-01-24 21:18:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tms330g-traffic-sign-reserved-parking-north-dakota-24-x-12-white>: HTTP status code is not handled or not allowed 2026-01-24 21:18:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tms336j-traffic-sign-reserved-parking-texas-sign-24-x-12-white->: HTTP status code is not handled or not allowed 2026-01-24 21:18:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tmas18h-traffic-sign-florida-250-fine-6-x-12-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:18:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tms312g-traffic-sign-reserved-parking-permit-florida-18-x-12-white>: HTTP status code is not handled or not allowed 2026-01-24 21:18:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tm506j-traffic-sign-aux-straight-arrow-15-x-21-white>: HTTP status code is not handled or not allowed 2026-01-24 21:18:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tmas12h-traffic-sign-parking-fine-minimum-california-9-x-12-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:18:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tmas10g-traffic-sign-violators-will-be-towed-6-x-12-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:18:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 23879 pages (at 98 pages/min), scraped 13067 items (at 23 items/min) 2026-01-24 21:18:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tmas15g-traffic-sign-maximum-penalty-250-9-x-12-white>: HTTP status code is not handled or not allowed 2026-01-24 21:18:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tmas16j-traffic-sign-parking-250-fine-6-x-12-white>: HTTP status code is not handled or not allowed 2026-01-24 21:18:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premier-built-in-undercounter-freezer-glass-door-abt-ucbi-0420ss-4-5-cu-ft>: HTTP status code is not handled or not allowed 2026-01-24 21:18:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dynarest-airfloat-deluxe-air-mattress-w-perimeter-80l-x-36w-x-11h>: HTTP status code is not handled or not allowed 2026-01-24 21:18:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premier-built-in-undercounter-refrigerator-abt-ucbi-0404ss-4-5-cu-ft>: HTTP status code is not handled or not allowed 2026-01-24 21:18:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/balancing-cream-unscented-1-gallon>: HTTP status code is not handled or not allowed 2026-01-24 21:18:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crescent-shaped-face-rest-covers-for-massage-table-white-bag-of-500>: HTTP status code is not handled or not allowed 2026-01-24 21:18:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/half-round-bolster-24-5-l-x-6-dia-burgundy>: HTTP status code is not handled or not allowed 2026-01-24 21:18:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sl-led-edison-light-bulb-6w-600-lumens-medium-base-dimmable-clear-glass-filament-6-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:18:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sl-led-light-bulb-14w-1500-lumens-twist-and-lock-base-dimmable-super-white-6-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:18:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/amscope-stereo-boom-microscope-20x-40x-magnification-w-5mp-camera-w-flourescent-light>: HTTP status code is not handled or not allowed 2026-01-24 21:18:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tenactiv-glove-w-blended-hppe-ultra-light-foam-nitrile-palm-ansi-a9-size-12>: HTTP status code is not handled or not allowed 2026-01-24 21:18:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/amscope-wf-microscope-eyepiece-10x-magnification-w-reticle-23mm>: HTTP status code is not handled or not allowed 2026-01-24 21:18:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tenactiv-cx-glove-blended-hppe-steel-micropore-nitrile-palm-a7-size-10>: HTTP status code is not handled or not allowed 2026-01-24 21:18:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mc-heavy-duty-tube-cutter-for-1-4-inch-to-1-5-8-inch-o-d-tubing>: HTTP status code is not handled or not allowed 2026-01-24 21:18:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anatomical-model-deluxe-osteoporosis-3-vertebrae>: HTTP status code is not handled or not allowed 2026-01-24 21:18:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chartpak-vinyl-letters-numbers-cha01010-1-2-h-black-helvetica-font-201-pcs>: HTTP status code is not handled or not allowed 2026-01-24 21:18:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mc-spartan-4-way-smart-manifold-with-clamps>: HTTP status code is not handled or not allowed 2026-01-24 21:18:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/snow_ice_removal/snow_blowers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:18:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vertical-all-purpose-band-saw-1-hp-220v-3-phase-ef1459>: HTTP status code is not handled or not allowed 2026-01-24 21:18:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vertical-all-purpose-band-saw-1-hp-440v-3-phase-ef1459>: HTTP status code is not handled or not allowed 2026-01-24 21:18:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oral-mint-flavor-18-inchl-packette-dental-floss-1000-qty>: HTTP status code is not handled or not allowed 2026-01-24 21:18:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/secure-digital-high-capacity-sdhc-ver96808-16-gb-storage-capcity-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:18:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ct-usa-foam-core-mattress-w-sealed-seams-36-inch-x-80-inch-x-6-inch>: HTTP status code is not handled or not allowed 2026-01-24 21:18:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/key-safe-with-keyed-lock-cb12488-160-key-cap-14-1-2-x-11-x-3-1-4-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:18:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flexible-pvc-double-impact-door-fdsc700bsm072108-6-w-x-9-h>: HTTP status code is not handled or not allowed 2026-01-24 21:18:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ct-usa-endurance-wall-mount-step-shelf-81-inch-x-34-1-2-inch-black>: HTTP status code is not handled or not allowed 2026-01-24 21:18:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fl-casual-work-three-quarter-smooth-leather-cognac-10-5eee>: HTTP status code is not handled or not allowed 2026-01-24 21:18:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/75214-mounted-grinding-point-metal-a15>: HTTP status code is not handled or not allowed 2026-01-24 21:18:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbk-athletic-work-shoe-dual-layer-stretch-mesh-upper-black-dark-grey-9-5m>: HTTP status code is not handled or not allowed 2026-01-24 21:18:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fl-casual-work-three-quarter-nubuck-leather-black-12d>: HTTP status code is not handled or not allowed 2026-01-24 21:18:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fl-casual-work-oxford-breathable-mesh-black-13eee>: HTTP status code is not handled or not allowed 2026-01-24 21:18:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/snow_ice_removal/snow_blowers>: HTTP status code is not handled or not allowed 2026-01-24 21:18:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rp-works-ruched-bootie-full-grain-leather-black-8w>: HTTP status code is not handled or not allowed 2026-01-24 21:18:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-jaw-puller-set-4216-6-ton-3way-12-jaws-20-pieces>: HTTP status code is not handled or not allowed 2026-01-24 21:18:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ia-6-inch-waterproof-work-boot-waterproof-leather-brown-8-5w>: HTTP status code is not handled or not allowed 2026-01-24 21:18:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbk-athletic-work-shoe-ultraknit-flexweave-grey-white-16m>: HTTP status code is not handled or not allowed 2026-01-24 21:18:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ia-6-inch-waterproof-work-boot-waterproof-leather-brown-9-5w>: HTTP status code is not handled or not allowed 2026-01-24 21:18:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss304-66010x04-1x12-class-150-hex-bushing-stainless-steel-304>: HTTP status code is not handled or not allowed 2026-01-24 21:18:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbk-athletic-work-shoe-ultraknit-flexweave-grey-white-7-5w>: HTTP status code is not handled or not allowed 2026-01-24 21:18:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rp-works-two-eye-tie-work-sneaker-breathable-knit-textile-leather-brown-tan-9-5m>: HTTP status code is not handled or not allowed 2026-01-24 21:18:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fl-casual-work-three-quarter-smooth-leather-cognac-9eee>: HTTP status code is not handled or not allowed 2026-01-24 21:18:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3m-performance-yellow-masking-tape-301-72-mm-x-55-m>: HTTP status code is not handled or not allowed 2026-01-24 21:18:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h-b-b60215-0000-durac-maximum-registeringautoclave-bi-metal-thermometer>: HTTP status code is not handled or not allowed 2026-01-24 21:18:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h14781-0000-polypropylene-60ml-utility-type-funnels-12pk>: HTTP status code is not handled or not allowed 2026-01-24 21:18:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f37807-0000-adjustable-handle-water-sample-dipper-3-to-9-long-handle>: HTTP status code is not handled or not allowed 2026-01-24 21:18:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-plus-2-step-steel-folding-step-ladder-4342001>: HTTP status code is not handled or not allowed 2026-01-24 21:18:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-round-pedestal-table-with-grey-nebula-top-round-silver-base-bistro-height>: HTTP status code is not handled or not allowed 2026-01-24 21:18:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-6-ft-hd-choker-sling-anchor>: HTTP status code is not handled or not allowed 2026-01-24 21:18:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commercial-all-purpose-scrubbing-dish-wand-refills-2-pack-pky-qty-6-1135>: HTTP status code is not handled or not allowed 2026-01-24 21:18:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u1558-2a-premium-locker-double-tier-15x15x36-2-doors-assembled-parchment>: HTTP status code is not handled or not allowed 2026-01-24 21:18:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u1588-2a-premium-locker-double-tier-15x18x36-2-doors-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:18:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ml-grip-tight-plus-molded-case-circuit-breaker-lockout-device-480-600v-red>: HTTP status code is not handled or not allowed 2026-01-24 21:18:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ml-grip-tight-plus-oversized-handle-circuit-breaker-lockout-480-600v-red>: HTTP status code is not handled or not allowed 2026-01-24 21:18:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crane-scale-shs300a-330-lb-capacity-0-04-lb-readability>: HTTP status code is not handled or not allowed 2026-01-24 21:18:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ml-a1106-anodized-aluminum-safety-padlock-1-1-2-inchw-1-1-2-inchtall-shackle-keyed-alike-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:18:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u1286-5-premium-locker-five-tier-12x18x12-5-doors-ready-to-assemble-parchment>: HTTP status code is not handled or not allowed 2026-01-24 21:18:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-chrome-3-tier-wire-shelving-add-on-unit-48w-x-36d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 21:18:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-chrome-3-tier-wire-shelving-starter-unit-48w-x-36d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 21:18:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tgbn4284-1bc-g-rr-ht-turnout-gear-locker-with-security-box-25x22x86-assembled-relay-red>: HTTP status code is not handled or not allowed 2026-01-24 21:18:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/awa282-1-gym-pe-locker-single-tier-12x18x72-assembled>: HTTP status code is not handled or not allowed 2026-01-24 21:18:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u3518-2hv-a-heavy-duty-ventilated-locker-double-tier-15x21x36-6-doors-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:18:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-chrome-2-tier-wire-shelf-basket-starter-unit-36w-x-24d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 21:18:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-chrome-3-tier-wire-shelving-starter-unit-36w-x-36d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 21:18:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sa-built-in-dishwasher-24-inch-wide-ada-compliant-top-controls-black-door>: HTTP status code is not handled or not allowed 2026-01-24 21:18:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lb18-bkit-2-18-ga-blue-steel-square-hole-pegboard-w-30-pc-lochook-bin-set>: HTTP status code is not handled or not allowed 2026-01-24 21:18:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oralite-48-inch-x-48-inch-super-bright-vivid-orange-roll-up-sign-right-lane-closed-ahead>: HTTP status code is not handled or not allowed 2026-01-24 21:19:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oralite-48-inch-x-48-inch-super-bright-vivid-orange-roll-up-sign-utility-work-ahead>: HTTP status code is not handled or not allowed 2026-01-24 21:19:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/100-jobmaster-rolling-cabinet-609-cu-ft-steel-tan>: HTTP status code is not handled or not allowed 2026-01-24 21:19:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-1-enclosure-mh3100-m1-12125h-x-12125w-x-6d-w-studs-18-ga-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:19:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transformer-tr100va005-100va-120-240-277-480-24v-1-hub-foot-mount-circuit-breaker>: HTTP status code is not handled or not allowed 2026-01-24 21:19:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-gauge-straight-finish-brad-nail-1-length-304-stainless-steel-pkg-of-50000-made-in-usa>: HTTP status code is not handled or not allowed 2026-01-24 21:19:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flap-disc-dw8358-type-27-4-1-2-x-5-8-11-80-grit-13300-rpm-5-pk>: HTTP status code is not handled or not allowed 2026-01-24 21:19:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sliding-door-handle-set-wood-handle-white-die-cast>: HTTP status code is not handled or not allowed 2026-01-24 21:19:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sliding-door-handle-flush-mount-white>: HTTP status code is not handled or not allowed 2026-01-24 21:19:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbwp-8-bushel-ups-fedex-able-antimicrobial-basket-all-swivel-casters-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:19:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flap-disc-dw8308-type-29-4-1-2-x-7-8-60-grit-13300-rpm-10-pk>: HTTP status code is not handled or not allowed 2026-01-24 21:19:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galaxie-wall-sconce-80w-beige-fc4800>: HTTP status code is not handled or not allowed 2026-01-24 21:19:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbwp-10-bushel-vinyl-basket-truck-all-swivel-casters-navy>: HTTP status code is not handled or not allowed 2026-01-24 21:19:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fly-pheromone-fa5000>: HTTP status code is not handled or not allowed 2026-01-24 21:19:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 23980 pages (at 101 pages/min), scraped 13089 items (at 22 items/min) 2026-01-24 21:19:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbwp-10-bushel-ups-fedex-able-antimicrobial-basket-all-swivel-casters-red>: HTTP status code is not handled or not allowed 2026-01-24 21:19:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tt-esd-rubber-matting-mt3072-30x72x0080-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:19:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbwp-12-bushel-ups-fedex-able-antimicrobial-basket-all-swivel-casters-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:19:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbwp-14-bushel-ups-fedex-able-vinyl-basket-all-swivel-casters-beige>: HTTP status code is not handled or not allowed 2026-01-24 21:19:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tt-esd-rubber-matting-mt4524-24x50x0080-green>: HTTP status code is not handled or not allowed 2026-01-24 21:19:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/horizon-mfg-over-the-seat-hard-hat-rack-5002-3l-7-3-4w-6h>: HTTP status code is not handled or not allowed 2026-01-24 21:19:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elite-watertight-case-with-pre-cubed-foam-el0904-10-3-16x9-9-16x4-5-8>: HTTP status code is not handled or not allowed 2026-01-24 21:19:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbwp-16-bushel-ups-fedex-able-antimicrobial-basket-all-swivel-casters-red>: HTTP status code is not handled or not allowed 2026-01-24 21:19:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elite-watertight-case-with-pre-cubed-foam-el012p-13-3-4x9-1-16x2-5-16>: HTTP status code is not handled or not allowed 2026-01-24 21:19:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbwp-16-bushel-antimicrobial-basket-all-swivel-casters-red>: HTTP status code is not handled or not allowed 2026-01-24 21:19:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fixed-non-ventilated-curb-5acc17fs-17-1-2-square-x-8h-galvanized-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:19:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roof-mount-damper-5acc23rd-23-x-23>: HTTP status code is not handled or not allowed 2026-01-24 21:19:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gs-filter-pipette-tip-0-1-10ul-universal-low-retention-graduated-45mm-extended-length>: HTTP status code is not handled or not allowed 2026-01-24 21:19:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-ft-outdoor-bench-with-back-vertical-steel-slat-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:19:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fixed-non-ventilated-curb-5acc20fs-20-1-2-square-x-8h-galvanized-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:19:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cnc-tool-holder-50-tapers-2-5w-x-5-5d-x-2h>: HTTP status code is not handled or not allowed 2026-01-24 21:19:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ajc-fullriver-hgl75-12a-12v-75ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-24 21:19:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motorola-rmm2050-rm-series-2-way-radio-5-channel-2-watt>: HTTP status code is not handled or not allowed 2026-01-24 21:19:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aberdeen-series-12-feet-boat-shaped-conference-table-gray-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:19:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/welding-strip-door-sd60-8-10x8-10w-x-8h-8-red-tint-pvc>: HTTP status code is not handled or not allowed 2026-01-24 21:19:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-6-diameter-x-15-l-air-chute-w-cap-ring-kpac2421-2>: HTTP status code is not handled or not allowed 2026-01-24 21:19:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/addvance-6380-metal-forming-lubricant-1-gallon-container>: HTTP status code is not handled or not allowed 2026-01-24 21:19:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/run-capacitor-45-plus-7-5-mfd-440v-round>: HTTP status code is not handled or not allowed 2026-01-24 21:19:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ajc-apc-back-ups-pro650-12v-12ah-ups-battery>: HTTP status code is not handled or not allowed 2026-01-24 21:19:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/46-s-5-5-8h-4-urethane-polypropylene-wheel-w-ttl-lock-brake-roller-bearing-swivel-caster>: HTTP status code is not handled or not allowed 2026-01-24 21:19:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ansi-grade-2-dual-key-pad-heavy-duty-electronic-lockset-brushed-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:19:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/76-series-8height-6-urethane-on-iron-wheel-rigid-caster>: HTTP status code is not handled or not allowed 2026-01-24 21:19:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/condenser-fan-motor-1-6-to-1-3-hp-1075-rpm-230v>: HTTP status code is not handled or not allowed 2026-01-24 21:19:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/55-2-nd-20-3-solid-disk-shaped-raw-magnet-79-diameter-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:19:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uvc-replacement-lamp-for-nq400-900e>: HTTP status code is not handled or not allowed 2026-01-24 21:19:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dti-frock-serged-seams-elastic-wrists-white-3x-30-qty>: HTTP status code is not handled or not allowed 2026-01-24 21:19:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/106l-long-single-foot-pedal-valve-bottom-mount-certified-nsfansi-372>: HTTP status code is not handled or not allowed 2026-01-24 21:19:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wern-arc-flash-construction-harness-w-dielectric-pass-thru-legs-back-hip-d-rings-m-l>: HTTP status code is not handled or not allowed 2026-01-24 21:19:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wern-basewear-construction-harness-w-tongue-buckle-legs-3-d-ring-s>: HTTP status code is not handled or not allowed 2026-01-24 21:19:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wern-d-bolt-anchor-5-8-inch-opening-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:19:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u18182-floor-mount-utility-sink-with-double-foot-pedal-valve>: HTTP status code is not handled or not allowed 2026-01-24 21:19:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wern-proform-f3-climbing-construction-harness-w-quick-connect-legs-m-l>: HTTP status code is not handled or not allowed 2026-01-24 21:19:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/225524-72-slot-cosmetic-display-for-pegboard-slatwall-13-x-1-5-acrylic>: HTTP status code is not handled or not allowed 2026-01-24 21:19:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-hand-truck-60138-dual-handle-solid-rubber-wheels-1000-lb-capacity-black>: HTTP status code is not handled or not allowed 2026-01-24 21:19:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wern-30-feet-2-man-rope-horizontal-lifeline-w-d-bolt-ratchet>: HTTP status code is not handled or not allowed 2026-01-24 21:19:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gear-metal-locker-70024-open-access-24w-x-24d-x-72h-tan-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 21:19:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/556810-acrylic-cylinder-8-x-1-clear>: HTTP status code is not handled or not allowed 2026-01-24 21:19:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-2-shelf-utility-cart-440lb-cap>: HTTP status code is not handled or not allowed 2026-01-24 21:19:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wern-proform-f3-construction-fall-protection-harness-w-tongue-buckle-legs-s>: HTTP status code is not handled or not allowed 2026-01-24 21:19:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/amscope-18mp-usb3-0-real-time-live-video-microscope-digital-camera>: HTTP status code is not handled or not allowed 2026-01-24 21:19:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/556099-acrylic-shelf-for-pegboard-slatwall-13-5-x-2-clear>: HTTP status code is not handled or not allowed 2026-01-24 21:19:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/21-110l-rigid-to-swing-adapter-spout-on-pre-rinse-body>: HTTP status code is not handled or not allowed 2026-01-24 21:19:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/21-113l-hose-adapter-3-8-x-3-8-male-npt-low-lead>: HTTP status code is not handled or not allowed 2026-01-24 21:19:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gm-exam-cape-30-inch-x-21-inch-mauve-100-case>: HTTP status code is not handled or not allowed 2026-01-24 21:19:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/703389-blu-4-sided-interlocking-pegboard-display-8-x-60-blue-opaque>: HTTP status code is not handled or not allowed 2026-01-24 21:19:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700508-pur-revolving-pegboard-countertop-display-unit-8-x-20-purple>: HTTP status code is not handled or not allowed 2026-01-24 21:19:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-i-beam-mount-fan-w-drop-cord-switch-2-speed-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:19:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-310l-royal-series-4-center-deck-mount-faucet-10-spout-low-lead>: HTTP status code is not handled or not allowed 2026-01-24 21:19:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vs-zx4-ir-system-vortex-mixer-with-foam-stands>: HTTP status code is not handled or not allowed 2026-01-24 21:19:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-812l-royal-series-3-4-full-flow-wall-mount-faucet-low-lead-compliant>: HTTP status code is not handled or not allowed 2026-01-24 21:19:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vs-overhead-stirrer-60-ncm-digital-system-w-stand-shaft-clamp-115v-60hz>: HTTP status code is not handled or not allowed 2026-01-24 21:19:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vs-arec-connect-ceramic-hot-plate-stirrer-115v-60hz>: HTTP status code is not handled or not allowed 2026-01-24 21:19:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/701415-blk-interlocking-pegboard-countertop-display-w-wheels-14-x-40-black-solid>: HTTP status code is not handled or not allowed 2026-01-24 21:19:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss-portable-pistol-security-safe-key-lock-tether-cable-0-12-cu-ft>: HTTP status code is not handled or not allowed 2026-01-24 21:19:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/330-piece-machine-screw-assortment-6-to-1-4-slotted-flat-head-zinc>: HTTP status code is not handled or not allowed 2026-01-24 21:19:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700223-clr-36-pegboard-revolving-floor-display-4-sided-clear-translucent>: HTTP status code is not handled or not allowed 2026-01-24 21:19:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keystock-1-8-x-1-4-x-6-ft-carbon-steel-plain-undersize-astm-a29>: HTTP status code is not handled or not allowed 2026-01-24 21:19:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mrp-high-density-rigid-pail-insert-5-gallon>: HTTP status code is not handled or not allowed 2026-01-24 21:19:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inplant-office-vinyl-cov-hdbd-8x10-4-wall-2-lgts-2-dplx-1-door-white>: HTTP status code is not handled or not allowed 2026-01-24 21:20:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/igg-gage-block-set-rectangular-36-pieces>: HTTP status code is not handled or not allowed 2026-01-24 21:20:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/igg-digital-electronic-angle-protractor-with-4-inch-7-inch-stainless-steel-blades-0-360>: HTTP status code is not handled or not allowed 2026-01-24 21:20:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inplant-office-pntd-steel-drywall-10x16-3-wall-3-lgt-3-dplx-1-dr-wht>: HTTP status code is not handled or not allowed 2026-01-24 21:20:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hygrade-blue-poly-tarp-mt2540-5-mil-25l-x-40w>: HTTP status code is not handled or not allowed 2026-01-24 21:20:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inplant-office-pntd-steel-drywall-16x20-2-wall-5-lgt-6-dplx-1-dr-wht>: HTTP status code is not handled or not allowed 2026-01-24 21:20:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hygrade-heavy-duty-super-cover-poly-tarp-mtgb2540-10-mil-greenblack-25l-x-40w>: HTTP status code is not handled or not allowed 2026-01-24 21:20:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/power_tools/drills_drivers_impact_wrenches/hammer_drills_rotary_hammers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:20:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnetic-nut-driver-2-9-16-l-x-5-16-cut-dia>: HTTP status code is not handled or not allowed 2026-01-24 21:20:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inplant-office-vinyl-cov-hdbd-20x20-3-wall-8-lgts-6-dplx-1-door-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:20:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/infinity-2-tier-2-door-locker-18-in-w-x-18-in-d-x-36-in-h-gray-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 21:20:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inplant-office-vinyl-cov-drywall-16x20-3-wall-5-lgt-6-dplx-1-dr-white>: HTTP status code is not handled or not allowed 2026-01-24 21:20:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 24079 pages (at 99 pages/min), scraped 13114 items (at 25 items/min) 2026-01-24 21:20:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastek-esd-workstation-organizer-universal-16-7-8l-8-1-8w-x-2-3-8h>: HTTP status code is not handled or not allowed 2026-01-24 21:20:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/astm-100-edu-formula-main-carbon-filter-for-mobile-ductless-fume-hood-edu-m>: HTTP status code is not handled or not allowed 2026-01-24 21:20:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-plastic-barrier-chain-in-a-pail-1-1-2-inch-x-300-feet-l-slate-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:20:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-mats-work-surface-male-snaps-23-1-2l-x-47-1-2w-x-1-16h>: HTTP status code is not handled or not allowed 2026-01-24 21:20:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-mats-work-surface-male-snaps-23-1-2l-x-35-1-2w-x-1-16h>: HTTP status code is not handled or not allowed 2026-01-24 21:20:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-electric-garage-space-heater-1500-watt-120v-with-thermostat-red>: HTTP status code is not handled or not allowed 2026-01-24 21:20:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-2-1-2-inch-medium-duty-stowable-stanchion-40-inch-h-white-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-24 21:20:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/circuit-board-shipping-and-storage-box-w-foam-11-1-2l-x-8-7-8w-x-2-3-4h>: HTTP status code is not handled or not allowed 2026-01-24 21:20:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-reflective-plastic-barrier-chain-2-inch-x-100-feet-l-slate-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:20:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/power_tools/drills_drivers_impact_wrenches/hammer_drills_rotary_hammers>: HTTP status code is not handled or not allowed 2026-01-24 21:20:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-2-1-2-inch-medium-duty-reflective-stanchion-kit-w-2-inch-x-50-feet-l-chain-white-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-24 21:20:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-3-inch-heavy-duty-stowable-stanchion-41-inch-h-blue-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-24 21:20:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-step-aluminum-rolling-ladder-16inw-grip-tread-21ind-top-step-w-o-handrails>: HTTP status code is not handled or not allowed 2026-01-24 21:20:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-3-inch-heavy-duty-stowable-stanchion-41-inch-h-safety-green-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-24 21:20:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-2-1-2-inch-medium-duty-stowable-stanchion-40-inch-h-white-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-24 21:20:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-step-aluminum-rolling-ladder-24inw-ribbed-tread-21ind-top-step-32in-handrails>: HTTP status code is not handled or not allowed 2026-01-24 21:20:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-utility-stakes-in-a-pail-9-inch-h-safety-green-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-24 21:20:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-series-swivel-plate-caster-408199459-rubber-on-polyolefin-8dia-675-lb-cap>: HTTP status code is not handled or not allowed 2026-01-24 21:20:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-series-swivel-plate-caster-405109459-rubber-on-polyolefin-5dia-500-lb-cap>: HTTP status code is not handled or not allowed 2026-01-24 21:20:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wash-bottle-ldpe-500ml-capacity-sodium-hypochlorite-4-pk-yellow-cap-wide-mouth>: HTTP status code is not handled or not allowed 2026-01-24 21:20:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heb201u-family-pdu-50a-1ph>: HTTP status code is not handled or not allowed 2026-01-24 21:20:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp100-11-radar-feedback-sign-ac-power-with-your-speed-sign-white-sign-1485-00065>: HTTP status code is not handled or not allowed 2026-01-24 21:20:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-2-6-x-5-1-2-hex-head-cap-screw-grade-8-partial-thread-unc-steel-plain>: HTTP status code is not handled or not allowed 2026-01-24 21:20:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fitness-ub521m-upper-body-ergometer-with-adjustable-seat-69l-x-28w-x-70h>: HTTP status code is not handled or not allowed 2026-01-24 21:20:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-exercise-putty-blue-firm-1-pound>: HTTP status code is not handled or not allowed 2026-01-24 21:20:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ashford-panel-moulding-corner-pml06x06as-618w-x-618h>: HTTP status code is not handled or not allowed 2026-01-24 21:20:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/seville-panel-moulding-corner-pml06x06se2-6w-x-6h-x-34d>: HTTP status code is not handled or not allowed 2026-01-24 21:20:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-putty-pear-xx-soft-4-ounce>: HTTP status code is not handled or not allowed 2026-01-24 21:20:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microwaveable-exercise-putty-firm-blue-6-ounce>: HTTP status code is not handled or not allowed 2026-01-24 21:20:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/southampton-capital-cap10x03x02so-1014w-x-3h-x-212dw-x-58>: HTTP status code is not handled or not allowed 2026-01-24 21:20:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/antimicrobial-exercise-putty-red-5-pound-soft>: HTTP status code is not handled or not allowed 2026-01-24 21:20:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/antimicrobial-exercise-putty-6-ounce-set-of-6>: HTTP status code is not handled or not allowed 2026-01-24 21:20:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/antimicrobial-exercise-putty-red-4-ounce-soft>: HTTP status code is not handled or not allowed 2026-01-24 21:20:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/antimicrobial-exercise-putty-black-3-ounce>: HTTP status code is not handled or not allowed 2026-01-24 21:20:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stability-trainer-16-diameter-platform-and-balance-disc>: HTTP status code is not handled or not allowed 2026-01-24 21:20:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/skb-2s-stainless-steel-replacement-blade>: HTTP status code is not handled or not allowed 2026-01-24 21:20:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-wide-designer-wood-locker-21364-single-tier-3-wide-15x24x72-maple-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 21:20:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/designer-wood-locker-31365-single-tier-3-wide-12w-x-15d-x-72h-gray-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 21:20:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/designer-wood-locker-32368-double-tier-3-wide-12w-x-18d-x-36h-gray-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 21:20:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cl-ex40-16st-16in-9-gallon-carpet-extractor-56265504>: HTTP status code is not handled or not allowed 2026-01-24 21:20:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15020035-led-corn-lamp-36w-4490-lumens-5000k-medium-e26-100w-hid-replacement>: HTTP status code is not handled or not allowed 2026-01-24 21:20:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-drop-key-control-box-4-3-8-w-x-3-1-4-d-x-7-1-4-h-keyed-alike-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:20:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14031427-perfo-15-piece-tool-holder-accessory-kit-pl31>: HTTP status code is not handled or not allowed 2026-01-24 21:20:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/workfit-sr-1-monitor-sit-stand-desktop-workstation-white>: HTTP status code is not handled or not allowed 2026-01-24 21:20:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-pe-foam-roller-half-round-6-dia-x-24l>: HTTP status code is not handled or not allowed 2026-01-24 21:20:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hand-exerciser-large-red-light>: HTTP status code is not handled or not allowed 2026-01-24 21:20:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/explosion-proof-com-pax-ial-blower-without-canister-9538-8-diameter-1-3hp-115v-900-cfm>: HTTP status code is not handled or not allowed 2026-01-24 21:20:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/satco-s9711-led-t5-replacement-25w-5000k>: HTTP status code is not handled or not allowed 2026-01-24 21:20:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disc-361f-2-inch-diameter-tsm-aluminum-oxide-60-grit>: HTTP status code is not handled or not allowed 2026-01-24 21:20:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inflatable-vestibular-seating-standing-disc-35-cm-14-green>: HTTP status code is not handled or not allowed 2026-01-24 21:20:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-series-inflatable-exercise-ball-65-cm-26-green>: HTTP status code is not handled or not allowed 2026-01-24 21:20:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inflatable-exercise-sensi-ball-red-38-95-cm>: HTTP status code is not handled or not allowed 2026-01-24 21:20:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-skate-board-rom-exerciser-resistance-system-only-9-weights-included>: HTTP status code is not handled or not allowed 2026-01-24 21:20:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inflatable-exercise-ball-with-feet-yellow-18-45-cm>: HTTP status code is not handled or not allowed 2026-01-24 21:20:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-liquid-laundry-detergent-intense-fresh-scent-100-oz-4-case-024200094218>: HTTP status code is not handled or not allowed 2026-01-24 21:20:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hitch-bike-rack-2-bike-capacity-double-sided-surface-mount-green-5040sm>: HTTP status code is not handled or not allowed 2026-01-24 21:20:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/loaded-gear-cx-200-triple-magazine-pouch-bi12276-od-green>: HTTP status code is not handled or not allowed 2026-01-24 21:20:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-stp-steel-safe-roll-lad-wgt-act-lock-16-w-serr-stp-org-wa062414g-o>: HTTP status code is not handled or not allowed 2026-01-24 21:20:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-stp-steel-safe-roll-lad-wgt-act-lock-16-w-exp-stp-org-wa052414x-o>: HTTP status code is not handled or not allowed 2026-01-24 21:20:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdpe-ground-protection-mat-3-x-6-clear-cm36>: HTTP status code is not handled or not allowed 2026-01-24 21:20:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/small-kent-floral-rosette-ros04x04x01kelw-412w-x-412h-x-58d>: HTTP status code is not handled or not allowed 2026-01-24 21:20:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stockport-bracket-bktm02x16x16stcrs-2w-x-16d-x-16h>: HTTP status code is not handled or not allowed 2026-01-24 21:20:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stiebel-eltron-mini-4-3-5-kw-point-of-use-tankless-electric-water-heater-220-240v>: HTTP status code is not handled or not allowed 2026-01-24 21:20:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibration-isolation-mounts-cylindrical-type-59-44-96-max-load-B1121165>: HTTP status code is not handled or not allowed 2026-01-24 21:20:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6in-x-3-8in-microfiber-mini-roller-covers-2-pack-24-case-6mf8038d>: HTTP status code is not handled or not allowed 2026-01-24 21:20:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/office-partition-panel-with-pass-thru-cable-24-1-4w-x-46h-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:20:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/office-partition-panel-with-pass-thru-cable-24-1-4w-x-76h-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:20:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4in-x-3-8in-100-polyester-roller-covers-2-pack-12-case-4ap038d>: HTTP status code is not handled or not allowed 2026-01-24 21:20:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hand-toolhose-assembly-upgrade-for-ex40-part-56265810>: HTTP status code is not handled or not allowed 2026-01-24 21:20:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hand-drive-assembly-ut3g-model-33336310>: HTTP status code is not handled or not allowed 2026-01-24 21:20:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kenda-smooth-tire-13x500-6-4pr>: HTTP status code is not handled or not allowed 2026-01-24 21:21:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slice-ceramic-blade-auto-retractable-box-cutter-10503>: HTTP status code is not handled or not allowed 2026-01-24 21:21:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/card5s-esd-dual-wheel-hard-floor-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:21:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/asimeto-602-02-1-articulating-arm-magnetic-base-60kg-280mm>: HTTP status code is not handled or not allowed 2026-01-24 21:21:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9551m-e-v-blk-integra-esd-vinyl-upholstered-chair-medium-back-aluminum-base-black>: HTTP status code is not handled or not allowed 2026-01-24 21:21:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7501d-dura-polyurethane-chair-black-nylon-base-mushroom-glides-black>: HTTP status code is not handled or not allowed 2026-01-24 21:21:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medstoremax-double-column-shelf-w-5-adjustable-dividers>: HTTP status code is not handled or not allowed 2026-01-24 21:21:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/32-device-charging-cart>: HTTP status code is not handled or not allowed 2026-01-24 21:21:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/riser-tapered-with-carpet-78l-x-18w-x-32h-black>: HTTP status code is not handled or not allowed 2026-01-24 21:21:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microcentrifuge-test-tube-rack-for-0-5-ml-tubes-requires-clip-fastener>: HTTP status code is not handled or not allowed 2026-01-24 21:21:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trailer-tire-480-8-6-ply>: HTTP status code is not handled or not allowed 2026-01-24 21:21:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shallow-shaking-water-bath-swb-15s-15-liters>: HTTP status code is not handled or not allowed 2026-01-24 21:21:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m4-x-0-7-x-20mm-security-machine-screw-flat-torx-head-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:21:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 24180 pages (at 101 pages/min), scraped 13132 items (at 18 items/min) 2026-01-24 21:21:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-24-x-1-security-cap-screw-flat-socket-head-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:21:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-16-x-1-security-cap-screw-flat-socket-head-18-8-stainless-steel-pkg-of-25>: HTTP status code is not handled or not allowed 2026-01-24 21:21:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-16-x-1-2-security-cap-screw-flat-socket-head-18-8-stainless-steel-pkg-of-25>: HTTP status code is not handled or not allowed 2026-01-24 21:21:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/liquid-bag-filter-poly-p-felt-4-1-8-dia-x-14-l-10-micron-plastic-sure-seal-ring-pkg-qty-50>: HTTP status code is not handled or not allowed 2026-01-24 21:21:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-x-1-security-sheet-metal-screw-pan-phillips-head-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:21:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-x-2-1-2-security-sheet-metal-screw-flat-phillips-head-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:21:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2mm-balldriver-screwdriver-long-10752>: HTTP status code is not handled or not allowed 2026-01-24 21:21:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-16-prohold-balldriver-screwdriver-74603>: HTTP status code is not handled or not allowed 2026-01-24 21:21:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sewn-liquid-bag-filter-polypropylene-felt-5-1-2-d-x-21-l-5-micron-steel-ring-pkg-50>: HTTP status code is not handled or not allowed 2026-01-24 21:21:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15mm-hex-l-wrench-long-12150>: HTTP status code is not handled or not allowed 2026-01-24 21:21:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rectangle-activity-table-72w-x-30l-x-24-31h-classic-oak>: HTTP status code is not handled or not allowed 2026-01-24 21:21:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-16-tamperproof-hex-key>: HTTP status code is not handled or not allowed 2026-01-24 21:21:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m5-x-0-8-x-16mm-security-cap-screw-flat-socket-head-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:21:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-1-2-x-17-1-2-x-16-clear-mobile-heavy-duty-giant-stack-container>: HTTP status code is not handled or not allowed 2026-01-24 21:21:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-freestanding-office-partition-panel-with-partial-window-36-1-4-w-x-43-1-2-h-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:21:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-office-partition-panel-with-partial-window-and-raceway-48-1-4w-x-47-1-2h-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:21:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roof-cable-de-icer-200-ft-120v-7-watts-per-foot>: HTTP status code is not handled or not allowed 2026-01-24 21:21:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/do-not-double-stack-2-w-x-110-yards-white-red>: HTTP status code is not handled or not allowed 2026-01-24 21:21:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-x-6-ft-nylon-canada-flag>: HTTP status code is not handled or not allowed 2026-01-24 21:21:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/canmeleon-aggregate-panel-ash-urn-side-open-38-gallon-tan>: HTTP status code is not handled or not allowed 2026-01-24 21:21:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-way-block-valve-1-4-male-sf-1-4-female-npt>: HTTP status code is not handled or not allowed 2026-01-24 21:21:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelving-30w-x-14d-x-63h-4-shelf-add-on-chrome>: HTTP status code is not handled or not allowed 2026-01-24 21:21:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-y-manifold-6mm-tube-x-1-8-swift-fit>: HTTP status code is not handled or not allowed 2026-01-24 21:21:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inline-flow-control-3-8-male-x-3-8-tube>: HTTP status code is not handled or not allowed 2026-01-24 21:21:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelf-level-36w-x-14d-chrome>: HTTP status code is not handled or not allowed 2026-01-24 21:21:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5t5000p05a-set-screw-w-full-ball-516-18-thread-x-12-thread-length>: HTTP status code is not handled or not allowed 2026-01-24 21:21:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ball-valve-1-2-female-bspp>: HTTP status code is not handled or not allowed 2026-01-24 21:21:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-pc-long-paddle-with-out-holes-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:21:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50700n-reducer-12mm-stud-x-10mm-tube>: HTTP status code is not handled or not allowed 2026-01-24 21:21:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ac-wetstep-drainable-mat-3ft-x-5ft-gray-789020035>: HTTP status code is not handled or not allowed 2026-01-24 21:21:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/activated-carbon-chemical-odor-removing-filter-sock-9feet>: HTTP status code is not handled or not allowed 2026-01-24 21:21:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultradewatering-bag-oil-and-sediment-model-3feet-x-4feet>: HTTP status code is not handled or not allowed 2026-01-24 21:21:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slide-seal-bag-10w-7l>: HTTP status code is not handled or not allowed 2026-01-24 21:21:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/i-mini-stirrer-with-telemodul-20-c-controller-100-240v>: HTTP status code is not handled or not allowed 2026-01-24 21:21:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/battery-connector-din-a320-female-connector-with-70mm-socket-contacts-handle>: HTTP status code is not handled or not allowed 2026-01-24 21:21:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-epoxy-wire-shelving-189254gz-60w-x-18d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 21:21:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transport-bag-tamper-evident-10w-14l>: HTTP status code is not handled or not allowed 2026-01-24 21:21:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reversible-sheeter-floor-model-24-x-47>: HTTP status code is not handled or not allowed 2026-01-24 21:21:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/invisible-solid-antiperspirant-deodorant-floral-scent-0-5-oz-36-sticks-case>: HTTP status code is not handled or not allowed 2026-01-24 21:21:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-step-plastic-step-stand-extra-large-black-2512w-33d-x-24h-ntxst-2-bk>: HTTP status code is not handled or not allowed 2026-01-24 21:21:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-step-nestable-plastic-step-stand-gray-26-w-x-33-d-x-20-h-nst-2-gy>: HTTP status code is not handled or not allowed 2026-01-24 21:21:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/amber-bag-open-ended-8w-14l>: HTTP status code is not handled or not allowed 2026-01-24 21:21:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-epoxy-wire-shelving-189217gz-30w-x-24d-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 21:21:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/werther-s-original-sugar-free-chewy-caramel-candy-1-46-oz-12-count-gr>: HTTP status code is not handled or not allowed 2026-01-24 21:21:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cel-gro-tissue-culture-rotator-drum-holds-76-26mm-tubes>: HTTP status code is not handled or not allowed 2026-01-24 21:21:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pegboard-panels-powdercoat-orange-16-x-32-2-pc>: HTTP status code is not handled or not allowed 2026-01-24 21:21:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-72-w-x-24-d-x-96-h-starter-3-shelf-levels-steel-deck-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:21:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-96-w-x-48-d-x-84-h-add-on-3-shelf-levels-steel-deck-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:21:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-tabletop-ironing-board-w-retractable-iron-rest-white-frame-aqua-blue-cover-06mm-wood-top>: HTTP status code is not handled or not allowed 2026-01-24 21:21:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-readybuilt-locker-12-w-x-18-d-x-78-h-725-hallowell-gray-single-tier-1-wide-assembled>: HTTP status code is not handled or not allowed 2026-01-24 21:21:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:21:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-48-w-x-36-d-x-72-h-add-on-3-shelf-levels-steel-deck-sand>: HTTP status code is not handled or not allowed 2026-01-24 21:21:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-48-w-x-24-d-x-72-h-starter-3-shelf-levels-wood-deck-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:21:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/digital-heating-shaking-drybath-120v-60hz>: HTTP status code is not handled or not allowed 2026-01-24 21:21:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/touch-screen-dry-bath-block-heater-1-block-capacity-100-120v-50-60hz>: HTTP status code is not handled or not allowed 2026-01-24 21:21:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/concrete-anchor-316-x-1-14-phillips-flat-head-pkg-of-225-24550>: HTTP status code is not handled or not allowed 2026-01-24 21:21:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/earth-self-stick-easel-pad-25w-x-30h-board-2-pk-fl1218207>: HTTP status code is not handled or not allowed 2026-01-24 21:21:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portfolio-clipboard-red-61632>: HTTP status code is not handled or not allowed 2026-01-24 21:22:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simplehuman-174-adjustable-shower-caddy-bt1098>: HTTP status code is not handled or not allowed 2026-01-24 21:22:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-value-max-locker-12-w-x-18-d-x-78-h-729-parchment-triple-tier-1-wide-assembled>: HTTP status code is not handled or not allowed 2026-01-24 21:22:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/westcott-15583-kleenearth-basic-plastic-handle-scissors-8-length-pointed-black>: HTTP status code is not handled or not allowed 2026-01-24 21:22:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-value-max-locker-36-w-x-15-d-x-78-h-729-parchment-double-tier-3-wide-assembled>: HTTP status code is not handled or not allowed 2026-01-24 21:22:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wesco-2-wheel-appliance-truck-750-lb-cap-59in-h-auto-rewind>: HTTP status code is not handled or not allowed 2026-01-24 21:22:04 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks landed on page that is not a product page. 2026-01-24 21:22:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-hi-tech-metal-bin-shelving-36-w-x-12-d-x-87-h-725-hallowell-gray-starter-unit-21-bins>: HTTP status code is not handled or not allowed 2026-01-24 21:22:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-hi-tech-metal-bin-shelving-36-w-x-18-d-x-87-h-725-hallowell-gray-add-on-unit-54-bins>: HTTP status code is not handled or not allowed 2026-01-24 21:22:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:22:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alba-pm1part-bo-cubicle-garment-peg-1-hook-1-15-x-1-38-x-4-310-metallic-grey>: HTTP status code is not handled or not allowed 2026-01-24 21:22:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kensington-62567-sp-17-17-laptop-sleeve-padded-interior-interiorexterior-pockets-black>: HTTP status code is not handled or not allowed 2026-01-24 21:22:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1021068-drain-flat-strain-1-1-2-inch-nps-for-standard-keil>: HTTP status code is not handled or not allowed 2026-01-24 21:22:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1901334-screw-truss-head-10-32-x-25-inch-for-bunn-o-matic>: HTTP status code is not handled or not allowed 2026-01-24 21:22:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 24275 pages (at 95 pages/min), scraped 13158 items (at 26 items/min) 2026-01-24 21:22:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/321421-o-ring-2-7-8-od-for-hobart>: HTTP status code is not handled or not allowed 2026-01-24 21:22:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s1101sgaf-clear-1000-series-clear-anti-fog-lens-20ea-cs>: HTTP status code is not handled or not allowed 2026-01-24 21:22:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010558-air-shutter-open-top-kit-for-garland-range>: HTTP status code is not handled or not allowed 2026-01-24 21:22:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alarm-lock-mortise-cylinder-pg10-pg21>: HTTP status code is not handled or not allowed 2026-01-24 21:22:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/abrasives/grinding_cutoff_wheels/depressed_center_wheels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:22:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-ply-embossed-roll-bathroom-tissue-4-x-4-550-sheets-roll-80-pk-gjo2508080>: HTTP status code is not handled or not allowed 2026-01-24 21:22:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sign-frame-two-acrylic-inserts-8-1-2-inch-by-11-inch>: HTTP status code is not handled or not allowed 2026-01-24 21:22:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks>: HTTP status code is not handled or not allowed 2026-01-24 21:22:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wisconsin-state-flag-3x5-nylon>: HTTP status code is not handled or not allowed 2026-01-24 21:22:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012926-relay-40a-solid-state-for-lincoln-manufacturing>: HTTP status code is not handled or not allowed 2026-01-24 21:22:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/incubated-refrigerated-environmental-chamber-humidity-control-29-cu-ft>: HTTP status code is not handled or not allowed 2026-01-24 21:22:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-96-w-x-24-d-x-96-h-add-on-3-shelf-levels-steel-deck-medium-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:22:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/igs400-general-protocol-microbiological-incubator-14-3-cu-ft-120v>: HTTP status code is not handled or not allowed 2026-01-24 21:22:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/32-1090-pump-seal-1-inch-outer-diameter>: HTTP status code is not handled or not allowed 2026-01-24 21:22:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012116-motor-for-vulcan-hart>: HTTP status code is not handled or not allowed 2026-01-24 21:22:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oms180-general-protocol-oven-mechanical-convection-6-3-cu-ft-120v>: HTTP status code is not handled or not allowed 2026-01-24 21:22:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1681263-insulation-front-seal-for-frymaster>: HTTP status code is not handled or not allowed 2026-01-24 21:22:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1901325-blade-mixing-for-bunn-o-matic>: HTTP status code is not handled or not allowed 2026-01-24 21:22:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/abrasives/grinding_cutoff_wheels/depressed_center_wheels>: HTTP status code is not handled or not allowed 2026-01-24 21:22:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-capacity-reach-in-co2-incubator-with-tc-sensor-120v-50-60hz>: HTTP status code is not handled or not allowed 2026-01-24 21:22:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010445-grommet-dispense-valve-for-bunn-o-matic>: HTTP status code is not handled or not allowed 2026-01-24 21:22:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-72-w-x-24-d-x-96-h-starter-3-shelf-levels-wire-deck-medium-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:22:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alera-alesd216030po-double-pedestal-steel-desk-metal-desk-60w-x-30d-x-29-12h-cherryputty>: HTTP status code is not handled or not allowed 2026-01-24 21:22:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alera-hplsd2460zp-double-pedestal-steel-credenza-60w-x-24d-x-29-12h-walnutblack>: HTTP status code is not handled or not allowed 2026-01-24 21:22:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-96-w-x-24-d-x-72-h-starter-3-shelf-levels-steel-deck-medium-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:22:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2041227-o-ring-silicone-1-1-2-inch-od-fda-for-hatco-corp>: HTTP status code is not handled or not allowed 2026-01-24 21:22:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/office_furniture/literature_mail_sorters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:22:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deflect-o-775383-euro-style-docuholder-4-12w-x-3-34d-x-8h-clear>: HTTP status code is not handled or not allowed 2026-01-24 21:22:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-shelf-level-for-bulk-storage-rack-72-w-x-24-d-wood-deck-medium-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:22:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-ton-high-reach-jack-stands-hw93511>: HTTP status code is not handled or not allowed 2026-01-24 21:22:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-1-2-ton-magic-lift-service-jack-27035>: HTTP status code is not handled or not allowed 2026-01-24 21:22:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chateau-series-mahogany-laminate-desking-709x148x365-hutch>: HTTP status code is not handled or not allowed 2026-01-24 21:22:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/261224-orifice>: HTTP status code is not handled or not allowed 2026-01-24 21:22:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oil-drip-pan-92510>: HTTP status code is not handled or not allowed 2026-01-24 21:22:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-40-14w-cabinet-12-drawer-174-compart-classic-blue-no-lock>: HTTP status code is not handled or not allowed 2026-01-24 21:22:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-terrain-six-wheel-dolly-1000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:22:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/450-lb-cap-low-profile-z-creeper-91452>: HTTP status code is not handled or not allowed 2026-01-24 21:22:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/walnut-laminate-office-suite-desking-59x148x36-hutch>: HTTP status code is not handled or not allowed 2026-01-24 21:22:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/office_furniture/literature_mail_sorters>: HTTP status code is not handled or not allowed 2026-01-24 21:22:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1100-lb-cap-low-profile-hydraulic-transmission-jack-41100c>: HTTP status code is not handled or not allowed 2026-01-24 21:22:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-40-14w-drawer-cabinet-5-drawer-51-compart-light-gray-no-lock>: HTTP status code is not handled or not allowed 2026-01-24 21:22:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-purpose-transmission-adapter-hw93721>: HTTP status code is not handled or not allowed 2026-01-24 21:22:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-40-14w-mobile-cabinet-7-drawers-94-compart-classic-blue-master-keyed>: HTTP status code is not handled or not allowed 2026-01-24 21:22:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-40-14w-mobile-cabinet-5-drawers-63-compart-light-gray-individual-lock>: HTTP status code is not handled or not allowed 2026-01-24 21:22:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011854-relay-solid-state-dual-for-turbochef>: HTTP status code is not handled or not allowed 2026-01-24 21:22:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-40-14w-mobile-cabinet-7-drawers-94-compart-light-gray-master-keyed>: HTTP status code is not handled or not allowed 2026-01-24 21:22:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-40-14w-cabinet-9-drawer-120-compart-light-gray-no-lock>: HTTP status code is not handled or not allowed 2026-01-24 21:22:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/-expander-blue-heavy>: HTTP status code is not handled or not allowed 2026-01-24 21:22:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-40-14w-drawer-cabinet10-drawer162-compart-bright-blue-individual-lock>: HTTP status code is not handled or not allowed 2026-01-24 21:22:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-40-14w-drawer-cabinet-5-drawer-51-compart-classic-blue-keyed-alike>: HTTP status code is not handled or not allowed 2026-01-24 21:22:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-ad824g-poly-green-divider-24d-x-8h-for-wire-shelves>: HTTP status code is not handled or not allowed 2026-01-24 21:22:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/263164-wire-guard>: HTTP status code is not handled or not allowed 2026-01-24 21:22:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-40-14w-cabinet-16-drawer-270-compart-light-gray-individual-lock>: HTTP status code is not handled or not allowed 2026-01-24 21:22:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2031136-weight-end-assy-for-globe-food-equipment>: HTTP status code is not handled or not allowed 2026-01-24 21:22:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/red-single-wall-heat-shrink-5670-100r-3-4-expanded-diameter-100-pcs>: HTTP status code is not handled or not allowed 2026-01-24 21:22:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foam-padded-sports-handle-for-exercise-band-tubing>: HTTP status code is not handled or not allowed 2026-01-24 21:22:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylon-solderless-butt-connector-flared-ends-162280-100-12-10-gauge-100-pcs>: HTTP status code is not handled or not allowed 2026-01-24 21:22:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/packaging/wrapping_equipment/shrink_wrap_systems already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:22:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2061250-hub-drive-for-robot-coupe>: HTTP status code is not handled or not allowed 2026-01-24 21:22:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unisex-blue-65-polyester-34-cotton-1-carbon-microstat-esd-short-coat-size-s>: HTTP status code is not handled or not allowed 2026-01-24 21:22:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interion-174-restaurant-lunchroom-square-bar-height-table-36-x-36-x-42-h-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:22:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/red-universal-leadhead-213552-001-2-0-gauge-1-pc>: HTTP status code is not handled or not allowed 2026-01-24 21:22:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unisex-ciel-blue-cotton-polyester-fashion-blend-reversible-scrub-pants-size-m>: HTTP status code is not handled or not allowed 2026-01-24 21:22:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-free-exercise-band-yellow-100-yard-roll-1-roll-box>: HTTP status code is not handled or not allowed 2026-01-24 21:22:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-free-exercise-band-yellow-50-yard-roll-1-roll-box>: HTTP status code is not handled or not allowed 2026-01-24 21:22:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/263801-rivet-springretainer-6-pkg-for-ice-o-matic>: HTTP status code is not handled or not allowed 2026-01-24 21:22:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-nylon-cable-ties-502104-100-85-long-100-pcs>: HTTP status code is not handled or not allowed 2026-01-24 21:23:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-free-exercise-band-blueberry-level-4-6-yard-roll-box>: HTTP status code is not handled or not allowed 2026-01-24 21:23:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009806-fuse-5-amp-tiny-i-oboard-for-henny-penny>: HTTP status code is not handled or not allowed 2026-01-24 21:23:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1731083-washer-flat-hd8799-hd8802-for-bloomfield>: HTTP status code is not handled or not allowed 2026-01-24 21:23:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/packaging/wrapping_equipment/shrink_wrap_systems>: HTTP status code is not handled or not allowed 2026-01-24 21:23:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/06159-basic-wall-bracket-black>: HTTP status code is not handled or not allowed 2026-01-24 21:23:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/29357-51-aluminum-handle-orange>: HTTP status code is not handled or not allowed 2026-01-24 21:23:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-free-exercise-band-plum-level-5-50-yard-roll-box>: HTTP status code is not handled or not allowed 2026-01-24 21:23:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lead-head-post-mold-213900-001-1-pc>: HTTP status code is not handled or not allowed 2026-01-24 21:23:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cover-guard-174-10-mil-temporary-surface-protection-72in-x-393ft>: HTTP status code is not handled or not allowed 2026-01-24 21:23:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/441616-contactor-2p-50-65a-208-240v>: HTTP status code is not handled or not allowed 2026-01-24 21:23:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-free-exercise-band-blueberry-level-4-50-yard-roll-box>: HTTP status code is not handled or not allowed 2026-01-24 21:23:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conntek-20601-040-40-30-amp-generator-power-extension-cord-with-nema-l14-30p-to-l14-30r>: HTTP status code is not handled or not allowed 2026-01-24 21:23:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 24369 pages (at 94 pages/min), scraped 13176 items (at 18 items/min) 2026-01-24 21:23:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-free-pre-cut-exercise-band-heavy-blue-48-l-strips-box-of-40>: HTTP status code is not handled or not allowed 2026-01-24 21:23:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/econoco-z-rack-with-osha-orange-base-rzk8rng-64l-x-24w-x-70h-black-with-orange-base>: HTTP status code is not handled or not allowed 2026-01-24 21:23:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-exercise-band-light-red-10-each>: HTTP status code is not handled or not allowed 2026-01-24 21:23:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/38-1084-signal-light-3-8-inch-x-1-5-16-inch-amber-125v>: HTTP status code is not handled or not allowed 2026-01-24 21:23:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leviton-23452-v-snap-in-lamp-support-clip-for-2g11-base-twin-tube-fluorescent-lampholder>: HTTP status code is not handled or not allowed 2026-01-24 21:23:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/321478-o-ring-3-4-inch-id-x-3-32-inch-width-for-hobart>: HTTP status code is not handled or not allowed 2026-01-24 21:23:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hager-4700-series-rim-exit-device2>: HTTP status code is not handled or not allowed 2026-01-24 21:23:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5491019-gasket-element>: HTTP status code is not handled or not allowed 2026-01-24 21:23:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sm-8000-p-rainier-pressure-balance-valve-trim>: HTTP status code is not handled or not allowed 2026-01-24 21:23:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sc-1210-af-commander-exposed-shower>: HTTP status code is not handled or not allowed 2026-01-24 21:23:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/56887-1-5-gallon-bucket-orange>: HTTP status code is not handled or not allowed 2026-01-24 21:23:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s-3219-wall-mounted-double-pedal-self-closing-mixing-valve>: HTTP status code is not handled or not allowed 2026-01-24 21:23:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-band-loop-30-long-red>: HTTP status code is not handled or not allowed 2026-01-24 21:23:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/53683-216-1-6-tube-brush-for-flex-rod-medium-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:23:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1681265-insulation-inner-front-lwr-for-frymaster>: HTTP status code is not handled or not allowed 2026-01-24 21:23:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metmax-acifort-full-safety-boots-w-metatarsal--ankle-protection-15h-size-10-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:23:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/huot-tap-dispenser-metric>: HTTP status code is not handled or not allowed 2026-01-24 21:23:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-high-z-bracket-rz0501-silver>: HTTP status code is not handled or not allowed 2026-01-24 21:23:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-floor-stand-10-1-2w-x-39-3-4d-x-30-1-8h-3-cylinder-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:23:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-free-exercise-band-5-band-heavy-blue-black-retail-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:23:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cylinder-pallet-stand-w-firewall-32w-x-48d-x-78h-12-cylinder-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:23:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rigpro174-purofort-full-safety-boots-steel-toe-12h-size-13-tan>: HTTP status code is not handled or not allowed 2026-01-24 21:23:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/93249-spray-gun-black>: HTTP status code is not handled or not allowed 2026-01-24 21:23:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/barnstead-full-size-ultrapure-cartridge-d8911-1-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:23:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/53654-216-0-8-tube-brush-for-flex-rod-medium-red>: HTTP status code is not handled or not allowed 2026-01-24 21:23:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-cylinder-tube-rack-1-wide-x-3-deep-16w-x-38d-x-30h3-cylinder-cap>: HTTP status code is not handled or not allowed 2026-01-24 21:23:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:23:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1831158-knob-ball-cover-1-1-2-inch-for-roundup-food-equipment>: HTTP status code is not handled or not allowed 2026-01-24 21:23:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1681278-shield-probe-for-frymaster>: HTTP status code is not handled or not allowed 2026-01-24 21:23:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-welded-frame-mirror-18-w-x-30-h>: HTTP status code is not handled or not allowed 2026-01-24 21:23:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reflective-delineator-post-49-in-h-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:23:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2631026-slide-door-24-1-2-inch-for-hobart>: HTTP status code is not handled or not allowed 2026-01-24 21:23:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/85-1164-fiberglass-sleeving-5-16-inch>: HTTP status code is not handled or not allowed 2026-01-24 21:23:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tote-box-without-lid-1>: HTTP status code is not handled or not allowed 2026-01-24 21:23:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/53755-216-0-4-tube-brush-stiff-white>: HTTP status code is not handled or not allowed 2026-01-24 21:23:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-swing-top-receptacle-13-gallon-1311htwh-white>: HTTP status code is not handled or not allowed 2026-01-24 21:23:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-w-x-30-d-production-workbench-stainless-steel-square-edge-with-drawers-shelf-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:23:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots>: HTTP status code is not handled or not allowed 2026-01-24 21:23:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m6-x-1-0-x-40mm-machine-screw-pan-head-phillips-steel-zinc-pkg-of-120>: HTTP status code is not handled or not allowed 2026-01-24 21:23:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powered-12-w-x-40-l-belt-conveyor-with-6-h-side-rails>: HTTP status code is not handled or not allowed 2026-01-24 21:23:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-20-x-2-1-4-machine-screw-pan-head-phillips-18-8-stainless-steel-ft-unc-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:23:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gn844-heavy-duty-push-pull-toggle-clamp-70evu4-asd-size-70-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:23:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-anti-static-bags-18w-x-24l-4-mil-pink-pack-of-250>: HTTP status code is not handled or not allowed 2026-01-24 21:23:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/53792-216-2-0-tube-brush-medium-green>: HTTP status code is not handled or not allowed 2026-01-24 21:23:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/bollards/safety_bollards_posts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:23:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-pallet-truck-pallet-jack-6600-lb-capacity-21-x-36>: HTTP status code is not handled or not allowed 2026-01-24 21:23:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/synthetic-winch-line-with-stainless-steel-thimble-58-dia-x-125l>: HTTP status code is not handled or not allowed 2026-01-24 21:23:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cam-follower-ecc-collar-full-comp-crowned-od-hex-hole-dbl-sealed-22mm-od-12mm-w-m10-x-10-thr>: HTTP status code is not handled or not allowed 2026-01-24 21:23:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/synthetic-winch-line-extension-with-ss-thimble-kevlar-tail-58-dia-x-75l>: HTTP status code is not handled or not allowed 2026-01-24 21:23:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010180-caster-3-in-swivelw-brake>: HTTP status code is not handled or not allowed 2026-01-24 21:23:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-16-x-4-machine-screw-flat-head-phillips-steel-zinc-ft-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:23:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/52-bollard-flex-with-sign-post-asphalt-installation-red-cover-black-tapes>: HTTP status code is not handled or not allowed 2026-01-24 21:23:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/52-bollard-flex-with-sign-post-asphalt-installation-white-cover-yellow-tapes>: HTTP status code is not handled or not allowed 2026-01-24 21:23:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m6-x-1-0-x-30mm-machine-screw-pan-head-phillips-steel-zinc-pkg-of-150>: HTTP status code is not handled or not allowed 2026-01-24 21:23:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009845-trim-ring-med-lg>: HTTP status code is not handled or not allowed 2026-01-24 21:23:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-conturaseries-surface-mounted-soap-dispenser>: HTTP status code is not handled or not allowed 2026-01-24 21:23:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/56607-dustpan-orange>: HTTP status code is not handled or not allowed 2026-01-24 21:23:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/red-hopper-front-bulk-truck-48-cu>: HTTP status code is not handled or not allowed 2026-01-24 21:23:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/bollards/safety_bollards_posts landed on page that is not a product page. 2026-01-24 21:23:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/31042-10-upright-broom-soft-stiff-green>: HTTP status code is not handled or not allowed 2026-01-24 21:23:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/31997-24-small-particle-push-broom-soft-orange>: HTTP status code is not handled or not allowed 2026-01-24 21:23:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-x-18-x-72-phenolic-locker-double-tier-graphite-grafix>: HTTP status code is not handled or not allowed 2026-01-24 21:23:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/palram-feria-patio-cover-kit-hg9234-34l-x-13w-clear-panel-white-frame>: HTTP status code is not handled or not allowed 2026-01-24 21:23:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-polyester-filter-24-w-x-30-h-x-1-d>: HTTP status code is not handled or not allowed 2026-01-24 21:23:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-polyester-filter-20-w-x-36-h-x-1-d>: HTTP status code is not handled or not allowed 2026-01-24 21:23:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/overhead_clearance_bars_door_guards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:23:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-shelf-liner-72w-x-12d-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:24:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-x-12-x-72-phenolic-locker-single-tier-silver-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:24:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-w-x-30-d-production-workbench-plastic-laminate-safety-edge-complete-bench-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:24:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-polyester-filter-14-w-x-20-h-x-1-d>: HTTP status code is not handled or not allowed 2026-01-24 21:24:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leviton-4s115-75e-75-velcro-bulk-roll-soft-cinch-lite-color-black>: HTTP status code is not handled or not allowed 2026-01-24 21:24:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-w-so-nylon-gate-arm-cover-black-cover-white-tapes>: HTTP status code is not handled or not allowed 2026-01-24 21:24:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/wire_cable/electrical_wire_cable already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:24:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brother-tn221y-toner-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:24:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-2-sjoow-vu-tron-yellow-250ft>: HTTP status code is not handled or not allowed 2026-01-24 21:24:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/31944-24-combo-push-broom-soft-stiff-red>: HTTP status code is not handled or not allowed 2026-01-24 21:24:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/31942-24-combo-push-broom-soft-stiff-green>: HTTP status code is not handled or not allowed 2026-01-24 21:24:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/overhead_clearance_bars_door_guards>: HTTP status code is not handled or not allowed 2026-01-24 21:24:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22-1011-2-1-2-inch-oven-thermostat-dial-300-650>: HTTP status code is not handled or not allowed 2026-01-24 21:24:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 24464 pages (at 95 pages/min), scraped 13196 items (at 20 items/min) 2026-01-24 21:24:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-fiberglass-filter-12-w-x-12-h-x-1-d>: HTTP status code is not handled or not allowed 2026-01-24 21:24:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-w-x-36-d-production-workbench-phenolic-resin-safety-edge-complete-bench-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:24:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-wire-shelf-truck-48x24x83-1200-pound-capacity-1>: HTTP status code is not handled or not allowed 2026-01-24 21:24:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-rated-access-door-walls-ceilings-18-x-18>: HTTP status code is not handled or not allowed 2026-01-24 21:24:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-hand-truck-loop-handle-semi-pneumatic-wheels>: HTTP status code is not handled or not allowed 2026-01-24 21:24:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42373-dish-brush-w-scraper-medium-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:24:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/wire_cable/electrical_wire_cable>: HTTP status code is not handled or not allowed 2026-01-24 21:24:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dinex-round-soup-bowl-lid-1000-cs-white-dxhh87>: HTTP status code is not handled or not allowed 2026-01-24 21:24:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/french-vanilla-coffee-96-pods>: HTTP status code is not handled or not allowed 2026-01-24 21:24:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-mesh-partition-security-room-10x10x10-with-roof-4-sides>: HTTP status code is not handled or not allowed 2026-01-24 21:24:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/breakfast-blend-coffee-12-pods>: HTTP status code is not handled or not allowed 2026-01-24 21:24:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/29583-50-basic-aluminum-handle-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:24:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/-rough-terrain-caster-kit-for-apex-1200-2000-4000-6500-portable-evaporative-cooler>: HTTP status code is not handled or not allowed 2026-01-24 21:24:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009529-orifice-58>: HTTP status code is not handled or not allowed 2026-01-24 21:24:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dinex-flat-tray-14-x-18-12-cs-onyx-dx1089i03>: HTTP status code is not handled or not allowed 2026-01-24 21:24:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96-x-36-maple-square-edge-work-bench-fixed-height-1-3-4-top>: HTTP status code is not handled or not allowed 2026-01-24 21:24:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-x-36-shop-top-work-bench-fixed-height-1-1-2-top>: HTTP status code is not handled or not allowed 2026-01-24 21:24:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carson-optical-lt-60-metal-linentest-magnifier>: HTTP status code is not handled or not allowed 2026-01-24 21:24:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fiberwood-dolly-36-l-x-18-w-700-lb-cap-3-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:24:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/35872-medium-hand-brush-medium-green>: HTTP status code is not handled or not allowed 2026-01-24 21:24:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-mesh-security-cage-48x24x72>: HTTP status code is not handled or not allowed 2026-01-24 21:24:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-mesh-security-cage-60x24x72>: HTTP status code is not handled or not allowed 2026-01-24 21:24:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40124-nylon-scraper-with-threaded-handle-red>: HTTP status code is not handled or not allowed 2026-01-24 21:24:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyol-ester-refrigeration-oil-1-gallon-150-sus>: HTTP status code is not handled or not allowed 2026-01-24 21:24:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wireway-husky-tear-drop-pallet-rack-add-on-96x36x96>: HTTP status code is not handled or not allowed 2026-01-24 21:24:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forkli-able-shipping-container-with-lid-44-x-29-5>: HTTP status code is not handled or not allowed 2026-01-24 21:24:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/americana-climbing-wind-safety-helmet-type-2-4-point-mega-ratchet-suspension-pink>: HTTP status code is not handled or not allowed 2026-01-24 21:24:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/261549-male-branch-tee>: HTTP status code is not handled or not allowed 2026-01-24 21:24:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/indoor-outdoor-quartz-electric-infrared-heater-och-46-240ve-240v-2000w-brown>: HTTP status code is not handled or not allowed 2026-01-24 21:24:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-floor-standing-stainless-steel-wet-wipe-station-starter-kit>: HTTP status code is not handled or not allowed 2026-01-24 21:24:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/americana-climbing-peak-safety-helmet-type-1-4-point-mega-ratchet-suspension-red>: HTTP status code is not handled or not allowed 2026-01-24 21:24:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-utility-cart-2-shelves-48x24>: HTTP status code is not handled or not allowed 2026-01-24 21:24:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1021129-strainer-drain-twistlock>: HTTP status code is not handled or not allowed 2026-01-24 21:24:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/350-10-3-dual-ramp-wheelchair-scale-with-handrail-1000-lb-x-0-2-lb>: HTTP status code is not handled or not allowed 2026-01-24 21:24:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/851214-tray-parts-6x45x11>: HTTP status code is not handled or not allowed 2026-01-24 21:24:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industries-tube-2-x-050-x-4-satin-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:24:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industries-handrail-bracket-for-15-tubing-satin-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:24:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/americana-climbing-wind-safety-helmet-type-2-4-point-mega-ratchet-suspension-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:24:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/80-quart-heavy-duty-stock-pot>: HTTP status code is not handled or not allowed 2026-01-24 21:24:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polar-chest-dry-ice-container-with-lid-casters-pb11dxx-43-l-x-27-1-2-w-x-39-1-2-h>: HTTP status code is not handled or not allowed 2026-01-24 21:24:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industries-tube-1-x-050-x-4-satin-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:24:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/262029-orifice-42-for-bakers-pride>: HTTP status code is not handled or not allowed 2026-01-24 21:24:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-carton-storage-truck-single-level-400lb>: HTTP status code is not handled or not allowed 2026-01-24 21:24:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/74-h-nexel-chrome-wire-shelving-add-on-60-w-x-30-d>: HTTP status code is not handled or not allowed 2026-01-24 21:24:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010629-hinge-insert-for-cleveland-range>: HTTP status code is not handled or not allowed 2026-01-24 21:24:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industries-ball-elbow-for-15-tubing-polished-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:24:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-cup-oval-measuring-scoop>: HTTP status code is not handled or not allowed 2026-01-24 21:24:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/original-wrist-and-ankle-weight-20-piece-set>: HTTP status code is not handled or not allowed 2026-01-24 21:24:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-cube-ice-maker-approx-496-lb-production-half-size-cube>: HTTP status code is not handled or not allowed 2026-01-24 21:24:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-handle-medicine-ball-24-lb-9-dia-23-cm-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:24:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-handle-medicine-ball-12-lb-9-dia-23-cm-red>: HTTP status code is not handled or not allowed 2026-01-24 21:24:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/soft-pliable-medicine-ball-11-lb-7-diameter-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:24:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1051010-motor-stir-110v-msd-10-20-30>: HTTP status code is not handled or not allowed 2026-01-24 21:24:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/721686-brush-disposable-1-1-2-inch-wide>: HTTP status code is not handled or not allowed 2026-01-24 21:24:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/relaxed-fit-twill-safety-cargo-utility-pants-34l-inseam-x-38-waist-navy>: HTTP status code is not handled or not allowed 2026-01-24 21:24:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/knife-switch-lockout-plastic-red>: HTTP status code is not handled or not allowed 2026-01-24 21:24:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industries-flush-elbow-fitting-side-outlet-for-15-tubing-satin-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:24:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-weight-bar-20-lb-black>: HTTP status code is not handled or not allowed 2026-01-24 21:24:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelite-vented-plastic-mat-shelf-truck-72l-x-24w-x-60h-polyurethane-brake-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:24:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yellow-cutting-board-1>: HTTP status code is not handled or not allowed 2026-01-24 21:24:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelite-vented-plastic-mat-shelf-truck-36l-x-21w-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:25:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-end-dome-with-drop-cord-and-2-speed-for-642-motor-only>: HTTP status code is not handled or not allowed 2026-01-24 21:25:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-pan-3-false-bottom>: HTTP status code is not handled or not allowed 2026-01-24 21:25:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ripstop-safety-rain-jacket-csa-class-2-level-2-3xl-fluorescent-green>: HTTP status code is not handled or not allowed 2026-01-24 21:25:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8014212-knob-4-armw-stud-m5-08x15>: HTTP status code is not handled or not allowed 2026-01-24 21:25:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010607-gasket-burner-boxto-tube-for-cleveland-range>: HTTP status code is not handled or not allowed 2026-01-24 21:25:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/servewell-economy-breath-guard-2>: HTTP status code is not handled or not allowed 2026-01-24 21:25:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/360-stretch-waist-flex-twill-cargo-pants-32l-inseam-x-34-12-35-12-waist-navy>: HTTP status code is not handled or not allowed 2026-01-24 21:25:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insulated-flex-safety-bib-overall-m-fluorescent-green>: HTTP status code is not handled or not allowed 2026-01-24 21:25:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-quarter-dome-polycarbonate-mirrors>: HTTP status code is not handled or not allowed 2026-01-24 21:25:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pull-on-ripstop-safety-rain-pants-csa-class-3-level-2-s-fluorescent-orange>: HTTP status code is not handled or not allowed 2026-01-24 21:25:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 24559 pages (at 95 pages/min), scraped 13220 items (at 24 items/min) 2026-01-24 21:25:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60x24-cabinet-workbench-with-hinged-doors>: HTTP status code is not handled or not allowed 2026-01-24 21:25:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pop-blind-rivet-1-8-x-4-2-button-head-up-to-1-8-grip-stainless-steel-steel-pkg-of-500>: HTTP status code is not handled or not allowed 2026-01-24 21:25:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/file_folders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:25:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pop-monobolt-blind-rivet-1-4-dome-head-080-375-grip-range-steel-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:25:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kennedy-8-drawer-roller-bearing-brown-1>: HTTP status code is not handled or not allowed 2026-01-24 21:25:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2151265-arm-pusher-head-left-for-redco-slicers>: HTTP status code is not handled or not allowed 2026-01-24 21:25:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/54-h-nexel-chrome-wire-shelving-72-w-x-18-d>: HTTP status code is not handled or not allowed 2026-01-24 21:25:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/54-h-nexel-chrome-wire-shelving-48-w-x-24-d>: HTTP status code is not handled or not allowed 2026-01-24 21:25:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smead-file-folders-13-cut-reinforced-top-tab-letter-green-100box>: HTTP status code is not handled or not allowed 2026-01-24 21:25:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clamps-wilton-21804>: HTTP status code is not handled or not allowed 2026-01-24 21:25:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/350125-jmd-15-milling-drilling-machine-w-newall-dp700-dro-1-hp>: HTTP status code is not handled or not allowed 2026-01-24 21:25:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/54-h-nexel-chrome-wire-shelving-add-on-48-w-x-36-d>: HTTP status code is not handled or not allowed 2026-01-24 21:25:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transparent-vinyl-welding-curtain--steel-frame-combo-kit-6w-x-8h-green>: HTTP status code is not handled or not allowed 2026-01-24 21:25:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/file_folders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:25:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/63-h-nexel-chrome-wire-shelving-add-on-24-w-x-18-d>: HTTP status code is not handled or not allowed 2026-01-24 21:25:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clamps-wilton14128>: HTTP status code is not handled or not allowed 2026-01-24 21:25:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/file_folders>: HTTP status code is not handled or not allowed 2026-01-24 21:25:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-stp-steel-safe-roll-lad-wgt-act-lock-16-w-perf-stp-org-wa072414p-o>: HTTP status code is not handled or not allowed 2026-01-24 21:25:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-stp-steel-safe-roll-lad-wgt-act-lock-16-w-serr-stp-org-wa052414g-o>: HTTP status code is not handled or not allowed 2026-01-24 21:25:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30004-5a-z-4-no-hub-floor-drain-body-5-round-nikaloy-strainer>: HTTP status code is not handled or not allowed 2026-01-24 21:25:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-tech-heavy-duty-outrigger-pad-24-x-24-x-1-thick-pad24241>: HTTP status code is not handled or not allowed 2026-01-24 21:25:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mounted-coat-rack-3-double-prong-hook-rail-nickel-medium-oak>: HTTP status code is not handled or not allowed 2026-01-24 21:25:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6in-x-1-2in-natural-sea-sponge-mini-roller-cover-12-case-6fp-050>: HTTP status code is not handled or not allowed 2026-01-24 21:25:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/non-electric-office-partition-panel-with-raceway-24-1-4w-x-76h-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:25:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/file_folders>: HTTP status code is not handled or not allowed 2026-01-24 21:25:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-inch-nylon-hard-floor-brush-accessory-for-ex40-part-56265177>: HTTP status code is not handled or not allowed 2026-01-24 21:25:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1200-lb-composite-davit-crane-with-110-ac-electric-winch>: HTTP status code is not handled or not allowed 2026-01-24 21:25:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ctp-adjustable-trolley-clamp-beam-width-3-to-7875-4000-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:25:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fixed-head-pad-seal-fits-8-x-9-door-with-12-projection-12-header-drop-curtain>: HTTP status code is not handled or not allowed 2026-01-24 21:25:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/983101-1-se9129-rounds9x12-flats-110v-1-ph>: HTTP status code is not handled or not allowed 2026-01-24 21:25:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7001d-dura-polyurethane-chair-black-nylon-base-mushroom-glides-black>: HTTP status code is not handled or not allowed 2026-01-24 21:25:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/977100-1-tb-16-bench-drill-press-5-8-drill-capacity-110v-only>: HTTP status code is not handled or not allowed 2026-01-24 21:25:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/professional-detailerfts-kit-sw84000>: HTTP status code is not handled or not allowed 2026-01-24 21:25:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-wall-tray-for-precision-shaking-water-baths>: HTTP status code is not handled or not allowed 2026-01-24 21:25:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8ab-x-1-2-security-self-drilling-screw-pan-6-lobe-head-410-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:25:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/liquid-bag-filter-nylon-mesh-4-1-8-dia-x-14-l-250-micron-steel-ring-pkg-qty-50>: HTTP status code is not handled or not allowed 2026-01-24 21:25:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/welded-liquid-bag-filter-polypropylene-felt-7-1-8-d-x-32-l-50-micron-plastic-flange-pkg-50>: HTTP status code is not handled or not allowed 2026-01-24 21:25:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-28-x-1-security-cap-screw-button-socket-head-18-8-stainless-steel-pkg-of-50>: HTTP status code is not handled or not allowed 2026-01-24 21:25:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sewn-liquid-bag-filter-polyester-multifilament-7-1-8-x-16-1-2-150-micronplastic-flange-pkg-50>: HTTP status code is not handled or not allowed 2026-01-24 21:25:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-45-torx-tamperproof-screwdriver>: HTTP status code is not handled or not allowed 2026-01-24 21:25:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-20-x-1-1-4-one-way-machine-screw-round-head-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:25:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4ab-x-1-2-tri-wing-sheet-metal-screw-pan-head-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:25:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/liquid-bag-filter-poly-p-felt-7-3-50-dia-x-32-l-100-micron-plastic-sure-seal-ring-pkg-qty-50>: HTTP status code is not handled or not allowed 2026-01-24 21:25:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-32-balldriver-power-bit-10805>: HTTP status code is not handled or not allowed 2026-01-24 21:25:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/welded-liquid-bag-filter-polypropylene-felt-7-1-8-d-x-32-l-100-micron-plastic-flange-pkg-50>: HTTP status code is not handled or not allowed 2026-01-24 21:25:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-20-x-2-security-machine-screw-flat-phillips-head-18-8-stainless-steel-pkg-of-50>: HTTP status code is not handled or not allowed 2026-01-24 21:25:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-x-3-1-2-security-sheet-metal-screw-pan-phillips-head-18-8-stainless-steel-pkg-of-50>: HTTP status code is not handled or not allowed 2026-01-24 21:25:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/square-activity-table-48w-x-48l-x-24-31h-classic-oak>: HTTP status code is not handled or not allowed 2026-01-24 21:25:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5mm-hex-tamper-resistant-l-wrench-48364>: HTTP status code is not handled or not allowed 2026-01-24 21:25:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/locker-six-tier-12x18x12-6-door-ready-to-assemble-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:25:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/19-7-8-x-41-7-8-x-17-1-2-rack-bin-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:25:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-1-2-x-10-7-8-x-3-1-2-clear-view-dividable-grid-containers>: HTTP status code is not handled or not allowed 2026-01-24 21:25:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-24-x-1-2-security-machine-screw-button-torx-head-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:25:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/freestanding-office-partition-panel-with-partial-window-48-1-4-w-x-42-h-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:25:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-electric-office-partition-panel-with-partial-window-36-1-4w-x-47-1-2h-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:25:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-tamperproof-spanner-bit-5-16-hex-shank>: HTTP status code is not handled or not allowed 2026-01-24 21:25:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trax-one-gallon>: HTTP status code is not handled or not allowed 2026-01-24 21:25:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/k15fs-36h-snap-in-baso-thermocouple>: HTTP status code is not handled or not allowed 2026-01-24 21:25:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelving-24w-x-14d-x-14h-1-shelf-starter-chrome>: HTTP status code is not handled or not allowed 2026-01-24 21:25:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/hvac_system_parts_components/heating_parts_components/thermocouples_thermopiles already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:25:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelving-36w-x-18d-x-63h-4-shelf-add-on-chrome>: HTTP status code is not handled or not allowed 2026-01-24 21:25:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-npt-poly-muffler-75-micron-145-psi-5-to-212-f>: HTTP status code is not handled or not allowed 2026-01-24 21:26:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nilfisk-gd-10-exhaust-filter>: HTTP status code is not handled or not allowed 2026-01-24 21:26:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nilfisk-gm-80-hepa-vacuum>: HTTP status code is not handled or not allowed 2026-01-24 21:26:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8n25p05a-set-screw-w-full-ball-m8-x-125-thread-25mm-thread-length>: HTTP status code is not handled or not allowed 2026-01-24 21:26:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/check-valve-3-8-nptf-male-viton>: HTTP status code is not handled or not allowed 2026-01-24 21:26:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelving-30w-x-18d-x-34h-2-shelf-add-on-chrome>: HTTP status code is not handled or not allowed 2026-01-24 21:26:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/komet-stir-bar-50-round-21mm-dia-x-50mm-l-1-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:26:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-reclosable-pharmacy-bag-2-mil-2w-3l>: HTTP status code is not handled or not allowed 2026-01-24 21:26:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/i-micro-stirrer-with-telemodul-20-c-controller-100-240v>: HTTP status code is not handled or not allowed 2026-01-24 21:26:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-mild-foam-hand-wash-1200ml-refill-unscented-2-refills-case>: HTTP status code is not handled or not allowed 2026-01-24 21:26:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/hvac_system_parts_components/heating_parts_components/thermocouples_thermopiles>: HTTP status code is not handled or not allowed 2026-01-24 21:26:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 24653 pages (at 94 pages/min), scraped 13245 items (at 25 items/min) 2026-01-24 21:26:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/discreet-incontinence-liners-very-light-long-44-liners-pack-3-packs-case>: HTTP status code is not handled or not allowed 2026-01-24 21:26:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-workbenchriser-60-w-x-10-d-x-12-h>: HTTP status code is not handled or not allowed 2026-01-24 21:26:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pegboard-panels-powdercoat-silver-16-x-32-2-pc>: HTTP status code is not handled or not allowed 2026-01-24 21:26:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-48-w-x-48-d-x-84-h-starter-3-shelf-levels-steel-deck-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:26:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/86500-4800s-12c-12-heavy-duty-copper-f-clamp>: HTTP status code is not handled or not allowed 2026-01-24 21:26:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1794224k-4224b-lathe-lamp-kit>: HTTP status code is not handled or not allowed 2026-01-24 21:26:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-readybuilt-locker-36-w-x-18-d-x-78-h-707-marine-blue-triple-tier-3-wide-assembled>: HTTP status code is not handled or not allowed 2026-01-24 21:26:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/diamond-plate-platform-kit-4-x-18-x-54>: HTTP status code is not handled or not allowed 2026-01-24 21:26:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-48-w-x-48-d-x-120-h-starter-3-shelf-levels-steel-deck-medium-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:26:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-coil-thread-repair-kit-for-metal-m4-07-x-15d>: HTTP status code is not handled or not allowed 2026-01-24 21:26:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-value-max-locker-12-w-x-12-d-x-78-h-725-hallowell-gray-double-tier-1-wide-assembled>: HTTP status code is not handled or not allowed 2026-01-24 21:26:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-coil-thread-repair-kit-for-metal-m3-05-x-15d>: HTTP status code is not handled or not allowed 2026-01-24 21:26:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-48-w-x-24-d-x-96-h-add-on-3-shelf-levels-wood-deck-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:26:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/legacy-double-lateral-low-credenza-ash-grey>: HTTP status code is not handled or not allowed 2026-01-24 21:26:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-value-max-locker-12-w-x-12-d-x-78-h-725-hallowell-gray-triple-tier-1-wide-knock-down>: HTTP status code is not handled or not allowed 2026-01-24 21:26:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kahlo-36-square-multipurpose-breakroom-tapered-leg-table-maple-black>: HTTP status code is not handled or not allowed 2026-01-24 21:26:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wesco-4-wheel-appliance-truck-1200-lb-cap-66in-h-auto-rewind>: HTTP status code is not handled or not allowed 2026-01-24 21:26:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-72-w-x-24-d-x-96-h-starter-3-shelf-levels-wood-deck-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:26:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:26:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-series-wheel-5000084592-ws-8-x-2-performa-rubber-1-2-straight-roller-bearing>: HTTP status code is not handled or not allowed 2026-01-24 21:26:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-48-w-x-24-d-x-120-h-starter-3-shelf-levels-steel-deck-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:26:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-series-wheel-20000553-5-x-1-1-4-polyolefin-3-8-delrin-bushing-black>: HTTP status code is not handled or not allowed 2026-01-24 21:26:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-48-w-x-24-d-x-84-h-add-on-3-shelf-levels-wood-deck-sand>: HTTP status code is not handled or not allowed 2026-01-24 21:26:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-72-w-x-48-d-x-96-h-starter-3-shelf-levels-wood-deck-medium-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:26:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ga7vsd-10-hp-variable-speed-rotary-screw-compressor-tankless-175-psig-3-phase-200v>: HTTP status code is not handled or not allowed 2026-01-24 21:26:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/281672-cap-outside-round>: HTTP status code is not handled or not allowed 2026-01-24 21:26:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-72-w-x-24-d-x-72-h-add-on-3-shelf-levels-wood-deck-sand>: HTTP status code is not handled or not allowed 2026-01-24 21:26:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-96-w-x-48-d-x-72-h-add-on-3-shelf-levels-steel-deck-sand>: HTTP status code is not handled or not allowed 2026-01-24 21:26:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-48-w-x-48-d-x-84-h-starter-3-shelf-levels-steel-deck-sand>: HTTP status code is not handled or not allowed 2026-01-24 21:26:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/261538-twin-basket9-3-8l-4-7-8w-5-3-8d-for-middleby>: HTTP status code is not handled or not allowed 2026-01-24 21:26:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010156-nut-for-vent-for-henny-penny>: HTTP status code is not handled or not allowed 2026-01-24 21:26:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011140-door-lh-ofe322-for-henny-penny>: HTTP status code is not handled or not allowed 2026-01-24 21:26:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-shelf-level-for-bulk-storage-rack-48-w-x-36-d-wood-deck-sand>: HTTP status code is not handled or not allowed 2026-01-24 21:26:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks>: HTTP status code is not handled or not allowed 2026-01-24 21:26:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010497-knob-burner>: HTTP status code is not handled or not allowed 2026-01-24 21:26:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roval-automatic-soap-dispenser>: HTTP status code is not handled or not allowed 2026-01-24 21:26:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-72-w-x-24-d-x-120-h-starter-3-shelf-levels-steel-deck-sand>: HTTP status code is not handled or not allowed 2026-01-24 21:26:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-72-w-x-36-d-x-96-h-add-on-3-shelf-levels-steel-deck-sand>: HTTP status code is not handled or not allowed 2026-01-24 21:26:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-72-w-x-48-d-x-120-h-add-on-3-shelf-levels-wire-deck-medium-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:26:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010715-radiant-for-american-range>: HTTP status code is not handled or not allowed 2026-01-24 21:26:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hutch-w-doors-mocha-71wx15dx36h>: HTTP status code is not handled or not allowed 2026-01-24 21:26:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sign-two-acrylic-inserts-8-1-2-inch-by-11-inch>: HTTP status code is not handled or not allowed 2026-01-24 21:26:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pl-c-lamp-cfq26wg24d830-21w-3000k>: HTTP status code is not handled or not allowed 2026-01-24 21:26:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-chromatography-refrigerator-abt-cs-16-16-cu-ft-white>: HTTP status code is not handled or not allowed 2026-01-24 21:26:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8015432-pump-assembly-emptying-for-rational-cooking-systems>: HTTP status code is not handled or not allowed 2026-01-24 21:26:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22624024n-flexible-conveyor-poly-skate-wheels-nylon-bushings-226-lb-per-ft>: HTTP status code is not handled or not allowed 2026-01-24 21:26:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-navy-flag-3x5-nylon>: HTTP status code is not handled or not allowed 2026-01-24 21:26:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alera-aleva353624mc-valencia-series-reversible-returnbridge-shell-35w-x-23-58d-medium-cherry>: HTTP status code is not handled or not allowed 2026-01-24 21:26:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/imh100-advanced-protocol-microbiological-incubator-3-67-cu-ft-120v>: HTTP status code is not handled or not allowed 2026-01-24 21:26:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/461829-thermostat-cont>: HTTP status code is not handled or not allowed 2026-01-24 21:26:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2241106-screw-blade-assembly-ez-wedger-for-nemco>: HTTP status code is not handled or not allowed 2026-01-24 21:26:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1781010-label-inch-iced-tea-inch-for-wilbur-curtis-co>: HTTP status code is not handled or not allowed 2026-01-24 21:26:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-inch-x-50-foot-medium-duty-pvc-discharge-hose-coupled-with-cxe-aluminum-cam-and-groove-fittings>: HTTP status code is not handled or not allowed 2026-01-24 21:26:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/441641-contactor-240v-40a-for-alto-shaam>: HTTP status code is not handled or not allowed 2026-01-24 21:26:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-inch-x-100-foot-pvc-lay-flat-discharge-hose-coupled-with-cxe-aluminum-cam-and-groove-fittings>: HTTP status code is not handled or not allowed 2026-01-24 21:26:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/241219-top-grate-for-american-range>: HTTP status code is not handled or not allowed 2026-01-24 21:26:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stack-on-hutch-door-kit-72w>: HTTP status code is not handled or not allowed 2026-01-24 21:26:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011492-pump-motor-for-hoshizaki-of-america>: HTTP status code is not handled or not allowed 2026-01-24 21:26:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012113-grate-for-vulcan-hart>: HTTP status code is not handled or not allowed 2026-01-24 21:26:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-gallon-6-5-peak-hp-stainless-steel-wet-dry-vacuum-with-cart-1610rv>: HTTP status code is not handled or not allowed 2026-01-24 21:26:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/biodegradable-hand-cleaner-stand-up-4-5-lb-pouch-4-pack-juniper-berry>: HTTP status code is not handled or not allowed 2026-01-24 21:26:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1711198-brush-replacement-medium-for-prince-castle>: HTTP status code is not handled or not allowed 2026-01-24 21:26:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shoulder-pulley-exerciser-with-tubing-and-handles-green-medium-1-set>: HTTP status code is not handled or not allowed 2026-01-24 21:26:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/263198-machine-screw>: HTTP status code is not handled or not allowed 2026-01-24 21:26:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/original-exercise-station-overhead-section>: HTTP status code is not handled or not allowed 2026-01-24 21:26:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-powder-exercise-band-gold-50-yard-roll-1-roll-box>: HTTP status code is not handled or not allowed 2026-01-24 21:26:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/physicianscare-1232-first-aid-adhesive-tape-12-x-10-yards>: HTTP status code is not handled or not allowed 2026-01-24 21:27:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/curad-cur961r-instant-cold-pack-2box>: HTTP status code is not handled or not allowed 2026-01-24 21:27:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2171134-pin-hinge-for-server-products>: HTTP status code is not handled or not allowed 2026-01-24 21:27:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-40-14w-drawer-cabinet-4-drawer-24-compart-bright-blue-no-lock>: HTTP status code is not handled or not allowed 2026-01-24 21:27:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1591118-brush-hand-black-w-scraper>: HTTP status code is not handled or not allowed 2026-01-24 21:27:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1971096-coupler-6-x-6-rj12-for-duke-manufacturing>: HTTP status code is not handled or not allowed 2026-01-24 21:27:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012210-adapter-1-4-p-2021-4-4s-for-cleveland-range>: HTTP status code is not handled or not allowed 2026-01-24 21:27:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fastener-large-5-1-2-10-each>: HTTP status code is not handled or not allowed 2026-01-24 21:27:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011968-door-gasket-for-rational-cooking-systems>: HTTP status code is not handled or not allowed 2026-01-24 21:27:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uv-black-poly-cable-ties-502260-100-8-100-pcs>: HTTP status code is not handled or not allowed 2026-01-24 21:27:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uv-black-standard-cable-ties-502210-100-75-100-pcs>: HTTP status code is not handled or not allowed 2026-01-24 21:27:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 24750 pages (at 97 pages/min), scraped 13266 items (at 21 items/min) 2026-01-24 21:27:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-tubing-with-handles-black-36>: HTTP status code is not handled or not allowed 2026-01-24 21:27:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unisex-ciel-blue-cotton-polyester-fashion-blend-non-reversible-scrub-pants-size-xs>: HTTP status code is not handled or not allowed 2026-01-24 21:27:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unisex-jade-cotton-polyester-fashion-blend-reversible-set-in-sleeve-scrub-shirt-size-xs>: HTTP status code is not handled or not allowed 2026-01-24 21:27:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/right-elbow-terminal-protector-5779-025r-red-25-pcs>: HTTP status code is not handled or not allowed 2026-01-24 21:27:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/wiring_devices/wire_clips_clamps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:27:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unisex-navy-cotton-polyester-fashion-blend-reversible-scrub-pants-size-m>: HTTP status code is not handled or not allowed 2026-01-24 21:27:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-tubing-with-handles-blue-18>: HTTP status code is not handled or not allowed 2026-01-24 21:27:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90-bent-magnalugs-6408-005fl-3-8-5-pcs>: HTTP status code is not handled or not allowed 2026-01-24 21:27:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/base-for-large-slide-racks-12d>: HTTP status code is not handled or not allowed 2026-01-24 21:27:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inplant-office-cat-5-phone-data-jack-plastic-white>: HTTP status code is not handled or not allowed 2026-01-24 21:27:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-nylon-split-loom-505110-100-2-id-100-ft>: HTTP status code is not handled or not allowed 2026-01-24 21:27:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/263785-leveler-cushion-glide>: HTTP status code is not handled or not allowed 2026-01-24 21:27:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metal-heavy-duty-modular-drawer-cabinet-8-drawer-counter-height-36-w-avalanche-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:27:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-band-blue-6-yard-roll>: HTTP status code is not handled or not allowed 2026-01-24 21:27:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/281613-cap-for-263742>: HTTP status code is not handled or not allowed 2026-01-24 21:27:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/wiring_devices/wire_clips_clamps>: HTTP status code is not handled or not allowed 2026-01-24 21:27:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electronic-tape-dispenser-for-1-2-3-water-activated-kraft-tape-with-free-case-of-tape>: HTTP status code is not handled or not allowed 2026-01-24 21:27:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ironwood-2-compartment-wardrobe-storage-cabinet-solid-door-cactus-star-color>: HTTP status code is not handled or not allowed 2026-01-24 21:27:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uv-black-heavy-duty-cable-ties-502222-100-1425-100-pcs>: HTTP status code is not handled or not allowed 2026-01-24 21:27:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ironwood-3-compartment-solid-door-wood-storage-cabinet-folkstone-color>: HTTP status code is not handled or not allowed 2026-01-24 21:27:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/cable_management/cable_ties_wraps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:27:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/281643-cap-inside-square>: HTTP status code is not handled or not allowed 2026-01-24 21:27:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1051015-tray-drip-m-msd-10-20-30>: HTTP status code is not handled or not allowed 2026-01-24 21:27:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/31666-12-upright-broom-medium-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:27:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1621171-screw-cap-burner-for-montague-oven>: HTTP status code is not handled or not allowed 2026-01-24 21:27:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1881204-disc-spray-raised-ctr-12-hole-for-bloomfield>: HTTP status code is not handled or not allowed 2026-01-24 21:27:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leviton-23351-medium-bi-pin-standard-fluorescent-lampholder-white>: HTTP status code is not handled or not allowed 2026-01-24 21:27:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leviton-167-medium-base-two-piece-keyless-incandescent-phenolic-lampholder>: HTTP status code is not handled or not allowed 2026-01-24 21:27:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/77514-10-foam-blade-squeegee-red>: HTTP status code is not handled or not allowed 2026-01-24 21:27:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1681305-relay-pump-control-for-frymaster>: HTTP status code is not handled or not allowed 2026-01-24 21:27:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/441129-contactor-4p-30-40a-24v-for-groen>: HTTP status code is not handled or not allowed 2026-01-24 21:27:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/econoco-adjustable-2-way-ballet-rack-psbbadj-41w-x-44-72h-x-22d-anthracite-grey>: HTTP status code is not handled or not allowed 2026-01-24 21:27:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-exercise-band-x-light-yellow-10-each>: HTTP status code is not handled or not allowed 2026-01-24 21:27:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/cable_management/cable_ties_wraps>: HTTP status code is not handled or not allowed 2026-01-24 21:27:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-free-exercise-band-green-6-yard-roll-box>: HTTP status code is not handled or not allowed 2026-01-24 21:27:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-exercise-band-black-6-yard-roll-box>: HTTP status code is not handled or not allowed 2026-01-24 21:27:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/71254-10-single-blade-ultra-hygiene-bench-squeegee-red>: HTTP status code is not handled or not allowed 2026-01-24 21:27:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tx4000-esd-cleanroom-apparel-soft-sole-boot-cover-s-white>: HTTP status code is not handled or not allowed 2026-01-24 21:27:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bungee-exercise-cord-with-attachments-4-cord-black>: HTTP status code is not handled or not allowed 2026-01-24 21:27:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s-3015-anystream-multi-function-shower-head>: HTTP status code is not handled or not allowed 2026-01-24 21:27:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-band-loop-30-long-silver>: HTTP status code is not handled or not allowed 2026-01-24 21:27:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/huot-tooltower-hsk-63a-cnc-holders>: HTTP status code is not handled or not allowed 2026-01-24 21:27:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metmax-acifort-full-safety-boots-w-metatarsal--ankle-protection-15h-size-12-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:27:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1911013-stud-center-plate-berkel>: HTTP status code is not handled or not allowed 2026-01-24 21:27:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-single-toilet-tissue-dispenser-non-controlled-delivery>: HTTP status code is not handled or not allowed 2026-01-24 21:27:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fieldpro-purofort-full-safety-boots-steel-toe-14h-size-7-black>: HTTP status code is not handled or not allowed 2026-01-24 21:27:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6400md5-32-oz-metal-detectable-hand-scoop-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:27:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-classicseries-vertical-paper-towel-dispenser-w-knob-latch>: HTTP status code is not handled or not allowed 2026-01-24 21:27:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-free-exercise-band-with-dispens-a-bandand-rack-50-yard-box-5-color-set>: HTTP status code is not handled or not allowed 2026-01-24 21:27:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/folding-platform-truck-with-telescopic-aluminum-deck-4-rubber-wheels>: HTTP status code is not handled or not allowed 2026-01-24 21:27:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/barnstead-2-3-size-mixed-bed-and-organic-removal-cartridge-d8951-1-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:27:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012107-relay-24vac>: HTTP status code is not handled or not allowed 2026-01-24 21:27:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/71704-28-single-blade-ultra-hygiene-squeegee-red>: HTTP status code is not handled or not allowed 2026-01-24 21:27:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/barnstead-ultrapure-cartridge-d0809-full-size-1-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:27:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1111270-insert-cold-monel-leadfree-for-ts-brass-bronze-works>: HTTP status code is not handled or not allowed 2026-01-24 21:27:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-surface-mounted-roll-towel-dispenser>: HTTP status code is not handled or not allowed 2026-01-24 21:27:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-shelf-liner-18w-x-12d-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:27:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ep20h120v3-20hp-two-stage-piston-compressor-120-gal-vertical-175-psi-3-phase-208-230460v>: HTTP status code is not handled or not allowed 2026-01-24 21:27:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/263835-cap-end-round-tubing>: HTTP status code is not handled or not allowed 2026-01-24 21:27:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/281546-cap-outside-round>: HTTP status code is not handled or not allowed 2026-01-24 21:27:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1301087-dial-infinite-control-0-3-for-low-temp-manufacturing>: HTTP status code is not handled or not allowed 2026-01-24 21:27:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/luxor-174-ec111hd-e-series-black-3-shelf-tub-cart-35-1-4-x-18-500-lb-cap>: HTTP status code is not handled or not allowed 2026-01-24 21:27:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lk1000cr-everyday-kitchen-faucet-1-hole-drilling>: HTTP status code is not handled or not allowed 2026-01-24 21:27:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/32-1210-packing-seal-kit>: HTTP status code is not handled or not allowed 2026-01-24 21:27:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/52-bollard-flex-asphalt-installation-white-cover-red-tapes>: HTTP status code is not handled or not allowed 2026-01-24 21:27:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-matrixseries-surface-mounted-multi-roll-tissue-dispenser>: HTTP status code is not handled or not allowed 2026-01-24 21:28:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5001010-relay-starter-36a-for-manitowoc-machines>: HTTP status code is not handled or not allowed 2026-01-24 21:28:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/221077-lever-handle-cold-for-ts-brass-bronze-works>: HTTP status code is not handled or not allowed 2026-01-24 21:28:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/welding/welding_soldering_supplies/welding_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:28:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/panel-sheet-mover-truck-with-plywood-steel-deck-1200-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:28:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-1-1-2-dia-straight-peened-grab-bar-42-w>: HTTP status code is not handled or not allowed 2026-01-24 21:28:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/red-hopper-front-security-bulk-truck-58-cu>: HTTP status code is not handled or not allowed 2026-01-24 21:28:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cubbie-storage-organizer-12-sections-black>: HTTP status code is not handled or not allowed 2026-01-24 21:28:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/264036-kason-10060007036-pilaster-standard-36-inch-for-kason>: HTTP status code is not handled or not allowed 2026-01-24 21:28:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelate-wire-shelf-truck-72x24x69-1200-pound-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:28:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-x-18-x-72-phenolic-locker-double-tier-graphite-grafix>: HTTP status code is not handled or not allowed 2026-01-24 21:28:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medium-duty-low-lint-wipes-12-x-12-100-bag-m-pr911>: HTTP status code is not handled or not allowed 2026-01-24 21:28:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 24845 pages (at 95 pages/min), scraped 13287 items (at 21 items/min) 2026-01-24 21:28:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/justrite-drum-lock-set-plastic-drums>: HTTP status code is not handled or not allowed 2026-01-24 21:28:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-polyester-filter-16-w-x-24-h-x-1-d>: HTTP status code is not handled or not allowed 2026-01-24 21:28:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/line-voltage-thermostat-single-pole-heat-only-aet9swts>: HTTP status code is not handled or not allowed 2026-01-24 21:28:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-polyester-filter-14-w-x-25-h-x-1-d>: HTTP status code is not handled or not allowed 2026-01-24 21:28:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/werner-174-4ft-fiberglass-tripod-step-ladder>: HTTP status code is not handled or not allowed 2026-01-24 21:28:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/welding/welding_soldering_supplies/welding_accessories landed on page that is not a product page. 2026-01-24 21:28:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-w-x-36-d-production-workbench-maple-butcher-block-square-edge-complete-bench-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:28:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-wire-shelf-truck-72x18x69-1200-pound-capacity-1>: HTTP status code is not handled or not allowed 2026-01-24 21:28:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-wire-shelf-truck-36x24x69-1200-pound-capacity-with-brakes-1>: HTTP status code is not handled or not allowed 2026-01-24 21:28:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-w-x-30-d-production-workbench-phenolic-resin-safety-edge-complete-bench-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:28:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/261228-orifice-67>: HTTP status code is not handled or not allowed 2026-01-24 21:28:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-awg-mtw-black-500-ft>: HTTP status code is not handled or not allowed 2026-01-24 21:28:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/innovera-compatible-remanufactured-ch562wn-61-ink-165-page-yield-tri-color>: HTTP status code is not handled or not allowed 2026-01-24 21:28:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conductive-51in-l-handle-black-c29903>: HTTP status code is not handled or not allowed 2026-01-24 21:28:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-utility-tray-dark-blue-26-l-x-12-1-2-w-x-4-1-2-h>: HTTP status code is not handled or not allowed 2026-01-24 21:28:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anti-static-broom-base-only-19-1-2in-black-c26155>: HTTP status code is not handled or not allowed 2026-01-24 21:28:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cortech-usa-cc116-4-garbage-laundry-cart-black-repro-pkg-qty-4>: HTTP status code is not handled or not allowed 2026-01-24 21:28:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cortech-usa-vm1824-mirror-18-x-24-ventura-pkg-qty-1>: HTTP status code is not handled or not allowed 2026-01-24 21:28:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durapan-light-gauge-two-thirds-size-pan-13-7-8-x-12-3-4-607232>: HTTP status code is not handled or not allowed 2026-01-24 21:28:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-rated-access-door-walls-ceilings-16-x-16>: HTTP status code is not handled or not allowed 2026-01-24 21:28:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/29109-stainless-steel-floor-scraper-black>: HTTP status code is not handled or not allowed 2026-01-24 21:28:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2051270-washer-worm-wheel-shaft-for-hobart>: HTTP status code is not handled or not allowed 2026-01-24 21:28:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dinex-55-china-bread-plate-55-bright-white-dx5acbp02a>: HTTP status code is not handled or not allowed 2026-01-24 21:28:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/29756-62-113-telescopic-aluminum-handle-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:28:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fir-plywood-premium-grade-crate-premium57-28-23-collapsible-inside-57l-x-28w-x-23>: HTTP status code is not handled or not allowed 2026-01-24 21:28:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/-replacement-media-set-for-apex153-4000-60w-x-8d-x-49-12h>: HTTP status code is not handled or not allowed 2026-01-24 21:28:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-access-door-12-x-12>: HTTP status code is not handled or not allowed 2026-01-24 21:28:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-mesh-partition-security-room-20x20x8-with-roof-4-sides>: HTTP status code is not handled or not allowed 2026-01-24 21:28:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dinex-fenwick-insulated-bowl-5-oz-48-cs-sage-dx520084>: HTTP status code is not handled or not allowed 2026-01-24 21:28:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/29582-50-basic-aluminum-handle-green>: HTTP status code is not handled or not allowed 2026-01-24 21:28:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1151038-adaptor-spout-riser-chicago-for-chicago-faucets>: HTTP status code is not handled or not allowed 2026-01-24 21:28:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-x-30-shop-top-work-bench-fixed-height-1-1-2-top>: HTTP status code is not handled or not allowed 2026-01-24 21:28:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5380-103-3-216-4-0-pipe-brush-medium-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:28:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dinex-fenwick-tumbler-9-oz-72-cs-clear-dxft907>: HTTP status code is not handled or not allowed 2026-01-24 21:28:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bott-double-straight-hooks-for-perfo-panels-package-of-5-1-inch-l>: HTTP status code is not handled or not allowed 2026-01-24 21:28:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6231014-screw-phlp-10-24thd-stainless-steel-25pk>: HTTP status code is not handled or not allowed 2026-01-24 21:28:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/41955-narrow-utility-brush-extra-stiff-white>: HTTP status code is not handled or not allowed 2026-01-24 21:28:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bolted-teardrop-pallet-rack-add-on-48inw-x-42ind-x-120inh-6590-lbs-shelf-cap>: HTTP status code is not handled or not allowed 2026-01-24 21:28:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40123-nylon-scraper-with-threaded-handle-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:28:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coil-king-1-gallon>: HTTP status code is not handled or not allowed 2026-01-24 21:28:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medium-cube-ice-machine-58lb>: HTTP status code is not handled or not allowed 2026-01-24 21:28:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2171190-knob-caramel-for-server-products>: HTTP status code is not handled or not allowed 2026-01-24 21:28:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-weight-bar-16-piece-set>: HTTP status code is not handled or not allowed 2026-01-24 21:28:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dinex-turnbury-insulated-dome-10dia-12-cs-latte-dx340031>: HTTP status code is not handled or not allowed 2026-01-24 21:28:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dinex-insulated-base-for-insulated-domes-9-1-2-d-12-cs-onyx-dx107703>: HTTP status code is not handled or not allowed 2026-01-24 21:28:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/diamond-vi-wind-front-brim-safety-helmet-adjustable-one-d-rotor-clamping-system-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:28:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009519-knob-thermostat-fryer-f-c-for-magikitchen-products>: HTTP status code is not handled or not allowed 2026-01-24 21:28:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2901402-gch8040-80v-pro-series-rapid-battery-charger>: HTTP status code is not handled or not allowed 2026-01-24 21:28:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conntek-20241-025-25-sjtw-14-3-outdoor-extension-cord-with-nema-5-15p-r>: HTTP status code is not handled or not allowed 2026-01-24 21:28:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-ss-nsf-wall-shelf-32>: HTTP status code is not handled or not allowed 2026-01-24 21:28:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-sectional-stackbin-unit-37-w-x-24-d-x-11-h-black-2-compartments>: HTTP status code is not handled or not allowed 2026-01-24 21:28:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/indoor-outdoor-quartz-electric-infrared-heater-och-57-240ve-240v-3000w-brown>: HTTP status code is not handled or not allowed 2026-01-24 21:28:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ccc10012-socket-set-screw>: HTTP status code is not handled or not allowed 2026-01-24 21:28:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stretch-unlined-bib-overall-l-black>: HTTP status code is not handled or not allowed 2026-01-24 21:28:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011034-thermostat-knob>: HTTP status code is not handled or not allowed 2026-01-24 21:28:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/45-compartment-sectional-unit-37-w-x-13-d-x-54-h-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:28:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009857-loose-parts-hardware-kit>: HTTP status code is not handled or not allowed 2026-01-24 21:28:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-5-quart-hd-sauce-pan-pl-handle>: HTTP status code is not handled or not allowed 2026-01-24 21:28:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a215-benchtop-ph-conductivity-meter-kit-stara2155>: HTTP status code is not handled or not allowed 2026-01-24 21:28:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/water-filter-replacement-cartridge-for-if1-if2-if3-and-if4-older-systems>: HTTP status code is not handled or not allowed 2026-01-24 21:28:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-wrist-and-ankle-weight-0-5-lb-walnut>: HTTP status code is not handled or not allowed 2026-01-24 21:28:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/long-handle-measuring-spoon>: HTTP status code is not handled or not allowed 2026-01-24 21:28:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industries-flush-tee-fitting-side-outlet-for-15-tubing-polished-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:28:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ladle-4-oz-1>: HTTP status code is not handled or not allowed 2026-01-24 21:28:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industries-5-foot-rail-kit-2-tube-satin-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:29:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industries-adjustable-saddle-ball-for-2-tubing-polished-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:29:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-cube-ice-maker-remote-cooled-approx-982-lb-production-full-size-cube>: HTTP status code is not handled or not allowed 2026-01-24 21:29:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011143-door-gasket-1675-inch-x205-inch-d-d-krp55>: HTTP status code is not handled or not allowed 2026-01-24 21:29:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cnc06010-machine-screw>: HTTP status code is not handled or not allowed 2026-01-24 21:29:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/soft-pliable-medicine-ball-4-lb-5-diameter-red>: HTTP status code is not handled or not allowed 2026-01-24 21:29:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/soft-pliable-medicine-ball-2-lb-5-diameter-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:29:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bin-nestable-6-5-8-w-x-11-5-8-d-x-4-h-beige>: HTTP status code is not handled or not allowed 2026-01-24 21:29:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industries-flush-center-post-for-15-tubing-polished-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:29:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poly-oxford-insulated-pull-on-safety-pants-l-black>: HTTP status code is not handled or not allowed 2026-01-24 21:29:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/511029-hot-stem-assembly-for-chicago-faucets>: HTTP status code is not handled or not allowed 2026-01-24 21:29:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fine-mesh-china-cap>: HTTP status code is not handled or not allowed 2026-01-24 21:29:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelite-vented-plastic-mat-shelf-truck-48l-x-21w-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:29:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 24944 pages (at 99 pages/min), scraped 13309 items (at 22 items/min) 2026-01-24 21:29:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cbb16-nylon-insert-lock-nut>: HTTP status code is not handled or not allowed 2026-01-24 21:29:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wooden-5-level-incline-board-14-x-18-surface>: HTTP status code is not handled or not allowed 2026-01-24 21:29:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ladle-12-oz-2>: HTTP status code is not handled or not allowed 2026-01-24 21:29:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelite-vented-plastic-mat-shelf-truck-30l-x-24w-x-80h-polyurethane-brake-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:29:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelite-vented-plastic-mat-shelf-truck-48l-x-24w-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:29:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cjc08020-machine-screw>: HTTP status code is not handled or not allowed 2026-01-24 21:29:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/561549-wall-mount-faucet-economy-w-14-inch-spout>: HTTP status code is not handled or not allowed 2026-01-24 21:29:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/relaxed-fit-flex-twill-cargo-pants-32l-inseam-x-46-waist-black>: HTTP status code is not handled or not allowed 2026-01-24 21:29:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-x-4-14-stud-wedge-anchor-304-stainless-steel-pkg-of-25>: HTTP status code is not handled or not allowed 2026-01-24 21:29:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/maple-hardwood-locker-bench-9-free-standing>: HTTP status code is not handled or not allowed 2026-01-24 21:29:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industries-glass-clip-for-1-2-glass-flat-polished-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:29:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industries-ball-elbow-for-2-tubing-polished-brass>: HTTP status code is not handled or not allowed 2026-01-24 21:29:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-variable-length-plastic-9-seal-black-100-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:29:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1711153-ring-egg-8-inch-dia-omelet-for-prince-castle>: HTTP status code is not handled or not allowed 2026-01-24 21:29:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-adjustable-one-piece-seal-yellow-100-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:29:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1518-8-l-guard-rail>: HTTP status code is not handled or not allowed 2026-01-24 21:29:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-file-folders-13-cut-first-position-one-ply-top-tab-letter-manila-100box>: HTTP status code is not handled or not allowed 2026-01-24 21:29:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pop-blind-rivet-1-4-x-8-4-button-head-up-to-1-4-grip-steel-steel-pkg-of-250>: HTTP status code is not handled or not allowed 2026-01-24 21:29:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-wire-hasp-one-piece-padlock-seal-black-1000-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:29:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/33lb-bench-and-floor-scale-cpw-plus-15p>: HTTP status code is not handled or not allowed 2026-01-24 21:29:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kennedy-3-drawer-roller-bearing-brown>: HTTP status code is not handled or not allowed 2026-01-24 21:29:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kennedy-5-drawer-mechanic-bearing-brown-2>: HTTP status code is not handled or not allowed 2026-01-24 21:29:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pop-blind-rivet-1-8-x-4-4-flange-head-up-to-1-4-grip-aluminum-steel-pkg-of-500>: HTTP status code is not handled or not allowed 2026-01-24 21:29:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pop-blind-rivet-3-32-x-3-4-button-head-up-to-1-4-grip-aluminum-aluminum-pkg-of-500>: HTTP status code is not handled or not allowed 2026-01-24 21:29:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pop-blind-rivet-1-8-x-4-8-button-head-up-to-1-2-grip-aluminum-aluminum-pkg-of-500>: HTTP status code is not handled or not allowed 2026-01-24 21:29:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/54-h-nexel-chrome-wire-shelving-42-w-x-14-d>: HTTP status code is not handled or not allowed 2026-01-24 21:29:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/54-h-nexel-chrome-wire-shelving-48-w-x-14-d>: HTTP status code is not handled or not allowed 2026-01-24 21:29:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/complete-computer-workstation-desk-with-storage-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:29:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smead-pressboard-classification-folders-letter-six-section-yellow-10box>: HTTP status code is not handled or not allowed 2026-01-24 21:29:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/54-h-nexel-chrome-wire-shelving-48-w-x-18-d>: HTTP status code is not handled or not allowed 2026-01-24 21:29:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/321571-gh-1880zx-large-spindle-bore-lathe-w-newall-dp700-dro-taper-attachment-7-1-2-hp>: HTTP status code is not handled or not allowed 2026-01-24 21:29:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hubbel-commercial-14-4-30>: HTTP status code is not handled or not allowed 2026-01-24 21:29:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clamps-wilton14301>: HTTP status code is not handled or not allowed 2026-01-24 21:29:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-hand-knob-w-ss-screw-1-4-20-thread-9-16-inch-thread-length-1-inch-knob-dia>: HTTP status code is not handled or not allowed 2026-01-24 21:29:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hubbel-industrial-duplex-outlet-70>: HTTP status code is not handled or not allowed 2026-01-24 21:29:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sinking-reed-switch-1-amp-tie-rod-3m-cbl>: HTTP status code is not handled or not allowed 2026-01-24 21:29:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-1894-door-strike-with-3-4-inch-to-1-5-8-inch-adjustable-offset>: HTTP status code is not handled or not allowed 2026-01-24 21:29:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hubbel-industrial-single-outlet-16-3-25>: HTTP status code is not handled or not allowed 2026-01-24 21:29:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-insulated-quilted-jacket-l-olive>: HTTP status code is not handled or not allowed 2026-01-24 21:29:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:29:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/63-h-nexel-chrome-wire-shelving-add-on-42-w-x-18-d>: HTTP status code is not handled or not allowed 2026-01-24 21:29:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-amp-24v-to-12v-dc-dc-converter>: HTTP status code is not handled or not allowed 2026-01-24 21:29:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clamps-wilton14375>: HTTP status code is not handled or not allowed 2026-01-24 21:29:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-professional-trauma-bag-red-10-104-red>: HTTP status code is not handled or not allowed 2026-01-24 21:29:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smead-3-12-accordion-expansion-drop-front-pocket-legal-manilaredrope-25box>: HTTP status code is not handled or not allowed 2026-01-24 21:29:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-63mm-bore-x-360mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 21:29:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/infant-bag-valve-mask-10-508>: HTTP status code is not handled or not allowed 2026-01-24 21:29:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/accessories-forneedle-scalers-replacement-parts-n307>: HTTP status code is not handled or not allowed 2026-01-24 21:29:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vises-wilton11104>: HTTP status code is not handled or not allowed 2026-01-24 21:29:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smead-3-12-accordion-expansion-file-tuff-pockets-straight-legal-redrope-10box>: HTTP status code is not handled or not allowed 2026-01-24 21:29:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-100mm-bore-x-850mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 21:29:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smead-tuff-hanging-folder-with-easy-slide-tab-letter-standard-green-20pack>: HTTP status code is not handled or not allowed 2026-01-24 21:29:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats>: HTTP status code is not handled or not allowed 2026-01-24 21:29:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-poly-oxford-3-in-1-waterproof-parka-jacket-3xl-black>: HTTP status code is not handled or not allowed 2026-01-24 21:29:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-wire-shelving-60-w-x-18-d-x-63-h>: HTTP status code is not handled or not allowed 2026-01-24 21:29:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-1-box-bottom-pressboard-hanging-folder-legal-standard-green-25box>: HTTP status code is not handled or not allowed 2026-01-24 21:29:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clamps-columbian-41409>: HTTP status code is not handled or not allowed 2026-01-24 21:29:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-wire-shelving-add-on-72-w-x-24-d-x-63-h>: HTTP status code is not handled or not allowed 2026-01-24 21:29:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1031100-screen-filter-12-inch-sq-ofst-hole-for-ultrafryer>: HTTP status code is not handled or not allowed 2026-01-24 21:29:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clamps-wilton-42510>: HTTP status code is not handled or not allowed 2026-01-24 21:29:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milwaukee-174-48-59-2819-28-volt-charger-m28-8482-v28-8482-batteries>: HTTP status code is not handled or not allowed 2026-01-24 21:29:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/type-r-class-2-heavy-duty-surveyor-safety-vest-zipper-xl-green-sv59z-2zgd-xl>: HTTP status code is not handled or not allowed 2026-01-24 21:30:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ankle-cuff-with-gravity-feed-cooler>: HTTP status code is not handled or not allowed 2026-01-24 21:30:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rwg560xs-axis-cut-resistant-polyurethane-palm-gloves-gray-black-xs-1-pair>: HTTP status code is not handled or not allowed 2026-01-24 21:30:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-awg-2c-105-strand-speaker-500-ft-box-white>: HTTP status code is not handled or not allowed 2026-01-24 21:30:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-poly-oxford-5-in-1-safety-jacket-5xl-fluorescent-orange>: HTTP status code is not handled or not allowed 2026-01-24 21:30:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chicago-pneumatic-heavy-duty-air-hammer-cp7111-6-7l-3000-bpm-7-2-joule-B1222332>: HTTP status code is not handled or not allowed 2026-01-24 21:30:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16awg-2c-solid-fire-alarm-cable-fplr-1000-ft-spool-red>: HTTP status code is not handled or not allowed 2026-01-24 21:30:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/type-r-class-2-two-tone-surveyor-safety-vest-xl-green-sv6g2xl>: HTTP status code is not handled or not allowed 2026-01-24 21:30:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/moist-heat-pack-cover-foam-filled-terry-standard-with-pocket>: HTTP status code is not handled or not allowed 2026-01-24 21:30:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-125mm-bore-x-50mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 21:30:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-ripstop-4-in-1-safety-jacket-4xl-black>: HTTP status code is not handled or not allowed 2026-01-24 21:30:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011874-door-gasket>: HTTP status code is not handled or not allowed 2026-01-24 21:30:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-100mm-bore-x-450mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 21:30:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5181012-seal-kit-for-ps-200metcraft-for-powersoak>: HTTP status code is not handled or not allowed 2026-01-24 21:30:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/base-bar-mounting-kit-2-bore-nfpa-cylinder>: HTTP status code is not handled or not allowed 2026-01-24 21:30:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 25040 pages (at 96 pages/min), scraped 13330 items (at 21 items/min) 2026-01-24 21:30:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-chrome-wire-shelf-cart-48x24-3-shelves-800-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:30:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bib-apron-28x27-three-pockets-adjustable-neckband-black>: HTTP status code is not handled or not allowed 2026-01-24 21:30:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-poly-oxford-5-in-1-safety-jacket-2xlt-fluorescent-green>: HTTP status code is not handled or not allowed 2026-01-24 21:30:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compress-neck-7-x-22-12-case>: HTTP status code is not handled or not allowed 2026-01-24 21:30:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-level-tapered-riser-carpet-60l-x-18w-8h-16h-24h-32h-grey>: HTTP status code is not handled or not allowed 2026-01-24 21:30:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-50mm-bore-x-820m-stroke>: HTTP status code is not handled or not allowed 2026-01-24 21:30:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-poly-oxford-5-in-1-safety-jacket-xl-fluorescent-green>: HTTP status code is not handled or not allowed 2026-01-24 21:30:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-ripstop-4-in-1-safety-jacket-xl-fluorescent-orange>: HTTP status code is not handled or not allowed 2026-01-24 21:30:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-63mm-bore-x-250mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 21:30:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-poly-oxford-5-in-1-safety-jacket-xlt-black>: HTTP status code is not handled or not allowed 2026-01-24 21:30:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2171086-collar-locking-cp1-3-1-4-1-6-for-server-products>: HTTP status code is not handled or not allowed 2026-01-24 21:30:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-wire-shelf-truck-60x24x69-1200-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:30:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-wire-shelf-truck-36x18x80-1200-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:30:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-wire-shelf-truck-72x24x92-1200-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:30:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-solid-galvanized-shelf-cart-48x18-2-shelves-800-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:30:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/full-terry-bar-mop-towel-16x19>: HTTP status code is not handled or not allowed 2026-01-24 21:30:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-size-fits-all-cup-dispenser>: HTTP status code is not handled or not allowed 2026-01-24 21:30:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fr-type-r-class-3-economy-mesh-vest-2xl-green-ssv97e-3vgmfr-2x>: HTTP status code is not handled or not allowed 2026-01-24 21:30:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-solid-galvanized-shelf-cart-36x24-3-shelves-800-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:30:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paraffin-bath-refill-6-lb-beads-in-bag-unscented-fragrance>: HTTP status code is not handled or not allowed 2026-01-24 21:30:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-safety-parka-jacket-3xlt-black>: HTTP status code is not handled or not allowed 2026-01-24 21:30:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-control-85958-08-08-knob-adj-12-tube-x-12-swift-fit-composite-body>: HTTP status code is not handled or not allowed 2026-01-24 21:30:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-control-89953-08-06-screw-adj-12-tube-x-38-swift-fit-metallic-collet>: HTTP status code is not handled or not allowed 2026-01-24 21:30:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/281303-wheel5-inch>: HTTP status code is not handled or not allowed 2026-01-24 21:30:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-6432-double-acting-magnetic-cylinder-25mm-bore-x-200mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 21:30:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/battery-powered-lift-counter-balanced-stacker-5>: HTTP status code is not handled or not allowed 2026-01-24 21:30:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-quilted-safety-freezer-jacket-m-black>: HTTP status code is not handled or not allowed 2026-01-24 21:30:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/threaded-y-85320-04-32-14-tube-x-10-32-unf-male-thread-composite-body>: HTTP status code is not handled or not allowed 2026-01-24 21:30:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72x30x52-5-cabinet-leg-workstation-w-4-drawers-powered-riser-shelf-butcher-block-top>: HTTP status code is not handled or not allowed 2026-01-24 21:30:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1711191-tool-emptying-pouch-mate-2-for-prince-castle>: HTTP status code is not handled or not allowed 2026-01-24 21:30:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2181252-lid-dispenser-6-qt-for-star-manufacturing>: HTTP status code is not handled or not allowed 2026-01-24 21:30:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/34-1436-toaster-element-120v-860w-11-1-4-inch-x-6-1-2-inch>: HTTP status code is not handled or not allowed 2026-01-24 21:30:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-80mm-bore-x-220mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 21:30:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-control-55915-8-38-knob-adj-8mm-tube-x-38-swift-fit-composite-body>: HTTP status code is not handled or not allowed 2026-01-24 21:30:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blue-vinyl-reusable-cold-pack-quarter-size-5-x-7-12-pk>: HTTP status code is not handled or not allowed 2026-01-24 21:30:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-heating-unit-m-2-with-12-standard-packs>: HTTP status code is not handled or not allowed 2026-01-24 21:30:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-control-85968-04-02-knob-adj-14-tube-x-18-swift-fit-composite-body>: HTTP status code is not handled or not allowed 2026-01-24 21:30:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-hh-pwfd-cable-stand>: HTTP status code is not handled or not allowed 2026-01-24 21:30:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-50mm-bore-x-550mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 21:30:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-control-55910-6-14-screw-adj-6mm-tube-x-14-swift-fit-composite-body>: HTTP status code is not handled or not allowed 2026-01-24 21:30:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-100mm-bore-x-240mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 21:30:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-2-drain-flush-bag>: HTTP status code is not handled or not allowed 2026-01-24 21:30:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-electric-75-3-8-cable-cutter>: HTTP status code is not handled or not allowed 2026-01-24 21:30:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-z-brite-wire-shelving-add-on-36-w-x-18-d-x-74-h>: HTTP status code is not handled or not allowed 2026-01-24 21:30:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-13>: HTTP status code is not handled or not allowed 2026-01-24 21:30:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flip-top-sandwich-bags-low-density-7-x-7-085-mil>: HTTP status code is not handled or not allowed 2026-01-24 21:30:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1901261-lid-brew-thru-f-1-gal-for-bunn-o-matic>: HTTP status code is not handled or not allowed 2026-01-24 21:30:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-75-5-8-flexi-male-female-end>: HTTP status code is not handled or not allowed 2026-01-24 21:30:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-male-89000-06-02-38-tube-x-18-swift-fit-univ-thread-metallic-collet>: HTTP status code is not handled or not allowed 2026-01-24 21:30:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-additional-level-wide-span-rack-96x24-wood-deck-800-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:30:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-100-3-4-flexi-male-female-end>: HTTP status code is not handled or not allowed 2026-01-24 21:30:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-125mm-bore-x-650mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 21:30:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-24>: HTTP status code is not handled or not allowed 2026-01-24 21:30:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beehive-style-serving-bowl-46569-double-wall-13-3-4-diameter>: HTTP status code is not handled or not allowed 2026-01-24 21:30:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-48-w-x-18-d-x-74-h>: HTTP status code is not handled or not allowed 2026-01-24 21:30:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/321873-filter-hose-1-ftw-1-2-inch-fittings>: HTTP status code is not handled or not allowed 2026-01-24 21:30:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-32mm-bore-x-80mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 21:30:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eagle-daub-can-metal-red-1-2-pt>: HTTP status code is not handled or not allowed 2026-01-24 21:30:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/moist-heat-pack-half-size-5-x-12>: HTTP status code is not handled or not allowed 2026-01-24 21:30:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-drum-li-tilt-60-high-dispensing-55-gal-drum-12v>: HTTP status code is not handled or not allowed 2026-01-24 21:30:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quilted-vest-with-sherpa-lining-nylonpolyester-2xl-black>: HTTP status code is not handled or not allowed 2026-01-24 21:30:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-male-89000-02-02-18-tube-x-18-swift-fit-univ-thread-metallic-collet>: HTTP status code is not handled or not allowed 2026-01-24 21:30:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0002-inch-red-plastic-color-coded-shim-stock-25-inch-x-50-inch-flat-sheet>: HTTP status code is not handled or not allowed 2026-01-24 21:30:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-6432-double-acting-magnetic-cylinder-16mm-bore-x-80mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 21:30:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-control-55905-6-14-knob-adj-6mm-tube-x-14-swift-fit-composite-body>: HTTP status code is not handled or not allowed 2026-01-24 21:30:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/union-60040-06-38-tube-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:30:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/union-60040-10-10mm-tube-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:31:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/finish-thompson-ss-centrifugal-pump-4-impeller-1hp-3ph-tefc-motor>: HTTP status code is not handled or not allowed 2026-01-24 21:31:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5mm-square-metric-keystock-gold-dichromate-finish-12-inch-length-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-24 21:31:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8402303-lower-hinge-for-hoshizaki-of-america>: HTTP status code is not handled or not allowed 2026-01-24 21:31:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/finish-thompson-40-drum-pump-kit-econo-pump-tube-ss-shaft-air-motor>: HTTP status code is not handled or not allowed 2026-01-24 21:31:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/finish-thompson-40-drum-pump-kit-econo-pump-tube-ss-shaft-odp-motor>: HTTP status code is not handled or not allowed 2026-01-24 21:31:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blending-hand-pad-7446b-6-inch-x-9-inch-med-grit-silicon-carbide>: HTTP status code is not handled or not allowed 2026-01-24 21:31:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-welded-heavy-duty-cabinet-11>: HTTP status code is not handled or not allowed 2026-01-24 21:31:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14mm-x-22mm-metric-keystock-plain-finish-1-meter-length>: HTTP status code is not handled or not allowed 2026-01-24 21:31:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1751159-hose-flex-return-line-5-inch-x22-inch-for-magikitchen-products>: HTTP status code is not handled or not allowed 2026-01-24 21:31:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelf-36x18-with-4-clips-1>: HTTP status code is not handled or not allowed 2026-01-24 21:31:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-piece-blue-tempered-shim-stock-assortment-6-inch-x-12-inch-sheets>: HTTP status code is not handled or not allowed 2026-01-24 21:31:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 25139 pages (at 99 pages/min), scraped 13351 items (at 21 items/min) 2026-01-24 21:31:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/artificial-christmas-tree-65-ft-flocked-snowy-pine>: HTTP status code is not handled or not allowed 2026-01-24 21:31:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-branch-tee-60215-4-14-4mm-tube-x-14-male-bspp-thread-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:31:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/little-giant-wrs-series-water-removal-system-115v-integral-4-7-level>: HTTP status code is not handled or not allowed 2026-01-24 21:31:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-20-ga-36-wx24-dx85-h-open-clip-style-5-shelf>: HTTP status code is not handled or not allowed 2026-01-24 21:31:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/263297-kason-11556000005-cartridge-assembly-for-kason>: HTTP status code is not handled or not allowed 2026-01-24 21:31:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-branch-tee-89210-04-02-14-tube-x-18-swift-fit-metallic-collet>: HTTP status code is not handled or not allowed 2026-01-24 21:31:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulkhead-union-60050-10-10mm-tube-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:31:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-20-ga-48-wx18-dx85-h-open-clip-style-6-shelf>: HTTP status code is not handled or not allowed 2026-01-24 21:31:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-male-elbow-89110-06-04-38-tube-x-14-swift-fit-metallic-collet>: HTTP status code is not handled or not allowed 2026-01-24 21:31:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-20-ga-36-wx18-dx97-h-open-clip-style-5-shelf>: HTTP status code is not handled or not allowed 2026-01-24 21:31:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thru-wall-package-drop-with-tilt-out-door-dvwm0062s-black>: HTTP status code is not handled or not allowed 2026-01-24 21:31:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-20-ga-48-wx24-dx97-h-open-clip-style-5-shelf>: HTTP status code is not handled or not allowed 2026-01-24 21:31:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2151370-blade-assembly-3-8-inch-dice-for-redco-slicers>: HTTP status code is not handled or not allowed 2026-01-24 21:31:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-20-ga-48-wx24-dx97-h-open-clip-style-6-shelf>: HTTP status code is not handled or not allowed 2026-01-24 21:31:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-56-epfc-3ph-3450-2850rpm-a>: HTTP status code is not handled or not allowed 2026-01-24 21:31:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/341138-strip-heater120v-250w-12-inch-x-1-1-2>: HTTP status code is not handled or not allowed 2026-01-24 21:31:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-184-dp-1ph-1740rpm-c184k17dk18aa>: HTTP status code is not handled or not allowed 2026-01-24 21:31:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/040mm-steel-shim-stock-150mm-x-25m-roll>: HTTP status code is not handled or not allowed 2026-01-24 21:31:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/51026-100-heavy-duty-plastic-barrier-chain-hdpe-2-x100-54mm-traffic-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:31:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1821081-support-basket-14-1-4x12-1-2-inch>: HTTP status code is not handled or not allowed 2026-01-24 21:31:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/53002-100-gothic-plastic-barrier-chain-hdpe-2-x100-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:31:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-56-tefc-1ph-3450rpm-m6k34fk16dd>: HTTP status code is not handled or not allowed 2026-01-24 21:31:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36x24x63-stainless-steel-solid-shelving-add-on>: HTTP status code is not handled or not allowed 2026-01-24 21:31:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96in-metal-mesh-bench-with-back-rest-red>: HTTP status code is not handled or not allowed 2026-01-24 21:31:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50054-100-plastic-barrier-chain-hdpe-2-x100-8-51mm-green>: HTTP status code is not handled or not allowed 2026-01-24 21:31:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-aquatic-exercise-kit-jogger-belt-hand-bars-medium-red>: HTTP status code is not handled or not allowed 2026-01-24 21:31:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-145-tefc-3ph-1725-850rpm-c145t48fb61dd>: HTTP status code is not handled or not allowed 2026-01-24 21:31:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b20hl-shieldedlined-hose-clamp-1-18-inch-1-58-inch-clamping-diameter-box-of-10>: HTTP status code is not handled or not allowed 2026-01-24 21:31:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pet-hair-eraser-cordless-handheld-vacuum-1782>: HTTP status code is not handled or not allowed 2026-01-24 21:31:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jet-tech-4-compartment-divider-insert-30087-rack-f-14>: HTTP status code is not handled or not allowed 2026-01-24 21:31:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tote-box-accessory-short-divider-3>: HTTP status code is not handled or not allowed 2026-01-24 21:31:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lineshaft-3-var-speed-drive-conveyor-lsds19x16-16-3-3-v25-60-05-cd-16bf-12-hp>: HTTP status code is not handled or not allowed 2026-01-24 21:31:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qr152h-quick-release-part-ss-hose-clamp-2-inch-10-inch-clamping-diam-10-pcs>: HTTP status code is not handled or not allowed 2026-01-24 21:31:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-male-elbow-60110-10-38-10mm-tube-x-38-bspt-thread-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:31:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/anti_fatigue_mats/wet_area_anti_fatigue_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:31:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6mm-tube-fitting-kit-50860n-6-60-assorted-fittings-durham-slide-drawer>: HTTP status code is not handled or not allowed 2026-01-24 21:31:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nut-eyebolt-7-8-9-x-12l-1-3-4-id-3-1-2-od-plain-pattern-304-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:31:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nut-eyebolt-5-8-11-x-6l-1-3-8-id-2-1-2-od-plain-pattern-316-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:31:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50323-10-s-hook-acetal-copolymer-2-purple-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 21:31:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hog-heaven-fashion-mat-seven-eighths-3x5-cobalt-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:31:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-20-ga-48-wx30-dx73-h-closed-clip-style-5-shelf>: HTTP status code is not handled or not allowed 2026-01-24 21:31:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keyboard-tray-235wx145d-mocha>: HTTP status code is not handled or not allowed 2026-01-24 21:31:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-tecgen-flame-resistant-henley-shirt-xs-tall-navy>: HTTP status code is not handled or not allowed 2026-01-24 21:31:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/union-89040-04-53-14-tube-x-532-tube-metallic-collet>: HTTP status code is not handled or not allowed 2026-01-24 21:31:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rear-caster-global-1-2-to-2-ton-portable-ac-s>: HTTP status code is not handled or not allowed 2026-01-24 21:31:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/floor_mats/anti_fatigue_mats/wet_area_anti_fatigue_mats>: HTTP status code is not handled or not allowed 2026-01-24 21:31:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/51766-10-heavy-duty-master-link-acetal-copolymer-2-blue-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 21:31:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-x-24-image-enclosed-letterboard-black-frame>: HTTP status code is not handled or not allowed 2026-01-24 21:31:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd40h-916-inch-band-heavyduty-3-pc-part-ss-hose-clamp-2-116-inch-3-inch-clmpgdiam-10-pcs>: HTTP status code is not handled or not allowed 2026-01-24 21:31:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22-1097-10-1-4-inch-door-latch-with-strike-straight-handle>: HTTP status code is not handled or not allowed 2026-01-24 21:31:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m3-x-0-5-x-8mm-socket-head-cap-screw-304-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:31:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-commercial-cutlery-bin-four-compartments-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:31:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m4-x-0-7-x-12mm-phillips-oval-head-machine-screw-304-stainless-steel-din-966-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:31:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tex-gold-fp-kinesiology-tape-2-x-34-yds-red-bulk-roll>: HTTP status code is not handled or not allowed 2026-01-24 21:31:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1771017-wheel-8-inch-w-delrin-bearing-for-lakeside-manufacturing>: HTTP status code is not handled or not allowed 2026-01-24 21:31:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-hex-drive-thru-dome-lid-black>: HTTP status code is not handled or not allowed 2026-01-24 21:31:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pre-cut-kinesiology-tape-wrist-case-of-20>: HTTP status code is not handled or not allowed 2026-01-24 21:31:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b96hsp-300-series-ss-hose-clamp-4-12-inch-6-12-inch-clamping-diameter-box-of-10>: HTTP status code is not handled or not allowed 2026-01-24 21:31:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b80hsp-300-series-ss-hose-clamp-4-58-inch-5-12-inch-clamping-diameter-box-of-10>: HTTP status code is not handled or not allowed 2026-01-24 21:31:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/321041-seal-assembly-for-blodgett-oven>: HTTP status code is not handled or not allowed 2026-01-24 21:31:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-tecgen-flame-resistant-work-shirt-5xl-short-navy>: HTTP status code is not handled or not allowed 2026-01-24 21:31:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-184-tefc-1ph-1140rpm-c184c11fb2cc>: HTTP status code is not handled or not allowed 2026-01-24 21:31:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-184-dp-1ph-1740rpm-p184c17dk3aa>: HTTP status code is not handled or not allowed 2026-01-24 21:32:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/261585-door-closerflush>: HTTP status code is not handled or not allowed 2026-01-24 21:32:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-6-dual-action-sander>: HTTP status code is not handled or not allowed 2026-01-24 21:32:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-tecgen-flame-resistant-work-shirt-3xl-long-tall-light-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:32:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dart-lift-n-lock-plastic-hot-cup-lids-fits-10-oz-cups-white-100bag>: HTTP status code is not handled or not allowed 2026-01-24 21:32:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-tecgen-flame-resistant-work-shirt-xl-long-tall-navy>: HTTP status code is not handled or not allowed 2026-01-24 21:32:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-56-tenv-3ph-3450-2850rpm-c6t34nc2ee>: HTTP status code is not handled or not allowed 2026-01-24 21:32:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011639-hinge-assembly-right-hand-old-style-for-continental-refrigerator>: HTTP status code is not handled or not allowed 2026-01-24 21:32:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lite-caulk-warmer-equipment-heated-pouch-pblcauw-85f-fixed>: HTTP status code is not handled or not allowed 2026-01-24 21:32:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-3-4-cu-yd-structural-foam-plastic-tilt-truck-2>: HTTP status code is not handled or not allowed 2026-01-24 21:32:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/light-filtering-cordless-cellular-shade-30-wide-x-48-drop-fossil>: HTTP status code is not handled or not allowed 2026-01-24 21:32:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1261528-caster-utility-cart-4-inch-metro-for-intermetro-industries>: HTTP status code is not handled or not allowed 2026-01-24 21:32:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b80hs-all-ss-worm-gear-hose-clamp-4-58-inch-5-12-inch-clamping-diameter-box-of-10>: HTTP status code is not handled or not allowed 2026-01-24 21:32:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-8482-medium-narcotics-cabinet-double-door-door-lock-16-w-x-8-d-x-24-h-beige>: HTTP status code is not handled or not allowed 2026-01-24 21:32:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kickspace-heater-114-1500750w-240120v-white>: HTTP status code is not handled or not allowed 2026-01-24 21:32:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 25239 pages (at 100 pages/min), scraped 13375 items (at 24 items/min) 2026-01-24 21:32:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-flame-resistant-dh-air153-work-shirt-xs-tall-navy>: HTTP status code is not handled or not allowed 2026-01-24 21:32:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-heater-174-15001125750w-240208v-white>: HTTP status code is not handled or not allowed 2026-01-24 21:32:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0005-inch-blue-plastic-color-coded-shim-stock-5-inch-x-20-inch-flat-sheet>: HTTP status code is not handled or not allowed 2026-01-24 21:32:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dart-conex-translucent-plastic-cold-cups-9-oz>: HTTP status code is not handled or not allowed 2026-01-24 21:32:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/boardwalk-full-length-polystyrene-cutlery-knife-black-1000carton>: HTTP status code is not handled or not allowed 2026-01-24 21:32:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/weekly-planner-whiteboard-steel-36-x-24>: HTTP status code is not handled or not allowed 2026-01-24 21:32:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-tecgen-flame-resistant-work-shirt-m-tall-light-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:32:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s9591-65w-a19-led-240-beam-spread-medium-base-3000k-non-dimmable>: HTTP status code is not handled or not allowed 2026-01-24 21:32:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10500-series-storage-cabinet-w-doors-36-w-x-20-d-x-29-1-2-h-harvest>: HTTP status code is not handled or not allowed 2026-01-24 21:32:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2151205-handle-pusher-head-onion-kng-for-redco-slicers>: HTTP status code is not handled or not allowed 2026-01-24 21:32:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/solo-cup-company-plastic-souffle-portion-cups-2oz-black-250bag>: HTTP status code is not handled or not allowed 2026-01-24 21:32:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gray-plastic-box-truck-16-bushel-medium-duty>: HTTP status code is not handled or not allowed 2026-01-24 21:32:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/utility-knives-irwin-2082101>: HTTP status code is not handled or not allowed 2026-01-24 21:32:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-inch-x-4-inch-x-0075-inch-stainless-steel-slotted-shim-pack-of-5>: HTTP status code is not handled or not allowed 2026-01-24 21:32:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s9497-65w-led-mr16-40-beam-spread-gu53-base-3500k-12v>: HTTP status code is not handled or not allowed 2026-01-24 21:32:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s9495-65w-mr16-led-40-beam-spread-gu53-base-2700k-12v>: HTTP status code is not handled or not allowed 2026-01-24 21:32:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2261138-ring-egg-4-inch-dia-4-section>: HTTP status code is not handled or not allowed 2026-01-24 21:32:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/bookcases already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:32:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-flame-resistant-dh-air153-work-shirt-4xl-regular-navy>: HTTP status code is not handled or not allowed 2026-01-24 21:32:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s9583-45w-a19-filalment-led-amber-medium-base-2200k>: HTTP status code is not handled or not allowed 2026-01-24 21:32:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carpenter-saws-irwin-marathon-2011201>: HTTP status code is not handled or not allowed 2026-01-24 21:32:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1021067-drain-spinseal-1-1-2-inch-35-inch-so>: HTTP status code is not handled or not allowed 2026-01-24 21:32:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-full-vision-show-case-white>: HTTP status code is not handled or not allowed 2026-01-24 21:32:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012609-heater-tube-104v-900w-15-3-8-inch>: HTTP status code is not handled or not allowed 2026-01-24 21:32:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/381793-kason-11808000000-led-light-fixture-for-kason>: HTTP status code is not handled or not allowed 2026-01-24 21:32:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-stacking-bin-clear-19>: HTTP status code is not handled or not allowed 2026-01-24 21:32:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-gantry-crane-adjustable-height-2000-lb-capacity-1>: HTTP status code is not handled or not allowed 2026-01-24 21:32:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rotom-270b-270-324-mfd-125-volts-start-round>: HTTP status code is not handled or not allowed 2026-01-24 21:32:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-tecgen-flame-resistant-coverall-3xl-tall-navy>: HTTP status code is not handled or not allowed 2026-01-24 21:32:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acrylic-eyewear-display-4-tier>: HTTP status code is not handled or not allowed 2026-01-24 21:32:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/bookcases landed on page that is not a product page. 2026-01-24 21:32:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-gantry-crane-adjustable-height-6000-lb-capacity-1>: HTTP status code is not handled or not allowed 2026-01-24 21:32:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-tecgen-flame-resistant-coverall-4xl-long-tall-tan>: HTTP status code is not handled or not allowed 2026-01-24 21:32:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/instant-alert-monitor-fcs11-for-fc-and-fcm-series>: HTTP status code is not handled or not allowed 2026-01-24 21:32:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rotating-countertop-display-18w-x-12d-x-21-1-2h-maple>: HTTP status code is not handled or not allowed 2026-01-24 21:32:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exo2-migb-04-lmotor-impact-glove>: HTTP status code is not handled or not allowed 2026-01-24 21:32:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-6-outlet-9ft-1200-joules>: HTTP status code is not handled or not allowed 2026-01-24 21:32:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0071-inch-diameter-music-wire-1-pound-coil>: HTTP status code is not handled or not allowed 2026-01-24 21:32:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/folding-shoe-rack-for-easy-storage-chrome-8-shelves>: HTTP status code is not handled or not allowed 2026-01-24 21:32:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-12-inch-id-x-2-18-inch-od-x-0047-inch-steel-arbor-shim-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 21:32:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/short-tube-for-20-ton-capacity-jack-stand>: HTTP status code is not handled or not allowed 2026-01-24 21:32:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/750va-compact-low-profile-standby-12-outlet-usb>: HTTP status code is not handled or not allowed 2026-01-24 21:32:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-inch-plastic-classroom-chair-set-of-five-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:32:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lightning-lift-heavy-duty-floor-jack-1-pc-handle-3-12-ton-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:32:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-dolly-27-l-x-16-w-with-solid-treadplate-deck-rubber-wheels>: HTTP status code is not handled or not allowed 2026-01-24 21:32:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-dolly-36-l-x-24-w-with-solid-treadplate-deck-steel-wheels>: HTTP status code is not handled or not allowed 2026-01-24 21:32:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anchor-extension-8l-o-ring-snaphook>: HTTP status code is not handled or not allowed 2026-01-24 21:32:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-dock-board-with-steel-curbs-7>: HTTP status code is not handled or not allowed 2026-01-24 21:32:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ce-airestream-fume-hood-with-explosion-proof-light-72w-24d-45h>: HTTP status code is not handled or not allowed 2026-01-24 21:32:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/save-a-stud-rethread-die-12mm-175>: HTTP status code is not handled or not allowed 2026-01-24 21:32:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-cloth-filter-18-gallon-wet-dry-squeegee-vacuum>: HTTP status code is not handled or not allowed 2026-01-24 21:32:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crevice-tool-18-gallon-wet-dry-squeegee-vacuum>: HTTP status code is not handled or not allowed 2026-01-24 21:32:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wet-pick-up-tool-18-gallon-wet-dry-squeegee-vacuum>: HTTP status code is not handled or not allowed 2026-01-24 21:32:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-32-x-3-8-button-socket-cap-screw-steel-black-oxide-unc-pkg-of-100-64018>: HTTP status code is not handled or not allowed 2026-01-24 21:32:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-32w-four-pin-verticalmount-socket>: HTTP status code is not handled or not allowed 2026-01-24 21:32:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-frame-for-7-ft-4-inch-high-door-or-room-divider>: HTTP status code is not handled or not allowed 2026-01-24 21:32:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evhc12-06l-led-high-lumen-emergency-unit-white-546-lumens-high-mounts>: HTTP status code is not handled or not allowed 2026-01-24 21:32:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11-panel-divider-6-ft-high-x-20-ft-5-inch-long-vinyl-granite>: HTTP status code is not handled or not allowed 2026-01-24 21:32:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/710702k-dc-1200vx-ck1-dust-collector-2hp-1ph-230v-2-micron-canister-kit>: HTTP status code is not handled or not allowed 2026-01-24 21:32:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keyless-porcelain-mogul-socket-mounting-screws-held-captive-2-wireways-1-2-strip-leads>: HTTP status code is not handled or not allowed 2026-01-24 21:32:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-outdoor-ashtray>: HTTP status code is not handled or not allowed 2026-01-24 21:32:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-defender-walk-ramp-39w-6l-4000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:32:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/augusta-6-outdoor-bench-with-back-without-side-armrests-surface-mount-brown>: HTTP status code is not handled or not allowed 2026-01-24 21:32:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-light-bent-glass-fixture-holder-w-pull-chain-switch-white-finish>: HTTP status code is not handled or not allowed 2026-01-24 21:32:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/outside-white-drum-glass-with-clear-sides-and-bottom-8-3-8-in-diameter>: HTTP status code is not handled or not allowed 2026-01-24 21:32:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprayed-inside-white-drum-glass-10-5-8-in-diameter>: HTTP status code is not handled or not allowed 2026-01-24 21:32:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-panel-display-tower-6-ft-5-inch-high-fabric-greysmoke>: HTTP status code is not handled or not allowed 2026-01-24 21:32:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/phenolic-candelabra-socket-with-paper-liner-4-in-screw-terminals-double-leg>: HTTP status code is not handled or not allowed 2026-01-24 21:32:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/richmond-4-leaning-rail-in-ground-mount-brown>: HTTP status code is not handled or not allowed 2026-01-24 21:32:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/halogen-socket-w-metal-plate-2-mounting-holes>: HTTP status code is not handled or not allowed 2026-01-24 21:32:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-gloves-11-mil-latex-12-inch-and-powder-free-medical-grade-xl>: HTTP status code is not handled or not allowed 2026-01-24 21:32:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009523-lid-gasket>: HTTP status code is not handled or not allowed 2026-01-24 21:33:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oxford-96-rectangular-picnic-table-with-umbrella-hole-surface-mount-black>: HTTP status code is not handled or not allowed 2026-01-24 21:33:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-panel-display-tower-6-ft-5-inch-high-fabric-beech>: HTTP status code is not handled or not allowed 2026-01-24 21:33:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-x-30-anti-static-top-ergo-line-workbench-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:33:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buna-n-x-profile-o-ring-dash-009-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:33:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buna-n-x-profile-o-ring-dash-218-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-24 21:33:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/miramar-10-saute-pan-without-lid-49413-2-1-4-quart-capacity-satin-finish>: HTTP status code is not handled or not allowed 2026-01-24 21:33:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-spill-grate-for-spill-pallets-workstations>: HTTP status code is not handled or not allowed 2026-01-24 21:33:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/madison-4-outdoor-bench-without-back-with-armrests-surface-mount-brown>: HTTP status code is not handled or not allowed 2026-01-24 21:33:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washdown-motor-56-tefc-1ph-3600rpm-aa1>: HTTP status code is not handled or not allowed 2026-01-24 21:33:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-shelf-nest-away-folding-truck>: HTTP status code is not handled or not allowed 2026-01-24 21:33:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 25338 pages (at 99 pages/min), scraped 13392 items (at 17 items/min) 2026-01-24 21:33:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-structural-flat-washer-1-1-8-i-d-steel-plain-f436-astm-a325-25-pk>: HTTP status code is not handled or not allowed 2026-01-24 21:33:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washdown-motor-213-tefc-3ph-1760-1450rpm-b>: HTTP status code is not handled or not allowed 2026-01-24 21:33:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/new-haven-8-ash-wood-bench-with-back-armrests-brown>: HTTP status code is not handled or not allowed 2026-01-24 21:33:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15ml-centrifuge-tube-snap-pop-lid-self-standing-resealable-bag-sterile-500-case>: HTTP status code is not handled or not allowed 2026-01-24 21:33:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-flat-washer-sae-9-32-i-d-steel-yellow-zinc-grade-8-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:33:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/infinity-locker-single-tier-12x12x72-3-door-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:33:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/infinity-locker-single-tier-12x18x72-3-door-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:33:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-split-lock-washer-385-377-i-d-094-thick-steel-zinc-grade-2-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 21:33:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fowler-xtra-value-depth-gage>: HTTP status code is not handled or not allowed 2026-01-24 21:33:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-round-table-with-4-imme-armless-chairs-coral-seatstudio-teak-top>: HTTP status code is not handled or not allowed 2026-01-24 21:33:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arkadia-sauce-pan-cover-7392-10-quart-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:33:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-med-hi-off-touch-switch>: HTTP status code is not handled or not allowed 2026-01-24 21:33:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelf-48x24-with-4-clips-2>: HTTP status code is not handled or not allowed 2026-01-24 21:33:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-square-table-with-2-imme-armless-barstools-black-seatmaple-top>: HTTP status code is not handled or not allowed 2026-01-24 21:33:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fowler-carbide-tipped-super-scriber>: HTTP status code is not handled or not allowed 2026-01-24 21:33:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/261772-baffle-filter-25-x-20-galvanized>: HTTP status code is not handled or not allowed 2026-01-24 21:33:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1ml-cf-cryogenic-vial-external-thread-self-standing-sterile-100-case>: HTTP status code is not handled or not allowed 2026-01-24 21:33:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fowler-premium-brand-1-2-3-blocks>: HTTP status code is not handled or not allowed 2026-01-24 21:33:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pack-of-100-buna-n-square-profile-o-rings-dash-012>: HTTP status code is not handled or not allowed 2026-01-24 21:33:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hunting-retractable-gear-tether-0tr0-025-large-48extention-od-green-carabiner>: HTTP status code is not handled or not allowed 2026-01-24 21:33:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ptfe-syringe-filter-0-45um-13mm-bulk-packed-non-sterile-100-case>: HTTP status code is not handled or not allowed 2026-01-24 21:33:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-round-table-with-2-imme-armless-barstools-white-seatstudio-teak-top>: HTTP status code is not handled or not allowed 2026-01-24 21:33:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tri-dex-label-holder-1>: HTTP status code is not handled or not allowed 2026-01-24 21:33:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-36x24x84-with-5-shelves-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:33:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-48x18x84-with-5-shelves-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:33:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1291128-kason-67001002520-filter-hood-for-kason>: HTTP status code is not handled or not allowed 2026-01-24 21:33:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-x-24-adjustable-height-shop-stand-2>: HTTP status code is not handled or not allowed 2026-01-24 21:33:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-foot-operated-hand-sanitizer-dispenser-use-with-gallon-bottles-w-pump>: HTTP status code is not handled or not allowed 2026-01-24 21:33:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/redco-tater-king-scooper-1801>: HTTP status code is not handled or not allowed 2026-01-24 21:33:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/851348-filter-paper-100-pk-for-bk-industries>: HTTP status code is not handled or not allowed 2026-01-24 21:33:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-48x12x60-with-5-shelves-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:33:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sec-alum-72-crossover-truckbox-single-lid-low-pro-deep-matblk-79211101>: HTTP status code is not handled or not allowed 2026-01-24 21:33:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traex-squeeze-dispensers-5232-13-standard-32-oz>: HTTP status code is not handled or not allowed 2026-01-24 21:33:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/key-times-to-wash-your-hands-sign>: HTTP status code is not handled or not allowed 2026-01-24 21:33:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sec-aluminum-63-crossover-truck-box-single-lid-black-79210988>: HTTP status code is not handled or not allowed 2026-01-24 21:33:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-2428-5-inch-swivel-plate-caster-300-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:33:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rfid-sticker-for-personal-device-charging-lockers>: HTTP status code is not handled or not allowed 2026-01-24 21:33:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/321565-gh-2280zx-large-spindle-bore-lathe-w-taper-attachment-7-1-2-hp>: HTTP status code is not handled or not allowed 2026-01-24 21:33:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-starter-rack-48x36x84-with-3-levels-wood-deck-1500lb-cap-per-shelf-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:33:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-plastic-storage-cabinet-36x18x37>: HTTP status code is not handled or not allowed 2026-01-24 21:33:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/circular-wire-flight-caddy-wr-1027-06-hold-3-glasses-7-x-11>: HTTP status code is not handled or not allowed 2026-01-24 21:33:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-super-wide-jumbo-cabinet-with-264-hook-on-bins>: HTTP status code is not handled or not allowed 2026-01-24 21:33:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arctic-cat-procross-xf-800-sno-pro-limited-794cc-replacement-battery-2013>: HTTP status code is not handled or not allowed 2026-01-24 21:33:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/341296-dishwasher-heater208v-10000w-2-1-4-inch-oc-for-champion-dishwasher>: HTTP status code is not handled or not allowed 2026-01-24 21:33:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g15-20-hp-rotary-screw-compressor-120-gal-horizontal-125-psig-3-phase-208-230-460v>: HTTP status code is not handled or not allowed 2026-01-24 21:33:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-portable-vehicle-axle-scale-pads-40000-lb-x-5-lb-4-pads>: HTTP status code is not handled or not allowed 2026-01-24 21:33:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-starter-rack-72x48x96-with-3-level-steel-deck-1000lb-cap-per-shelf-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:33:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-starter-rack-72x24x96-with-3-levels-wire-deck-1000lb-cap-per-shelf-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:33:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-starter-rack-72x48x96-with-3-levels-wire-deck-1000lb-cap-per-shelf-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:33:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8015433-kason-1810lct400-led-light-fixture-for-kason>: HTTP status code is not handled or not allowed 2026-01-24 21:33:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/341302-sink-heater208v-6000w-26-inch-x-17-inch>: HTTP status code is not handled or not allowed 2026-01-24 21:33:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dirty-sox-small-a-c-system-odor-neutralizer-cleaner-20-oz-1-qty>: HTTP status code is not handled or not allowed 2026-01-24 21:33:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mirage-induction-rethermalizer-74110110-11-quart-silver>: HTTP status code is not handled or not allowed 2026-01-24 21:33:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8013048-element-120v-1100w-for-sealed-well-for-advance-tabco>: HTTP status code is not handled or not allowed 2026-01-24 21:33:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-add-on-rack-48x48x84-with-3-levels-steel-deck-1500lb-cap-per-level-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:33:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/axle-kit-magliner-hand-truck-with-10-wheels>: HTTP status code is not handled or not allowed 2026-01-24 21:33:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/580001-control-pro-170-hea-piston-pump-paint-sprayer>: HTTP status code is not handled or not allowed 2026-01-24 21:33:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arkadia-10-non-stick-fry-pan-n7010-8-gauge-1-3-4-depth>: HTTP status code is not handled or not allowed 2026-01-24 21:33:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-add-on-rack-72x36x96-with-3-levels-wood-deck-1000-lb-cap-per-level-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:33:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-wall-insulated-serving-bowl-4659260-6-9-quart-black-black>: HTTP status code is not handled or not allowed 2026-01-24 21:33:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009353-element-assembly208-220v-for-moffat>: HTTP status code is not handled or not allowed 2026-01-24 21:33:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-add-on-rack-96x36x96-with-3-levels-wood-deck-800-lb-cap-per-level-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:33:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobilepro-anti-fatigue-mat-3-4-thick-2-x-1-5-black>: HTTP status code is not handled or not allowed 2026-01-24 21:33:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cm-lodestar-electric-chain-hoist-with-chain-container-500-lb-capacity-7>: HTTP status code is not handled or not allowed 2026-01-24 21:33:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluted-double-wall-insulated-serving-bowls-47685-0-75-quart-round>: HTTP status code is not handled or not allowed 2026-01-24 21:33:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hl-series-lateral-file-36-wide-4-drawer-lava-red>: HTTP status code is not handled or not allowed 2026-01-24 21:33:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-level-96x48-high-capacity-rack-wood-deck-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:34:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/front-loading-digital-lock-depository-safe-2-0-cu-ft-off-white-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:34:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/queue-153-solutions-barricade-cart-30-barricades-barr-cart>: HTTP status code is not handled or not allowed 2026-01-24 21:34:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-pan-v-stainless-steam-table-pan-30162-6-depth-2-3-size>: HTTP status code is not handled or not allowed 2026-01-24 21:34:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/easy-adjust-boltless-shelf-truck-with-laminate-shelves-20>: HTTP status code is not handled or not allowed 2026-01-24 21:34:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-insulated-disconnect-f-12-10-awg-yellow-250-tab-50-pk>: HTTP status code is not handled or not allowed 2026-01-24 21:34:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-one-plastic-stock-utility-procart-40-l-x-19-1-2-w-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:34:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/norton-66623395022-neon-fiber-disc-5-inch-x-7-8-inch-aluminum-oxide-120-grit>: HTTP status code is not handled or not allowed 2026-01-24 21:34:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tap-holder-w-clutch-5-32-inch-no-8-m4-at5508>: HTTP status code is not handled or not allowed 2026-01-24 21:34:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cayenne-18-manually-controlled-gas-griddle-40719-28000-btu-18-x-27-x-16>: HTTP status code is not handled or not allowed 2026-01-24 21:34:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-square-table-with-4-imme-armless-caster-chairs-navy-seatdesigner-white-top>: HTTP status code is not handled or not allowed 2026-01-24 21:34:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bookcases_hutches already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:34:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-grid-wall-peg-hook>: HTTP status code is not handled or not allowed 2026-01-24 21:34:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/norton-66252823603-gemini-right-angle-cut-off-wheel-5-inch-x-045-inch-x-7-8-inch-36-grit-aluminum-oxide-type-1>: HTTP status code is not handled or not allowed 2026-01-24 21:34:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-durable-floor-tape-red-2-x-100-3-pack-wear-resistant-pvc>: HTTP status code is not handled or not allowed 2026-01-24 21:34:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 25435 pages (at 97 pages/min), scraped 13410 items (at 18 items/min) 2026-01-24 21:34:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/263896-baffle-filter-25-x-16-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:34:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10500-series153-laminate-bookcase-36w-x-13d-x-43-34h-kingswood-walnut>: HTTP status code is not handled or not allowed 2026-01-24 21:34:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-housing-die-grinder>: HTTP status code is not handled or not allowed 2026-01-24 21:34:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slant-shelf-36x24>: HTTP status code is not handled or not allowed 2026-01-24 21:34:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merit-69957351584-abrasotex-non-woven-quick-change-disc-1-inch-dia-med-grit-aluminum-oxide-type-iii>: HTTP status code is not handled or not allowed 2026-01-24 21:34:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/skullerz-saga-frameless-safety-glasses-matte-black-in-outdoor-lens>: HTTP status code is not handled or not allowed 2026-01-24 21:34:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gearmotor-ac-parallel-60rpm-cm38p17fz12c>: HTTP status code is not handled or not allowed 2026-01-24 21:34:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/norton-66252842202-gemini-right-angle-cut-off-wheel-6-inch-x-047-inch-x-7-8-inch-24-grit-aluminum-oxide-type-27>: HTTP status code is not handled or not allowed 2026-01-24 21:34:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bookcases_hutches>: HTTP status code is not handled or not allowed 2026-01-24 21:34:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/redco-blade-assembly-only-15065-4-section-wedge-tabletop-wall-mount>: HTTP status code is not handled or not allowed 2026-01-24 21:34:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chill-its-multi-band-face-cover-neck-gaiter-khaki-heather>: HTTP status code is not handled or not allowed 2026-01-24 21:34:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/norton-66623395005-neon-fiber-disc-4-inch-x-5-8-inch-aluminum-oxide-100-grit>: HTTP status code is not handled or not allowed 2026-01-24 21:34:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pushgard-174-push-in-connector-3-port-10-pk>: HTTP status code is not handled or not allowed 2026-01-24 21:34:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grid-wall-bin-accessory-label-holder-sold-pack-of-6-1>: HTTP status code is not handled or not allowed 2026-01-24 21:34:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gearmotor-dc-parallel-125rpm-cm34d25nz56b>: HTTP status code is not handled or not allowed 2026-01-24 21:34:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bin-nestable-1>: HTTP status code is not handled or not allowed 2026-01-24 21:34:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2221340-jar-w-blndg-asy-mx1000-1100xt-for-waring-products>: HTTP status code is not handled or not allowed 2026-01-24 21:34:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-giant-stackable-storage-bin-18-3-8x29x14-7-8-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:34:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/norton-66261045600-bear-tex-non-woven-hand-pad-6-inch-x-9-inch-none-non-abrasive>: HTTP status code is not handled or not allowed 2026-01-24 21:34:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plus-cut-resistant-gloves-nano-foam-nitrile-coated-ansi-a4-xs-black-12-pair>: HTTP status code is not handled or not allowed 2026-01-24 21:34:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-storage-cabinet-9300-series-2-door-fixed-shelves-light-grey>: HTTP status code is not handled or not allowed 2026-01-24 21:34:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12ft-fiberglass-leansafe-ladder-w-plastic-tool-tray-375-lb-capacity-l7312>: HTTP status code is not handled or not allowed 2026-01-24 21:34:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hanger-coupler-for-runway-rails-for-hubbell-50-lb-capacity-tool-cranes>: HTTP status code is not handled or not allowed 2026-01-24 21:34:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/generators_inverters_power_distribution_units/power_inverters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:34:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tennsco-locker-number-plate-51-75-aluminum-with-1-2-black-lettering-lnp-a-75-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-24 21:34:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/full-mask-low-pressure-system-3-workers-100-hose>: HTTP status code is not handled or not allowed 2026-01-24 21:34:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-buffet-station-with-wire-grill-4667580-natural-21-x-16-x-7-1-2>: HTTP status code is not handled or not allowed 2026-01-24 21:34:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/small-buffet-station-with-wire-grill-4667475-black-16-x-16-x-7-1-2>: HTTP status code is not handled or not allowed 2026-01-24 21:34:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/263395-closer-assembly-door-for-norlake-refrigeration>: HTTP status code is not handled or not allowed 2026-01-24 21:34:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/264328-door-closer-flush>: HTTP status code is not handled or not allowed 2026-01-24 21:34:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scribe-engraving-chisel-kit>: HTTP status code is not handled or not allowed 2026-01-24 21:34:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/visiontek-soundcube-wireless-bluetooth-speaker-black>: HTTP status code is not handled or not allowed 2026-01-24 21:34:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zurn-double-lab-faucet-with-8-gooseneck-and-lever-handles-lead-free>: HTTP status code is not handled or not allowed 2026-01-24 21:34:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/check-holders-2518-18-long-brushed-aluminum>: HTTP status code is not handled or not allowed 2026-01-24 21:34:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-work-tent-8-x-8-2-door>: HTTP status code is not handled or not allowed 2026-01-24 21:34:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/261773-baffle-filter-16-x-20-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:34:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10x20-auto-shelter-1-3-8-inch-frame-tan-cover>: HTTP status code is not handled or not allowed 2026-01-24 21:34:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-sign-no-parking-unauthorized-vehicles-08mm-thick-1>: HTTP status code is not handled or not allowed 2026-01-24 21:34:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/generators_inverters_power_distribution_units/power_inverters>: HTTP status code is not handled or not allowed 2026-01-24 21:34:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/341123-urn-heater-w-cut-out480v-7000w>: HTTP status code is not handled or not allowed 2026-01-24 21:34:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-sign-no-parking-anytime-080mm-thick>: HTTP status code is not handled or not allowed 2026-01-24 21:34:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12x20x9-barn-shelter-grey>: HTTP status code is not handled or not allowed 2026-01-24 21:34:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vpl910lensled-sign-projector-lens-only>: HTTP status code is not handled or not allowed 2026-01-24 21:34:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1571106-connector-gas-3-4-inch-x48-inch-for-dormont-mfg>: HTTP status code is not handled or not allowed 2026-01-24 21:34:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-x-72-table-with-6-imme-armless-barstools-white-seatespresso-top>: HTTP status code is not handled or not allowed 2026-01-24 21:34:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drum-heater-60-to-250-degrees-fahrenheit-1920-watts>: HTTP status code is not handled or not allowed 2026-01-24 21:34:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/81-h-kiosk-unit-with-power-outlet-beige>: HTTP status code is not handled or not allowed 2026-01-24 21:34:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ignition-mobile-guest-reception-stacking-chair-designer-white-frameblack-seat>: HTTP status code is not handled or not allowed 2026-01-24 21:34:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mcslhegvsp-gas-cylinder-shoulder-label>: HTTP status code is not handled or not allowed 2026-01-24 21:34:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/441657-mercury-contactor-for-frymaster>: HTTP status code is not handled or not allowed 2026-01-24 21:34:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/341981-heating-element-2200-watt-265-volt-for-nieco-corp>: HTTP status code is not handled or not allowed 2026-01-24 21:34:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ptp206gn-tough-mark-strip>: HTTP status code is not handled or not allowed 2026-01-24 21:34:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1791001-shelf-ceramic-m-r21ht-for-sharp>: HTTP status code is not handled or not allowed 2026-01-24 21:34:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2570-grade-2-cylindrical-lock-classroom-2-34-us26d-wtn-scc-kd-asa>: HTTP status code is not handled or not allowed 2026-01-24 21:34:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-lexington-bench-perforated-72-wide-25-deep-black>: HTTP status code is not handled or not allowed 2026-01-24 21:34:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tote-box-heat-resistant-40-to-300-degrees-53>: HTTP status code is not handled or not allowed 2026-01-24 21:34:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tote-box-heat-resistant-40-to-300-degrees-58>: HTTP status code is not handled or not allowed 2026-01-24 21:34:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ignition-20-4-way-stretch-task-chair-mid-back-17-20h-seat-basalt>: HTTP status code is not handled or not allowed 2026-01-24 21:34:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ignition-20-4-way-stretch-task-chair-black-adj-lumbar-support-17-21h-seat-carolina>: HTTP status code is not handled or not allowed 2026-01-24 21:34:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-platter-cart-10-trays-12-inch>: HTTP status code is not handled or not allowed 2026-01-24 21:34:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-w-x-15-3-4-d-x-7-1-2-h-stackbox-black>: HTTP status code is not handled or not allowed 2026-01-24 21:34:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-x-72-table-with-6-imme-armless-caster-chairs-coral-seatmaple-top>: HTTP status code is not handled or not allowed 2026-01-24 21:34:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010828-heating-element-208v-8300w-for-middleby>: HTTP status code is not handled or not allowed 2026-01-24 21:34:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kee-klamp-straight-coupling-1>: HTTP status code is not handled or not allowed 2026-01-24 21:34:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gml101rd-gauge-marking-decal>: HTTP status code is not handled or not allowed 2026-01-24 21:34:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-x-72-table-with-6-imme-armless-caster-chairs-black-seatespresso-top>: HTTP status code is not handled or not allowed 2026-01-24 21:34:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6x100-4-mil-clear-poly-sheeting>: HTTP status code is not handled or not allowed 2026-01-24 21:34:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tac501-red-tag-tracking-center>: HTTP status code is not handled or not allowed 2026-01-24 21:34:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-x-72-table-with-6-imme-armless-caster-chairs-light-gray-seatespresso-top>: HTTP status code is not handled or not allowed 2026-01-24 21:34:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kee-klamp-side-outlet-tee-2>: HTTP status code is not handled or not allowed 2026-01-24 21:34:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6x10-2-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-24 21:35:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-w-x-18-d-x-6-h-hopper-box-black>: HTTP status code is not handled or not allowed 2026-01-24 21:35:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/noheat-hand-dryer-with-noise-reduction-nozzle-graphite-die-cast-zinc-alloy-hepa-110-120v>: HTTP status code is not handled or not allowed 2026-01-24 21:35:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eye-face-wash-deck-mounted-90-degree-swivel-right-hand-mounting-g1806>: HTTP status code is not handled or not allowed 2026-01-24 21:35:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ptp211bu-tough-mark-strip>: HTTP status code is not handled or not allowed 2026-01-24 21:35:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-led-canopy-light-clcn11-40w-raa-br50>: HTTP status code is not handled or not allowed 2026-01-24 21:35:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kee-klamp-corner-swivel-socket-1>: HTTP status code is not handled or not allowed 2026-01-24 21:35:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kee-klamp-two-socket-tee-90-1>: HTTP status code is not handled or not allowed 2026-01-24 21:35:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010816-replacement-motorassembly-for-montague-oven>: HTTP status code is not handled or not allowed 2026-01-24 21:35:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2311020-grid-top-w-hndl-22-1-4x24-5-8-for-magikitchen-products>: HTTP status code is not handled or not allowed 2026-01-24 21:35:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 25532 pages (at 97 pages/min), scraped 13428 items (at 18 items/min) 2026-01-24 21:35:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/front-cover-plate-for-wham10-welding-helmet-4-12l-x-4w-clear-pack-of-5>: HTTP status code is not handled or not allowed 2026-01-24 21:35:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10194891-harness-back-d-ring-tongue-buckle-leg-straps-super-extra-large>: HTTP status code is not handled or not allowed 2026-01-24 21:35:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/snap-lock-hard-hat-red>: HTTP status code is not handled or not allowed 2026-01-24 21:35:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rotating-drum-cradle-with-polyolefin-wheels>: HTTP status code is not handled or not allowed 2026-01-24 21:35:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enclosure-kit-slide-door-24-x-72-x-18-light-grey>: HTTP status code is not handled or not allowed 2026-01-24 21:35:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blickman-chrome-5-leg-iv-height-adjustable-stand-52-1-2-93-1-2-4-hook>: HTTP status code is not handled or not allowed 2026-01-24 21:35:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cantilever-rack-double-sided-upright-17>: HTTP status code is not handled or not allowed 2026-01-24 21:35:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-boltless-shelving-starter-3600-lb-cap-96in-w-x-48in-d-x-96in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:35:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ridge-scraper-mat-3-ft-x-5-ft-black>: HTTP status code is not handled or not allowed 2026-01-24 21:35:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-14-x-30-red-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:35:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/comfort-rest-pebble-foam-mat-hd-4-ft-x-6-ft-coal>: HTTP status code is not handled or not allowed 2026-01-24 21:35:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spillmagic-sm-biohazard-spillmagic-biohazard-cleanup-kit>: HTTP status code is not handled or not allowed 2026-01-24 21:35:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10197219-harness-back-shoulder-d-rings-tongue-buckle-leg-straps-standard>: HTTP status code is not handled or not allowed 2026-01-24 21:35:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-14-x-72-white-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:35:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spillmagic-97110-spillmagic-powder-10-lb-bag>: HTTP status code is not handled or not allowed 2026-01-24 21:35:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/superflow-mat-4-ft-x-6-ft-red>: HTTP status code is not handled or not allowed 2026-01-24 21:35:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergo-mat-3-ft-x-5-ft-black>: HTTP status code is not handled or not allowed 2026-01-24 21:35:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010249-expansion-valve-for-manitowoc-machines>: HTTP status code is not handled or not allowed 2026-01-24 21:35:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10197436-harness-back-chest-hip-d-rings-qwik-fit-leg-straps-standard>: HTTP status code is not handled or not allowed 2026-01-24 21:35:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/881078-thermal-expansion-valvetex2-r22-less-orfice>: HTTP status code is not handled or not allowed 2026-01-24 21:35:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10194862-harness-back-chest-d-rings-quick-connect-leg-straps-super-extra-large>: HTTP status code is not handled or not allowed 2026-01-24 21:35:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-task-light-pivoting-head-40-degree-flood-3x3-watt-led-24v-ac-dc-w-out-transformer>: HTTP status code is not handled or not allowed 2026-01-24 21:35:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10194947-harness-back-d-ring-quick-connect-leg-straps-super-extra-large>: HTTP status code is not handled or not allowed 2026-01-24 21:35:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-24-x-30-dark-yellow-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:35:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-stanchion-safety-green>: HTTP status code is not handled or not allowed 2026-01-24 21:35:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/packard-contactor-4-pole-40-208240>: HTTP status code is not handled or not allowed 2026-01-24 21:35:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-extra-heavy-duty-boltless-shelving-starter-48in-w-x-24in-d-x-96in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:35:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-retention-load-pin-kit-for-1-weblok-assembly>: HTTP status code is not handled or not allowed 2026-01-24 21:35:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-wide-boltless-shelving-starter-60in-w-x-36in-d-x-60in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:35:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2241266-block-pusher-1-2-inch-ez-let-cut-for-nemco>: HTTP status code is not handled or not allowed 2026-01-24 21:35:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-14-x-36-white-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:35:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-add-on-72in-w-x-48in-d-x-84in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:35:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/packard-fan-center-spdt-relay-120v>: HTTP status code is not handled or not allowed 2026-01-24 21:35:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/wiring_devices/contactors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:35:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-plastic-utility-truck-800-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:35:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dynamic-high-pressure-ball-valve-1-1-16-12-npt-thread-5800-psi>: HTTP status code is not handled or not allowed 2026-01-24 21:35:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/upholstery-cleaning-tool-kit-with-8-ft-hose-for-fivestar-and-comet-models-357ac>: HTTP status code is not handled or not allowed 2026-01-24 21:35:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flipak-attached-lid-container-14>: HTTP status code is not handled or not allowed 2026-01-24 21:35:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30x6x24-telescoping-inner-boxes>: HTTP status code is not handled or not allowed 2026-01-24 21:35:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lab-workstation-cpu-shelf-12-wide-23-deep-30-high-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:35:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24x24x20-double-wall-boxes>: HTTP status code is not handled or not allowed 2026-01-24 21:35:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/impact-xps-153-impact-socket-set-w-std-socket-adapter-8-pc-1-4-drive-6pt-sae>: HTTP status code is not handled or not allowed 2026-01-24 21:35:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-x2150-2-mil-poly-tubing>: HTTP status code is not handled or not allowed 2026-01-24 21:35:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-1292-3-piece-1-8-inch-npt-bj-flange-nipple-kit>: HTTP status code is not handled or not allowed 2026-01-24 21:35:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dynamic-hydraulic-gear-pump-0-48-cu-in-rev-1-2-dia-straight-drive-shaft-10-39-gpm-5000-rpm>: HTTP status code is not handled or not allowed 2026-01-24 21:35:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/wiring_devices/contactors>: HTTP status code is not handled or not allowed 2026-01-24 21:35:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-x2150-2-mil-poly-tubing>: HTTP status code is not handled or not allowed 2026-01-24 21:35:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-18-x-36-dark-yellow-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:35:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-foot-wheel-tire-chock-security-chain>: HTTP status code is not handled or not allowed 2026-01-24 21:35:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-add-on-48in-w-x-24in-d-x-60in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:35:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dynamic-2-1-2-fluid-glycerine-filled-pressure-gage-stem-2000-psi>: HTTP status code is not handled or not allowed 2026-01-24 21:35:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/locking-storage-cabinet-36-w-x-18-d-x-72-h-with-removable-bins-2>: HTTP status code is not handled or not allowed 2026-01-24 21:35:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/table-top-lectern-24-wide-19-deep-13-high-medium-oak>: HTTP status code is not handled or not allowed 2026-01-24 21:35:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/table-top-lectern-24-wide-19-deep-13-high-natural-oak>: HTTP status code is not handled or not allowed 2026-01-24 21:35:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-machine-stand-30-wide-19-deep-36-high-mahogany>: HTTP status code is not handled or not allowed 2026-01-24 21:35:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/outdoor-enclosed-bulletin-board-cabinet-1-door-2-feet-h-x-1-5-feet-w-silver-trim-platinum>: HTTP status code is not handled or not allowed 2026-01-24 21:35:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vin-tak-tackboard-with-aluminum-trim-4-feet-h-x-8-feet-w-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:35:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/desk-top-organizer-29-wide-2-shelves-29-wide-12-deep-18-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:35:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-add-on-48in-w-x-48in-d-x-72in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:35:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coffee-bar-21-wide-17-deep-30-high-medium-oak>: HTTP status code is not handled or not allowed 2026-01-24 21:35:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-30-x-72-blue-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:35:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powered-scissor-lift-table-with-hand-control-42-x-42-2000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:35:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-universal-pad-light-weight-15-x-18-gray-100-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:36:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-add-on-60in-w-x-36in-d-x-84in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:36:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durastripe-supreme-v-2-x-100-black-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:36:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/461480-t-stat-ea5>: HTTP status code is not handled or not allowed 2026-01-24 21:36:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-30-x-48-blue-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:36:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/shelving_accessories/shelving_enclosure_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:36:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergomat-complete-smooth-3-x-12>: HTTP status code is not handled or not allowed 2026-01-24 21:36:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10x12-red-packing-list-enclosed-envelopes-pl435>: HTTP status code is not handled or not allowed 2026-01-24 21:36:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pennsylvania-heavy-duty-counting-scale-2lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:36:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-hi-vis-rain-bib-overall-ansi-class-e-yellow-xl>: HTTP status code is not handled or not allowed 2026-01-24 21:36:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/boning-5-nar-flex-blk-fibrox>: HTTP status code is not handled or not allowed 2026-01-24 21:36:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/46-1494-thermostat-type-rx-temperature-200-375-f-36-inch-capillary>: HTTP status code is not handled or not allowed 2026-01-24 21:36:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/shelving_accessories/shelving_enclosure_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:36:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torq-bar-blender-48-oz-stainless-steel-jar-toggle-switch-2-speed-2-hp>: HTTP status code is not handled or not allowed 2026-01-24 21:36:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 25627 pages (at 95 pages/min), scraped 13450 items (at 22 items/min) 2026-01-24 21:36:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-heavy-duty-boltless-shelving-starter-36inw-x-24ind-x-60inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:36:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enclosure-kit-slide-door-18-x-24-x-18-white>: HTTP status code is not handled or not allowed 2026-01-24 21:36:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1171403-valve-water-120v-3-4-inch-asco>: HTTP status code is not handled or not allowed 2026-01-24 21:36:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/shelving/shelving_accessories/shelving_enclosure_kits>: HTTP status code is not handled or not allowed 2026-01-24 21:36:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-14-x-60-dark-yellow-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:36:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-starter-48inw-x-18ind-x-84inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:36:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-starter-48inw-x-48ind-x-72inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:36:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5x12-2-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-24 21:36:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/end-base-cap-1>: HTTP status code is not handled or not allowed 2026-01-24 21:36:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durastripe-32-octagone-sign-authorized-personnel-only>: HTTP status code is not handled or not allowed 2026-01-24 21:36:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-wx18-d-gray-dolly-base>: HTTP status code is not handled or not allowed 2026-01-24 21:36:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durastripe-36x27-rectangle-empty-drum-storage-area>: HTTP status code is not handled or not allowed 2026-01-24 21:36:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-lab-coat-ultrasoft-3xl-navy-c09uplc>: HTTP status code is not handled or not allowed 2026-01-24 21:36:18 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/shelving_accessories/shelving_enclosure_kits landed on page that is not a product page. 2026-01-24 21:36:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-work-shirt-xl-ln-royal-blue-tcg01130223>: HTTP status code is not handled or not allowed 2026-01-24 21:36:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/prime-flame-resistant-long-pants-xl-gray-df2-cm-762-plp-rg-xl>: HTTP status code is not handled or not allowed 2026-01-24 21:36:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-terrain-6-wheel-hand-cart-slv1000hc6y-1000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:36:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enclosure-kit-slide-door-18-x-36-x-18-black>: HTTP status code is not handled or not allowed 2026-01-24 21:36:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-30-x-41-light-grey-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:36:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-2-double-solenoid-valve-pilot-g-1-4-220v-ac-5va-coil-black-connection>: HTTP status code is not handled or not allowed 2026-01-24 21:36:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-high-capacity-bulk-rack-with-wood-decking-72x36x120-starter>: HTTP status code is not handled or not allowed 2026-01-24 21:36:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9904-deluxe-supplied-air-shieldhelmet-no-flow-valve>: HTTP status code is not handled or not allowed 2026-01-24 21:36:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scott-100-percent-recycled-fiber-hard-roll-towels-white-8x800ft-12-carton>: HTTP status code is not handled or not allowed 2026-01-24 21:36:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durastripe-24x18-rectangle-stop>: HTTP status code is not handled or not allowed 2026-01-24 21:36:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1381252-thermometer-infrared-w-boot-for-taylor-precision-products-lp>: HTTP status code is not handled or not allowed 2026-01-24 21:36:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enclosure-kit-slide-door-24-x-36-x-18-light-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:36:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7100-contour-knee-pad>: HTTP status code is not handled or not allowed 2026-01-24 21:36:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kit2sc112x12-12-cal-arc-flash-kit-w-short-coat-bib-overall-in-ultrasoft-2xl-sz-12>: HTTP status code is not handled or not allowed 2026-01-24 21:36:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enclosure-kit-slide-door-24-x-60-x-18-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:36:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-open-cntr-dbl-solenoid-valve-pilot-spr-cntred-1-2-nptf-24v-dc-2w-coil-black-conn>: HTTP status code is not handled or not allowed 2026-01-24 21:36:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-open-cntr-dbl-solenoid-valve-pilot-spring-cntred-1-4-nptf-24v-dc-3w-coil-led-conn>: HTTP status code is not handled or not allowed 2026-01-24 21:36:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-high-capacity-bulk-rack-with-steel-decking-60x48x72-add-on>: HTTP status code is not handled or not allowed 2026-01-24 21:36:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/881210-pressure-dual-control-for-johnson-controls>: HTTP status code is not handled or not allowed 2026-01-24 21:36:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-open-cntr-dbl-solenoid-valve-pilot-spr-cntred-1-8-nptf-220v-ac-5va-coil-led-conn>: HTTP status code is not handled or not allowed 2026-01-24 21:36:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-pres-cntr-dbl-solenoid-vlv-pilot-spr-cntred-1-4-nptf-24v-dc-2w-coil-led-conn>: HTTP status code is not handled or not allowed 2026-01-24 21:36:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-starter-42inw-x-30ind-x-96inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:36:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-12-x-48-light-blue-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:36:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/st-drawer-layout-12-compartments-2-h>: HTTP status code is not handled or not allowed 2026-01-24 21:36:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-unlined-bib-overall-32-x-30-navy-bib6dnv32xl30>: HTTP status code is not handled or not allowed 2026-01-24 21:36:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-high-capacity-bulk-rack-without-decking-96x36x120-starter>: HTTP status code is not handled or not allowed 2026-01-24 21:36:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72006-titan-18-hvls-fan-18-ft-dia-1-ph-200-240v>: HTTP status code is not handled or not allowed 2026-01-24 21:36:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/511206-pilot-assembly>: HTTP status code is not handled or not allowed 2026-01-24 21:36:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wgr44-d-double-headed-rack-protector-44>: HTTP status code is not handled or not allowed 2026-01-24 21:36:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wgr48l-l-left-side-rack-protector-lite-48>: HTTP status code is not handled or not allowed 2026-01-24 21:36:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dw-drawer-layout-9-compartments-5-h>: HTTP status code is not handled or not allowed 2026-01-24 21:36:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-high-capacity-bulk-rack-with-wire-decking-72x36x96-starter>: HTTP status code is not handled or not allowed 2026-01-24 21:36:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-2-single-solenoid-valve-pilot-spr-return-1-4-nptf-24v-ac-5va-coil-led-connection>: HTTP status code is not handled or not allowed 2026-01-24 21:36:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-high-capacity-bulk-rack-with-wire-decking-96x24x96-starter>: HTTP status code is not handled or not allowed 2026-01-24 21:36:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/translucent-liner-21-x-42>: HTTP status code is not handled or not allowed 2026-01-24 21:36:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-unlined-bib-overall-34-x-32-brown-bib6dcm34x32>: HTTP status code is not handled or not allowed 2026-01-24 21:36:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2551010-blade-for-shaver-specialty-king-cutter-vegetable-potato-cutter>: HTTP status code is not handled or not allowed 2026-01-24 21:36:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enclosure-kit-slide-door-18-x-48-x-18-white>: HTTP status code is not handled or not allowed 2026-01-24 21:36:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-2-spt-1-brown>: HTTP status code is not handled or not allowed 2026-01-24 21:36:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-g-1-8-double-solenoid-valve-pilot-220v-ac-5va-coil-black-connection>: HTTP status code is not handled or not allowed 2026-01-24 21:36:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-2-single-solenoid-pilot-valve-spring-return-1-4-24v-dc-2w-coil-black-connection>: HTTP status code is not handled or not allowed 2026-01-24 21:36:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010808-thermostat-for-vollrath-idea-company>: HTTP status code is not handled or not allowed 2026-01-24 21:36:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kit2cv11ngb-sm-12-cal-ultrasoft-arc-flash-kit-fr-coverall-balaclava-sm-no-gloves>: HTTP status code is not handled or not allowed 2026-01-24 21:36:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zetex-plus-mitten-14l-yellow-jumbo-m60zprh01014>: HTTP status code is not handled or not allowed 2026-01-24 21:36:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/amana-06-cu-ft-1800-watt-hd-commercial-microwave-1>: HTTP status code is not handled or not allowed 2026-01-24 21:36:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-powered-clamp-2-inch-bung-agitator-1-half-hp-2-4-cl-propellers>: HTTP status code is not handled or not allowed 2026-01-24 21:36:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ball-mountkit-2-inch-drop-2-inch-ball-chrome>: HTTP status code is not handled or not allowed 2026-01-24 21:36:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/preassembled-heat-cable-fg1-12p-12-ft-120v>: HTTP status code is not handled or not allowed 2026-01-24 21:36:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/k93215-filter-regulator-38-npt>: HTTP status code is not handled or not allowed 2026-01-24 21:36:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-economy-bookcase-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:36:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-inch-black-grommet-for-round-recessed-lights>: HTTP status code is not handled or not allowed 2026-01-24 21:36:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-closed-single-solenoid-valve-ext-pilot-g-1-8-110v-ac-5va-coil-black-connection>: HTTP status code is not handled or not allowed 2026-01-24 21:36:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f2024-1-outlet-x-2-12-female-npt>: HTTP status code is not handled or not allowed 2026-01-24 21:37:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/461102-thermosta-tk-5-16-x-7-1-2-42>: HTTP status code is not handled or not allowed 2026-01-24 21:37:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kit2cv11bm08-12-cal-ultrasoft-arc-flash-kit-fr-coverall-balaclava-m-glove-sz-08>: HTTP status code is not handled or not allowed 2026-01-24 21:37:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-g-1-2-double-solenoid-valve-pilot-24v-dc-2w-coil-black-connection>: HTTP status code is not handled or not allowed 2026-01-24 21:37:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-starter-36inw-x-24ind-x-96inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:37:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/staplers_staples/staples already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:37:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-heavy-duty-boltless-shelving-starter-48inw-x-18ind-x-60inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:37:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/synchro-leather-task-stool-360-footrest-with-armrests-1>: HTTP status code is not handled or not allowed 2026-01-24 21:37:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-inch-x-36-inch-x-36-inch-corrugated-boxes>: HTTP status code is not handled or not allowed 2026-01-24 21:37:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/421910-rocker-switch-lighted-for-american-permanent-ware>: HTTP status code is not handled or not allowed 2026-01-24 21:37:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-32-x-1-4-phillips-flat-head-machine-screw-pkg-of-100-dwa66008>: HTTP status code is not handled or not allowed 2026-01-24 21:37:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-gasket-813-15-1-1-2>: HTTP status code is not handled or not allowed 2026-01-24 21:37:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-4-4-work-shirt-2xl-t-navy-blue-df2-cm-450c-ls-nb-2xlt>: HTTP status code is not handled or not allowed 2026-01-24 21:37:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-closed-single-solenoid-valve-ext-pilot-g-1-8-12v-dc-3w-coil-black-connection>: HTTP status code is not handled or not allowed 2026-01-24 21:37:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 25722 pages (at 95 pages/min), scraped 13466 items (at 16 items/min) 2026-01-24 21:37:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-pressure-cntr-g1-8-dbl-solenoid-valve-pilot-spr-cntred-24v-dc-2w-coil-black-conn>: HTTP status code is not handled or not allowed 2026-01-24 21:37:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultrawedge-wedge-anchor-3-8-16-x-5-304-stainless-steel-pkg-of-50-616080>: HTTP status code is not handled or not allowed 2026-01-24 21:37:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-high-three-way-kit-for-two-46-in-low-panel-with-cable>: HTTP status code is not handled or not allowed 2026-01-24 21:37:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-add-on-60inw-x-36ind-x-96inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:37:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/68694-4-mm-x-4-mm-juliennedisc-r28-for-robot-coupe>: HTTP status code is not handled or not allowed 2026-01-24 21:37:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/structural-plastic-shelving-66x16x72-adjustable>: HTTP status code is not handled or not allowed 2026-01-24 21:37:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/staplers_staples/staples landed on page that is not a product page. 2026-01-24 21:37:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2241226-yoke-frykutter-m-n55450-for-nemco>: HTTP status code is not handled or not allowed 2026-01-24 21:37:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-x-5-x-4-corrugated-mailers>: HTTP status code is not handled or not allowed 2026-01-24 21:37:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sleeve-anchor-1-2-x-2-1-4-hex-nut-head-304-stainless-steel-pkg-of-25-618040>: HTTP status code is not handled or not allowed 2026-01-24 21:37:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-industrial-fog-nozzle-520-2011-2-npsh-brass>: HTTP status code is not handled or not allowed 2026-01-24 21:37:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009498-extension-probe-2-1-2-inch-for-cleveland-range>: HTTP status code is not handled or not allowed 2026-01-24 21:37:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fr-control-2-0-long-sleeve-t-shirt-xl-khaki-c52jksrlsxl>: HTTP status code is not handled or not allowed 2026-01-24 21:37:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/doghouse-hotdog-cooker>: HTTP status code is not handled or not allowed 2026-01-24 21:37:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-starter-96inw-x-36ind-x-72inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:37:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-starter-96inw-x-36ind-x-96inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:37:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-hose-reel-cover-138-3-18-x-6-10-oz-red-vinyl-for-1430-1>: HTTP status code is not handled or not allowed 2026-01-24 21:37:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-polyester-mesh-wristlet-xl-black-s01mc9xxl>: HTTP status code is not handled or not allowed 2026-01-24 21:37:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/resin-thermal-transfer-ribbon-80mm-x-300m-axr7-plus-outside-ink-resin>: HTTP status code is not handled or not allowed 2026-01-24 21:37:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25-drawcord-cotton-duck-natural-straight>: HTTP status code is not handled or not allowed 2026-01-24 21:37:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-vis-work-shirt-type-r-class-3-m-t-fluorescent-yellow-df2-ax3-324ls-hy-mdt>: HTTP status code is not handled or not allowed 2026-01-24 21:37:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-dbl-solenoid-vlv-ext-pilot-open-cntr-spr-cntred-1-4-nptf-110v-ac-5va-coil-black>: HTTP status code is not handled or not allowed 2026-01-24 21:37:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/open-sided-wire-exchange-truck-five-wire-shelves-800-lb-cap-3>: HTTP status code is not handled or not allowed 2026-01-24 21:37:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/321782-jar-poly-7-inch-shallow-for-server-products>: HTTP status code is not handled or not allowed 2026-01-24 21:37:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gardner-bender-repl-leads-gdt-292a-gdt-293a>: HTTP status code is not handled or not allowed 2026-01-24 21:37:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-cal-ultrasoft-flame-resistant-coverall-2xl-x-32-navy-c88up2xl32>: HTTP status code is not handled or not allowed 2026-01-24 21:37:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-closed-g-1-2-single-solenoid-valve-pilot-spr-return-24v-dc-3w-coil-led-connection>: HTTP status code is not handled or not allowed 2026-01-24 21:37:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-58-208-230v-850rpm>: HTTP status code is not handled or not allowed 2026-01-24 21:37:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/homak-34-professional-3-drawer-service-cart-black>: HTTP status code is not handled or not allowed 2026-01-24 21:37:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-58-enclosed-blower-115v-1075rpm-14hp>: HTTP status code is not handled or not allowed 2026-01-24 21:37:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-holder-cartridge-kit-d50228-4-pk>: HTTP status code is not handled or not allowed 2026-01-24 21:37:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-open-g-1-2-single-solenoid-valve-pilot-spr-return-24v-ac-5va-coil-led-connection>: HTTP status code is not handled or not allowed 2026-01-24 21:37:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-2-single-solenoid-valve-pilot-spr-return-1-4-nptf-24v-dc-3w-coil-blk-connection>: HTTP status code is not handled or not allowed 2026-01-24 21:37:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-add-on-72inw-x-36ind-x-72inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:37:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-s-brass-pot-kettle-filler>: HTTP status code is not handled or not allowed 2026-01-24 21:37:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-heavy-duty-boltless-shelving-add-on-36inw-x-18ind-x-60inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:37:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-s-brass-easyinstall-lavatory-faucet>: HTTP status code is not handled or not allowed 2026-01-24 21:37:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/haws-horizontal-or-vertical-drench-shower-1012-ss-showerhead>: HTTP status code is not handled or not allowed 2026-01-24 21:37:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-work-shirt-5x-gray-tcg01150234>: HTTP status code is not handled or not allowed 2026-01-24 21:37:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/262835-bearing-for-hobart>: HTTP status code is not handled or not allowed 2026-01-24 21:37:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sw-10l-18w-straight-galvanized-steel-skatewheel-conveyor-16-bf-16-wpf>: HTTP status code is not handled or not allowed 2026-01-24 21:37:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-open-1-8-nptf-single-solenoid-valve-pilot-spr-return-110v-ac-5va-coil-blk-conn>: HTTP status code is not handled or not allowed 2026-01-24 21:37:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5411001-jar-4-sided-red-w-lid-75-oz-for-blendtec>: HTTP status code is not handled or not allowed 2026-01-24 21:37:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-shaft-1625rpm-115v-13hp>: HTTP status code is not handled or not allowed 2026-01-24 21:37:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-starter-36inw-x-18ind-x-72inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:37:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-work-shirt-3xl-ln-tan-tcg01120229>: HTTP status code is not handled or not allowed 2026-01-24 21:37:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-single-shaft-open-blower-115v-1625rpm-112hp>: HTTP status code is not handled or not allowed 2026-01-24 21:37:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-capacity-33-lbs-cylinders-heavy-duty-steel-gas-cylinder-cage-30w-x-70d-x-30h-white>: HTTP status code is not handled or not allowed 2026-01-24 21:37:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-18-ga-36-wx30-dx97-h-open-clip-style-7-shelf>: HTTP status code is not handled or not allowed 2026-01-24 21:37:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stanchion-kit-red-6pk-50-of-2-chain-w-c-hooks-incl-2-5-pole-14-base-40-height>: HTTP status code is not handled or not allowed 2026-01-24 21:37:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lightweight-flame-resistant-t-shirt-3xl-navy-blue-df2-cm-446ts-nb-3xl>: HTTP status code is not handled or not allowed 2026-01-24 21:37:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-dbl-solenoid-valve-pilot-spring-cntred-closed-cntr-g-1-4-12v-dc-3w-coil-led-conn>: HTTP status code is not handled or not allowed 2026-01-24 21:37:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-closed-1-2-nptf-single-solenoid-valve-pilot-spr-return-24v-dc-3w-coil-blk-conn>: HTTP status code is not handled or not allowed 2026-01-24 21:37:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-flame-resistant-work-shirt-m-light-blue-tcgsswn00119mdrg00>: HTTP status code is not handled or not allowed 2026-01-24 21:37:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010965-screw-carrier-for-hobart>: HTTP status code is not handled or not allowed 2026-01-24 21:37:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-gallon-wheeled-overpack-salvage-drum-with-lid-a95over-wd-9-polyethylene-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:37:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-dbl-solenoid-valve-pilot-spring-cntred-closed-cntr-g-1-2-24v-ac-5va-coil-led-conn>: HTTP status code is not handled or not allowed 2026-01-24 21:37:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-3740-charge-port-4-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:37:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-58-outdoor-ball-fan-115v-825rpm-13hp>: HTTP status code is not handled or not allowed 2026-01-24 21:37:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pegboard-hobby-craft-organizer-storage-kit-yellow-white-32-x-32-x-9>: HTTP status code is not handled or not allowed 2026-01-24 21:37:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-dock-seal-40-ounce-8-w-x-10-h-hd-wear-pleats>: HTTP status code is not handled or not allowed 2026-01-24 21:38:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/264264-reducing-coupling-7-8-inch-ftg-x-3-4-inch>: HTTP status code is not handled or not allowed 2026-01-24 21:38:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/264269-tee-3-8-inch>: HTTP status code is not handled or not allowed 2026-01-24 21:38:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-add-on-72inw-x-48ind-x-72inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:38:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/welded-cabinet-drawer-combination-36w-x-18d-x-66h-1-drawer2-shelf-cabinet-light-grey>: HTTP status code is not handled or not allowed 2026-01-24 21:38:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pegboard-pack-2-panels-yellow-metal-32-x-32-x-3-4>: HTTP status code is not handled or not allowed 2026-01-24 21:38:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-dbl-solenoid-valve-pilot-spr-cntred-closed-cntr-g1-8-24v-dc-3w-coil-led-conn>: HTTP status code is not handled or not allowed 2026-01-24 21:38:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-2-double-solenoid-valve-pilot-1-4-nptf-110v-ac-5va-coil-led-connection>: HTTP status code is not handled or not allowed 2026-01-24 21:38:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metal-indoor-outdoor-stackable-chair-red>: HTTP status code is not handled or not allowed 2026-01-24 21:38:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metal-indoor-outdoor-stackable-chair-orange>: HTTP status code is not handled or not allowed 2026-01-24 21:38:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-2-double-solenoid-valve-pilot-1-4-nptf-220v-ac-5va-coil-black-connection>: HTTP status code is not handled or not allowed 2026-01-24 21:38:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-add-on-72inw-x-30ind-x-60inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:38:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quick-adjust-wire-shelf-truck-60x24x69-1200-pound-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:38:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 25823 pages (at 101 pages/min), scraped 13493 items (at 27 items/min) 2026-01-24 21:38:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pegboard-utility-tool-storage-kit-galvanized-red-32-x-32-x-6>: HTTP status code is not handled or not allowed 2026-01-24 21:38:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-2-single-solenoid-valve-ext-pilot-spring-return-g-1-8-24v-dc-2w-coil-black-conn>: HTTP status code is not handled or not allowed 2026-01-24 21:38:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quick-adjust-wire-shelving-48x24x86>: HTTP status code is not handled or not allowed 2026-01-24 21:38:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quick-adjust-hook-4-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:38:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pegboard-standard-tool-storage-kit-orange-blue-48-x-32-x-9>: HTTP status code is not handled or not allowed 2026-01-24 21:38:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-person-box-locker-with-legs-12w-x-18d-x-12h-assembled-light-grey>: HTTP status code is not handled or not allowed 2026-01-24 21:38:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-heavy-duty-boltless-shelving-starter-36inw-x-12ind-x-96inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:38:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009116-720719-baso-kit-36-48>: HTTP status code is not handled or not allowed 2026-01-24 21:38:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roll-of-clear-packaging-tape-3-x-165>: HTTP status code is not handled or not allowed 2026-01-24 21:38:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/262855-key1-8-x-1-1-8-for-hobart>: HTTP status code is not handled or not allowed 2026-01-24 21:38:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-heavy-duty-boltless-shelving-starter-36inw-x-18ind-x-96inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:38:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nv222-portable-balance-220g-x-0-01g-5-7-dia-platform>: HTTP status code is not handled or not allowed 2026-01-24 21:38:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-280-1-light-ceiling-flush-mount-w-alabaster-glass-old-bronze-11-5w-x-4-5h>: HTTP status code is not handled or not allowed 2026-01-24 21:38:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:38:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mixing-paddle-4-75x9-75-paddle-48x1-tubular-handle>: HTTP status code is not handled or not allowed 2026-01-24 21:38:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-32-x-5-8-security-machine-screw-flat-hex-socket-head-alloy-steel-black-oxide-ft-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 21:38:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-36-w-x-24-d-x-30-to-36-h-adj-height-shop-stand-16-ga-430-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:38:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-w-x-20-h-x-1-d-pleated-merv-8-standard-capacity-air-filter-global-industrial>: HTTP status code is not handled or not allowed 2026-01-24 21:38:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/faultless-swivel-plate-caster-4-polyolefin-wheel-1>: HTTP status code is not handled or not allowed 2026-01-24 21:38:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/281889-lock-ring-pushbutton-for-accu-temp>: HTTP status code is not handled or not allowed 2026-01-24 21:38:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/faultless-stainless-steel-swivel-plate-caster-5-tpr-wheel-with-brake>: HTTP status code is not handled or not allowed 2026-01-24 21:38:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-5044-patton-3-light-ceiling-semi-flush-frosted-brushed-nickel-13w-x-11h>: HTTP status code is not handled or not allowed 2026-01-24 21:38:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/utility-scoop-84-oz-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:38:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-swing-panel-pegboard-24w-x-1-12d-x-48h-black>: HTTP status code is not handled or not allowed 2026-01-24 21:38:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-plate-caster-8-plastic-wheel-800-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:38:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-king-sk3000-pallet-rack-108w-x-42d-x-96h-4040-lbs-cap-per-level-add-on-unit>: HTTP status code is not handled or not allowed 2026-01-24 21:38:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wide-span-rack-60x24x96-with-3-shelves-wood-deck-1200-lb-capacity-per-level>: HTTP status code is not handled or not allowed 2026-01-24 21:38:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8ab-x-3-8-security-sheet-metal-screw-pan-spanner-head-18-8-stainless-steel-ft-usa-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 21:38:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots>: HTTP status code is not handled or not allowed 2026-01-24 21:38:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2721309-clutch-feeder-drive-assembly-for-nieco-corp>: HTTP status code is not handled or not allowed 2026-01-24 21:38:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spice-womens-safety-work-boots-steel-toe-7h-size-95w-mystic-dune-brown>: HTTP status code is not handled or not allowed 2026-01-24 21:38:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inertia-womens-safety-work-boots-steel-toe-7h-size-7w-matte-black>: HTTP status code is not handled or not allowed 2026-01-24 21:38:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/horizon-womens-safety-work-boots-alloy-toe-8h-size-12-desert-tan>: HTTP status code is not handled or not allowed 2026-01-24 21:38:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010928-shaft-upper-wheel-for-hobart>: HTTP status code is not handled or not allowed 2026-01-24 21:38:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/digital-power-distribution-unit-switched-rm-w-plug-retention-20a>: HTTP status code is not handled or not allowed 2026-01-24 21:38:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/77-136-1-light-wall-mount-w-glass-cylinder-satin-aluminum-4-75w-x-8h>: HTTP status code is not handled or not allowed 2026-01-24 21:38:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-green-epoxy-wire-shelving-36w-x-12-d-x-86-h>: HTTP status code is not handled or not allowed 2026-01-24 21:38:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nova-eh-20-womens-safety-work-boots-steel-toe-18h-size-9-black>: HTTP status code is not handled or not allowed 2026-01-24 21:38:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fr-mesh-vest-c2-lime-s-m>: HTTP status code is not handled or not allowed 2026-01-24 21:38:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-green-epoxy-wire-shelving-add-on-48w-x-12-d-x-74-h>: HTTP status code is not handled or not allowed 2026-01-24 21:38:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:38:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-15-16-inch-2-3-16-inch-stainless-steel-ultra-t-bolt-clamp-ut-193>: HTTP status code is not handled or not allowed 2026-01-24 21:38:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnetic-glass-whiteboard-72-x-48-black>: HTTP status code is not handled or not allowed 2026-01-24 21:38:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-x-5-8-security-machine-screw-flat-torx-plus-head-18-8-stainless-steel-ft-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 21:38:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-7-16-inch-3-3-4-inch-stainless-steel-ultra-t-bolt-clamp-ut-343>: HTTP status code is not handled or not allowed 2026-01-24 21:38:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-4974-boxwood-1-light-outdoor-post-clear-beveled-white-8w-x-19-25h>: HTTP status code is not handled or not allowed 2026-01-24 21:38:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spac-trac-5-unit-storage-shelving-chrome-36x21x74>: HTTP status code is not handled or not allowed 2026-01-24 21:38:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wide-span-rack-48x36x84-with-3-shelves-no-deck-1200-lb-capacity-per-level>: HTTP status code is not handled or not allowed 2026-01-24 21:38:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rotating-multi-directional-machine-dolly-rl8-6-13817>: HTTP status code is not handled or not allowed 2026-01-24 21:38:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/77-533-1-light-ceiling-mount-w-white-glass-cylinder-4-25w-x-8h>: HTTP status code is not handled or not allowed 2026-01-24 21:38:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibration-shock-absorption-mount-conical-1-02-h-205-max-load-254-32-5-16x18-15-9-55>: HTTP status code is not handled or not allowed 2026-01-24 21:38:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots>: HTTP status code is not handled or not allowed 2026-01-24 21:38:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/265862-fence-lower-carriage-stainless-steel-for-hobart>: HTTP status code is not handled or not allowed 2026-01-24 21:38:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/valence-womens-safety-work-boots-steel-toe-7h-size-85-vegan-midnight-black>: HTTP status code is not handled or not allowed 2026-01-24 21:38:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-8-sds-plus-rotary-hammer-w-vc>: HTTP status code is not handled or not allowed 2026-01-24 21:38:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kwik-sc-galvanized-steel-kwik-kit-railing-system-6-feet-by-6-feet-corner-kit>: HTTP status code is not handled or not allowed 2026-01-24 21:38:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fk22-2x2-flange-kit-for-lay-in-troffer-accessory-for-jt-series>: HTTP status code is not handled or not allowed 2026-01-24 21:38:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tearaway-standard-scratch-guard-bulk-roll-8w-x-8h-clear-18-strips-per-roll>: HTTP status code is not handled or not allowed 2026-01-24 21:38:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crane-forklift-emptying-device-570-lbs-2-642-842-0>: HTTP status code is not handled or not allowed 2026-01-24 21:38:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/walkie-tugger-t15>: HTTP status code is not handled or not allowed 2026-01-24 21:38:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pyramid-sign-base-with-wheels-and-98-foot-high-post-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:38:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/77-898-1-light-post-coach-lantern-w-brass-trimmed-acrylic-panels-black-7-75w-x-12-5h>: HTTP status code is not handled or not allowed 2026-01-24 21:39:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plc-03610-32oz-natural-hdpe-wide-mouth-round-bottle-with-53-400-white-pp-cap-case-of-12>: HTTP status code is not handled or not allowed 2026-01-24 21:39:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/235998-oil-free-round-aluminum-drying-pans-102mm-dia-x-5-16d-case-of-500>: HTTP status code is not handled or not allowed 2026-01-24 21:39:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2721317-belt-wire-mpb94-for-nieco-corp>: HTTP status code is not handled or not allowed 2026-01-24 21:39:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ride-on-propane-sweeper-with-warning-beacon-km-150-500-r-lpg-6-987-085-0>: HTTP status code is not handled or not allowed 2026-01-24 21:39:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/horizon-womens-safety-work-boots-alloy-toe-8h-size-6-desert-tan>: HTTP status code is not handled or not allowed 2026-01-24 21:39:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2661049-o-ring-3-8-inch-od-for-taylor-freezer>: HTTP status code is not handled or not allowed 2026-01-24 21:39:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/free-standing-clothes-rack-3-shelf-48-w-x-24-d-x-74-h>: HTTP status code is not handled or not allowed 2026-01-24 21:39:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010737-label-for-star-manufacturing>: HTTP status code is not handled or not allowed 2026-01-24 21:39:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/76-600-1-light-ceiling-flush-mount-small-white-mushroom-old-bronze-7-5w-x-5h>: HTTP status code is not handled or not allowed 2026-01-24 21:39:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010807-hinge-kit-for-vollrath-idea-company>: HTTP status code is not handled or not allowed 2026-01-24 21:39:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-stem-caster-3-dia-element-with-1-3-8-h-stem>: HTTP status code is not handled or not allowed 2026-01-24 21:39:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011127-kason-11245000076-hinge-1-3-4-ofst-camlift>: HTTP status code is not handled or not allowed 2026-01-24 21:39:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-expansion-stem-caster-4-dia-element-derlin-bearings-top-lk-brk>: HTTP status code is not handled or not allowed 2026-01-24 21:39:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 25920 pages (at 97 pages/min), scraped 13518 items (at 25 items/min) 2026-01-24 21:39:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5031015-filter-condenser-10-inch-x-12-inch-for-hoshizaki-of-america>: HTTP status code is not handled or not allowed 2026-01-24 21:39:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lux-ssetpbk-o4x-class-2-black-bottom-t-shirt-w-pocket-orange-4x>: HTTP status code is not handled or not allowed 2026-01-24 21:39:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kwikmb-window-mount-installation-kits-for-kuhl-ym-and-em-models>: HTTP status code is not handled or not allowed 2026-01-24 21:39:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kwcfm-replacement-carbon-filters-3-pack-for-kuhl-sm-em-and-ym-models>: HTTP status code is not handled or not allowed 2026-01-24 21:39:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/71-round-blow-molded-plastic-folding-table-speckled-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:39:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hss-tmx-jewelers-saw-3-inch-dia-x-032-inch-face-x-1-2-inch-hole-x-168-teeth>: HTTP status code is not handled or not allowed 2026-01-24 21:39:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-acrylic-desk-divider-36-x-24-freestanding>: HTTP status code is not handled or not allowed 2026-01-24 21:39:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/science-lab-table-30h-with-black-phenolic-resin-top-60-24>: HTTP status code is not handled or not allowed 2026-01-24 21:39:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-boltless-shelving-starter-36inw-x-12ind-x-96inh-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:39:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/120-x-34-extra-long-folding-assembly-bench>: HTTP status code is not handled or not allowed 2026-01-24 21:39:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-4911-adams-1-light-outdoor-hanging-frosted-white-6-125w-x-15-75h>: HTTP status code is not handled or not allowed 2026-01-24 21:39:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drink-mat-20523-mlb-los-angeles-dodgers-3-1-4-x-24-x-1>: HTTP status code is not handled or not allowed 2026-01-24 21:39:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1961080-switch-toggle-spdt-on-on-tab-for-frymaster>: HTTP status code is not handled or not allowed 2026-01-24 21:39:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-bench-drawer-with-lock>: HTTP status code is not handled or not allowed 2026-01-24 21:39:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-shank-straight-flute-tmx-chucking-reamer-decimal-size-249-6-flute>: HTTP status code is not handled or not allowed 2026-01-24 21:39:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c426v1-cap-start-motor>: HTTP status code is not handled or not allowed 2026-01-24 21:39:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-boltless-shelving-add-on-48inw-x-24ind-x-72inh-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:39:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011290-harvest-assist-kit-for-scotsman>: HTTP status code is not handled or not allowed 2026-01-24 21:39:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/letter-file-storage-drawer-locking-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:39:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/starter-unit-legal-31-inch-45-1-2-inch-71-1-2-inch-6-opening-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:39:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-boltless-shelving-add-on-36inw-x-24ind-x-96inh-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:39:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/starter-unit-legal-31-inch-45-1-2-inch-82-inch-7-opening-black>: HTTP status code is not handled or not allowed 2026-01-24 21:39:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-edge-work-bench-top-shop-top-48-w-x-30-d-x-1-3-4-thick>: HTTP status code is not handled or not allowed 2026-01-24 21:39:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-4928-banyan-1-light-outdoor-post-clear-water-rustic-bronze-6-125w-x-17-25h>: HTTP status code is not handled or not allowed 2026-01-24 21:39:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010814-thermometer-digital-for-delfield>: HTTP status code is not handled or not allowed 2026-01-24 21:39:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-adjustable-flip-top-table-crank-handle-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:39:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-boltless-shelving-add-on-36inw-x-24ind-x-72inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:39:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8781-ultra-containment-wall-69-feet-by-69-feet-by-1-foot-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:39:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8762-ultra-containment-wall-2-feet-high-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:39:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hss-tmx-plain-teeth-side-milling-cutter-5-inch-dia-x-3-8-inch-face-x-1-1-4-inch-hole-x-24-teeth>: HTTP status code is not handled or not allowed 2026-01-24 21:39:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-3265-empire-2-light-vanity-wall-frosted-white-brushed-nickel-14-875w-x-6-125h>: HTTP status code is not handled or not allowed 2026-01-24 21:39:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-boltless-shelving-add-on-36inw-x-24ind-x-96inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:39:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-24-w-11-step-steel-rolling-ladder-21-d-top-step>: HTTP status code is not handled or not allowed 2026-01-24 21:39:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-liner-for-small-ultra-filter-pad-4-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:39:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-m-style-brass-coupler-3-8-x-1-4-female-npt-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 21:39:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-310-6-light-vanity-wall-polished-brass-36w-x-4-75h>: HTTP status code is not handled or not allowed 2026-01-24 21:39:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/neoprene-machinery-leveling-mount-3-long-3-wide-2>: HTTP status code is not handled or not allowed 2026-01-24 21:39:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/outdoor-ratio-led-floodlight-14000l-124w-50k-wide-dist-knuckle-mt-120-277v>: HTTP status code is not handled or not allowed 2026-01-24 21:39:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-awg-3-cond-45-ft-20-amp-duplex-gfci-outlet-with-cord-29-lbs>: HTTP status code is not handled or not allowed 2026-01-24 21:39:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-shelf-single-rivet-laminate-deck-48inw-x-18ind-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:39:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cyclone-hand-chain-hoist-on-plain-trolley-2-ton-20-ft-lift>: HTTP status code is not handled or not allowed 2026-01-24 21:39:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gs-restroom-bowl-cleaner-concentrate-2-oz-packs-2385108>: HTTP status code is not handled or not allowed 2026-01-24 21:39:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lodestar-electric-chain-hoist-w-chain-container-3-ton-10-ft-lift-18-11-fpm-460v>: HTTP status code is not handled or not allowed 2026-01-24 21:39:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-24v-10a-battery-charger-641411>: HTTP status code is not handled or not allowed 2026-01-24 21:39:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3500-series-safe-t-alert-back-up-alarm-87-112db-12-24-volts-self-adjusting>: HTTP status code is not handled or not allowed 2026-01-24 21:39:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-23-33-medium-duty-rigid-caster-33-4-rt-rubber-mold-on-4-dia-200-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:39:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cyclone-hand-chain-hoist-on-plain-trolley-10-ton-10-ft-lift>: HTTP status code is not handled or not allowed 2026-01-24 21:39:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electronic-code-switch-psl-12-90v-1-code>: HTTP status code is not handled or not allowed 2026-01-24 21:39:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alliance-hose-rubber-snubber-hose-3-8x24-with-1-4-male-and-male-swivel>: HTTP status code is not handled or not allowed 2026-01-24 21:39:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-boltless-shelving-starter-48inw-x-18ind-x-72inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:39:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-boltless-shelving-starter-48inw-x-18ind-x-84inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:39:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/serrated-24-w-5-step-steel-rolling-ladder-10-d-top-step-kdsr105242>: HTTP status code is not handled or not allowed 2026-01-24 21:39:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/serrated-24-w-9-step-steel-rolling-ladder-10-d-top-step-kdsr109242>: HTTP status code is not handled or not allowed 2026-01-24 21:39:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-packaging-tape>: HTTP status code is not handled or not allowed 2026-01-24 21:39:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-boltless-shelving-starter-36inw-x-18ind-x-96inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:39:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storage-tank-self-standing-220-gallon-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:39:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dry-erase-fabric-combination-board-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:39:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inside-stational-deck-dome-oven-square-exterior-59-gas-97500-btu-120v-digital-control>: HTTP status code is not handled or not allowed 2026-01-24 21:39:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/meteorlite-5-high-profile-strobe-light-12-80-volts-permanent-mount-amber>: HTTP status code is not handled or not allowed 2026-01-24 21:39:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m18-cordless-xc-high-capacity-lithium-ion-battery>: HTTP status code is not handled or not allowed 2026-01-24 21:39:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergi-lite-wireguard-ksm27-2-f>: HTTP status code is not handled or not allowed 2026-01-24 21:39:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-boltless-shelving-add-on-36inw-x-12ind-x-60inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:40:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyurethane-aluminum-wheel-ac-804-ra-4-dia-1-1-2w-675-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:40:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergi-lite-circuit-board-12ksm54-2>: HTTP status code is not handled or not allowed 2026-01-24 21:40:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-wall-container-mesh-23-3-4-x15-3-4-x8-1-4>: HTTP status code is not handled or not allowed 2026-01-24 21:40:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8ft-aluminum-team-bench-with-back-galvanized-frame-in-ground-mount>: HTTP status code is not handled or not allowed 2026-01-24 21:40:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-wall-container-mesh-sides-solid-base-23-3-4-x15-3-4-x16-1-8>: HTTP status code is not handled or not allowed 2026-01-24 21:40:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15ft-aluminum-team-bench-with-back-galvanized-frame-portable>: HTTP status code is not handled or not allowed 2026-01-24 21:40:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e-series-platform-truck-e-98-q-3060-poe-hardwood-30-x-60-8-wheels-1400-lb-cap>: HTTP status code is not handled or not allowed 2026-01-24 21:40:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6ft-aluminum-team-bench-without-back-galvanized-frame-in-ground-mount>: HTTP status code is not handled or not allowed 2026-01-24 21:40:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/561390-service-valve>: HTTP status code is not handled or not allowed 2026-01-24 21:40:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-bicycle-300-cap-low-gravity-front-basket-men-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:40:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/dollies/lever_dollies already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:40:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blue-curtain-wall-partition-with-clear-vision-strip-12-x-10>: HTTP status code is not handled or not allowed 2026-01-24 21:40:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-boltless-shelving-add-on-48inw-x-12ind-x-96inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:40:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 26015 pages (at 95 pages/min), scraped 13539 items (at 21 items/min) 2026-01-24 21:40:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/266234-socket-leg-steel-weld-mount>: HTTP status code is not handled or not allowed 2026-01-24 21:40:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wes-1000>: HTTP status code is not handled or not allowed 2026-01-24 21:40:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/461576-control-temperature>: HTTP status code is not handled or not allowed 2026-01-24 21:40:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vinyl-upholstered-shop-stool-steel-frame-1>: HTTP status code is not handled or not allowed 2026-01-24 21:40:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/881469-valve-body-expansion-valve-for-parker-hannifin>: HTTP status code is not handled or not allowed 2026-01-24 21:40:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-ft-square-outdoor-tree-bench-expanded-metal-surface-mount-black>: HTTP status code is not handled or not allowed 2026-01-24 21:40:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cloth-dry-eraser-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-24 21:40:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/versa-2-shelf-mobile-rack-cabinet-with-locking-panels-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:40:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/carts-trucks/dollies/lever_dollies>: HTTP status code is not handled or not allowed 2026-01-24 21:40:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-3-shelf-boltless-shelving-add-on-48inw-x-12ind-x-60inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:40:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-uncaged-fixed-access-ladder-yellow-5>: HTTP status code is not handled or not allowed 2026-01-24 21:40:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-3-shelf-boltless-shelving-add-on-36inw-x-24ind-x-84inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:40:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/342291-channel-heater-wire-stainless-steel-6>: HTTP status code is not handled or not allowed 2026-01-24 21:40:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-safety-view-locker-six-tier-12x18x12-6-doors-unassembled-tan>: HTTP status code is not handled or not allowed 2026-01-24 21:40:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-4-shelf-boltless-shelving-add-on-48inw-x-12ind-x-84inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:40:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sugar-free-electrolyte-replacement-powder-pouches>: HTTP status code is not handled or not allowed 2026-01-24 21:40:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caged-walk-through-fixed-access-ladder-gray-5>: HTTP status code is not handled or not allowed 2026-01-24 21:40:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-63-inch-high-post-chrome-4-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:40:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/datum-tekstak-laptop-storage-charging-locker-5-tier-key-lock-7>: HTTP status code is not handled or not allowed 2026-01-24 21:40:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/automatic-hand-dryer-115-volt-2>: HTTP status code is not handled or not allowed 2026-01-24 21:40:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pressure-relief-valve-3-4-npt-for-champion-104649>: HTTP status code is not handled or not allowed 2026-01-24 21:40:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lcd105blk-32-60-low-profile-monitor-wall-mount-black>: HTTP status code is not handled or not allowed 2026-01-24 21:40:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-brute-waste-receptacle-caddy-bag>: HTTP status code is not handled or not allowed 2026-01-24 21:40:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pink-anti-static-poly-tubing-12in-x-500ft>: HTTP status code is not handled or not allowed 2026-01-24 21:40:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-density-printed-seafood-bag-12in-x-18in-pkg-qty-1000>: HTTP status code is not handled or not allowed 2026-01-24 21:40:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-outside-tuck-mailers-7-18l-x-6-58w-x-6-12h-white-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-24 21:40:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tuf-r-heavy-lldpe-flat-bag-12in-x-16in-2-mil-pkg-qty-500>: HTTP status code is not handled or not allowed 2026-01-24 21:40:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-edge-work-bench-top-esd-72-w-x-36-d-x-1-1-4-thick>: HTTP status code is not handled or not allowed 2026-01-24 21:40:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-double-wall-corrugated-boxes-12l-x-9w-x-9h-kraft-pack-of-15>: HTTP status code is not handled or not allowed 2026-01-24 21:40:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-mailers-6l-x-3w-x-2h-kraft-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-24 21:40:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fb31t8-850-31w-fluorescent-w-medium-bi-pin-base-daylight>: HTTP status code is not handled or not allowed 2026-01-24 21:40:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ht7040t-intense-ir-hd-tvi-indooroutdoor-bullet-camera-1080p-2mp-28-12mm-lens>: HTTP status code is not handled or not allowed 2026-01-24 21:40:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zipl4d1-4-channel-plug-play-nvr-and-ip-kit-4-dome-cameras-1tb>: HTTP status code is not handled or not allowed 2026-01-24 21:40:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-density-poly-tubing-16in-6-mil-x-725ft>: HTTP status code is not handled or not allowed 2026-01-24 21:40:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dispense-rite-5-section-wire-rack-cup-and-lid-organizer>: HTTP status code is not handled or not allowed 2026-01-24 21:40:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/printed-thank-you-take-out-bag-with-bell-top-handle-16-5in-x-14in-pkg-qty-500>: HTTP status code is not handled or not allowed 2026-01-24 21:40:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-mailers-6l-x-5w-x-4h-kraft-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-24 21:40:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2x3-arrow-trax-mat-charcoal-black>: HTTP status code is not handled or not allowed 2026-01-24 21:40:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/266151-cover-hinge-r42>: HTTP status code is not handled or not allowed 2026-01-24 21:40:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/protap-cutting-fluid-1-gallon-pkg-of-2-30128p>: HTTP status code is not handled or not allowed 2026-01-24 21:40:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-gauge-heavy-duty-36-w-x-18-d-x-78-h-cabinet>: HTTP status code is not handled or not allowed 2026-01-24 21:40:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-profile-steady-burning-led-24-12-male-pipe-red>: HTTP status code is not handled or not allowed 2026-01-24 21:40:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cardboard-corrugated-boxes-16l-x-12w-x-11h-kraft-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-24 21:40:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-mailers-8l-x-3w-x-3h-kraft-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-24 21:40:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8915-clean-removal-filament-tape-1-x-60-yds-6-mil-clear>: HTTP status code is not handled or not allowed 2026-01-24 21:40:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ad-paper-vacuum-bag-for-nilfisk-ad-carpetriever-28-large-area-upright>: HTTP status code is not handled or not allowed 2026-01-24 21:40:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/federal-signal-explosion-proof-strobe-light-24-green>: HTTP status code is not handled or not allowed 2026-01-24 21:40:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-cal-shark-tooth-mat-rubber-floor-protection-mat-blk-3-4-thick-x-2-x-3>: HTTP status code is not handled or not allowed 2026-01-24 21:40:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/d-e-hepa-filter-for-d-e-defendair-hepa-500>: HTTP status code is not handled or not allowed 2026-01-24 21:40:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15a19-27-gu24v-15w-compact-fluorescent-w-twist-turn-base-warm>: HTTP status code is not handled or not allowed 2026-01-24 21:40:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18t2gu24v-27-squat-18w-compact-fluorescent-w-twist-turn-base-warm>: HTTP status code is not handled or not allowed 2026-01-24 21:40:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cardboard-corrugated-boxes-36l-x-12w-x-8h-kraft-pack-of-20>: HTTP status code is not handled or not allowed 2026-01-24 21:40:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/turfex-160lb-capacity-heavy-duty-push-spreader-stainless-steel-frame>: HTTP status code is not handled or not allowed 2026-01-24 21:40:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cf5ds-e-841-5w-compact-fluorescent-w-2g7-base-coolwhite>: HTTP status code is not handled or not allowed 2026-01-24 21:40:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/import-cam-operated-5c-horizontalvertical-collet-fixture>: HTTP status code is not handled or not allowed 2026-01-24 21:40:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-w-x-30-d-mobile-workbench-shop-top-square-edge-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:40:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cf7ds-e-841-7w-compact-fluorescent-w-2g7-base-coolwhite>: HTTP status code is not handled or not allowed 2026-01-24 21:40:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-height-wobble-stool-swivel-chair-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:40:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-wall-gaylord-bottom-36l-x-36w-x-36h-kraft-pack-of-5>: HTTP status code is not handled or not allowed 2026-01-24 21:40:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-locker-number-plate-kit-pkg-of-200-numbered-1100-1299>: HTTP status code is not handled or not allowed 2026-01-24 21:40:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/homak-bl04041092-pro-ii-series-41-w-x-24-1-2-d-x-39-h-9-drawer-blue-roller-tool-cabinet>: HTTP status code is not handled or not allowed 2026-01-24 21:40:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-steel-table-1-shelf-72x36-1200-lb-capacity-unassembled-1>: HTTP status code is not handled or not allowed 2026-01-24 21:40:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hon5711sb11t-volt-armless-task-chair-black>: HTTP status code is not handled or not allowed 2026-01-24 21:40:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-shelf-unassembled-service-cart-48-x-30-1200-lb-capacity-2>: HTTP status code is not handled or not allowed 2026-01-24 21:40:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pic-gauges-2-1-2-pressure-gauge-liquid-filled-200-psi-ss-case-center-back-mount-pro-202l-254g>: HTTP status code is not handled or not allowed 2026-01-24 21:41:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bin-nestable-6-5-8-w-x-11-5-8-d-x-4-d-red>: HTTP status code is not handled or not allowed 2026-01-24 21:41:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dspbl-powder-free-glove-sm-ltx-white-100bx>: HTTP status code is not handled or not allowed 2026-01-24 21:41:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gray-caressoft-vinyl-medical-stool-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:41:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modern-executive-vinyl-conference-chair-black>: HTTP status code is not handled or not allowed 2026-01-24 21:41:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/loop-end-ecomop-lg-5-recycled-pet-gre-12>: HTTP status code is not handled or not allowed 2026-01-24 21:41:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bin-nestable-11-1-8-w-x-17-5-8-d-x-4-h-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:41:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s2500c-otg-anti-fog-safety-glasses-black-frame-clear-lens>: HTTP status code is not handled or not allowed 2026-01-24 21:41:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ft55hl-830-env-55w-compact-fluorescent-w-2g11-base-warm-white>: HTTP status code is not handled or not allowed 2026-01-24 21:41:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/purell-sanitizing-hand-wipes>: HTTP status code is not handled or not allowed 2026-01-24 21:41:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-decontamination-shower-16-nozzle-porta300-16e>: HTTP status code is not handled or not allowed 2026-01-24 21:41:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-shower-treadle-t-style-ts-foot-ef>: HTTP status code is not handled or not allowed 2026-01-24 21:41:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-easy-fold-mailers-24l-x-24w-x-2h-kraft-pack-of-20>: HTTP status code is not handled or not allowed 2026-01-24 21:41:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/75-ton-ironworker-3-phase-230v-iw75-3p230>: HTTP status code is not handled or not allowed 2026-01-24 21:41:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/75-ton-ironworker-3-phase-208v-iw75-3p208>: HTTP status code is not handled or not allowed 2026-01-24 21:41:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-container-without-lid-31>: HTTP status code is not handled or not allowed 2026-01-24 21:41:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 26114 pages (at 99 pages/min), scraped 13558 items (at 19 items/min) 2026-01-24 21:41:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-ton-ironworker-3-phase-208v-w-hydraulic-accessory-pack-iw60-3p208-ac500>: HTTP status code is not handled or not allowed 2026-01-24 21:41:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cf13dd-e-827-13w-compact-fluorescent-w-g24vq-1-base-soft-white>: HTTP status code is not handled or not allowed 2026-01-24 21:41:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gen-purp-micro-fiber-cloth-red>: HTTP status code is not handled or not allowed 2026-01-24 21:41:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glass-bead-size-60-100-526000b-50>: HTTP status code is not handled or not allowed 2026-01-24 21:41:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-nitrile-gloves-medium-powder-free>: HTTP status code is not handled or not allowed 2026-01-24 21:41:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/282360-drain-pan-for-hoshizaki-of-america>: HTTP status code is not handled or not allowed 2026-01-24 21:41:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-edison-globe-light-bulb-85w-2700k-6-1116l-clear-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-24 21:41:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-153-deluxe-thermoplastic-coated-32-gallon-mesh-receptacle-w-rain-bonnet-lid-base-green>: HTTP status code is not handled or not allowed 2026-01-24 21:41:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e57-3-36w-miniatures-w-minature-bayonet-base>: HTTP status code is not handled or not allowed 2026-01-24 21:41:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/38-1272-plug-nema-6-20p>: HTTP status code is not handled or not allowed 2026-01-24 21:41:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rlph1-long-sleeve-pullover-hoodie-with-uv-protection-class-3-xl-hi-vis-lime>: HTTP status code is not handled or not allowed 2026-01-24 21:41:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mini-abrasive-blast-cabinet-101691r-a>: HTTP status code is not handled or not allowed 2026-01-24 21:41:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kdw-flush-access-doors-for-drywall-surf-lock-8wx8h-kdwp88l>: HTTP status code is not handled or not allowed 2026-01-24 21:41:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60a-y-60w-incandescent-w-medium-base>: HTTP status code is not handled or not allowed 2026-01-24 21:41:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flexvolt-174-60v-max-cordless-grinder-w-brake-kit-2-batteries-brushless-4-5-6>: HTTP status code is not handled or not allowed 2026-01-24 21:41:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-polish-water-base>: HTTP status code is not handled or not allowed 2026-01-24 21:41:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-auto-grip-lift-truck-attachment-1500-lb-cap>: HTTP status code is not handled or not allowed 2026-01-24 21:41:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/loom-clamps-stainless-steel-4>: HTTP status code is not handled or not allowed 2026-01-24 21:41:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manual-chain-hoist-stainless-steel-1-ton-capacity-20-lift>: HTTP status code is not handled or not allowed 2026-01-24 21:41:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cut-resistant-gloves-micro-foam-nitrile-coated-ansi-a4-xl-graypurple>: HTTP status code is not handled or not allowed 2026-01-24 21:41:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/counter-weight-drum-handler-battery-powered-90-h-counter-weighted>: HTTP status code is not handled or not allowed 2026-01-24 21:41:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/x18t5-18w-miniature-sealed-beams-w-mini-wedge-base>: HTTP status code is not handled or not allowed 2026-01-24 21:41:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kdw-flush-access-doors-for-drywall-surf-lock-24wx24h-kdwp2424l>: HTTP status code is not handled or not allowed 2026-01-24 21:41:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8008777-ss-temp-assembly-control>: HTTP status code is not handled or not allowed 2026-01-24 21:41:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/panel-leg-workbench-with-fixed-height-legs-32>: HTTP status code is not handled or not allowed 2026-01-24 21:41:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-cantilever-rack-single-sided-add-on-heavy-duty-72-w-x-50-d-x-10-h-with-lip>: HTTP status code is not handled or not allowed 2026-01-24 21:41:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/cable_management/corrugated_tubing_fittings already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:41:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guardian-booth-8-x-10-guard-booth-economy-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:41:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6411-10w-miniature-sealed-beams-w-sv8-5-8-base>: HTTP status code is not handled or not allowed 2026-01-24 21:41:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uhmw-polyethylene-plastic-sheet-1-4-thick-x-48-wide-x-96-long>: HTTP status code is not handled or not allowed 2026-01-24 21:41:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uhmw-polyethylene-plastic-sheet-1-4-thick-x-48-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-24 21:41:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-30-computer-workstation-single-sided>: HTTP status code is not handled or not allowed 2026-01-24 21:41:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-step-stairway-slope-work-platform-w-handrails-36in-x-72in-platform>: HTTP status code is not handled or not allowed 2026-01-24 21:41:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tripower-classical-section-molded-notch-vbelt-gates-bx38>: HTTP status code is not handled or not allowed 2026-01-24 21:41:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-sheet-1-8-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 21:41:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibco-silent-pneumatic-turbine-vibrator-vs-190>: HTTP status code is not handled or not allowed 2026-01-24 21:41:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fleetrunner-microv-serpentine-drive-belt-gates-k060935hd>: HTTP status code is not handled or not allowed 2026-01-24 21:41:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scratch-resistant-strip-door-curtain-10-w-x-10-h>: HTTP status code is not handled or not allowed 2026-01-24 21:41:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/cable_management/corrugated_tubing_fittings>: HTTP status code is not handled or not allowed 2026-01-24 21:41:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-hc-narrow-section-molded-notch-vbelt-gates-3vx630>: HTTP status code is not handled or not allowed 2026-01-24 21:41:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibco-silent-pneumatic-turbine-vibrator-mhi-32-poly>: HTTP status code is not handled or not allowed 2026-01-24 21:41:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hipower-ii-classical-section-wrapped-vbelt-gates-a31>: HTTP status code is not handled or not allowed 2026-01-24 21:41:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibco-pneumatic-end-mounted-piston-vibrator-10-1-1-4s>: HTTP status code is not handled or not allowed 2026-01-24 21:41:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-drum-hardcover-spillpallet-with-drain-66-gallon-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:41:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15f10-15w-incandescent-w-candelabra-base>: HTTP status code is not handled or not allowed 2026-01-24 21:41:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/folding-duck-pond-mini-berm-containment-3-x-3-x-6-5633-ye-f>: HTTP status code is not handled or not allowed 2026-01-24 21:41:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speedy-duffel-spill-kit-oil-only-10-gal-capacity-yellow-1312-ye>: HTTP status code is not handled or not allowed 2026-01-24 21:41:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibco-silent-pneumatic-turbine-vibrator-mltss-32>: HTTP status code is not handled or not allowed 2026-01-24 21:41:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-purpose-drum-stacker-4-drum-system-no-drain>: HTTP status code is not handled or not allowed 2026-01-24 21:41:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/511170-coil-solenoid-220-240v>: HTTP status code is not handled or not allowed 2026-01-24 21:41:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-12-x-10-standard-clear-strip-strip-curtain-doors>: HTTP status code is not handled or not allowed 2026-01-24 21:41:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-x119-orange-strip>: HTTP status code is not handled or not allowed 2026-01-24 21:41:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibco-heavy-duty-electric-vibrator-2p-100-1>: HTTP status code is not handled or not allowed 2026-01-24 21:41:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hipower-ii-classical-section-wrapped-vbelt-gates-b96>: HTTP status code is not handled or not allowed 2026-01-24 21:41:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibco-heavy-duty-electric-vibrator-2pcd-1050-3>: HTTP status code is not handled or not allowed 2026-01-24 21:41:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibco-12-volt-electric-vibrator-dc-60>: HTTP status code is not handled or not allowed 2026-01-24 21:41:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/100g40-w-100w-incandescent-w-medium-base>: HTTP status code is not handled or not allowed 2026-01-24 21:41:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/import-177-256capacity-rubber-collet-for-tapping-heads>: HTTP status code is not handled or not allowed 2026-01-24 21:41:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/250-7-16-dia-6x36-extra-improved-plow-steel-galvanized-wire-rope-independent-wire-rope-center>: HTTP status code is not handled or not allowed 2026-01-24 21:41:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50r20-a-50w-incandescent-w-medium-base>: HTTP status code is not handled or not allowed 2026-01-24 21:41:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/generac-7224-guardian-14kw-120-240v-1-phase-air-cooled-standby-generator-ng-lp-wifi-enabled>: HTTP status code is not handled or not allowed 2026-01-24 21:41:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-1-thick-x-1-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 21:41:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-security-lcd-computer-cabinet-complete-bundle-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:41:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uhmw-polyethylene-plastic-bar-1-2-thick-x-1-1-2-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 21:42:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/13510-filter-cartridge-for-cuno-inc>: HTTP status code is not handled or not allowed 2026-01-24 21:42:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibco-pneumatic-piston-vibrator-55-1-1-2>: HTTP status code is not handled or not allowed 2026-01-24 21:42:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t9-circline-fluorescent-bulb-g10q-base-40w-2600-lumens-3000k-warm-white-pk-of-4>: HTTP status code is not handled or not allowed 2026-01-24 21:42:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/250-5-8-dia-6x19-improved-plow-stee-galvanized-wire-rope-independent-wire-rope-center>: HTTP status code is not handled or not allowed 2026-01-24 21:42:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/duplex-switch-cover-plate-1-gang-silver-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-24 21:42:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/45-orange-ez-grab-delineator-post-w-2-ea-3-hi-reflective>: HTTP status code is not handled or not allowed 2026-01-24 21:42:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-1-1-2-thick-x-4-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 21:42:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nest-only-container-21-x-15-1-8-x-5-1-8-gray-closed-handle>: HTTP status code is not handled or not allowed 2026-01-24 21:42:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-sheet-1-2-thick-x-6-wide-x-6-long>: HTTP status code is not handled or not allowed 2026-01-24 21:42:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-sport-cone-flourescent-orange>: HTTP status code is not handled or not allowed 2026-01-24 21:42:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ushio-jcv120v-1000wc4-gx6-35-2cc-8>: HTTP status code is not handled or not allowed 2026-01-24 21:42:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-steel-reel-holder-workbench-60-w>: HTTP status code is not handled or not allowed 2026-01-24 21:42:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 26209 pages (at 95 pages/min), scraped 13578 items (at 20 items/min) 2026-01-24 21:42:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-adjustable-led-emergency-light-fixture-4w-200-lumens-120-277v>: HTTP status code is not handled or not allowed 2026-01-24 21:42:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010699-hi-limit-for-hatco-corp>: HTTP status code is not handled or not allowed 2026-01-24 21:42:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-rod-2-1-2-diameter-x-1-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 21:42:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milton-s-99787w-t-style-automotive-safety-coupler-1-4-fnpt>: HTTP status code is not handled or not allowed 2026-01-24 21:42:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ushio-cf26de-841-doubletube>: HTTP status code is not handled or not allowed 2026-01-24 21:42:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-1-4-thick-x-1-1-2-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 21:42:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foam-hoagie-container-with-removable-lid-9-45x5-310x3-310-white-125bag>: HTTP status code is not handled or not allowed 2026-01-24 21:42:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-plastic-rod-1-diameter-x-6-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 21:42:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milton-s-1837w-h-style-industrial-plug-3-8-mnpt>: HTTP status code is not handled or not allowed 2026-01-24 21:42:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-b11-chandelier-light-bulb-e26-base-55w-600-lumens-50k-super-white-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-24 21:42:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ushio-ush-103d-hg100w>: HTTP status code is not handled or not allowed 2026-01-24 21:42:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/epoxy-coated-steel-storage-basket-for-wire-and-wall-shelving-17-3-8w-x-7-5d-x-5h-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:42:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-a19-colored-light-bulb-e26-base-3w-1800-lumens-yellow-pack-of-3>: HTTP status code is not handled or not allowed 2026-01-24 21:42:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rapidwire-welded-wire-4-sided-cage-w-3-hinged-door-ceiling-12-6-x-12-6-x-12-5-1-4h>: HTTP status code is not handled or not allowed 2026-01-24 21:42:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-forks-9-inches-black-144case>: HTTP status code is not handled or not allowed 2026-01-24 21:42:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-acetal-plastic-bar-1-4-thick-x-1-1-2-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-24 21:42:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/76-1158-inline-carbon-block-water-filter-cartridge-il-35-1-micron-rated-3000-gallon>: HTTP status code is not handled or not allowed 2026-01-24 21:42:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rapidwire-welded-wire-4-sided-cage-w-5-sliding-door-ceiling-10-4-x-10-4-x-10-5-1-4h>: HTTP status code is not handled or not allowed 2026-01-24 21:42:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/new-sandensankyo-sd7h15-compressor-w-clutch-four-seasons-78597>: HTTP status code is not handled or not allowed 2026-01-24 21:42:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2531354-wire-hi-temp-16-ga-tan-100>: HTTP status code is not handled or not allowed 2026-01-24 21:42:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-function-merchandise-lift-with-shelf-safety-orange>: HTTP status code is not handled or not allowed 2026-01-24 21:42:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/840-style-3-sided-driver-cage-4w-x-10d-x-8h>: HTTP status code is not handled or not allowed 2026-01-24 21:42:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-gsc-012-ss>: HTTP status code is not handled or not allowed 2026-01-24 21:42:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arsenal-5876-small-clamshell-organizer-black-13220>: HTTP status code is not handled or not allowed 2026-01-24 21:42:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/incandescent-bulb-with-e12-candelabra-base-7w-120v-orange-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-24 21:42:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-two-piece-clamping-collar-gm2c-25-b>: HTTP status code is not handled or not allowed 2026-01-24 21:42:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chillits-6710ct-evaporative-cooling-triangle-hat-with-cooling-towel-solid-blue-12587>: HTTP status code is not handled or not allowed 2026-01-24 21:42:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdpe-plastic-sheet-1-thick-x-24-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-24 21:42:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/801-2506-3-4-inch-npt-lead-free-vacuum-breaker-125-psi-180-f-maximum-temperature>: HTTP status code is not handled or not allowed 2026-01-24 21:42:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8356frhl-class-3-fr-modacrylic-vest-lime-22215>: HTTP status code is not handled or not allowed 2026-01-24 21:42:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-plastic-sheet-1-8-thick-x-24-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-24 21:42:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012393-hi-limit-for-imperial-cooking-equipment>: HTTP status code is not handled or not allowed 2026-01-24 21:42:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chair-mat-for-carpet-medium-pile-48-x-72-rectangle-clear-beveled-edge>: HTTP status code is not handled or not allowed 2026-01-24 21:42:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/feeding-chair-5-inch-seat-height>: HTTP status code is not handled or not allowed 2026-01-24 21:42:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-rack-wall-bracket-6-es-2>: HTTP status code is not handled or not allowed 2026-01-24 21:42:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wireway-pallet-rack-tear-drop-upright-frame-240x42>: HTTP status code is not handled or not allowed 2026-01-24 21:42:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/import-individual-transfer-punch-5-16>: HTTP status code is not handled or not allowed 2026-01-24 21:42:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-set-screw-collar-gmc-30-b>: HTTP status code is not handled or not allowed 2026-01-24 21:42:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/85870-maintenance-sorbent-roll-medium-capacity-mcm-25-in-x-150-1-rollcase>: HTTP status code is not handled or not allowed 2026-01-24 21:42:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/srp-chem-chemical-sorbent-spill-response-pack-35-gallon-capacity-3-packscase>: HTTP status code is not handled or not allowed 2026-01-24 21:42:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-gsc-181-z>: HTTP status code is not handled or not allowed 2026-01-24 21:42:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/38-1549-120-inch-sjtow-appliance-power-cord-14-gauge-wire>: HTTP status code is not handled or not allowed 2026-01-24 21:42:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0423rblk2xl-womens-softshell-quilted-jacket-black-2xl>: HTTP status code is not handled or not allowed 2026-01-24 21:42:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60x30-stainless-square-edge-power-apron-production-bench-tan-32313687>: HTTP status code is not handled or not allowed 2026-01-24 21:42:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-g1sc-125-ss>: HTTP status code is not handled or not allowed 2026-01-24 21:42:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-ca12-chandelier-light-bulb-e26-base-7w-500-lumens-2700k-soft-white-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-24 21:42:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72x36-esd-square-edge-power-apron-production-bench-tan-32313715>: HTTP status code is not handled or not allowed 2026-01-24 21:42:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-inch-speed-squeegymoss-rubber-with-steel-straight-frame>: HTTP status code is not handled or not allowed 2026-01-24 21:42:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-inch-l-tube-cleaning-brush-with-3-4-inch-dia-nylon-brush-and-stainless-steel-handle>: HTTP status code is not handled or not allowed 2026-01-24 21:42:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-stainless-steel-shelf-for-model-100-ss>: HTTP status code is not handled or not allowed 2026-01-24 21:42:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-w-x-30-d-mobile-production-workbench-with-power-apron-shop-top-square-edge-blue-32313740>: HTTP status code is not handled or not allowed 2026-01-24 21:42:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60x30-esd-safety-edge-mobile-power-apron-production-bench-blue-32313742>: HTTP status code is not handled or not allowed 2026-01-24 21:42:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-set-screw-collar-gmc-06-b>: HTTP status code is not handled or not allowed 2026-01-24 21:42:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compact-led-exit-sign-with-90-minute-battery-power-back-up-025w-120-277v-white>: HTTP status code is not handled or not allowed 2026-01-24 21:42:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-g1sc-200-b>: HTTP status code is not handled or not allowed 2026-01-24 21:42:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-leg-telescoping-easel-with-pad-retainer-adjusts-34-to-64-aluminum-silver>: HTTP status code is not handled or not allowed 2026-01-24 21:42:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/561344-valve-steam-safety-3-4>: HTTP status code is not handled or not allowed 2026-01-24 21:42:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/761394-filter-cartridge-4hc-h>: HTTP status code is not handled or not allowed 2026-01-24 21:42:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-br30-recessed-light-bulb-e26-base-105w-800-lumens-30k-warm-white-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-24 21:42:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/combination-metal-storage-cabinet-36x18x72-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:42:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90-torsion-spring-0-989-coil-dia-0-125-wire-dia-wound-right-music-wire>: HTTP status code is not handled or not allowed 2026-01-24 21:42:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-set-screw-collar-gmc-12-ss>: HTTP status code is not handled or not allowed 2026-01-24 21:42:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reebokrb8694-mens-stealth-6-boot-with-side-zipper-desert-tan-size-13-m>: HTTP status code is not handled or not allowed 2026-01-24 21:42:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009429-thermostat-kp>: HTTP status code is not handled or not allowed 2026-01-24 21:42:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trailer-spotter-with-3-way-ball-for-30l-forks-600-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:42:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009510-temperature-controller>: HTTP status code is not handled or not allowed 2026-01-24 21:42:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drainage-mat-interlocking-tile-4>: HTTP status code is not handled or not allowed 2026-01-24 21:42:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72x36-maple-square-edge-power-apron-production-bench-blue-32313832>: HTTP status code is not handled or not allowed 2026-01-24 21:42:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/faultless-swivel-plate-caster-5-polyolefin-wheel-2>: HTTP status code is not handled or not allowed 2026-01-24 21:42:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ulti-mate-garage-2-0-series-9-piece-cabinet-set-142w-x-21-14d-wall-x-80h-black>: HTTP status code is not handled or not allowed 2026-01-24 21:42:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-w-x-30-d-stainless-steel-square-edge-with-drawer-riser>: HTTP status code is not handled or not allowed 2026-01-24 21:42:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-point-mechanical-pencil-09-mm-2-per-set>: HTTP status code is not handled or not allowed 2026-01-24 21:42:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m20x1-0-hss-iso-metric-tmx-round-adjustable-die-fine-thread-1-1-2-inch-od>: HTTP status code is not handled or not allowed 2026-01-24 21:43:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/511358-spring-regulator-conv>: HTTP status code is not handled or not allowed 2026-01-24 21:43:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/54-1035-type-vr8204a-gas-valve-kit-natural-gas-1-2-inch-gas-in-out-3-16-inch-pilot-out>: HTTP status code is not handled or not allowed 2026-01-24 21:43:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/32-bottle-dakota-wine-rack-with-display-top-mahogany-39-1-2h>: HTTP status code is not handled or not allowed 2026-01-24 21:43:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sed10074-10-hp-oil-less-scroll-compressor-tankless-116-psi-3-phase-460v>: HTTP status code is not handled or not allowed 2026-01-24 21:43:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2891054-sensor-thermistor-6-inch-l-44-inch-leads-for-marshall-air-products>: HTTP status code is not handled or not allowed 2026-01-24 21:43:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-signs-notice-all-visitors-must-register-rigid-plastic-7-h-x-10-w>: HTTP status code is not handled or not allowed 2026-01-24 21:43:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crespa-lantern-style-outdoor-wall-sconce-light-fixture-e26-medium-base-40w-black>: HTTP status code is not handled or not allowed 2026-01-24 21:43:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-valve-solenoid-1-8-nptf-3-port-2-pos-normally-clsd-120vac-free-to-atmosphere-exh>: HTTP status code is not handled or not allowed 2026-01-24 21:43:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-valve-solenoid-1-8-nptf-3-port-2-pos-normally-clsd-240vac-free-to-atmosphere-exh>: HTTP status code is not handled or not allowed 2026-01-24 21:43:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009733-hubbell-ss-gang-platesingle>: HTTP status code is not handled or not allowed 2026-01-24 21:43:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-g1sc-087-b>: HTTP status code is not handled or not allowed 2026-01-24 21:43:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/umbrellas_awnings_shades/outdoor_umbrellas_bases/umbrellas already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:43:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-compartment-sink-t304-ss-20l-x-20w-x-12d-bowls-2-18-drainboards>: HTTP status code is not handled or not allowed 2026-01-24 21:43:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 26304 pages (at 95 pages/min), scraped 13588 items (at 10 items/min) 2026-01-24 21:43:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-valve-solenoid-1-4-nptf-2-port-2-pos-normally-clsd-240vac-no-exh>: HTTP status code is not handled or not allowed 2026-01-24 21:43:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/umbrellas_awnings_shades/outdoor_umbrellas_bases/umbrellas already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:43:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cargo-load-binder-straight-style-e-type-12-long>: HTTP status code is not handled or not allowed 2026-01-24 21:43:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-valve-solenoid-1-4-nptf-3-port-2-pos-normally-clsd-or-normally-open-240vac-piped-exh>: HTTP status code is not handled or not allowed 2026-01-24 21:43:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cylinder-sngl-acting-4-bore-2-stroke-dbl-side-lug-mount>: HTTP status code is not handled or not allowed 2026-01-24 21:43:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-gsc-018-b>: HTTP status code is not handled or not allowed 2026-01-24 21:43:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009339-thermostat-kit-bjwa-for-southbend-range>: HTTP status code is not handled or not allowed 2026-01-24 21:43:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-24-w-5-step-steel-rolling-ladder-14-d-top-step-w-handrails-orange-w-cal-osha-handrail-cal-fsh526p-o>: HTTP status code is not handled or not allowed 2026-01-24 21:43:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aah-48rbmag2-db-storage-cabinet-all-welded-48-wx18-dx78-h-denim-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:43:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-ceramic-utility-knife-blades-pointed-tips-10528-pack-of-3>: HTTP status code is not handled or not allowed 2026-01-24 21:43:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/outdoor_furniture/umbrellas_awnings_shades/outdoor_umbrellas_bases/umbrellas>: HTTP status code is not handled or not allowed 2026-01-24 21:43:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-ctrl-in-line-1-8-npt-male-port-ctrlled-flow-inlet-barb-for-1-4-od-tube-free-flow-inlet>: HTTP status code is not handled or not allowed 2026-01-24 21:43:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/outdoor_furniture/umbrellas_awnings_shades/outdoor_umbrellas_bases/umbrellas>: HTTP status code is not handled or not allowed 2026-01-24 21:43:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aah-30rb-l-grn-storage-cabinet-all-welded-30-wx18-dx72-h-leaf-green>: HTTP status code is not handled or not allowed 2026-01-24 21:43:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/made-in-usa-cobalt-3-flute-countersink-set-82-1-4-3-4>: HTTP status code is not handled or not allowed 2026-01-24 21:43:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/comfort-park-avenue-bench-recycled-plastic-6-ft-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:43:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/werner-ladder-fiberglass-extension-with-250-lb-capacity-36-h>: HTTP status code is not handled or not allowed 2026-01-24 21:43:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ia5017-groundbreaker-6-external-met-guard-work-boot-steel-toe-men-s-sz-9-5-w-wide-brown>: HTTP status code is not handled or not allowed 2026-01-24 21:43:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/541164-gas-safety-valve-1-2-inch>: HTTP status code is not handled or not allowed 2026-01-24 21:43:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-table-recycled-plastic-6-ft-black-frame-cedar-ada>: HTTP status code is not handled or not allowed 2026-01-24 21:43:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-step-extra-heavy-duty-steel-rolling-safety-ladder-heavy-duty-serrated-grating-w-cal-osha-handrail-cal-ml063221g>: HTTP status code is not handled or not allowed 2026-01-24 21:43:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-24-w-8-step-steel-rolling-ladder-21-d-top-step-w-handrails-lock-step-orange-w-cal-osha-handrail-cal-fs083221p-o>: HTTP status code is not handled or not allowed 2026-01-24 21:43:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-24-w-11-step-steel-rolling-ladder-21-d-top-step-w-handrails-lock-step-orange-w-cal-osha-handrail-cal-fs113221p-o>: HTTP status code is not handled or not allowed 2026-01-24 21:43:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tub-cart-black-32-x-18-3-shelves-2-outrigger-utility-cart-bins-100-lbs-shelf-cap>: HTTP status code is not handled or not allowed 2026-01-24 21:43:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-valve-5-port-2-pos-sngl-solenoid-1-4-nptf-120vac>: HTTP status code is not handled or not allowed 2026-01-24 21:43:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conntek-30123-bk-20-to-15-20-amp-generator-locking-adapter-with-nema-l5-20p-to-5-15-20r-black>: HTTP status code is not handled or not allowed 2026-01-24 21:43:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heritage-backless-bench-recycled-plastic-4-ft-green-frame-cedar>: HTTP status code is not handled or not allowed 2026-01-24 21:43:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/push-pull-valve-5-port-2-pos-1-4-npt>: HTTP status code is not handled or not allowed 2026-01-24 21:43:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/toggle-switch-plate-2-gang-ivory-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-24 21:43:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-step-24-wx59-d-stainless-steel-rolling-safety-ladder-serrated-grating-w-cal-osha-handrail-cal-ss063221g>: HTTP status code is not handled or not allowed 2026-01-24 21:43:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-step-24-w-steel-safety-angle-orange-rolling-ladder-w-handrails-perforated-tread-w-cal-osha-handrail-cal-sw1232p-o>: HTTP status code is not handled or not allowed 2026-01-24 21:43:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-natural-cable-ties-100-pieces>: HTTP status code is not handled or not allowed 2026-01-24 21:43:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-deck-truck-48-l-x-30-w-2400-lb-capacity-with-ergo-handle>: HTTP status code is not handled or not allowed 2026-01-24 21:43:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gtek-nitrile-microsurface-nylon-grip-gloves-large>: HTTP status code is not handled or not allowed 2026-01-24 21:43:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bussing-tub-22-l-x-17-w-x-5-h-flat-bottom-bus-box-polypropylene-black>: HTTP status code is not handled or not allowed 2026-01-24 21:43:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/using-fire-extinguishers-dvd-program>: HTTP status code is not handled or not allowed 2026-01-24 21:43:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-24-w-7-step-steel-rolling-ladder-14-d-top-step-w-handrails-lock-step-orange-w-cal-osha-handrail-cal-fs073014p-o>: HTTP status code is not handled or not allowed 2026-01-24 21:43:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gray-plastic-shopping-basket-22-liter-with-black-plastic-handle-pkg-qty-12>: HTTP status code is not handled or not allowed 2026-01-24 21:43:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hardwood-platform-truck-54-x-27-2400-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:43:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aah-30rb-ts-storage-cabinet-all-welded-30-wx18-dx72-h-tropic-sand>: HTTP status code is not handled or not allowed 2026-01-24 21:43:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/genplus-black-plastic-rolling-basket-68-liter->: HTTP status code is not handled or not allowed 2026-01-24 21:43:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bca-365218-h-grn-bookcase-all-welded-36-wx18-dx52-h-hunter-green>: HTTP status code is not handled or not allowed 2026-01-24 21:43:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergotron-t-slot-bracket>: HTTP status code is not handled or not allowed 2026-01-24 21:43:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/umbrellas_awnings_shades/outdoor_umbrellas_bases/umbrellas already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:43:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergotron-notebook-arm-mount-tray>: HTTP status code is not handled or not allowed 2026-01-24 21:43:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-w-x-12-h-x-1-d-pleated-merv-8-standard-capacity-air-filter-global-industrial>: HTTP status code is not handled or not allowed 2026-01-24 21:43:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zip-a-duct-fixing-strap-with-stainless-steel-hardwarefor-20-to-36-inch-diameter-ducts>: HTTP status code is not handled or not allowed 2026-01-24 21:43:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-w-x-18-h-x-1-d-pleated-merv-8-standard-capacity-air-filter-global-industrial>: HTTP status code is not handled or not allowed 2026-01-24 21:43:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/13-step-extra-heavy-duty-steel-rolling-safety-ladder-perforated-tread-w-cal-osha-handrail-cal-ml134021p>: HTTP status code is not handled or not allowed 2026-01-24 21:43:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bca-365213-r-bookcase-all-welded-36-wx13-dx52-h-red>: HTTP status code is not handled or not allowed 2026-01-24 21:43:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/filter-md450mbab42a-12-npt-5-0200-psig-170-scfm>: HTTP status code is not handled or not allowed 2026-01-24 21:43:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/connecting-clampmounting-bracket-ra118105m-for-md4-frl-components>: HTTP status code is not handled or not allowed 2026-01-24 21:43:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-sign-wheel-chair-ramp-080-mil-thick>: HTTP status code is not handled or not allowed 2026-01-24 21:43:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/valor-2000-v22xwe1501t-water-resistant-scale-3-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:43:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/outdoor_furniture/umbrellas_awnings_shades/outdoor_umbrellas_bases/umbrellas>: HTTP status code is not handled or not allowed 2026-01-24 21:43:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-w-x-20-h-x-1-d-pleated-merv-8-high-capacity-air-filter-global-industrial>: HTTP status code is not handled or not allowed 2026-01-24 21:43:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fixed-tv-wall-mount-with-built-in-level-for-17-in-42-in-tvs>: HTTP status code is not handled or not allowed 2026-01-24 21:43:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/communicating-indoor-air-handler-with-a-coil-4-ton-1-ph-230v>: HTTP status code is not handled or not allowed 2026-01-24 21:43:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/imported-type-c-tool-holder-bushing-2od-x-1-12id>: HTTP status code is not handled or not allowed 2026-01-24 21:43:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-w-x-16-h-x-4-d-pleated-merv-8-standard-capacity-air-filter-global-industrial>: HTTP status code is not handled or not allowed 2026-01-24 21:43:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss-hose-braid-w-150-fixed-x-floating-fl-4-x-12>: HTTP status code is not handled or not allowed 2026-01-24 21:43:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/suction-brush-for-aero-series-black>: HTTP status code is not handled or not allowed 2026-01-24 21:43:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forney-1-series-type-101-heavy-duty-acetylene-cutting-tip-size-3>: HTTP status code is not handled or not allowed 2026-01-24 21:43:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aah-30rb-db-storage-cabinet-all-welded-30-wx18-dx72-h-denim-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:43:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8000000-aluminum-tripod-adjustable-5-7l-350-cap-lbs>: HTTP status code is not handled or not allowed 2026-01-24 21:43:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss-hose-braid-w-male-npt-nipple-1-x-18>: HTTP status code is not handled or not allowed 2026-01-24 21:43:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/walker-ski-glides-black>: HTTP status code is not handled or not allowed 2026-01-24 21:43:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/safety_cabinets/flammable_materials_cabinets/drum_storage_flammable_safety_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:43:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-step-24-w-steel-safety-angle-rolling-ladder-w-handrails-grip-tread-w-cal-osha-handrail-cal-sw1032g>: HTTP status code is not handled or not allowed 2026-01-24 21:43:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yellow-line-striper-spray-paint>: HTTP status code is not handled or not allowed 2026-01-24 21:43:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polymer-pilaster-w-shoe-18w-x-82h-folkstone-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:43:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-heavy-duty-wheel-chock-polyurethane-17l-x-14w-x-14h>: HTTP status code is not handled or not allowed 2026-01-24 21:44:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1231470-y-lanyard-2l-310-cap-lbs>: HTTP status code is not handled or not allowed 2026-01-24 21:44:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e-lite-plastic-equipment-pad-32l-x-32w-x-3h-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:44:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/infinity-locker-single-tier-12x18x60-3-door-assembled-tan>: HTTP status code is not handled or not allowed 2026-01-24 21:44:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-light-full-electric-bed-with-full-length-side-rails-spring-mattress>: HTTP status code is not handled or not allowed 2026-01-24 21:44:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-step-24-w-steel-safety-angle-orange-rolling-ladder-w-handrails-perforated-tread-w-cal-osha-handrail-cal-sw630p-o>: HTTP status code is not handled or not allowed 2026-01-24 21:44:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-pilaster-w-shoe-12w-x-82h-burgundy>: HTTP status code is not handled or not allowed 2026-01-24 21:44:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/cabinets/safety_cabinets/flammable_materials_cabinets/drum_storage_flammable_safety_cabinets>: HTTP status code is not handled or not allowed 2026-01-24 21:44:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48x48-slatwall-panel-with-aluminum-inserts>: HTTP status code is not handled or not allowed 2026-01-24 21:44:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-locker-single-tier-12x18x72-with-legs-3-doors-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:44:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1340101-pro-stretch-shock-absorbing-lanyard-6l-130-310-cap-lbs>: HTTP status code is not handled or not allowed 2026-01-24 21:44:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22-inch-viper-plus-gt-flip-back-detachable-adj-ht-full-arm-elevating-legrest>: HTTP status code is not handled or not allowed 2026-01-24 21:44:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 26403 pages (at 99 pages/min), scraped 13607 items (at 19 items/min) 2026-01-24 21:44:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-locker-six-tier-12x18x12-without-legs-18-doors-assembled-sand>: HTTP status code is not handled or not allowed 2026-01-24 21:44:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forney-sil-phos-brazing-rod-18-dia>: HTTP status code is not handled or not allowed 2026-01-24 21:44:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1202842-rope-lifeline-100l-310-cap-lbs>: HTTP status code is not handled or not allowed 2026-01-24 21:44:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-inch-fan-shroud-merv-6-air-filter-global-industrial>: HTTP status code is not handled or not allowed 2026-01-24 21:44:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7403060-securaspan-hll-lifeline-assembly-w-zorbit-energy-absorber-60l>: HTTP status code is not handled or not allowed 2026-01-24 21:44:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-step-24-w-steel-safety-angle-orange-rolling-ladder-w-handrails-serrated-tread-w-cal-osha-handrail-cal-sw530g-o>: HTTP status code is not handled or not allowed 2026-01-24 21:44:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8510826-heavy-duty-zinc-plated-sleeve-cap>: HTTP status code is not handled or not allowed 2026-01-24 21:44:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-locker-double-tier-12x15x36-6-door-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:44:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-squeegee-front-for-nobles-tennant-1228905>: HTTP status code is not handled or not allowed 2026-01-24 21:44:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/valencia-bow-front-desk-shell-71wx35-12d-to-41-38dx29-58h-espresso>: HTTP status code is not handled or not allowed 2026-01-24 21:44:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-pilaster-w-shoe-4w-x-82h-red>: HTTP status code is not handled or not allowed 2026-01-24 21:44:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/valencia-series-hutch-with-doors-70-58wx15dx35-12h-espresso>: HTTP status code is not handled or not allowed 2026-01-24 21:44:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forney-tweco-style-mig-welding-nozzle-for-85339-size-12>: HTTP status code is not handled or not allowed 2026-01-24 21:44:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-strapping-kit-1-2-steel-strapping>: HTTP status code is not handled or not allowed 2026-01-24 21:44:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/seat-cushion-for-file-pedestals-14-78x19-18x2-18-tan-quartz>: HTTP status code is not handled or not allowed 2026-01-24 21:44:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-squeegee-set-urethane-for-nilfisk-advance-56115335-nilfisk-advance-56112252>: HTTP status code is not handled or not allowed 2026-01-24 21:44:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forney-plasma-cutter-shield-cup-40a>: HTTP status code is not handled or not allowed 2026-01-24 21:44:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/analog-gauge-push-pull-dynamometer-100-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:44:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1802-17-64-cobalt-heavy-duty-straw-135-split-point-jobber-length-drill>: HTTP status code is not handled or not allowed 2026-01-24 21:44:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clip-label-3l-x-1-14h-gray-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-24 21:44:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1801-5-16-hss-heavy-duty-steam-oxide-135-split-point-jobber-length-drill>: HTTP status code is not handled or not allowed 2026-01-24 21:44:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/559-ta-1-2-cobalt-heavy-duty-tialn-135-split-point-screw-machine-drill>: HTTP status code is not handled or not allowed 2026-01-24 21:44:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s102022-wire-rope-sling-3-4-x-12-eye-and-eye-8200-11200-22400-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-24 21:44:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bradley-showerhorizontal-drench-station-s19-120>: HTTP status code is not handled or not allowed 2026-01-24 21:44:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-squeegee-front-for-nobles-tennant-1025334>: HTTP status code is not handled or not allowed 2026-01-24 21:44:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-pilaster-w-shoe-3w-x-82h-dark-khaki>: HTTP status code is not handled or not allowed 2026-01-24 21:44:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electronic-push-pull-dynamometer-50-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:44:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-pilaster-w-shoe-10w-x-82h-pepper-dust>: HTTP status code is not handled or not allowed 2026-01-24 21:44:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacment-drive-wheel-for-alto-clarke-439704>: HTTP status code is not handled or not allowed 2026-01-24 21:44:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyester-strapping-kit-psk48-1-2-inches-x-0-025-inches-x-5800-feet-black-16-inches-x-6-inches-core>: HTTP status code is not handled or not allowed 2026-01-24 21:44:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-pilaster-w-shoe-6w-x-82h-burgundy>: HTTP status code is not handled or not allowed 2026-01-24 21:44:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b151094-10-double-leg-chain-sling-w-grab-hook>: HTTP status code is not handled or not allowed 2026-01-24 21:44:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-piece-hand-evaluation-set-red>: HTTP status code is not handled or not allowed 2026-01-24 21:44:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/concrete-planter-forklift-knockouts-72lx24wx30h-rectangle-gray-limestone>: HTTP status code is not handled or not allowed 2026-01-24 21:44:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-pilaster-w-shoe-18w-x-82h-burgundy>: HTTP status code is not handled or not allowed 2026-01-24 21:44:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forney-tweco-style-gas-diffuser-size-2-916>: HTTP status code is not handled or not allowed 2026-01-24 21:44:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/duraline-economy-tubular-dunnage-rack-24-inch-w-36-inch-l-12-inch-h>: HTTP status code is not handled or not allowed 2026-01-24 21:44:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacment-squeegee-set-for-nilfisk-advance-56409413>: HTTP status code is not handled or not allowed 2026-01-24 21:44:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eye-face-wash-eye-wash-fountain-2>: HTTP status code is not handled or not allowed 2026-01-24 21:44:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-data-sheets-in-hazwoper-environments-dvd-program>: HTTP status code is not handled or not allowed 2026-01-24 21:44:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1805-3-8-hss-heavy-duty-steam-oxide-135-aircraft-extension-drill>: HTTP status code is not handled or not allowed 2026-01-24 21:44:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1818-1-4-4in-oal-hss-heavy-duty-sand-blasted-118-point-carbide-tipped-masonry-drill>: HTTP status code is not handled or not allowed 2026-01-24 21:44:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s152030-8-poly-web-sling-eye-and-eye-2560-3200-6400-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-24 21:44:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-pad-lok-pad-driver-complete-assembly-for-nilfisk-advance-l08812466>: HTTP status code is not handled or not allowed 2026-01-24 21:44:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/articulating-keyboard-assembly-for-1-25-diameter-columns>: HTTP status code is not handled or not allowed 2026-01-24 21:44:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/l-bracket-containment-berm-5l-x-5w-x-12h>: HTTP status code is not handled or not allowed 2026-01-24 21:44:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1821-1-4-4in-oal-hss-h-d-sand-blasted-118-point-carbide-tipped-sds-plus-2-masonry-drill>: HTTP status code is not handled or not allowed 2026-01-24 21:44:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1877-39-64-black-gold-118-point-3-flat-1-2-reduced-shank-silver-deming-drill>: HTTP status code is not handled or not allowed 2026-01-24 21:44:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-pad-driver-for-pioneer-eclipse-mp478400-powr-flite-tl19>: HTTP status code is not handled or not allowed 2026-01-24 21:44:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1821-5-16-6in-oal-hss-h-d-sand-blasted-118-point-carbide-tipped-sds-plus-2-masonry-drill>: HTTP status code is not handled or not allowed 2026-01-24 21:44:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/en53375-solid-five-lobed-knob-14-20-3tag6m>: HTTP status code is not handled or not allowed 2026-01-24 21:44:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polymer-pilaster-w-shoe-7w-x-82h-black-confetti>: HTTP status code is not handled or not allowed 2026-01-24 21:44:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-industry-standard-clamping-couplings-w-keyway-climax-2iscc-125-125kw>: HTTP status code is not handled or not allowed 2026-01-24 21:44:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s152057-12-poly-web-sling-endless-2500-3200-6400-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-24 21:44:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polymer-inward-swing-partition-door-26w-black-confetti>: HTTP status code is not handled or not allowed 2026-01-24 21:44:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galvanized-frame-wall-exhaust-shutter-with-aluminum-blades-rear-flange-60-556-std-60>: HTTP status code is not handled or not allowed 2026-01-24 21:44:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-piece-repel-rain-suit-m-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:44:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-piece-repel-rain-suit-5xl-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:44:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s152012-4-poly-web-sling-eye-and-eye-2560-3200-6400-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-24 21:44:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/phenolic-black-core-partition-panel-wo-brackets-5412w-dove-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:44:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/werner-10-type-1a-fiberglass-podium-ladder-pd6210>: HTTP status code is not handled or not allowed 2026-01-24 21:44:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-brush-kit-mal-grit-for-nilfisk-advance-l08837066>: HTTP status code is not handled or not allowed 2026-01-24 21:45:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-heated-fleece-hoodie-jacket-with-detachable-hood-4xl-charcoal-mix>: HTTP status code is not handled or not allowed 2026-01-24 21:45:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacment-skirt-for-nilfisk-advance-56315626>: HTTP status code is not handled or not allowed 2026-01-24 21:45:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-industry-standard-clamping-couplings-w-keyway-climax-2iscc-137-100kw>: HTTP status code is not handled or not allowed 2026-01-24 21:45:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s152043-20-poly-web-sling-eye-and-eye-5120-6400-12800-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-24 21:45:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/collar-plug>: HTTP status code is not handled or not allowed 2026-01-24 21:45:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacment-t-handle-expandable-drain-plug-for-crusader-2020>: HTTP status code is not handled or not allowed 2026-01-24 21:45:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/papr-unit-belt-mounted-tr-302n>: HTTP status code is not handled or not allowed 2026-01-24 21:45:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mounting-bracket-2>: HTTP status code is not handled or not allowed 2026-01-24 21:45:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/superscrape-plus-entrance-mat-316-thick-4w-x-6l-black>: HTTP status code is not handled or not allowed 2026-01-24 21:45:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/three-sided-frame-30x60>: HTTP status code is not handled or not allowed 2026-01-24 21:45:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standoff-34-dia-x-1-12l-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:45:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-griffin-low-form-plastic-beaker-1000ml-case-of-12>: HTTP status code is not handled or not allowed 2026-01-24 21:45:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-safety-cargo-pant-50-waist-x-34l-inseam-navy>: HTTP status code is not handled or not allowed 2026-01-24 21:45:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 26500 pages (at 97 pages/min), scraped 13628 items (at 21 items/min) 2026-01-24 21:45:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1899-33-hss-general-purpose-steam-oxide-118-point-jobber-length-drill>: HTTP status code is not handled or not allowed 2026-01-24 21:45:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/southwire-67963002-nm-b-pcs-10-2-g-16-2-pegy-1000r-romex-simpull-nm-b-pcs-duo-cable-1000>: HTTP status code is not handled or not allowed 2026-01-24 21:45:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacment-squeegee-set-for-viper-vf91113>: HTTP status code is not handled or not allowed 2026-01-24 21:45:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polymer-partition-panel-wo-brackets-5912w-charcoal>: HTTP status code is not handled or not allowed 2026-01-24 21:45:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-rectangular-strip-carbide-blank-18x58x12-stb420l>: HTTP status code is not handled or not allowed 2026-01-24 21:45:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-unground-carbide-blank-18x12x34-2210>: HTTP status code is not handled or not allowed 2026-01-24 21:45:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-flame-resistant-heavyweight-pullover-hoodie-xl-black>: HTTP status code is not handled or not allowed 2026-01-24 21:45:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/157-9-hss-general-purpose-bright-118-point-screw-machine-drill>: HTTP status code is not handled or not allowed 2026-01-24 21:45:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-climax-2c-156-z>: HTTP status code is not handled or not allowed 2026-01-24 21:45:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-x-22-1-2-x-1-4-jumbo-fold-over-mailers>: HTTP status code is not handled or not allowed 2026-01-24 21:45:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-hose-assembly-smooth-for-pacific-steamex-67512690>: HTTP status code is not handled or not allowed 2026-01-24 21:45:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-industry-standard-clamping-couplings-climax-2iscc-137-137-s>: HTTP status code is not handled or not allowed 2026-01-24 21:45:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-x-4-x-60-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 21:45:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-drain-hose-w-drain-cap-for-minuteman-245452>: HTTP status code is not handled or not allowed 2026-01-24 21:45:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polymer-partition-panel-wo-brackets-5412w-olive>: HTTP status code is not handled or not allowed 2026-01-24 21:45:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-climax-1c-225-s>: HTTP status code is not handled or not allowed 2026-01-24 21:45:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-x-7-x-8-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 21:45:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polymer-pilaster-w-shoe-5w-x-82h-hunter-green>: HTTP status code is not handled or not allowed 2026-01-24 21:45:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-g-10-fr-4-garolite-rod-1-2-diameter-x-1-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 21:45:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alera-iv-series-guest-chairs-2480-x-2283-x-3228-black-2carton>: HTTP status code is not handled or not allowed 2026-01-24 21:45:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50350-odin-safety-glasses-kryptek-frameyellow-lense>: HTTP status code is not handled or not allowed 2026-01-24 21:45:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-climax-h2c-375>: HTTP status code is not handled or not allowed 2026-01-24 21:45:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/559-9-64-cobalt-heavy-duty-straw-135-split-point-screw-machine-drill>: HTTP status code is not handled or not allowed 2026-01-24 21:45:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-flame-resistant-button-up-safety-shirt-cottonnylon-3xl-hi-vis-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:45:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h1-replacement-chin-strap-novent-bivent>: HTTP status code is not handled or not allowed 2026-01-24 21:45:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-xx-garolite-sheet-1-8-thick-x-24-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 21:45:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-microfiber-cleaning-cloths-bluewhite-stripes-12-x-12-600pack>: HTTP status code is not handled or not allowed 2026-01-24 21:45:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/versasure-cleaner-disinfectant-wipes-1-ply-6-34-x-8-white-85-towelscan>: HTTP status code is not handled or not allowed 2026-01-24 21:45:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/protective-cap-staz-on-suspension-lamp-bracket-cord-holder-std-natural-tan>: HTTP status code is not handled or not allowed 2026-01-24 21:45:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-shovel-10-in-blade-purple>: HTTP status code is not handled or not allowed 2026-01-24 21:45:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/virex-all-purpose-disinfectant-cleaner-lemon-scent-32-oz-spray-bottle-4carton>: HTTP status code is not handled or not allowed 2026-01-24 21:45:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-x-11-x-7-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 21:45:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-climax-1c-175-a>: HTTP status code is not handled or not allowed 2026-01-24 21:45:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medaphene-plus-disinfectant-spray-155-oz-aerosol-spray-12carton>: HTTP status code is not handled or not allowed 2026-01-24 21:45:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-x-14-x-8-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 21:45:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-laminate-pilaster-w-shoe-12w-x-82h-taupe>: HTTP status code is not handled or not allowed 2026-01-24 21:45:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o-ir-higher-performance-polyolefin-cf-shrink-film-100-ga-10w-x-2620l-clear-1-roll>: HTTP status code is not handled or not allowed 2026-01-24 21:45:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/squids-3775-can-bottle-holder-and-trap-black>: HTTP status code is not handled or not allowed 2026-01-24 21:45:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cast-iron-multistage-booster-pump-1-2-hp>: HTTP status code is not handled or not allowed 2026-01-24 21:45:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metal-frame-for-full-brim-helmets>: HTTP status code is not handled or not allowed 2026-01-24 21:45:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrosion-resistant-set-screw-collar-cr-climax-crc-015-s>: HTTP status code is not handled or not allowed 2026-01-24 21:45:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-1-2-x-12-1-2-x-6-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 21:45:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/visor-propionate-clear-molded-8-inch-w-x-17-inch-l-x-098-inch-h>: HTTP status code is not handled or not allowed 2026-01-24 21:45:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ppr-2r-free-flow-pourers-red>: HTTP status code is not handled or not allowed 2026-01-24 21:45:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-climax-h2c-450>: HTTP status code is not handled or not allowed 2026-01-24 21:45:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-x-16-x-12-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 21:45:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pprc-1w-pp-round-storage-container-polypropelyne-white>: HTTP status code is not handled or not allowed 2026-01-24 21:45:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wb-8-wine-bucket-8-qt-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:45:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-padding-set-for-v-gard-h2-safety-helmet-black>: HTTP status code is not handled or not allowed 2026-01-24 21:45:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hand-scoop-32-oz-green>: HTTP status code is not handled or not allowed 2026-01-24 21:45:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wbkg-9-rectangular-black-wire-fruit-basket>: HTTP status code is not handled or not allowed 2026-01-24 21:45:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v-gard-h2-safety-helmet-vented-climbing-class-c-type-2-white>: HTTP status code is not handled or not allowed 2026-01-24 21:45:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cordless-5-random-orbital-sander-bare-tool-24v>: HTTP status code is not handled or not allowed 2026-01-24 21:45:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dwa8933-xp-ceramic-metal-grinding-wheels-type-27-4-1-2-inch-x-7-8-inch-24-grit-ceramic>: HTTP status code is not handled or not allowed 2026-01-24 21:45:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silver-duct-tape-2-88-x-30-yards>: HTTP status code is not handled or not allowed 2026-01-24 21:45:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dwa8062f-hp-metal-cutting-wheels-type-1-4-1-2-inch-x-7-8-inch-aluminum-oxide-60-grit->: HTTP status code is not handled or not allowed 2026-01-24 21:45:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nashville-wire-steel-security-cart--48w-x-24d-x-70h>: HTTP status code is not handled or not allowed 2026-01-24 21:45:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stack-pack-height-adjustment-rack-rail-26-916l-x-4-34w-x-10-14h-black-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-24 21:45:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-glue-15-grams>: HTTP status code is not handled or not allowed 2026-01-24 21:45:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-laminate-pilaster-w-shoe-6w-x-82h-almond>: HTTP status code is not handled or not allowed 2026-01-24 21:45:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/original-glue-8-ounces>: HTTP status code is not handled or not allowed 2026-01-24 21:45:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ps-32-utility-scoop-32-oz-clear-polycarbonate>: HTTP status code is not handled or not allowed 2026-01-24 21:45:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-x-24-x-20-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 21:45:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-climax-1c-187-s>: HTTP status code is not handled or not allowed 2026-01-24 21:45:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/28-x-6-x-6-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 21:45:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tmt-rf4-dial-refrigerator-freezer-thermometer>: HTTP status code is not handled or not allowed 2026-01-24 21:45:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/patriot-fully-framed-ta-50-locker-wsolid-doors-sides-36wx24dx78h-redall-welded>: HTTP status code is not handled or not allowed 2026-01-24 21:45:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/psb-08c-squeeze-bottles-8-oz>: HTTP status code is not handled or not allowed 2026-01-24 21:45:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/patriot-fully-framed-ta-50-locker-wsolid-doors-sides-36wx24dx78h-blueall-welded>: HTTP status code is not handled or not allowed 2026-01-24 21:45:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/patriot-fully-framed-ta-50-locker-wexpanded-metal-sides-42wx24dx78h-rdall-welded>: HTTP status code is not handled or not allowed 2026-01-24 21:45:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-x-24-x-12-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 21:46:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/w-series-8-direct-heat-foot-sealer-standing-operation-15mm-meshed-seal-width>: HTTP status code is not handled or not allowed 2026-01-24 21:46:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-guardrail-stack-rack-kit-11-guardails-and-12-ucc-bases-5l-rails-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:46:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o-201-general-purpose-polyolefin-cf-shrink-film-75-ga-10w-x-3500l-clear-1-roll>: HTTP status code is not handled or not allowed 2026-01-24 21:46:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-linen-le-phenolic-sheet-0-750-thick-48-x-48>: HTTP status code is not handled or not allowed 2026-01-24 21:46:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-fluid-level-sight-glass-w-o-reflector-g-1-1-4-pipe-thread-r09-b>: HTTP status code is not handled or not allowed 2026-01-24 21:46:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rx723019rcgnbk-250-professional-72-19-drawer-green-triple-bank-roller-cabinet>: HTTP status code is not handled or not allowed 2026-01-24 21:46:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnetic-glass-board-36-x-24>: HTTP status code is not handled or not allowed 2026-01-24 21:46:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-canvas-ce-phenolic-rod-1-750-dia-x-48-000-inch>: HTTP status code is not handled or not allowed 2026-01-24 21:46:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/patriot-fully-framed-gear-locker-w-security-box-48wx24dx72h-red-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 21:46:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-canvas-ce-phenolic-sheet-0-187-thick-48-x-96>: HTTP status code is not handled or not allowed 2026-01-24 21:46:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelf-vented-sides-24wx18dx76h-red-all-welded>: HTTP status code is not handled or not allowed 2026-01-24 21:46:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smartair-ductless-mini-split-ac-heater-wi-fi-control-12000-btu-25-installation-kit-white>: HTTP status code is not handled or not allowed 2026-01-24 21:46:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anti-static-tube-brush-black-80-x-95-x-119mm-c27132>: HTTP status code is not handled or not allowed 2026-01-24 21:46:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 26603 pages (at 103 pages/min), scraped 13648 items (at 20 items/min) 2026-01-24 21:46:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o-201-general-purpose-polyolefin-cf-shrink-film-60-ga-8w-x-4375l-clear-1-roll>: HTTP status code is not handled or not allowed 2026-01-24 21:46:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-laminate-pilaster-w-shoe-5w-x-82h-silver-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:46:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/patriot-fully-framed-gear-locker-w-2-coat-rods-36wx24dx72h-blue-all-welded>: HTTP status code is not handled or not allowed 2026-01-24 21:46:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-climax-2c-243>: HTTP status code is not handled or not allowed 2026-01-24 21:46:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16in-high-speed-surfacing-machine-1-5-hp-3450-rpm-496898>: HTTP status code is not handled or not allowed 2026-01-24 21:46:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-2-x-18-kra-crimped-end-tubes-case>: HTTP status code is not handled or not allowed 2026-01-24 21:46:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelf-vented-sides-24wx18dx72h-blue-all-welded>: HTTP status code is not handled or not allowed 2026-01-24 21:46:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-x-24-kra-3-piece-telescoping-tubes-case>: HTTP status code is not handled or not allowed 2026-01-24 21:46:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-nyloil-fg-cast-nylon-tube-6-500-id-x-7-000-od-ft>: HTTP status code is not handled or not allowed 2026-01-24 21:46:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/work-table-24-wide-top-w-turned-up-edge-at-rear-30-long-14-gauge-1-12-rear-splash>: HTTP status code is not handled or not allowed 2026-01-24 21:46:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/patriot-fully-framed-duty-locker-24wx24dx90h-blue-all-welded>: HTTP status code is not handled or not allowed 2026-01-24 21:46:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/upright-controlled-room-temperature-cabinet-glass-door-10-5-cu-ft>: HTTP status code is not handled or not allowed 2026-01-24 21:46:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/built-in-undercounter-controlled-room-temperature-cabinet-left-hinged-solid-door-4-6-cu-ft>: HTTP status code is not handled or not allowed 2026-01-24 21:46:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/step-down-clamp-on-adaptor-sda-series-climax-sda-075>: HTTP status code is not handled or not allowed 2026-01-24 21:46:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4th-level-add-on-for-tapered-transport-riser-gray-carpet-with-black-aluminum-frame>: HTTP status code is not handled or not allowed 2026-01-24 21:46:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11-920-8-coated-work-gloves-nitrile-grip-15-gauge-size-7-medium-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:46:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-x-18-white-mailing-tubes-with-caps-case>: HTTP status code is not handled or not allowed 2026-01-24 21:46:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/step-down-clamp-on-adaptor-sda-series-climax-sda-100>: HTTP status code is not handled or not allowed 2026-01-24 21:46:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/surface_property_testing already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:46:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-paper-phenolic-sheet-0-375-thick-36-x-48>: HTTP status code is not handled or not allowed 2026-01-24 21:46:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rigid-coupling-4-set-screws-2-90-rc4h-series-climax-rc-050-4h90>: HTTP status code is not handled or not allowed 2026-01-24 21:46:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/window-door-siding-100-silicone-rubber-sealant-9-8-oz-black-7079808642>: HTTP status code is not handled or not allowed 2026-01-24 21:46:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-two-piece-clamping-collar-climax-m2c-19>: HTTP status code is not handled or not allowed 2026-01-24 21:46:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tool-kit-blue-7079809125>: HTTP status code is not handled or not allowed 2026-01-24 21:46:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-46-octagonal-outdoor-steel-picnic-table-expanded-metal-black>: HTTP status code is not handled or not allowed 2026-01-24 21:46:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/100-waterproof-window-door-trim-sealant-10-1-oz-almond-7079818288>: HTTP status code is not handled or not allowed 2026-01-24 21:46:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelf-security-box-18wx18dx72h-red-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 21:46:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qf-clamp-with-pin-6-dia-galvanized-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:46:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reborn-collection-lightweight-backpack-polyester-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:46:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/testing_inspection_instruments/surface_property_testing>: HTTP status code is not handled or not allowed 2026-01-24 21:46:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0034-11-stanford-table-fork>: HTTP status code is not handled or not allowed 2026-01-24 21:46:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0034-18-stanford-hollow-handle-table-knife>: HTTP status code is not handled or not allowed 2026-01-24 21:46:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11-7-8-x-11-7-8-stock-pad>: HTTP status code is not handled or not allowed 2026-01-24 21:46:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelf-footlocker-18wx18dx78h-blue-all-welded>: HTTP status code is not handled or not allowed 2026-01-24 21:46:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/through-the-wall-depository-drop-box-wdd-180e-electronic-lock-14w-x-12d-x-15h-beige>: HTTP status code is not handled or not allowed 2026-01-24 21:46:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/king-economy-8-section-with-t-hinge-privacy-screen>: HTTP status code is not handled or not allowed 2026-01-24 21:46:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/strapping-protectors-4-x-4-x-2-0-160-thickness-480-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:46:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-sided-bulk-truck-mesh-sides-4-lipped-shelves-30-48-6-poly-wheels>: HTTP status code is not handled or not allowed 2026-01-24 21:46:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-climax-2c-168-z>: HTTP status code is not handled or not allowed 2026-01-24 21:46:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2075r-reinforced-drag-fork>: HTTP status code is not handled or not allowed 2026-01-24 21:46:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/doral-task-chair-mid-back-articulating-tilt-21-26h-seat-blackchrome>: HTTP status code is not handled or not allowed 2026-01-24 21:46:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/welded-5-shelf-service-cart-lipped-shelves-1200-lbs-18-32-poly-wheels>: HTTP status code is not handled or not allowed 2026-01-24 21:46:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/calci-solv-descaler-for-hvac-acidic-scent-gallon-4-case-nl001-g4>: HTTP status code is not handled or not allowed 2026-01-24 21:46:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0016-02-winston-teaspoon>: HTTP status code is not handled or not allowed 2026-01-24 21:46:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-radex-1x1-2-reducing-nipple>: HTTP status code is not handled or not allowed 2026-01-24 21:46:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelate-silver-epoxy-wire-shelf-48x14>: HTTP status code is not handled or not allowed 2026-01-24 21:46:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/44rd-stainless-steel-scoop-16-oz>: HTTP status code is not handled or not allowed 2026-01-24 21:46:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cortina-28-pop-up-traffic-cone-led-light-orange-03-500-80>: HTTP status code is not handled or not allowed 2026-01-24 21:46:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bus-smoking-shelter-flat-roof-3-side-open-front-with-gray-5-gallon-outdoor-ashtray-6-5-wx3-8-dx7-h>: HTTP status code is not handled or not allowed 2026-01-24 21:46:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:46:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/97603-l-connect-all-strap-large>: HTTP status code is not handled or not allowed 2026-01-24 21:46:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0485-chillbreaker-high-bib-overall-regular-navy-4xl>: HTTP status code is not handled or not allowed 2026-01-24 21:46:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-threaded-clamping-collar-recessed-screw-climax-tc-087-09>: HTTP status code is not handled or not allowed 2026-01-24 21:46:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bspb-13-perforated-basting-spoon-w-bakelite-handle-13l-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:46:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/everlast-task-chair-mid-back-articulating-tilt-19-24h-seat-bluechrome>: HTTP status code is not handled or not allowed 2026-01-24 21:46:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-inch-brick-corner-edge-guard-protector-48l-x-24w-x-9h-orange>: HTTP status code is not handled or not allowed 2026-01-24 21:46:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/132c-ice-logger-boot-regular-tan-6-5>: HTTP status code is not handled or not allowed 2026-01-24 21:46:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-silenced-cool-air-tube-assembly>: HTTP status code is not handled or not allowed 2026-01-24 21:46:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergodyne-glowear-8205hl-eco-recycled-hi-vis-safety-vest-sm-lime>: HTTP status code is not handled or not allowed 2026-01-24 21:46:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-gx4-with-10ppm-co-sensor>: HTTP status code is not handled or not allowed 2026-01-24 21:46:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0488-insulated-quilted-sweatshirt-regular-navy-2xl>: HTTP status code is not handled or not allowed 2026-01-24 21:46:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelf-21x72-nexelon>: HTTP status code is not handled or not allowed 2026-01-24 21:46:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-threaded-clamping-collar-recessed-screw-climax-tc-087-14>: HTTP status code is not handled or not allowed 2026-01-24 21:46:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bioair-plus-air-scrubber-negative-air-machine-with-dual-uv-lights-1800-cfm>: HTTP status code is not handled or not allowed 2026-01-24 21:46:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 21:46:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/azar-300195-vertical-horizontal-door-sign-holder-5-w-x-1-4-d-x-7-h>: HTTP status code is not handled or not allowed 2026-01-24 21:47:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/max-lift-standard-scissor-lift-table-hand-operated-2000-lb-capacity-64l-x-48w>: HTTP status code is not handled or not allowed 2026-01-24 21:47:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interion-174-36-round-steel-mesh-outdoor-caf-table-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:47:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cbh-1824-cutting-board-18l-24w-3-4h-white>: HTTP status code is not handled or not allowed 2026-01-24 21:47:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tl-kit-with-low-profile-yellow-hood-he-filters-extended-battery>: HTTP status code is not handled or not allowed 2026-01-24 21:47:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/maxicut-ultra-micro-foam-nitrile-coated-palm-fingers-gloves-ansi-a4-xs-gray-6-dozencase>: HTTP status code is not handled or not allowed 2026-01-24 21:47:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbw7000-resin-wicker-outdoor-storage-deck-box-71-7-gallon-47l-x-23w-x-21h>: HTTP status code is not handled or not allowed 2026-01-24 21:47:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-minute-cylinder-quick-connect-aluminum-with-air>: HTTP status code is not handled or not allowed 2026-01-24 21:47:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-one-piece-clamping-collar-climax-m1c-45-s>: HTTP status code is not handled or not allowed 2026-01-24 21:47:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-zlink-kit-fr-shoulder-cape-c40>: HTTP status code is not handled or not allowed 2026-01-24 21:47:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bash-soft-face-sledge-hammer-25-lb-head-12l-unbreakable-steel-core-handle>: HTTP status code is not handled or not allowed 2026-01-24 21:47:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-two-piece-clamping-collar-climax-m2c-03>: HTTP status code is not handled or not allowed 2026-01-24 21:47:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0442-chillbreaker-parka-regular-navy-3xl>: HTTP status code is not handled or not allowed 2026-01-24 21:47:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/azar-700728-gre-two-sided-pegboard-floor-display-w-studio-base-14-1-2-w-x-14-d-x-44-h>: HTTP status code is not handled or not allowed 2026-01-24 21:47:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 26699 pages (at 96 pages/min), scraped 13667 items (at 19 items/min) 2026-01-24 21:47:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-motorola-cable-multi-pin>: HTTP status code is not handled or not allowed 2026-01-24 21:47:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dxs2153-metric-deep-impact-socket-17mm-drive-black>: HTTP status code is not handled or not allowed 2026-01-24 21:47:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gray-wall-bracket>: HTTP status code is not handled or not allowed 2026-01-24 21:47:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/little-bin-plastic-stacking-bins-1-3-4-x-3-1-4-x-3-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:47:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safe-t-gard-white-1-2-fold-toilet-seat-covers-250-covers-pack-20-packs-case>: HTTP status code is not handled or not allowed 2026-01-24 21:47:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-one-piece-clamping-collar-climax-m1c-54>: HTTP status code is not handled or not allowed 2026-01-24 21:47:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/azar-700906-display-bowl-14-w-x-14-d-x-7-1-2-h>: HTTP status code is not handled or not allowed 2026-01-24 21:47:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-industry-standard-clamping-couplings-climax-iscc-137-137-s>: HTTP status code is not handled or not allowed 2026-01-24 21:47:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/respiratory_protection/powered_air_purifying_respirator_systems already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:47:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-dividable-plastic-tote-box-22-l-x-17-w-x-10-h-2>: HTTP status code is not handled or not allowed 2026-01-24 21:47:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/azar-700912-bowl-for-pegboard-slatwall-8-w-x-13-1-4-d-x-7-1-2-h>: HTTP status code is not handled or not allowed 2026-01-24 21:47:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-x-14-x-8-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 21:47:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/post-clamp-7-16-space-black>: HTTP status code is not handled or not allowed 2026-01-24 21:47:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-nova-3-cool-tube-xl-leather-cape>: HTTP status code is not handled or not allowed 2026-01-24 21:47:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jlp-150a-20-lever-hoist-1-12-ton-capacity-20-lift>: HTTP status code is not handled or not allowed 2026-01-24 21:47:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-gallon-solution-tank-floor-machines>: HTTP status code is not handled or not allowed 2026-01-24 21:47:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sanding-belt-aluminum-oxide-60-grit-48l-x-2w-pack-of-3>: HTTP status code is not handled or not allowed 2026-01-24 21:47:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0344l2-hivis-minus-50-suit-regular-hivis-lime-yellow-xl>: HTTP status code is not handled or not allowed 2026-01-24 21:47:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-zlink-tychem-2000-face-seal>: HTTP status code is not handled or not allowed 2026-01-24 21:47:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/respiratory_protection/powered_air_purifying_respirator_systems>: HTTP status code is not handled or not allowed 2026-01-24 21:47:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lightweight-set-screw-collar-l-climax-lc-125>: HTTP status code is not handled or not allowed 2026-01-24 21:47:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/28-gallon-square-rubbermaid-brute-waste-receptacles-red>: HTTP status code is not handled or not allowed 2026-01-24 21:47:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-zlink-respirator-with-chin-seal>: HTTP status code is not handled or not allowed 2026-01-24 21:47:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dxs2153-metric-deep-impact-socket-24mm-drive-black>: HTTP status code is not handled or not allowed 2026-01-24 21:47:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conference-arm-chair-with-casters-vinyl-black>: HTTP status code is not handled or not allowed 2026-01-24 21:47:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-shelf-machine-table-60-x-30>: HTTP status code is not handled or not allowed 2026-01-24 21:47:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-climax-c-018-dt>: HTTP status code is not handled or not allowed 2026-01-24 21:47:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/129c-enduramax-boot-regular-black-8>: HTTP status code is not handled or not allowed 2026-01-24 21:47:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-inch-x-5-yards-self-adhesive-bandage-non-latex-dark-purple-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-24 21:47:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-behind-head-band-quiet-slim-earmuffs>: HTTP status code is not handled or not allowed 2026-01-24 21:47:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ics-1-ice-cream-spreader>: HTTP status code is not handled or not allowed 2026-01-24 21:47:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/work-height-platform-truck-with-wood-deck-24-x-48-fixed-height>: HTTP status code is not handled or not allowed 2026-01-24 21:47:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s90-150-15-hand-chain-hoist-1-12-ton-capacity-15-lift>: HTTP status code is not handled or not allowed 2026-01-24 21:47:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hs003f-heartsine-360p-full-auto-defibrillator-package>: HTTP status code is not handled or not allowed 2026-01-24 21:47:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-inch-x-5-yards-self-adhesive-bandage-non-latex-tan-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-24 21:47:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cantilever-rack-single-sided-upright-43>: HTTP status code is not handled or not allowed 2026-01-24 21:47:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-by-24-stage-package-24-inch-h-hardboard-floor-black-shirred-pleat-skirting>: HTTP status code is not handled or not allowed 2026-01-24 21:47:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cmt-1217-octagonal-serving-tray-17l-chrome-gadroon-edge-w-engraving>: HTTP status code is not handled or not allowed 2026-01-24 21:47:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-recessed-screw-climax-h1c-081-s>: HTTP status code is not handled or not allowed 2026-01-24 21:47:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-x-39-40-45-gallon-hot-water-soluble-bag>: HTTP status code is not handled or not allowed 2026-01-24 21:47:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swa-1-steak-weight-8-1-4l-4-1-4w-wood-handle-cast-aluminum>: HTTP status code is not handled or not allowed 2026-01-24 21:47:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-one-piece-clamping-collar-climax-m1c-20>: HTTP status code is not handled or not allowed 2026-01-24 21:47:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cantilever-rack-horizontal-brace-set-34>: HTTP status code is not handled or not allowed 2026-01-24 21:47:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g-104-oil-vinegar-cruets-with-cone-tops>: HTTP status code is not handled or not allowed 2026-01-24 21:47:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g-105-paneled-shakers-w-mushroom-tops>: HTTP status code is not handled or not allowed 2026-01-24 21:47:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-climax-c-112-dt>: HTTP status code is not handled or not allowed 2026-01-24 21:47:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-locker-single-tier-12x12x72-without-legs-3-doors-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:47:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1114-regulator-38-npt-250-psi-gauge-not-included>: HTTP status code is not handled or not allowed 2026-01-24 21:47:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1113-regulator-14-npt-250-psi-gauge-not-included>: HTTP status code is not handled or not allowed 2026-01-24 21:47:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1308600-universal-nylon-markers>: HTTP status code is not handled or not allowed 2026-01-24 21:47:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swa-9-steak-weight-9l-5-1-4w-wood-handle-cast-iron>: HTTP status code is not handled or not allowed 2026-01-24 21:47:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-green-2-tier-wire-shelving-starter-unit-36-w-x-12-d-x-34-h>: HTTP status code is not handled or not allowed 2026-01-24 21:47:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/23070-cut-off-wheel-type-1-a60t-4-inch-x-035-inch-x-3-8-inch-60-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-24 21:47:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-climax-c-156>: HTTP status code is not handled or not allowed 2026-01-24 21:47:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-roll-no-adhesive-60a-12-thick-x-36-wide-x-8-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 21:47:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/outdoor_lighting_fixtures/wall_pack_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:47:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/d217rb-osha-sign-danger-no-smoking-matches-or-open-flames-10-x-14-whiteredblack>: HTTP status code is not handled or not allowed 2026-01-24 21:47:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-sheet-no-adhesive-60a-12-thick-x-24-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 21:47:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ccs-8f-china-cap-strainer-fine-mesh>: HTTP status code is not handled or not allowed 2026-01-24 21:47:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-sheet-no-adhesive-60a-14-thick-x-36-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 21:47:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-green-2-tier-wire-shelving-add-on-unit-24-w-x-24-d-x-34-h>: HTTP status code is not handled or not allowed 2026-01-24 21:47:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50128-3a-fiber-disc-5-inch-x-7-8-inch-36-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-24 21:47:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/28115-bench-wheel-vitrified-7-inch-x-1-inch-x-1-inch-120-grit-silicon-carbide>: HTTP status code is not handled or not allowed 2026-01-24 21:47:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9240016-universal-spinner>: HTTP status code is not handled or not allowed 2026-01-24 21:47:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-tier-shelving-add-on-unit-solid-galvanized-steel-54w-x-24d-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 21:48:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-cord-spring-rewind-reels-duplex-industrial-receptacle-75-cord-12-awg>: HTTP status code is not handled or not allowed 2026-01-24 21:48:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/outdoor_lighting_fixtures/wall_pack_lights>: HTTP status code is not handled or not allowed 2026-01-24 21:48:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mpd-24-mixing-paddle>: HTTP status code is not handled or not allowed 2026-01-24 21:48:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/hose_cord_reels/power_cord_reels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:48:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22073-depressed-center-wheel-t27-5-inch-x-045-inch-x-7-8-inch-ceramic-alum-oxide>: HTTP status code is not handled or not allowed 2026-01-24 21:48:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20015-depressed-center-wheel-t27-a24n-4-inch-x-1-8-inch-x-5-8-inch-24-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-24 21:48:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/viton-rubber-roll-no-adhesive-75a-116-thick-x-36-wide-x-6-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 21:48:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mas-4r-round-ashtray-4d-red-plastic>: HTTP status code is not handled or not allowed 2026-01-24 21:48:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-threaded-clamping-collar-climax-istc-037-24>: HTTP status code is not handled or not allowed 2026-01-24 21:48:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/viton-rubber-roll-no-adhesive-75a-316-thick-x-36-wide-x-10-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 21:48:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0005-10-dots-table-spoon>: HTTP status code is not handled or not allowed 2026-01-24 21:48:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-industry-standard-clamping-couplings-climax-iscc-062-062-s>: HTTP status code is not handled or not allowed 2026-01-24 21:48:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 26794 pages (at 95 pages/min), scraped 13688 items (at 21 items/min) 2026-01-24 21:48:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/viton-rubber-roll-no-adhesive-75a-132-thick-x-36-wide-x-8-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 21:48:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/viton-rubber-roll-no-adhesive-75a-116-thick-x-36-wide-x-10-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 21:48:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-industry-standard-clamping-couplings-climax-iscc-075-050>: HTTP status code is not handled or not allowed 2026-01-24 21:48:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/abbc-6-6-tier-bus-box-cart>: HTTP status code is not handled or not allowed 2026-01-24 21:48:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dandux-vinyl-basket-truck-bulk-truck-6-bushel-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:48:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classroom-connector-school-to-home-folders-purple-25-box>: HTTP status code is not handled or not allowed 2026-01-24 21:48:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/hose_cord_reels/power_cord_reels landed on page that is not a product page. 2026-01-24 21:48:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-5-swivel-plate-caster-with-hardware-includes-1-caster-4-lock-nuts-and-4-plow-bolts>: HTTP status code is not handled or not allowed 2026-01-24 21:48:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/viton-rubber-roll-no-adhesive-75a-14-thick-x-36-wide-x-9-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 21:48:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:48:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spring-rewind-hose-reel-for-air-water-1-2-i-d-25-hose-capacity-less-hose-300-psi>: HTTP status code is not handled or not allowed 2026-01-24 21:48:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fiberglass-stacking-box-gray-18-3-4x11-7-8x1-3-4>: HTTP status code is not handled or not allowed 2026-01-24 21:48:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-b-cart-cm-machinist-tool-cart-1500-lb-cap-24w-x-12d-x-12h-blueblack>: HTTP status code is not handled or not allowed 2026-01-24 21:48:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fiberglass-stacking-box-green-16-1-2x11-3-8x4-5-8>: HTTP status code is not handled or not allowed 2026-01-24 21:48:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50172-az-x-fiber-disc-7-inch-x-7-8-inch-50-grit-zirconium>: HTTP status code is not handled or not allowed 2026-01-24 21:48:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alxp-1826-aluminum-sheet-pan>: HTTP status code is not handled or not allowed 2026-01-24 21:48:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-sheet-with-acrylic-adhesive-70a-116-thick-x-36-wide-x-36-long>: HTTP status code is not handled or not allowed 2026-01-24 21:48:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-recessed-screw-climax-h1c-131-s>: HTTP status code is not handled or not allowed 2026-01-24 21:48:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-climax-c-068>: HTTP status code is not handled or not allowed 2026-01-24 21:48:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-climax-c-068-a>: HTTP status code is not handled or not allowed 2026-01-24 21:48:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-spring-rewind-hose-reel-34-i-d-50-hose-capacity-less-hose-300-psi>: HTTP status code is not handled or not allowed 2026-01-24 21:48:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-cord-spring-rewind-reels-single-industrial-receptacle-50-cord-12-awg>: HTTP status code is not handled or not allowed 2026-01-24 21:48:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spring-rewind-static-discharge-cable-reel-100-stainless-steel-cable>: HTTP status code is not handled or not allowed 2026-01-24 21:48:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/84255-water-proof-sanding-sheets-9-inch-x-11-inch-240-grit-silicon-carbide>: HTTP status code is not handled or not allowed 2026-01-24 21:48:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 21:48:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-cord-spring-rewind-reels-100-cord-12-awg-less-accessory>: HTTP status code is not handled or not allowed 2026-01-24 21:48:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/csc-8-quietmaster-caster-with-swivel-plate-and-brake-4-x-4-1-2-plate-600-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:48:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fiberglass-tote-lid-42-1-2x20-white-2>: HTTP status code is not handled or not allowed 2026-01-24 21:48:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/81820-da-f-shop-roll-2-inch-x-50-yds-180-grit-handy-roll-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-24 21:48:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zirconium-oxide-belt-2-x-48-180-grit-pack-of-3>: HTTP status code is not handled or not allowed 2026-01-24 21:48:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-roll-with-acrylic-adhesive-50a-14-thick-x-36-w-x-10-ft-l>: HTTP status code is not handled or not allowed 2026-01-24 21:48:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-sheet-with-acrylic-adhesive-60a-116-thick-x-36-wide-x-36-long>: HTTP status code is not handled or not allowed 2026-01-24 21:48:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/84252-water-proof-sanding-sheets-9-inch-x-11-inch-150-grit-silicon-carbide>: HTTP status code is not handled or not allowed 2026-01-24 21:48:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tempco-immersion-heater-circulation-18000-480>: HTTP status code is not handled or not allowed 2026-01-24 21:48:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tempco-immersion-heater-tank-4000-240v-304-ss>: HTTP status code is not handled or not allowed 2026-01-24 21:48:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-recessed-screw-climax-h2c-156-s>: HTTP status code is not handled or not allowed 2026-01-24 21:48:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-tier-shoe-rack-espresso>: HTTP status code is not handled or not allowed 2026-01-24 21:48:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hook-swivel-w-latch-3-4t-wll-carbon>: HTTP status code is not handled or not allowed 2026-01-24 21:48:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50160-2a-fiber-disc-4-1-2-inch-x-7-8-inch-80-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-24 21:48:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-row-national-rep-tip-and-roll-aluminum-bleacher-15-wide-double-footboard>: HTTP status code is not handled or not allowed 2026-01-24 21:48:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50198-9s-fiber-disc-4-1-2-inch-x-7-8-inch-80-grit-ceramic>: HTTP status code is not handled or not allowed 2026-01-24 21:48:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-piece-cotton-coil-baskets-black>: HTTP status code is not handled or not allowed 2026-01-24 21:48:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-strip-with-acrylic-adhesive-60a-38-thick-x-6-wide-x-5-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 21:48:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-one-piece-clamping-collar-climax-m1c-15-s>: HTTP status code is not handled or not allowed 2026-01-24 21:48:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cantilever-rack-double-sided-upright-47>: HTTP status code is not handled or not allowed 2026-01-24 21:48:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/84913-da-f-sanding-sheets-9-inch-x-11-inch-daf-120-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-24 21:48:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-terrain-vise153-with-swivel-base-5-jaw-width-4-14-throat-depth>: HTTP status code is not handled or not allowed 2026-01-24 21:48:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-stacking-bin-16-1-2-x-18-x-11-black>: HTTP status code is not handled or not allowed 2026-01-24 21:48:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-strip-with-acrylic-adhesive-60a-18-thick-x-4-wide-x-5-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 21:48:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-row-national-rep-aluminum-bleacher-15-wide-double-footboard>: HTTP status code is not handled or not allowed 2026-01-24 21:48:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-42-removable-handle-global-aluminum-platform-trucks>: HTTP status code is not handled or not allowed 2026-01-24 21:48:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-roll-no-adhesive-70a-18-thick-x-36-wide-x-8-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 21:48:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/450p-multi-purpose-bench-vise-with-swivel-base-4-12-jaw-width-2-14-throat-depth>: HTTP status code is not handled or not allowed 2026-01-24 21:48:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-diamond-deck-truck-56-x-42-1000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:48:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/digital-veterinary-scale-330-lb-x-2-lb-with-wheels-and-handle>: HTTP status code is not handled or not allowed 2026-01-24 21:48:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-standard-soft-face-hammer-281-lb-head-12-34-steel-handle>: HTTP status code is not handled or not allowed 2026-01-24 21:48:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-sheet-with-acrylic-adhesive-50a-18-thick-x-36-wide-x-36-long>: HTTP status code is not handled or not allowed 2026-01-24 21:48:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/l100-1500wo-10-hand-chain-hoist-with-overload-protection-15-ton-capacity-10-lift>: HTTP status code is not handled or not allowed 2026-01-24 21:48:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acid-resistant-spray-applicator-2-gallon-capacity-1-case>: HTTP status code is not handled or not allowed 2026-01-24 21:48:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/din-580-lifting-eye-bolts-ss-shoulder-matte-m12-2-1-16-h-1-2-t-13-16-l>: HTTP status code is not handled or not allowed 2026-01-24 21:48:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g-209a-galvanized-alloy-shackle-spa-1-12-1-2t-wll>: HTTP status code is not handled or not allowed 2026-01-24 21:48:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/end-table-medium-oak>: HTTP status code is not handled or not allowed 2026-01-24 21:48:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/desk-calendar-refill-with-tabs-3-1-2-x-6-2014>: HTTP status code is not handled or not allowed 2026-01-24 21:48:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gn-1524-swing-bolts-steel-w-extended-thread-length-blackened-m8-2-3-8-l-5-16-i-d>: HTTP status code is not handled or not allowed 2026-01-24 21:48:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/d1-6-4-jaw-chuck-for-elite-17-geared-head-lathe-12-dia>: HTTP status code is not handled or not allowed 2026-01-24 21:48:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gd-12254-garage-door-cable-clamps-12-galvanizedpack-of-2>: HTTP status code is not handled or not allowed 2026-01-24 21:48:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/din-6319-spherical-washers-steel-dished-type-blackened-m16-1-4-t-3-4-i-d>: HTTP status code is not handled or not allowed 2026-01-24 21:48:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-342-alloy-masterlink-oblong-wide-1-x-7-5-x-4-3-26000-lbs-wll>: HTTP status code is not handled or not allowed 2026-01-24 21:48:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leeson-3-phase-washguard-duty-motor-5hp-3515rpm-184tc-tefc-208-230-460v-60-50hz-rigid-c>: HTTP status code is not handled or not allowed 2026-01-24 21:48:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/112-quad-ring-xring-12id-x-1116od-70-duro-round-black-pkg-100>: HTTP status code is not handled or not allowed 2026-01-24 21:48:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-strip-with-acrylic-adhesive-50a-38-thick-x-2-wide-x-5-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 21:48:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/118-quad-ring-xring-78id-x-1116od-70-duro-round-black-pkg-100>: HTTP status code is not handled or not allowed 2026-01-24 21:48:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n-6549-closet-door-bottom-guide-1-left-1-right>: HTTP status code is not handled or not allowed 2026-01-24 21:48:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/220-quad-ring-xring-138id-x-158od-70-duro-round-black-pkg-100>: HTTP status code is not handled or not allowed 2026-01-24 21:48:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gn-749-threaded-pipe-plugs-steel-zinc-plated-with-nbr-synthetic-rubber-seal-g-1-2>: HTTP status code is not handled or not allowed 2026-01-24 21:49:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-cart-poly-green-24l-x-24w-x-43h-pneumatic-rigid-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:49:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/biohazard-specimen-bags-12x15-1000-case-3-ply>: HTTP status code is not handled or not allowed 2026-01-24 21:49:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-cart-poly-green-36l-x-18w-x-40h-polyurethane-brake-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:49:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n-6519-closet-door-roller-1-left-1-right-78-inch-nylon-ball-bearing-wheel>: HTTP status code is not handled or not allowed 2026-01-24 21:49:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wn-751-clevis-fork-joints-steel-w-snap-on-spring-pin-zinc-pltd-3-8-x-24l-1-9-16-oal>: HTTP status code is not handled or not allowed 2026-01-24 21:49:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-sheet-with-high-temp-adhesive-60a-12-thick-x-36-wide-x-36-long>: HTTP status code is not handled or not allowed 2026-01-24 21:49:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-dolly-40-x-48-floor-locks>: HTTP status code is not handled or not allowed 2026-01-24 21:49:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leeson-3-phase-washguard-duty-motor-3-lb-ft-coupler-brake-56c-nema4x-ip55-bissc-575v>: HTTP status code is not handled or not allowed 2026-01-24 21:49:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-sheet-with-high-temp-adhesive-60a-18-thick-x-36-wide-x-36-long>: HTTP status code is not handled or not allowed 2026-01-24 21:49:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/retaining-pin-guard-for-dxs-2236-series-air-impact-wrench>: HTTP status code is not handled or not allowed 2026-01-24 21:49:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delfin-5-3-gallon-hepa-vacuum-2-7-hp-mistral-202-ds>: HTTP status code is not handled or not allowed 2026-01-24 21:49:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-forged-head-stake-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:49:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delfin-5-3-gallon-hepa-vacuum-1-hp-mistral-201-d1>: HTTP status code is not handled or not allowed 2026-01-24 21:49:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 26893 pages (at 99 pages/min), scraped 13700 items (at 12 items/min) 2026-01-24 21:49:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gear-driven-drill-press-1-5-hp-single-phase-110v-dp-1000g>: HTTP status code is not handled or not allowed 2026-01-24 21:49:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-variable-speed-bench-top-drill-press-0-75-hp-3-phase-110v-dp-4016b-vs>: HTTP status code is not handled or not allowed 2026-01-24 21:49:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gear-driven-dual-miter-band-saw-2-hp-single-phase-220v-bs-350sa>: HTTP status code is not handled or not allowed 2026-01-24 21:49:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-sided-bulk-truck-tubular-steel-sides-30-x-48-polyurethane-wheels>: HTTP status code is not handled or not allowed 2026-01-24 21:49:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/viton-rubber-sheet-with-high-temp-adhesive-75a-116-thick-x-6-wide-x-6-long>: HTTP status code is not handled or not allowed 2026-01-24 21:49:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-inch-capacity-pistol-grip-reversible-keyed-chuck-w-1800-rpm>: HTTP status code is not handled or not allowed 2026-01-24 21:49:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-roll-no-adhesive-50a-132-thick-x-36-wide-x-5-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 21:49:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-tarp-joint-side-mount-3011866>: HTTP status code is not handled or not allowed 2026-01-24 21:49:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/front-base-12-w-x-6-h-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:49:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-sheet-no-adhesive-50a-116-thick-x-36-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 21:49:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leeson-single-phase-explosion-proof-motor-1-3hp-1140rpm-56-epfc-60hz-automatic-1-0sf-rigid>: HTTP status code is not handled or not allowed 2026-01-24 21:49:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drop-shelf-truck-mesh-sides-30-x-48>: HTTP status code is not handled or not allowed 2026-01-24 21:49:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rock-medium-coat-hook-clear-anodized-aluminum>: HTTP status code is not handled or not allowed 2026-01-24 21:49:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-roll-no-adhesive-60a-316-thick-x-36-wide-x-4-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 21:49:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/desktop-cnc-engraver-17-x-17-dem-1717>: HTTP status code is not handled or not allowed 2026-01-24 21:49:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bubble-wrap-cushioning-material-12-x-10-3-16-thick>: HTTP status code is not handled or not allowed 2026-01-24 21:49:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-moving-storage-boxes-18l-x-18w-x-24h-brown>: HTTP status code is not handled or not allowed 2026-01-24 21:49:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elephantwall-topple-barrier-14-ft-2-in-h-x-16-ft-l>: HTTP status code is not handled or not allowed 2026-01-24 21:49:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pem-aluminum-saddle-threshold-48-l-x-6-w-1-4-h-mill-aluminum>: HTTP status code is not handled or not allowed 2026-01-24 21:49:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrislynn-10-32x190-locking-helical-inserts-bulk-pack-of-100-inserts>: HTTP status code is not handled or not allowed 2026-01-24 21:49:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classic-moving-boxes8-sm-15l-x-12w-x-10h-4-med-18l-x-15w-x-14h>: HTTP status code is not handled or not allowed 2026-01-24 21:49:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-sheet-no-adhesive-70a-332-thick-x-36-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 21:49:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=manual-roll-bender-2-hp-3-phase-220v-r-m10e already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:49:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-general-security-laminated-padlocks-no-2blf>: HTTP status code is not handled or not allowed 2026-01-24 21:49:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-strip-no-adhesive-70a-132-thick-x-6-wide-x-5-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 21:49:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-general-security-laminated-padlocks-no-25kalj>: HTTP status code is not handled or not allowed 2026-01-24 21:49:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-high-security-brass-solid-body-padlocks-no-6840b>: HTTP status code is not handled or not allowed 2026-01-24 21:49:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-roll-no-adhesive-60a-132-thick-x-36-wide-x-4-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 21:49:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-strip-no-adhesive-50a-116-thick-x-4-wide-x-5-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 21:49:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:49:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smi11-general-purpose-laboratory-incubator-double-doors-10-8-cu-ft-306-l-115v>: HTTP status code is not handled or not allowed 2026-01-24 21:49:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/47-7-l-x-7-2-w-vinyl-wall-plank-gray-oak-7-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:49:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:49:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-intensity-infrared-patio-heater-with-transformer-liquid-propane-60000-btu-120v>: HTTP status code is not handled or not allowed 2026-01-24 21:49:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/630-516-516-3-hollow-shaft-nut-driver-w-cushion-grip>: HTTP status code is not handled or not allowed 2026-01-24 21:49:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=manual-roll-bender-2-hp-3-phase-220v-r-m10e>: HTTP status code is not handled or not allowed 2026-01-24 21:49:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-shelf-truck-4-lip-shelves-30-x-60>: HTTP status code is not handled or not allowed 2026-01-24 21:49:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-sheet-no-adhesive-50a-132-thick-x-12-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 21:49:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-high-security-steel-solid-body-padlocks-no-6230>: HTTP status code is not handled or not allowed 2026-01-24 21:49:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manual-chain-hoist-10-foot-li-6-000-pound-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:49:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-cold-cups-20-oz-clear-1000-carton>: HTTP status code is not handled or not allowed 2026-01-24 21:49:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips>: HTTP status code is not handled or not allowed 2026-01-24 21:49:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-cold-cups-9-oz-clear-1000-carton>: HTTP status code is not handled or not allowed 2026-01-24 21:49:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls>: HTTP status code is not handled or not allowed 2026-01-24 21:49:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-work-height-platform-truck-30-x-60-mold-on-rubber-wheels>: HTTP status code is not handled or not allowed 2026-01-24 21:49:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-roll-no-adhesive-50a-316-thick-x-36-wide-x-9-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 21:49:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-roll-no-adhesive-50a-116-thick-x-36-wide-x-4-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 21:49:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/antimicrobial-bonded-leather-reception-loveseat-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:49:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-shelving-open-shelving-36x18x75-7>: HTTP status code is not handled or not allowed 2026-01-24 21:49:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-high-security-steel-solid-body-padlocks-no-7030kalt>: HTTP status code is not handled or not allowed 2026-01-24 21:49:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s78129-class-3-hooded-sweatshirt-fluorescent-orange-2xl>: HTTP status code is not handled or not allowed 2026-01-24 21:49:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s78129-class-3-hooded-sweatshirt-fluorescent-orange-4xl>: HTTP status code is not handled or not allowed 2026-01-24 21:49:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kd-jumbo-wardrobe-cabinet-48-w-x-24-d-x-78-h-medium-grey>: HTTP status code is not handled or not allowed 2026-01-24 21:49:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s78129-class-3-hooded-sweatshirt-fluorescent-orange-lg>: HTTP status code is not handled or not allowed 2026-01-24 21:49:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-welded-combination-cabinet-36-w-x-18-d-x-72-h-black>: HTTP status code is not handled or not allowed 2026-01-24 21:49:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-tall-bolt-down-safety-netting-bollard-removable-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:49:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-roll-no-adhesive-70a-12-thick-x-36-wide-x-4-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 21:49:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:49:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-x40-fall-safety-net-with-debris-liner-black>: HTTP status code is not handled or not allowed 2026-01-24 21:49:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-insulated-replacement-cover-for-n1tw>: HTTP status code is not handled or not allowed 2026-01-24 21:50:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o12008-magnaprene-plain-front-overall-green-md>: HTTP status code is not handled or not allowed 2026-01-24 21:50:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-ft-l-hytrol-slider-bed-conveyor-24-inch-w-belt>: HTTP status code is not handled or not allowed 2026-01-24 21:50:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/colored-carton-sealing-tape-2-x-110-yds-22-mil-blue-6pack>: HTTP status code is not handled or not allowed 2026-01-24 21:50:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluidity-03f-hung-diaphragm-solenoid-valve-2-2-nc-nbr-seal-1-2-nptf-12-mm-24v-dc>: HTTP status code is not handled or not allowed 2026-01-24 21:50:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modine-hot-dawg-100000-btu-gas-fired-unit-heater>: HTTP status code is not handled or not allowed 2026-01-24 21:50:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wood-deck-72-w-x-18-d-x-5-8-th>: HTTP status code is not handled or not allowed 2026-01-24 21:50:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-roll-with-acrylic-adhesive-50a-38-thick-x-36-wide-x-10-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 21:50:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o22007-iron-eagle-plain-front-overall-gold-snaplock-buckles-3xl>: HTTP status code is not handled or not allowed 2026-01-24 21:50:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets>: HTTP status code is not handled or not allowed 2026-01-24 21:50:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-strip-with-acrylic-adhesive-60a-316-thick-x-2-wide-x-5-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 21:50:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 26986 pages (at 93 pages/min), scraped 13727 items (at 27 items/min) 2026-01-24 21:50:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/j26002-bomber-hooded-jacket-fluorescent-yellowgreenblack-5xl>: HTTP status code is not handled or not allowed 2026-01-24 21:50:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluidity-02f-direct-acting-solenoid-valve-2-2-nc-epdm-seal-3-8-nptf-3-mm-110v-ac>: HTTP status code is not handled or not allowed 2026-01-24 21:50:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-raised-circular-stair-tread-square-nose-204375-x-72-brown>: HTTP status code is not handled or not allowed 2026-01-24 21:50:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-part-carbonless-inventory-tag-pre-wired-4000-4499-8-6-14l-x-3-18w-500pack>: HTTP status code is not handled or not allowed 2026-01-24 21:50:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s78122-class-3-hooded-sweatshirt-fluorescent-lime-2xl>: HTTP status code is not handled or not allowed 2026-01-24 21:50:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-208-blowgun-startip-with-4-tube-1-4-npt-inlet-connection>: HTTP status code is not handled or not allowed 2026-01-24 21:50:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/600-hot-melt-tape-2-x-1000-yds-16-mil-clear-6pack>: HTTP status code is not handled or not allowed 2026-01-24 21:50:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thermoplastic-rubber-wall-base-4-in-x-48-in-slate>: HTTP status code is not handled or not allowed 2026-01-24 21:50:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/colored-carton-sealing-tape-2-x-55-yds-22-mil-orange-6pack>: HTTP status code is not handled or not allowed 2026-01-24 21:50:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-sheet-no-adhesive-70a-116-thick-x-36-wide-x-36-long>: HTTP status code is not handled or not allowed 2026-01-24 21:50:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-sheet-no-adhesive-70a-132-thick-x-36-wide-x-36-long>: HTTP status code is not handled or not allowed 2026-01-24 21:50:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o23122-vision-snap-fly-front-overall-fluorescent-lime-3xl>: HTTP status code is not handled or not allowed 2026-01-24 21:50:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3m-ss-304304l-2-half-coupling-npt-female-plain>: HTTP status code is not handled or not allowed 2026-01-24 21:50:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-315-coupling-1-4-hose-barb-asian-interchange>: HTTP status code is not handled or not allowed 2026-01-24 21:50:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-roll-no-adhesive-50a-38-thick-x-36-wide-x-8-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 21:50:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-heavy-duty-pegboard-cabinet-12-gauge-60-w-x-24-d-x-78-h>: HTTP status code is not handled or not allowed 2026-01-24 21:50:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vinyl-ribbed-stair-tread-round-nose-125-x-36-light-brown>: HTTP status code is not handled or not allowed 2026-01-24 21:50:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stordrawer-steel-plus153-file-storage-drawers-24l-x-12w-x-10h-white-6pack>: HTTP status code is not handled or not allowed 2026-01-24 21:50:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:50:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merit-red-brass-seamless-standard-38-3-npt-male>: HTTP status code is not handled or not allowed 2026-01-24 21:50:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/advantage-pro-max-pallet-jack-truck-27w-x-64l-11000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:50:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heat-wagon-25ft-long-heater-duct-14-in-diameter>: HTTP status code is not handled or not allowed 2026-01-24 21:50:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-peek-film-0-005-thick-x-24-w-x-10-l>: HTTP status code is not handled or not allowed 2026-01-24 21:50:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-roll-no-adhesive-40a-38-thick-x-36-wide-x-7-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 21:50:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lid-17-1-4x11-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:50:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merit-ss-304-barstock-hex-head-bushing-3-2-npt-male-female>: HTTP status code is not handled or not allowed 2026-01-24 21:50:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tyvek174-shipping-tag-pre-wired-2-34l-x-1-38w-orange-100pack>: HTTP status code is not handled or not allowed 2026-01-24 21:50:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-sheet-no-adhesive-50a-12-thick-x-36-wide-x-36-long>: HTTP status code is not handled or not allowed 2026-01-24 21:50:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medium-vortex-tube-25-scfm-1700-btu-hr>: HTTP status code is not handled or not allowed 2026-01-24 21:50:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:50:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets>: HTTP status code is not handled or not allowed 2026-01-24 21:50:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mb924b-neoprene-steel-toe-snugleg-boots-brick-redbrown-size-13>: HTTP status code is not handled or not allowed 2026-01-24 21:50:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1736s10-osha-stairway-36-width-10-stairs>: HTTP status code is not handled or not allowed 2026-01-24 21:50:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-clear-lid-18x12>: HTTP status code is not handled or not allowed 2026-01-24 21:50:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/j32007-american-storm-fly-front-jacket-yellow-3xl>: HTTP status code is not handled or not allowed 2026-01-24 21:50:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tyvek174-shipping-tag-2-34l-x-1-38w-red-100pack>: HTTP status code is not handled or not allowed 2026-01-24 21:50:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slatwall-counter-top-cube-display-fixture-white-w-spinner-base>: HTTP status code is not handled or not allowed 2026-01-24 21:50:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-strap-guards-2-3-4x2-1-8>: HTTP status code is not handled or not allowed 2026-01-24 21:50:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-tag-pre-strung-7-5-34l-x-2-78w-white-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 21:50:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-sheet-no-adhesive-60a-12-thick-x-36-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 21:50:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-1-2x7-8-16-grit-alumox-resin-fibre-disc>: HTTP status code is not handled or not allowed 2026-01-24 21:50:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-delrin-homopolymer-rod-0-250-dia>: HTTP status code is not handled or not allowed 2026-01-24 21:50:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls>: HTTP status code is not handled or not allowed 2026-01-24 21:50:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-tag-7-5-34l-x-2-78w-manilla-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 21:50:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s75522-class-3-long-sleeve-tshirt-fluorescent-yellowgreen-3xl>: HTTP status code is not handled or not allowed 2026-01-24 21:50:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-shelf-steel-shelving-with-36-akrobins-36x12x39>: HTTP status code is not handled or not allowed 2026-01-24 21:50:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tyvek174-shipping-tag-4-14l-x-2-18w-green-100pack>: HTTP status code is not handled or not allowed 2026-01-24 21:50:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/j22208-iron-eagle-storm-fly-front-jacket-green-hood-snaps-2xl>: HTTP status code is not handled or not allowed 2026-01-24 21:50:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-gallon-fluid-extractor>: HTTP status code is not handled or not allowed 2026-01-24 21:50:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2x3x5-8-11-type-17>: HTTP status code is not handled or not allowed 2026-01-24 21:50:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-24-18-gauge-stainless-steel-workbench-w-backsplash>: HTTP status code is not handled or not allowed 2026-01-24 21:50:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fda-bunan-rubber-strip-with-acrylic-adhesive-50a-14-thick-x-12-wide-x-10-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 21:50:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-1-pneumatic-grease-pump-35-lb>: HTTP status code is not handled or not allowed 2026-01-24 21:50:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grease-system-35-lb>: HTTP status code is not handled or not allowed 2026-01-24 21:50:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/l-70100-123-9g-12-awg-3-conductor-x-100ft-15-amp-tri-tap-gfci-outlet-with-cord>: HTTP status code is not handled or not allowed 2026-01-24 21:50:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-pipe-adapter-for-ev-5100>: HTTP status code is not handled or not allowed 2026-01-24 21:50:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamper-kit-gray-vinyl-standard-pocket>: HTTP status code is not handled or not allowed 2026-01-24 21:50:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamper-kit-red-vinyl-enviro-pocket>: HTTP status code is not handled or not allowed 2026-01-24 21:50:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/expandable-add-on-rack-60x24x84-gray-with-3-level-wood-deck-1000lb-cap-per-level>: HTTP status code is not handled or not allowed 2026-01-24 21:50:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/expandable-add-on-rack-72x48x84-gray-with-3-level-wood-deck-750lb-cap-per-level>: HTTP status code is not handled or not allowed 2026-01-24 21:50:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-basket-truck-24-bu-white-vinyl-wood-base-2-rg-2-sw>: HTTP status code is not handled or not allowed 2026-01-24 21:50:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-nema-4x-thermostat-model-tf115023>: HTTP status code is not handled or not allowed 2026-01-24 21:51:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylatron-nsm-sheet-1-250-thick-x-24-w-x-48-l>: HTTP status code is not handled or not allowed 2026-01-24 21:51:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-sheet-no-adhesive-60a-116-thick-x-6-wide-x-6-long>: HTTP status code is not handled or not allowed 2026-01-24 21:51:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-locker-bench-hardwood-top-w-steel-tube-pedestals-bolt-down-48-x-9-1-2-x-17>: HTTP status code is not handled or not allowed 2026-01-24 21:51:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylatron-gsm-sheet-cast-1-750-thick-x-24-w-x-24-l>: HTTP status code is not handled or not allowed 2026-01-24 21:51:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavenly-choice-double-layer-1-ply-toilet-paper-850-sheets-per-roll>: HTTP status code is not handled or not allowed 2026-01-24 21:51:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-gallon-rubbermaid-plastic-wastebasket-black>: HTTP status code is not handled or not allowed 2026-01-24 21:51:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/j22168-iron-eagle-storm-fly-front-hooded-jacket-green-3xl>: HTTP status code is not handled or not allowed 2026-01-24 21:51:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-sheet-no-adhesive-60a-38-thick-x-36-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 21:51:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/expandable-starter-rack-48x18x84-gray-with-3-levels-no-deck-1500lb-cap-per-level>: HTTP status code is not handled or not allowed 2026-01-24 21:51:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-sheet-no-adhesive-60a-14-thick-x-12-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 21:51:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 27081 pages (at 95 pages/min), scraped 13752 items (at 25 items/min) 2026-01-24 21:51:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aprilaire-whole-house-humidifier-model-500-built-in-bypass-dampers>: HTTP status code is not handled or not allowed 2026-01-24 21:51:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-welded-cabinet-clearview-doors-four-shelves-60-inch-w-x-24-inch-d-x-74-inch-h-black>: HTTP status code is not handled or not allowed 2026-01-24 21:51:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ketron-hpv-bearing-grade-peek-0-500-dia-x-3-28-l>: HTTP status code is not handled or not allowed 2026-01-24 21:51:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-rack-wire-deck-starter-4-levels-gray-15>: HTTP status code is not handled or not allowed 2026-01-24 21:51:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylatron-gsm-sheet-cast-2-000-thick-x-24-w-x-24-l>: HTTP status code is not handled or not allowed 2026-01-24 21:51:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perm-liner-truck-16-bu-red-vinyl-wood-base-2-rg-2-sw>: HTTP status code is not handled or not allowed 2026-01-24 21:51:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/35123-workbrutes-10-work-boots-yellow-cleated-outsole-2xl>: HTTP status code is not handled or not allowed 2026-01-24 21:51:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:51:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/j22207-iron-eagle-storm-fly-front-jacket-gold-hood-snaps-sm>: HTTP status code is not handled or not allowed 2026-01-24 21:51:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o31107-webdri-snap-fly-front-overall-yellow-lg>: HTTP status code is not handled or not allowed 2026-01-24 21:51:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/accuglide-2-taping-head-lower-taping-head-2>: HTTP status code is not handled or not allowed 2026-01-24 21:51:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-tag-pre-strung-5-4-34l-x-2-38w-fluorescent-red-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 21:51:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gorbel-ld-free-standing-tool-solutions-jib-crane-4-ft-span-and-10-ft-height-under-boom-150-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:51:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heatstar-hs400ag-commercial-greenhouse-heater-propane-natural-gas-dual-fuel-400000-btu-120v>: HTTP status code is not handled or not allowed 2026-01-24 21:51:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c56207-durascrim-coat-yellow-48-2-patch-pockets-hood-snaps-lg>: HTTP status code is not handled or not allowed 2026-01-24 21:51:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tygon-se-200-tube-ajd00017-0-250-id-x-375-od-x-50-l>: HTTP status code is not handled or not allowed 2026-01-24 21:51:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-level-60x36-high-capacity-rack-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:51:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machine-table-2-shelves-18-x-32>: HTTP status code is not handled or not allowed 2026-01-24 21:51:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-tag-pre-strung-8-6-14l-x-3-18w-fluorescent-yellow-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 21:51:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls>: HTTP status code is not handled or not allowed 2026-01-24 21:51:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/del-vacuum-hose-white-30-l>: HTTP status code is not handled or not allowed 2026-01-24 21:51:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-30-glass-filled-cast-nylon-rod-4-dia>: HTTP status code is not handled or not allowed 2026-01-24 21:51:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-his-sheet-0-093-thick-x-48-w-x-96-l>: HTTP status code is not handled or not allowed 2026-01-24 21:51:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-heavy-duty-7-gauge-steel-workbench-drawer-30-x-36>: HTTP status code is not handled or not allowed 2026-01-24 21:51:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gorbel-steel-gantry-crane-15-ft-span-and-17-ft-20-ft-adjustable-height-6000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:51:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silbrade-braided-silicone-tube-0-312-id-x-592-od-x-100-l-roll>: HTTP status code is not handled or not allowed 2026-01-24 21:51:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gorbel-hd-free-standing-jib-crane-16-ft-span-and-12-ft-height-under-boom-2000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:51:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s56307-durascrim-3-pc-suit-yellow-detachable-hood-4xl>: HTTP status code is not handled or not allowed 2026-01-24 21:51:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usb-wired-optical-quiet-click-mouse-for-mac-pc-space-gray-black-dynamousesg>: HTTP status code is not handled or not allowed 2026-01-24 21:51:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-mat-shelf-18x42-nexelon>: HTTP status code is not handled or not allowed 2026-01-24 21:51:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gorbel-hd-free-standing-jib-crane-12-ft-span-and-10-ft-height-under-boom-3000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:51:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-ultra-slim-usb-wired-keyboard-for-mac-and-pc-acekeya>: HTTP status code is not handled or not allowed 2026-01-24 21:51:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/boom-swing-wall-mount-ws50-10-ft-50-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:51:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-undercarriage-cart>: HTTP status code is not handled or not allowed 2026-01-24 21:51:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cubby-locker-welded-34-1-2w-x-13-1-2d-x-66h-15-compartments-champagne-putty>: HTTP status code is not handled or not allowed 2026-01-24 21:51:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/apparel-boxes-19l-x-12w-x-3h-white-50pack>: HTTP status code is not handled or not allowed 2026-01-24 21:51:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-infinity-ventilated-steel-locker-six-tier-3-wide-12x18x12-unassembled-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:51:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rsb-restorer-dilutable-pine-gallon-bottle-4-bottles-904923>: HTTP status code is not handled or not allowed 2026-01-24 21:51:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gloveworks-hd-diamond-textured-nitrile-industrial-gloves-m-black-100-per-box-10-box-per-cs>: HTTP status code is not handled or not allowed 2026-01-24 21:51:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-top-machine-table-24-x-36-x-36-w-accessory-drawer>: HTTP status code is not handled or not allowed 2026-01-24 21:51:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hang-stack-bins-w-two-12-rails-six-bins-4-1-8w-x-7-3-8d-x-3h-red>: HTTP status code is not handled or not allowed 2026-01-24 21:51:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-height-welded-workbench-w-open-base-hardboard-over-steel-36-x-72>: HTTP status code is not handled or not allowed 2026-01-24 21:51:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eva-tubing-shrink-wrapped-coil-w-label-3-4-i-d-x-1-8-wall-50-feet>: HTTP status code is not handled or not allowed 2026-01-24 21:51:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eva-tubing-w-nylon-braid-reinforcement-bulk-3-4-i-d-300-feet>: HTTP status code is not handled or not allowed 2026-01-24 21:51:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/906-st-m20x1-5-gpc-steel-threaded-plug-with-m20-x-1-5-tapered-thread-w-gpc-coating>: HTTP status code is not handled or not allowed 2026-01-24 21:51:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smokescreen-odor-elminator-16-oz-can-pleasant-scent-12-cans-case>: HTTP status code is not handled or not allowed 2026-01-24 21:51:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/906-ni-m8x1-a-stainless-threaded-plug-with-m8-x-1-0-tapered-thread>: HTTP status code is not handled or not allowed 2026-01-24 21:51:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/walls_ceilings/ceiling_tiles already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:51:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hang-stack-bins-w-two-12-rails-four-bins-5-1-2w-x-10-7-8d-x-5h-red>: HTTP status code is not handled or not allowed 2026-01-24 21:51:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/great-lakes-tin-ludington-2-x-2-nail-up-tin-ceiling-tile-in-matte-white-t57-01>: HTTP status code is not handled or not allowed 2026-01-24 21:51:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/orange-response-industrial-citrus-degreaser-and-deodorizer-55-gallon-drum>: HTTP status code is not handled or not allowed 2026-01-24 21:51:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grade-a-binderless-glass-microfiber-filter-0-3-mm-thick-2-1-cm-dia-1-6-um-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 21:51:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/libman-commercial-18-inch-microfiber-mop-refill>: HTTP status code is not handled or not allowed 2026-01-24 21:51:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2517-flatback-tape-1-x-60-yds-65-mil-kraft-12pack>: HTTP status code is not handled or not allowed 2026-01-24 21:51:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g7-145-aff-10hp-oil-injected-rotary-screw-floor-145psi-3ph-208-230-460v-w-dryer>: HTTP status code is not handled or not allowed 2026-01-24 21:51:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/361823098s-clip-s3-basic-rack-unit-98-h-x-36-w-x-18-d-6xms230-shelves-galvanized>: HTTP status code is not handled or not allowed 2026-01-24 21:51:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/libman-commercial-lobby-broom>: HTTP status code is not handled or not allowed 2026-01-24 21:51:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/libman-commercial-janitor-corn-broom>: HTTP status code is not handled or not allowed 2026-01-24 21:51:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-gift-box-bottoms-10l-x-10w-x-6h-white-50pack>: HTTP status code is not handled or not allowed 2026-01-24 21:51:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grade-cfp3-qualitative-cellulose-filter-paper-0-32-mm-thick-5-5-cm-dia-6-um-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 21:51:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/great-lakes-tin-ludington-2-x-2-nail-up-tin-ceiling-tile-in-penny-vein-t57-05>: HTTP status code is not handled or not allowed 2026-01-24 21:51:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/walls_ceilings/ceiling_tiles>: HTTP status code is not handled or not allowed 2026-01-24 21:51:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sl300-hk2090-sl300-series-2-to-3-watt-uhf-2-channel-analog-two-way-radio>: HTTP status code is not handled or not allowed 2026-01-24 21:51:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-purpose-synthetic-grease-nlgi-0-with-syncolon-174-ptfe-30-lb-pail>: HTTP status code is not handled or not allowed 2026-01-24 21:51:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/54h-nexel-chrome-wire-shelving-24wx21d>: HTTP status code is not handled or not allowed 2026-01-24 21:51:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/libman-commercial-floor-scrub>: HTTP status code is not handled or not allowed 2026-01-24 21:51:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/supreme-sliptech-mat-11-16-thick-4-wide-cut-length-up-to-60-black>: HTTP status code is not handled or not allowed 2026-01-24 21:51:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/supreme-sliptech-mat-11-16-thick-36x60-black-w-yellow-border>: HTTP status code is not handled or not allowed 2026-01-24 21:51:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-22-4030-straight-cutting-aviation-snips>: HTTP status code is not handled or not allowed 2026-01-24 21:52:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/49-00-5460-blade-4l-wood-5-pk>: HTTP status code is not handled or not allowed 2026-01-24 21:52:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-with-18-shelf-bins-7-shelves-36x18x39-5>: HTTP status code is not handled or not allowed 2026-01-24 21:52:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-with-10-hopper-bins-36x18x75-6>: HTTP status code is not handled or not allowed 2026-01-24 21:52:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/libman-commercial-short-handle-utility-brush-white>: HTTP status code is not handled or not allowed 2026-01-24 21:52:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/urinal-screen-w-non-para-deodorizer-block-cherry-scent-white-pink-12-blocks-case>: HTTP status code is not handled or not allowed 2026-01-24 21:52:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tank-cover-for-p1800-series-ultrasonic-part-cleaners>: HTTP status code is not handled or not allowed 2026-01-24 21:52:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/illinois-engineered-products-single-folding-gate-3w-to-4w-and-66h>: HTTP status code is not handled or not allowed 2026-01-24 21:52:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultrasonic-table-top-parts-cleaning-system-digital-timer-heat-3-25-gal-45-khz-230v>: HTTP status code is not handled or not allowed 2026-01-24 21:52:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/great-lakes-tin-chicago-2-x-2-nail-up-tin-ceiling-tile-in-clear-t60-04>: HTTP status code is not handled or not allowed 2026-01-24 21:52:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spitfire-power-cleaner-liquid-32-oz-spray-bottle-fresh-pine-scent-12-case>: HTTP status code is not handled or not allowed 2026-01-24 21:52:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/illinois-engineered-products-single-folding-gate-8w-to-9w-and-7h>: HTTP status code is not handled or not allowed 2026-01-24 21:52:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/illinois-engineered-products-double-folding-gate-6w-to-8w-and-66h>: HTTP status code is not handled or not allowed 2026-01-24 21:52:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 27179 pages (at 98 pages/min), scraped 13771 items (at 19 items/min) 2026-01-24 21:52:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/illinois-engineered-products-double-folding-gate-6w-to-8w-and-7h>: HTTP status code is not handled or not allowed 2026-01-24 21:52:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-z-brite-wire-shelving-48wx14dx86h>: HTTP status code is not handled or not allowed 2026-01-24 21:52:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jamaica-beach-adj-sling-chaise-lounge-orange-sling-white-resin-frame-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-24 21:52:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/copper-block-1230r-5234711>: HTTP status code is not handled or not allowed 2026-01-24 21:52:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bntl2230-22-x-30-exposed-flange-key-operated-lock>: HTTP status code is not handled or not allowed 2026-01-24 21:52:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-cabinet-shop-desk-with-pigeonhole-compartment-riser-34-1-2-w-x-30-d-x-51-1-2-h-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:52:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foaming-crystal-industrial-cleaner-and-degreaser-20-oz-aerosol-12-cans-case>: HTTP status code is not handled or not allowed 2026-01-24 21:52:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-cabinet-shop-desk-w-pigeonhole-riser-pegboard-upper-cabinet-34-1-2-w-x-30-d-x-80-h-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:52:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-8-stainless-steel-brazier-w-cover-20-qt>: HTTP status code is not handled or not allowed 2026-01-24 21:52:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/probite-174-3-4-x-3-4-x-3-4-lead-free-brass-tee>: HTTP status code is not handled or not allowed 2026-01-24 21:52:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2221vs-key-5510316>: HTTP status code is not handled or not allowed 2026-01-24 21:52:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-foam-skin-cleanser-with-moisturizers-citrus-scent-1-5-l-refill-2-case>: HTTP status code is not handled or not allowed 2026-01-24 21:52:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aquaba-38-round-resin-pedestal-table-w-umbrella-hole-ranch-on-bronze-frame>: HTTP status code is not handled or not allowed 2026-01-24 21:52:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/panel-kit-72-w-workbench-with-18-w-whiteboard-36-w-louver-mounting-rai-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:52:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-vs-drill-press-screw-5513511>: HTTP status code is not handled or not allowed 2026-01-24 21:52:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bushing-1230r-5235771>: HTTP status code is not handled or not allowed 2026-01-24 21:52:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adaptor-inert-arc>: HTTP status code is not handled or not allowed 2026-01-24 21:52:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-dock-plate-60-w-x-48-l-2660-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:52:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hose-repair-kit-w-c-5-tool>: HTTP status code is not handled or not allowed 2026-01-24 21:52:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3500-series-4b-vertical-mailbox-6-doors-surface-mounted-bronze-usps-access>: HTTP status code is not handled or not allowed 2026-01-24 21:52:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bu-th-109-tank-holder>: HTTP status code is not handled or not allowed 2026-01-24 21:52:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-wire-shelf-truck-48x18x80-1200-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:52:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-dunnage-rack-24-w-x-18-d-2>: HTTP status code is not handled or not allowed 2026-01-24 21:52:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-dunnage-rack-30-w-x-18-d-4>: HTTP status code is not handled or not allowed 2026-01-24 21:52:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-lehigh-bench-eco-friendly-synthetic-wood-in-coastal-teak>: HTTP status code is not handled or not allowed 2026-01-24 21:52:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/702-corrugated-switchboard-matting-36x5-black>: HTTP status code is not handled or not allowed 2026-01-24 21:52:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-keys-global-floor-scrubbers-sweepers>: HTTP status code is not handled or not allowed 2026-01-24 21:52:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conductive-trash-liners-44-gallon-2-mil-pink-pack-of-50-wbas44-lp>: HTTP status code is not handled or not allowed 2026-01-24 21:52:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flammable-cabinet-with-manual-close-double-door-90-gallon>: HTTP status code is not handled or not allowed 2026-01-24 21:52:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-latex-bootshoe-covers-yellow-lg-25-pairscase>: HTTP status code is not handled or not allowed 2026-01-24 21:52:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reechcraft-pl-xm-compact-base-powerlift-145-316-platform-height-350-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:52:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heater-base-legs-wbl100>: HTTP status code is not handled or not allowed 2026-01-24 21:52:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-storage-drawer-cabinet-30x27x40-2>: HTTP status code is not handled or not allowed 2026-01-24 21:52:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-element-kit-for-arrow-midflow-filters>: HTTP status code is not handled or not allowed 2026-01-24 21:52:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/furniture-polish-lemon-14-2-oz-aerosol-spray>: HTTP status code is not handled or not allowed 2026-01-24 21:52:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-straight-grab-bar-satin-stainless-steel-36-w-x-1-1-4-dia>: HTTP status code is not handled or not allowed 2026-01-24 21:52:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cross-linked-polyethylene-shoe-covers-water-resistant-white-lg-100bag-3-bagscase>: HTTP status code is not handled or not allowed 2026-01-24 21:52:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-straight-grab-bar-peened-stainless-steel-42-w-x-1-1-2-dia>: HTTP status code is not handled or not allowed 2026-01-24 21:52:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-breather-vent>: HTTP status code is not handled or not allowed 2026-01-24 21:52:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cfswdl3510-ip69-1hp-1730rpm-1ph-60hz-56c-3540l-tefc-f1-n>: HTTP status code is not handled or not allowed 2026-01-24 21:52:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7020-screw-12-x-1-12-5712931>: HTTP status code is not handled or not allowed 2026-01-24 21:52:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-muffler-3-8-female>: HTTP status code is not handled or not allowed 2026-01-24 21:52:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flextec-650x-multi-process-welder-815a-5060-hz-red>: HTTP status code is not handled or not allowed 2026-01-24 21:52:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-dusters-refill-dust-lock-fiber-yellow-6-box-4-box-case>: HTTP status code is not handled or not allowed 2026-01-24 21:52:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-drawer-unit-divider-kit-8-compartment-1>: HTTP status code is not handled or not allowed 2026-01-24 21:52:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4002-switch-key-5641171>: HTTP status code is not handled or not allowed 2026-01-24 21:52:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vss-1000-1-100-gallon-sidewalk-sprayer>: HTTP status code is not handled or not allowed 2026-01-24 21:52:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-drawer-unit-divider-kit-12-compartment-12>: HTTP status code is not handled or not allowed 2026-01-24 21:52:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8201-v-belt-a51-5513816>: HTTP status code is not handled or not allowed 2026-01-24 21:52:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8201-door-5513835>: HTTP status code is not handled or not allowed 2026-01-24 21:52:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-outdoor-bi-level-drinking-fountain-with-bottle-filling-station-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:52:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-tray-cubby-storage-unassembled-w-assorted-trays-50-3-4w-x-12d-x-33-7-8h>: HTTP status code is not handled or not allowed 2026-01-24 21:52:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-tray-cubby-storage-unassembled-w-o-trays-50-3-4w-x-12d-x-33-7-8h>: HTTP status code is not handled or not allowed 2026-01-24 21:52:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/x-tractor-mini-portable-welding-fume-extractor-120v-1-ph-60hz-black>: HTTP status code is not handled or not allowed 2026-01-24 21:52:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjusting-block-5783641>: HTTP status code is not handled or not allowed 2026-01-24 21:52:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd-sms-coverall-elastic-wrists-ankles-zipper-front-single-collar-white-3xl-25case>: HTTP status code is not handled or not allowed 2026-01-24 21:52:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/indestructable-too-folding-table-60-round-charcoal-1200-series>: HTTP status code is not handled or not allowed 2026-01-24 21:52:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lightweight-polypropylene-bouffant-cap-100-latex-free-orange-21-100bag>: HTTP status code is not handled or not allowed 2026-01-24 21:52:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/genesis-polycarbonate-light-panels-2ft-w-x-4ft-l-opal-10-case-708-02>: HTTP status code is not handled or not allowed 2026-01-24 21:52:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/frameless-safety-glasses-scratch-resistant-amber-lens>: HTTP status code is not handled or not allowed 2026-01-24 21:52:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7020-acme-nut-bttn-for-hbs-7040vs-5712541>: HTTP status code is not handled or not allowed 2026-01-24 21:52:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/decorative-bollard-cover-stone-finish-8-in-dia-60-in-h>: HTTP status code is not handled or not allowed 2026-01-24 21:52:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pedestal-table-with-height-adjustable-legs-36x72-3>: HTTP status code is not handled or not allowed 2026-01-24 21:52:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jamaica-beach-folding-chair-straw-sling-w-white-resin-frame-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-24 21:52:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16oz-natural-pp-jar-with-89-400-white-pp-unlined-cap-24pk>: HTTP status code is not handled or not allowed 2026-01-24 21:52:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lyon-mobile-storage-cabinet-60x24x68-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:52:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergodyne-620-knee-sleeve-open-patella-spiral-stays-black-x-large>: HTTP status code is not handled or not allowed 2026-01-24 21:53:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stock-stop-5711321>: HTTP status code is not handled or not allowed 2026-01-24 21:53:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/center-pulley-shaft-5625991>: HTTP status code is not handled or not allowed 2026-01-24 21:53:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foodsafe-cfswdm3709t-e-ip69-75hp-3500rpm-3ph-60hz-213tc-3734m-tefc>: HTTP status code is not handled or not allowed 2026-01-24 21:53:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jamaica-beach-folding-chair-orange-sling-w-white-resin-frame-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-24 21:53:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40-h-mobile-post-beige>: HTTP status code is not handled or not allowed 2026-01-24 21:53:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8oz-clr-glass-standard-wide-mouth-bottle-w-58-400-white-pp-sturdeeseal-pe-foam-cap-24pk>: HTTP status code is not handled or not allowed 2026-01-24 21:53:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/409996-medium-carrying-case-black>: HTTP status code is not handled or not allowed 2026-01-24 21:53:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-seal-kit-for-manual-pallet-jack-truck-hl-99-fits-lift-model-hp25l>: HTTP status code is not handled or not allowed 2026-01-24 21:53:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16oz-clear-safety-coated-glass-straight-sided-round-jar-89-400-neck-finish-12-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:53:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nut-38-16-6295740>: HTTP status code is not handled or not allowed 2026-01-24 21:53:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-drawer-cabinet-workbench-plastic>: HTTP status code is not handled or not allowed 2026-01-24 21:53:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ct70-ac-circuit-load-tester-240-ac-voltage>: HTTP status code is not handled or not allowed 2026-01-24 21:53:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hvbs-712-7-x-12-horizontal-vertical-bandsaw-3-4-hp-115-230v-1ph>: HTTP status code is not handled or not allowed 2026-01-24 21:53:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 27278 pages (at 99 pages/min), scraped 13790 items (at 19 items/min) 2026-01-24 21:53:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jet-jtm-949evs-mill-3-axis-acu-rite-200s-dro-knee-x-y-and-z-axis-powerfeeds-698178>: HTTP status code is not handled or not allowed 2026-01-24 21:53:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dm220-mini-pocket-multimeter-wnoncontact-voltage-detector-orangegreen-ac-capable>: HTTP status code is not handled or not allowed 2026-01-24 21:53:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-t-bar-3-shelf-rack-20-wx60-hx72-l>: HTTP status code is not handled or not allowed 2026-01-24 21:53:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-18w-x-20l-1-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 21:53:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=jet-jtm-2-mill-newall-dp500-dro-x-and-y-axis-powerfeeds-690949 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:53:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/screw-5713321>: HTTP status code is not handled or not allowed 2026-01-24 21:53:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/70-400-white-metal-cap-with-pulp-pe-liner-packed-in-bags-of-12-144pk>: HTTP status code is not handled or not allowed 2026-01-24 21:53:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jet-jtm-1050evs2-230-mill-3-axis-newall-dp700-dro-knee-x-and-y-axis-powerfeeds-690548>: HTTP status code is not handled or not allowed 2026-01-24 21:53:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8205z-class-2-super-econo-vest-lime-2xl-3xl>: HTTP status code is not handled or not allowed 2026-01-24 21:53:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8210hl-class-2-economy-vest-orange-s-m>: HTTP status code is not handled or not allowed 2026-01-24 21:53:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triumph-style-t1hd-hss-jobbers-drill-black-bronze-oxide-564-12>: HTTP status code is not handled or not allowed 2026-01-24 21:53:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-adjustable-t-bar-shelf-24-wx60-l>: HTTP status code is not handled or not allowed 2026-01-24 21:53:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8295-class-2-polo-shirt-lime-medium>: HTTP status code is not handled or not allowed 2026-01-24 21:53:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triumph-style-t1d-hss-jobbers-drill-black-oxide-732-12>: HTTP status code is not handled or not allowed 2026-01-24 21:53:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-step-steel-orange-tilt-and-roll-ladder-with-serrated-grating-tr-3g-o>: HTTP status code is not handled or not allowed 2026-01-24 21:53:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8320z-class-3-standard-vest-orange-2xl-3xl>: HTTP status code is not handled or not allowed 2026-01-24 21:53:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triumph-style-t1hd-hss-jobbers-drill-black-bronze-oxide-1564-12>: HTTP status code is not handled or not allowed 2026-01-24 21:53:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=jet-jtm-1050evs2-230-mill-acu-rite-200s-dro-x-and-y-axis-powerfeeds-690544 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:53:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=jet-jtm-2-mill-newall-dp500-dro-x-and-y-axis-powerfeeds-690949>: HTTP status code is not handled or not allowed 2026-01-24 21:53:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-38w-x-48l-2-mil-clear-200pack>: HTTP status code is not handled or not allowed 2026-01-24 21:53:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-solid-wallshelf-12-wx48-l-12-ga>: HTTP status code is not handled or not allowed 2026-01-24 21:53:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/29-25-x-81mm-10-dram-clear-squat-vial-w-24-400-black-phenolic-polycone-lined-cap-144pk>: HTTP status code is not handled or not allowed 2026-01-24 21:53:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bollard-post-sleeve-4-d-x-52-h-yellow-with-red-tape-hdpe>: HTTP status code is not handled or not allowed 2026-01-24 21:53:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/short-sleevet-shirt-segmented-tape-hi-vis-yellow-medium>: HTTP status code is not handled or not allowed 2026-01-24 21:53:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-4000-single-strap-wrist-support-tan-medium-right>: HTTP status code is not handled or not allowed 2026-01-24 21:53:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-4000-single-strap-wrist-support-tan-large-right>: HTTP status code is not handled or not allowed 2026-01-24 21:53:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/weathermaster-twin-black-post-retracting-belt-barrier-ada-compliant-11-ft-black-belt>: HTTP status code is not handled or not allowed 2026-01-24 21:53:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cart-cover-400-denier-nylon-for-24-x-60-x-63-unit-hook-loop-fastener-closure-red>: HTTP status code is not handled or not allowed 2026-01-24 21:53:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scale-ring-jsl-14dp17fse-10604503>: HTTP status code is not handled or not allowed 2026-01-24 21:53:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=jet-jtm-1050evs2-230-mill-acu-rite-200s-dro-x-and-y-axis-powerfeeds-690544>: HTTP status code is not handled or not allowed 2026-01-24 21:53:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cart-cover-400-denier-nylon-for-24-x-48-x-74-unit-hook-loop-fastener-closure-red>: HTTP status code is not handled or not allowed 2026-01-24 21:53:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8925-class-e-thermal-pants-lime-4-x-large>: HTTP status code is not handled or not allowed 2026-01-24 21:53:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:53:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8001-arm-leg-band-lime-s-m>: HTTP status code is not handled or not allowed 2026-01-24 21:53:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/893-filament-tape-2-x-60-yds-6-mil-clear-6pack>: HTTP status code is not handled or not allowed 2026-01-24 21:53:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-breathable-pants-hi-viz-yellow-xl>: HTTP status code is not handled or not allowed 2026-01-24 21:53:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/506-24-screwhd-washer-10106s24>: HTTP status code is not handled or not allowed 2026-01-24 21:53:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-4010-double-strap-wrist-support-tan-x-large-left>: HTTP status code is not handled or not allowed 2026-01-24 21:53:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/full-zip-hoodie-sweatshirt-hi-vis-yellow-xl>: HTTP status code is not handled or not allowed 2026-01-24 21:53:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/17-x-60mm-2-dram-amber-glass-vial-w-15-425-green-thermoset-cap-vacuum-144pk>: HTTP status code is not handled or not allowed 2026-01-24 21:53:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/21-x-70mm-4-dram-amber-borosilicate-vial-with-18-400-neck-finish-vial-only-144pk>: HTTP status code is not handled or not allowed 2026-01-24 21:53:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-unit-with-two-enclosures-chrome-24w-x-36l-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 21:53:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-unit-with-one-enclosure-chrome-24w-x-48l-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 21:53:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions>: HTTP status code is not handled or not allowed 2026-01-24 21:53:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25-x-95mm-8-dram-vial-w-22-400-green-thermoset-f217-ptfe-lined-cap-cleaned-144pk>: HTTP status code is not handled or not allowed 2026-01-24 21:53:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprng-hndl-mount-rear-5710981>: HTTP status code is not handled or not allowed 2026-01-24 21:53:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classic-flame-resistant-short-sleeve-t-shirt-ansi-hi-viz-yellow-3xl>: HTTP status code is not handled or not allowed 2026-01-24 21:53:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/decor_linens/frames_posters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:53:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-12w-x-24l-1-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 21:53:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-round-message-stamp-check-mark-pre-inked-re-inkable-red>: HTTP status code is not handled or not allowed 2026-01-24 21:53:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cable-18-100mm-41002-95>: HTTP status code is not handled or not allowed 2026-01-24 21:53:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insul-tube-6l-3-4-thick-7-8>: HTTP status code is not handled or not allowed 2026-01-24 21:53:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-one-comfort-grip-deluxe-plus-d-ring-view-binder-5-capacity-8-1-2-x-11-white>: HTTP status code is not handled or not allowed 2026-01-24 21:53:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lateral-file-letter-legal-size-fireproof-4-drawers-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:53:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/occumitts-support-gloves-1-pair-large-navy>: HTTP status code is not handled or not allowed 2026-01-24 21:53:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-30w-x-42l-3-mil-clear-100pack>: HTTP status code is not handled or not allowed 2026-01-24 21:53:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/outdoor-metal-trash-container-blue-2>: HTTP status code is not handled or not allowed 2026-01-24 21:53:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-32w-x-32l-3-mil-clear-100pack>: HTTP status code is not handled or not allowed 2026-01-24 21:53:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/decor_linens/frames_posters>: HTTP status code is not handled or not allowed 2026-01-24 21:53:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/protective-goggle-clearframe-polycarbonate-lens-2220>: HTTP status code is not handled or not allowed 2026-01-24 21:53:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/operating-plate-437-2>: HTTP status code is not handled or not allowed 2026-01-24 21:53:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metal-spreaders-steel-plastic-3-w-x-1-3-d-x-4-1-2-h>: HTTP status code is not handled or not allowed 2026-01-24 21:53:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/office-grade-guillotine-paper-trimmer-15-cutting-length-15-sheet-capacity-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:53:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-bushel-yellow-vinyl-basket-truck-bulk-truck>: HTTP status code is not handled or not allowed 2026-01-24 21:53:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/205675-9-hyflex-ultra-lghtweight-assembly-glove-11-801-9>: HTTP status code is not handled or not allowed 2026-01-24 21:53:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-van-stand-steel-16-w-x-17-d-x-24-h>: HTTP status code is not handled or not allowed 2026-01-24 21:53:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lock-multi-position-caster-lock-bolt-on-type>: HTTP status code is not handled or not allowed 2026-01-24 21:53:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-nitrimed-xtra-nitrile-glove-powder-free-6012l>: HTTP status code is not handled or not allowed 2026-01-24 21:54:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flange-b-233-1>: HTTP status code is not handled or not allowed 2026-01-24 21:54:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flex-tuff-10-gage-bluelatex-ctd-palm-9680l>: HTTP status code is not handled or not allowed 2026-01-24 21:54:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/valve-block-jsg-0716n-302-0018>: HTTP status code is not handled or not allowed 2026-01-24 21:54:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bearing-separator-3-16-12-mm-to-3-75-mm-open-used-with-18-1-pulling-device>: HTTP status code is not handled or not allowed 2026-01-24 21:54:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plate-front-4224-6295799>: HTTP status code is not handled or not allowed 2026-01-24 21:54:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/205571-8-hyflex-ultra-lightweight-assembly-glove-11-800-8>: HTTP status code is not handled or not allowed 2026-01-24 21:54:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/danger-restricted-area-authorized-employees-only-10x14-rigid-plastic>: HTTP status code is not handled or not allowed 2026-01-24 21:54:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/411kaltprp-safety-padlock-1-12w-x-3h-shackle-purple>: HTTP status code is not handled or not allowed 2026-01-24 21:54:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/int-extractor-set-wth-counterstay-slide-ha-mmer-1-1-8-28-mm-to-2-3-4-70-mm>: HTTP status code is not handled or not allowed 2026-01-24 21:54:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/notebooks_pads already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:54:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-13w-x-28l-3-mil-clear-500pack>: HTTP status code is not handled or not allowed 2026-01-24 21:54:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 27375 pages (at 97 pages/min), scraped 13812 items (at 22 items/min) 2026-01-24 21:54:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-extinguisher-sign-18x4-aluminum>: HTTP status code is not handled or not allowed 2026-01-24 21:54:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/192243-8-chemi-pro-hvy-dty-neoprene-nat-latex-224-8>: HTTP status code is not handled or not allowed 2026-01-24 21:54:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-8w-x-14l-3-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 21:54:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/miracool-velcro-collar-bandanas-blue-denim-12-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:54:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-weight-hot-mill-glove-w-gauntlet-1bc40721>: HTTP status code is not handled or not allowed 2026-01-24 21:54:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-5w-x-38l-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 21:54:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-restrainer-563038a>: HTTP status code is not handled or not allowed 2026-01-24 21:54:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-16w-x-48l-3-mil-clear-250pack>: HTTP status code is not handled or not allowed 2026-01-24 21:54:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s33ltprp-snap-lock-safety-padlock-1-38w-x-3h-purple>: HTTP status code is not handled or not allowed 2026-01-24 21:54:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/needle-cover-f25-27>: HTTP status code is not handled or not allowed 2026-01-24 21:54:18 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/paper_stationery/notebooks_pads landed on page that is not a product page. 2026-01-24 21:54:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/410kagrn-safety-padlock-1-12h-shackle-green>: HTTP status code is not handled or not allowed 2026-01-24 21:54:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/meteors-disposable-earplugs-uncorded-nrr-33-6870>: HTTP status code is not handled or not allowed 2026-01-24 21:54:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/main-shaft-housing-cos18-112>: HTTP status code is not handled or not allowed 2026-01-24 21:54:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-12w-x-15l-4-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 21:54:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/205573-10-hyflex-ultra-lghtweight-assembly-glove-11-800-10>: HTTP status code is not handled or not allowed 2026-01-24 21:54:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-flame-resistant-shoulder-length-two-way-midnight-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:54:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/diamond-blade-for-mbse>: HTTP status code is not handled or not allowed 2026-01-24 21:54:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/id-label-ap3-19>: HTTP status code is not handled or not allowed 2026-01-24 21:54:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reverse-switch-ghd-20tpft-et-1221>: HTTP status code is not handled or not allowed 2026-01-24 21:54:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-rain-poncho-80-l-one-size-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:54:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cart-fitler-combo-am-map100-7584p100>: HTTP status code is not handled or not allowed 2026-01-24 21:54:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chain-plastic-1-1-2-links-on-a-reel-red-200-feet-trade-size-6>: HTTP status code is not handled or not allowed 2026-01-24 21:54:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-exterior-42h-platform-2-rails-spiral-stair-kit-60dia-11-3-4h-alum-handrail>: HTTP status code is not handled or not allowed 2026-01-24 21:54:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-d-ring-extension-8928-18inbk>: HTTP status code is not handled or not allowed 2026-01-24 21:54:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deflectors-24-1155-sta2f-z3036>: HTTP status code is not handled or not allowed 2026-01-24 21:54:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n95-particulate-respirator-plus-nuisance-oz-2400n95>: HTTP status code is not handled or not allowed 2026-01-24 21:54:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3lhblu-laminated-steel-safety-padlock-1-916w-x-2h-shackle-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:54:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hayden-36h-platform-rail-spiral-stair-kit-60dia-12-5-6h-blk>: HTTP status code is not handled or not allowed 2026-01-24 21:54:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s1105prp-aluminum-safety-padlock-1-12w-x-1h-shackle-purple>: HTTP status code is not handled or not allowed 2026-01-24 21:54:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heaters-34-1868-tru801805>: HTTP status code is not handled or not allowed 2026-01-24 21:54:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s1106ylw-aluminum-safety-padlock-1-12w-x-1-12h-shackle-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:54:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-premium-locker-single-tier-12x15x72-1-door-ready-assemble-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:54:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/406kas12orj-safety-padlock-orange-12set>: HTTP status code is not handled or not allowed 2026-01-24 21:54:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gaskets-32-1387-cle100330>: HTTP status code is not handled or not allowed 2026-01-24 21:54:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/410kas3red-safety-padlock-1-12h-shackle-red-3set>: HTTP status code is not handled or not allowed 2026-01-24 21:54:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pathway-hd-code-compliant-wheelchair-ramp-kit-36-x-48-w-guards-3-pk>: HTTP status code is not handled or not allowed 2026-01-24 21:54:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-13w-x-18l-4-mil-clear-500pack>: HTTP status code is not handled or not allowed 2026-01-24 21:54:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-premium-locker-double-tier-12x15x30-6-door-parchment>: HTTP status code is not handled or not allowed 2026-01-24 21:54:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-14w-x-28l-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 21:54:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cable-choker-sling-7x193-8-stainless-steel-w-5900550>: HTTP status code is not handled or not allowed 2026-01-24 21:54:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elements-34-1409-apw82530100>: HTTP status code is not handled or not allowed 2026-01-24 21:54:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-premium-locker-double-tier-12x12x36-2-door-assembled-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:54:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cords-38-1304-apw85638>: HTTP status code is not handled or not allowed 2026-01-24 21:54:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-40w-x-60l-2-mil-clear-100pack>: HTTP status code is not handled or not allowed 2026-01-24 21:54:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thermostats-46-1155-sta2t-30402-08>: HTTP status code is not handled or not allowed 2026-01-24 21:54:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nuts-26-2118-altnu-2187>: HTTP status code is not handled or not allowed 2026-01-24 21:54:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-x-30-solid-deck-hardwood-dolly-t-15-1830-5ph-fb-1500-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:54:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/automatic-wire-twister-rubber-handle>: HTTP status code is not handled or not allowed 2026-01-24 21:54:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-premium-locker-single-tier-12x12x60-3-door-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:54:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/handle-sleeve-gh-1340w-gh1440w-07-26>: HTTP status code is not handled or not allowed 2026-01-24 21:54:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jet-jcdc-2-cyclone-dust-collector-2hp-230v>: HTTP status code is not handled or not allowed 2026-01-24 21:54:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vent-plug-hbs814gh-800-48>: HTTP status code is not handled or not allowed 2026-01-24 21:54:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-12w-x-16l-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 21:54:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-plug-body-with-high-flow-profile-1-2-fnpt-connection-used-with-esi-coupling>: HTTP status code is not handled or not allowed 2026-01-24 21:54:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/doors-28-2372-man040001717>: HTTP status code is not handled or not allowed 2026-01-24 21:54:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mesh-task-chair-fabric-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:54:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s1-safety-quick-coupling-1-4-industrial-interchange-with-1-4-fnpt-connection>: HTTP status code is not handled or not allowed 2026-01-24 21:54:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wormgear-adjustable-bull-float-bracket>: HTTP status code is not handled or not allowed 2026-01-24 21:54:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-drainage-matting-standard-tile-12-x-12-36-per-case>: HTTP status code is not handled or not allowed 2026-01-24 21:54:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-step-24in-w-steel-safety-angle-rolling-ladder-grip-strut-gray-kdec109242>: HTTP status code is not handled or not allowed 2026-01-24 21:54:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-nexelon-24w-x-48l-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 21:54:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5c-collet-23-32-inch-round-tmx>: HTTP status code is not handled or not allowed 2026-01-24 21:55:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-unit-5-tier-poly-z-brite-21w-x-54l-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 21:55:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-fnpt-filter-regulator>: HTTP status code is not handled or not allowed 2026-01-24 21:55:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camlock-studs-d1-4-5-8-diameter-metric-thread-m10x1>: HTTP status code is not handled or not allowed 2026-01-24 21:55:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camlock-studs-d1-3-9-16-diameter-metric-thread-m10x1>: HTTP status code is not handled or not allowed 2026-01-24 21:55:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/screw-spacer-hp35a-60>: HTTP status code is not handled or not allowed 2026-01-24 21:55:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crow-bar-w-29>: HTTP status code is not handled or not allowed 2026-01-24 21:55:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/soft-blank-jaws-for-bison-scroll-chuck-6-inch-3-piece-set>: HTTP status code is not handled or not allowed 2026-01-24 21:55:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/13-step-24in-w-30in-d-top-step-steel-rolling-ladder-perforated-tread-36in-handrail-kdsr113246-d3>: HTTP status code is not handled or not allowed 2026-01-24 21:55:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-jaw-2-piece-scroll-chuck-forged-steel-body-camlock-d-12-inch-d1-6>: HTTP status code is not handled or not allowed 2026-01-24 21:55:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/28-543-wood-handle-stiff-scraper-knife-3-inch>: HTTP status code is not handled or not allowed 2026-01-24 21:55:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 27473 pages (at 98 pages/min), scraped 13838 items (at 26 items/min) 2026-01-24 21:55:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-5-hp-roof-ventilator>: HTTP status code is not handled or not allowed 2026-01-24 21:55:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ceiling-kit-1-1-ton-1-5-ton-air-conditioners-1>: HTTP status code is not handled or not allowed 2026-01-24 21:55:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f4424rtgv-econo-trough-type-3r-400x400x2400-galvanized>: HTTP status code is not handled or not allowed 2026-01-24 21:55:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ceiling-kit-5-ton-10-ton-air-conditioners-1>: HTTP status code is not handled or not allowed 2026-01-24 21:55:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a1614nfal-junction-box-type-4x-clamp-cover-1600x1400x600>: HTTP status code is not handled or not allowed 2026-01-24 21:55:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/minigrip-slider-grip-reclosable-poly-bags-14-12w-x-16l-27-mil-clear250pk>: HTTP status code is not handled or not allowed 2026-01-24 21:55:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-systems-denali-series-multipurpose-rack-black>: HTTP status code is not handled or not allowed 2026-01-24 21:55:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tongue-groove-plier-sets-pc-1>: HTTP status code is not handled or not allowed 2026-01-24 21:55:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-24w-x-28l-2-mil-clear-250pack>: HTTP status code is not handled or not allowed 2026-01-24 21:55:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-chrome-18w-x-42l-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 21:55:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/awdh2420n4ss-deep-hinged-window-kit-2219-x-1614-ss>: HTTP status code is not handled or not allowed 2026-01-24 21:55:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-60-w-x-18-d-x-63-h>: HTTP status code is not handled or not allowed 2026-01-24 21:55:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cspb2020-concept8482-swing-out-panel-fits-2000x2000>: HTTP status code is not handled or not allowed 2026-01-24 21:55:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-30w-x-30l-2-mil-clear-250pack>: HTTP status code is not handled or not allowed 2026-01-24 21:55:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-step-steel-caged-walk-through-fixed-access-ladder-gray-wlfc1118>: HTTP status code is not handled or not allowed 2026-01-24 21:55:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0-150mm-0-6-x-0-05mm-and-1-128-vernier-caliper>: HTTP status code is not handled or not allowed 2026-01-24 21:55:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gusseted-poly-bags-8w-x-3d-x-20h-3-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 21:55:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f44b360-barriers-qty-5-fits-400x400-steelgray>: HTTP status code is not handled or not allowed 2026-01-24 21:55:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-unit-5-tier-chrome-24w-x-24l-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 21:55:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_tools/cleaning_supply_dispensers_organizers/broom_holders_shadow_boards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:55:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-purpose-vinyl-stool-black>: HTTP status code is not handled or not allowed 2026-01-24 21:55:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-5w-x-18l-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 21:55:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a6636t1t-wire-trough-type-1-600x600x3600-steelgray>: HTTP status code is not handled or not allowed 2026-01-24 21:55:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-mm-od-nylon-tubing-natural-color-100-roll-160-500-psi>: HTTP status code is not handled or not allowed 2026-01-24 21:55:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-unit-5-tier-poly-z-brite-24w-x-24l-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 21:55:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-9w-x-15l-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 21:55:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-od-polyurethane-tubing-blue-color-100-roll-125-200-psi>: HTTP status code is not handled or not allowed 2026-01-24 21:55:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fixed-steel-ladder-guard-door-for-entry-onto-fixed-ladder-safety-yellow-opfs03-y>: HTTP status code is not handled or not allowed 2026-01-24 21:55:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/garden-hoes-h-103fg>: HTTP status code is not handled or not allowed 2026-01-24 21:55:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-nexelon-18w-x-48l-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 21:55:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_tools/cleaning_supply_dispensers_organizers/broom_holders_shadow_boards>: HTTP status code is not handled or not allowed 2026-01-24 21:55:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-enclosure-opaque-cover-no-knockouts-8-66-in-x-5-9-in-x-11-81-in>: HTTP status code is not handled or not allowed 2026-01-24 21:55:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-6w-x-18l-6-mil-clear-500pack>: HTTP status code is not handled or not allowed 2026-01-24 21:55:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-poly-z-brite-24w-x-24l-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 21:55:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/model-535-replacement-parts-47570>: HTTP status code is not handled or not allowed 2026-01-24 21:55:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bird-spikes-adhesive-10oz>: HTTP status code is not handled or not allowed 2026-01-24 21:55:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-step-plastic-step-stand-21-w-x-19-1-2-d-x-24-1-2-h-black>: HTTP status code is not handled or not allowed 2026-01-24 21:55:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-threading-model-1224-machine-die-heads-dies-33077>: HTTP status code is not handled or not allowed 2026-01-24 21:55:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f44wa-telescope-ftg-400x400>: HTTP status code is not handled or not allowed 2026-01-24 21:55:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-36x24x72-with-5-shelves-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:55:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/healthsmart-folding-walking-stick-soft-comfort-grip-collapsible-walking-stick-adjustable-folding-walking-cane-lavender>: HTTP status code is not handled or not allowed 2026-01-24 21:55:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anti-static-flat-poly-bags-18w-x-30l-6-mil-pink-200pack>: HTTP status code is not handled or not allowed 2026-01-24 21:55:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gear-lever-brckt-ghd-2020pf-2401066>: HTTP status code is not handled or not allowed 2026-01-24 21:55:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-36x24x72-with-4-shelves-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:55:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c06d-310-9-amp-mini-contactor-non-reversing-24v-coil>: HTTP status code is not handled or not allowed 2026-01-24 21:55:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sharpshooter-hammer-tacker-pht150c>: HTTP status code is not handled or not allowed 2026-01-24 21:55:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/model-no-404-400-series-instrument-bender-14-tube-size-58-bend-radius>: HTTP status code is not handled or not allowed 2026-01-24 21:55:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f44w24-straight-section-400x400x2400>: HTTP status code is not handled or not allowed 2026-01-24 21:55:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mabis-precision-series-aneroid-sphygmomanometer-manual-blood-pressure-monitor-with-calibrated-blue-nylon-cuff-and-carrying-case-large-adult>: HTTP status code is not handled or not allowed 2026-01-24 21:55:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-nexelon-24w-x-48l-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 21:55:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f66le9a-90-elbowtop-open-600x600-steelgray>: HTTP status code is not handled or not allowed 2026-01-24 21:55:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-poly-z-brite-21w-x-60l-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 21:55:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c06d-301-9-amp-mini-contactor-non-reversing-24v-coil>: HTTP status code is not handled or not allowed 2026-01-24 21:55:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-shelving-36x12x72-with-5-shelves-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:55:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22mm-non-metallic-non-illuminated-turn-to-release-round-mushroom-button>: HTTP status code is not handled or not allowed 2026-01-24 21:55:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1300-strapping-tape-2-x-60-yds-39-mil-clear-24pack>: HTTP status code is not handled or not allowed 2026-01-24 21:55:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-shelving-48x12x72-with-4-shelves-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:55:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-shelving-48x24x72-with-4-shelves-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 21:55:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnetic-corner-bumper-guard-a-39-inches-long-by-2inches-wide-black-yellow-60-6910>: HTTP status code is not handled or not allowed 2026-01-24 21:55:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1300-strapping-tape-38-x-60-yds-39-mil-clear-96pack>: HTTP status code is not handled or not allowed 2026-01-24 21:55:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-cap-bolt-ss-8nss-12n-ss8n-9>: HTTP status code is not handled or not allowed 2026-01-24 21:55:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clamp-7-inch-diameter-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:55:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gusseted-poly-bags-6w-x-4d-x-15h-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 21:55:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clamp-together-90-deg-elbow-15-clr-6-inch-diameter-glv-22-gage>: HTTP status code is not handled or not allowed 2026-01-24 21:55:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fastdraw-freedom-portable-dispensing-system-1-gal-min-4-gal-min-12-case-91290>: HTTP status code is not handled or not allowed 2026-01-24 21:55:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clamp-together-adjustable-sleeve-w-o-ring-3-inch-diameter-galvanized-24-gage>: HTTP status code is not handled or not allowed 2026-01-24 21:55:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-bulk-box-6-bushel-truck-steel-chassis-base-white>: HTTP status code is not handled or not allowed 2026-01-24 21:55:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clamp-together-90-deg-tubed-elbow-25-clr-4-inch-diameter-ss-16-gage>: HTTP status code is not handled or not allowed 2026-01-24 21:55:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/worm-gear-1230r-5233771>: HTTP status code is not handled or not allowed 2026-01-24 21:56:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-lab-coat-1-pocket-knit-wrist-collar-snap-closure-x-large-25-case>: HTTP status code is not handled or not allowed 2026-01-24 21:56:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mounted-hair-dryer-black>: HTTP status code is not handled or not allowed 2026-01-24 21:56:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/j-hook-close-ring-steel>: HTTP status code is not handled or not allowed 2026-01-24 21:56:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/janitorial-storage-cabinet-30x15x66-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:56:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/model-no-18-straight-pipe-wrenches-18-2-12-pipe-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:56:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/table-4-chair-set-black-polypropylene-cafe-chairs-36w-x-29h-round-natural-table>: HTTP status code is not handled or not allowed 2026-01-24 21:56:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-bulk-box-6-bushel-truck-steel-chassis-base-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:56:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mid-back-white-mesh-swivel-ergonomic-task-office-chair-with-flip-up-arms>: HTTP status code is not handled or not allowed 2026-01-24 21:56:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gear-shaft-16t-ghb-1340-4212>: HTTP status code is not handled or not allowed 2026-01-24 21:56:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/model-no-2-strap-wrench-11-34l-1-18-strap-width-5-12-od-tube-capacity1>: HTTP status code is not handled or not allowed 2026-01-24 21:56:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-8w-x-18l-8-mil-clear-500pack>: HTTP status code is not handled or not allowed 2026-01-24 21:56:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 27575 pages (at 102 pages/min), scraped 13860 items (at 22 items/min) 2026-01-24 21:56:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-x-24-plastic-square-edge-bench-top>: HTTP status code is not handled or not allowed 2026-01-24 21:56:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-1-2-medium-orange-brush-sw77350>: HTTP status code is not handled or not allowed 2026-01-24 21:56:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shaft-worm-gear-3706002>: HTTP status code is not handled or not allowed 2026-01-24 21:56:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shaft-gear-3710007>: HTTP status code is not handled or not allowed 2026-01-24 21:56:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/erinwal-sorbo-air-gel-anti-fatigue-insoles-d-4-5-heel-arch-gel-inserts-antibacterial>: HTTP status code is not handled or not allowed 2026-01-24 21:56:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1400-strapping-tape-38-x-60-yds-43-mil-clear-96pack>: HTTP status code is not handled or not allowed 2026-01-24 21:56:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/workbench-riser-96-w-x-10-1-2-d-x-12-h-black>: HTTP status code is not handled or not allowed 2026-01-24 21:56:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/impactoe-steel-toe-cap-overshoes-med-8-5-11-vulcanized-rbbr-pvc-free-waterproof-antislip>: HTTP status code is not handled or not allowed 2026-01-24 21:56:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-39in-h-retractable-stanchion-with-6-1-2-ft-red-belt>: HTTP status code is not handled or not allowed 2026-01-24 21:56:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kneeling-mat-8-x-16-x-1-waterproof-foam-carrying-handle-meets-flammability-standard>: HTTP status code is not handled or not allowed 2026-01-24 21:56:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/taco-bronze-pressure-reducing-valve-34-npt>: HTTP status code is not handled or not allowed 2026-01-24 21:56:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wiremold-wspc320bk-desktop-power-center-3-tamper-resistant-outlets-2-3a-usb-ports-black>: HTTP status code is not handled or not allowed 2026-01-24 21:56:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-aluminum-half-pallet-36x24>: HTTP status code is not handled or not allowed 2026-01-24 21:56:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rendezvous-eyewear-amber-lens-with-black-frame>: HTTP status code is not handled or not allowed 2026-01-24 21:56:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-half-skid-40x24>: HTTP status code is not handled or not allowed 2026-01-24 21:56:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spacing-ring-5mm-1806211>: HTTP status code is not handled or not allowed 2026-01-24 21:56:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/solid-steel-shelf-48-w-x-18-d>: HTTP status code is not handled or not allowed 2026-01-24 21:56:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-base-assy-c0-2905330>: HTTP status code is not handled or not allowed 2026-01-24 21:56:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-cabinets-fs-36-b-storage-cabinet-assembled-36x18x72-black>: HTTP status code is not handled or not allowed 2026-01-24 21:56:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lewiston-equine-mailbox-no-address-plates-w-horsehead-finial-decorative-fluted-base-black>: HTTP status code is not handled or not allowed 2026-01-24 21:56:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-foot-x-12-foot-blue-tarp>: HTTP status code is not handled or not allowed 2026-01-24 21:56:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-chain-1-link-100-l-hdpe-traffic-orange>: HTTP status code is not handled or not allowed 2026-01-24 21:56:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ecotrap-insert-case-of-6>: HTTP status code is not handled or not allowed 2026-01-24 21:56:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-precision-ansi-nickel-plated-roller-chain-120-1np-1-1-2-pitch-connecting-link>: HTTP status code is not handled or not allowed 2026-01-24 21:56:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-precision-iso-metric-double-roller-chain-24b-2-1-1-2-pitch-connecting-link>: HTTP status code is not handled or not allowed 2026-01-24 21:56:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/complete-packaging-workbench-plastic-safety-edge-60-x-30>: HTTP status code is not handled or not allowed 2026-01-24 21:56:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stronghold-stainless-steel-storage-cabinet-36-x-20-x-72>: HTTP status code is not handled or not allowed 2026-01-24 21:56:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/offices-to-go-bridge-48wx24dx29-1-2h-walnut>: HTTP status code is not handled or not allowed 2026-01-24 21:56:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-chain-3-4-link-100-l-hdpe-safety-pink>: HTTP status code is not handled or not allowed 2026-01-24 21:56:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4956-double-sided-foam-tape-12-x-5-yds-62-mil-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:56:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stlflx-airguardz-astm-level-1-surgical-mask>: HTTP status code is not handled or not allowed 2026-01-24 21:56:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/041-372-24-cat-5e-24-port-110-idc-patch-panel>: HTTP status code is not handled or not allowed 2026-01-24 21:56:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/locknut-assy-300s-2904060>: HTTP status code is not handled or not allowed 2026-01-24 21:56:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-profile-auto-shift-hydraulic-scissor-lift-cart-900-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:56:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/front-strip-ghb-1340-7141>: HTTP status code is not handled or not allowed 2026-01-24 21:56:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/074-877-50bl-cat-6-utp-mold-injection-snagless-patch-cord-50-ft-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:56:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bearing-cap-jsm-748-30111>: HTTP status code is not handled or not allowed 2026-01-24 21:56:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-foot-x-25-foot-blue-tarp>: HTTP status code is not handled or not allowed 2026-01-24 21:56:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-foot-x-5-foot-light-duty-industrial-drain-tarp>: HTTP status code is not handled or not allowed 2026-01-24 21:56:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4947f-double-sided-foam-tape-34-x-5-yds-45-mil-black>: HTTP status code is not handled or not allowed 2026-01-24 21:56:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22-7513w-6xs-cut-resistant-gloves-nitrile-micro-foam-coating-a2-b-size-6>: HTTP status code is not handled or not allowed 2026-01-24 21:56:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-x-36-x-36-double-shelf-shop-stand-maple-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:56:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/airspade-ht126-25-scfm-nozzle>: HTTP status code is not handled or not allowed 2026-01-24 21:56:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/airspade-ht220-coupling-for-series-2000>: HTTP status code is not handled or not allowed 2026-01-24 21:56:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/minigrip-white-block-reclosable-poly-bags-5w-x-8l-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 21:56:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-foot-x-20-foot-heavy-duty-silver-tarp>: HTTP status code is not handled or not allowed 2026-01-24 21:56:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-packaging-workbench-plastic-safety-edge-72-x-30-with-lower-shelf-kit>: HTTP status code is not handled or not allowed 2026-01-24 21:56:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/purofort-full-safety-mens-work-boots-size-16-green>: HTTP status code is not handled or not allowed 2026-01-24 21:56:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/airestream-fume-hood-with-explosion-proof-light-48w-x-30d-x-59h>: HTTP status code is not handled or not allowed 2026-01-24 21:56:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/resealable-poly-bags-9w-x-12l-15-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 21:56:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-tarp-repair-tape-2-inch-x-35-foot>: HTTP status code is not handled or not allowed 2026-01-24 21:56:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-precision-ansi-cottered-pin-roller-chain-50-1c-5-8-pitch-100ft-reel>: HTTP status code is not handled or not allowed 2026-01-24 21:56:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-precision-ansi-double-roller-chain-40-2r-1-2-pitch-100ft-reel>: HTTP status code is not handled or not allowed 2026-01-24 21:56:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/main-screw-screw-jack-sj15t-04>: HTTP status code is not handled or not allowed 2026-01-24 21:56:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40099-60-grit-aluminum-oxide-25-lbs>: HTTP status code is not handled or not allowed 2026-01-24 21:56:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-inch-deep-top-step-kit-grip-strut>: HTTP status code is not handled or not allowed 2026-01-24 21:56:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-hanging-ranch-sign-with-post-decorative-ornate-base-and-horsehead-finial-in-black>: HTTP status code is not handled or not allowed 2026-01-24 21:56:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-step-50-incline-ladder-24-inch-w-grip-strut>: HTTP status code is not handled or not allowed 2026-01-24 21:56:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30230su-fb32sp8506-32w-fluorescent-t8-ubend-tube-bulb-mini-bipin-super-white>: HTTP status code is not handled or not allowed 2026-01-24 21:56:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/throttle-valve-pin-jet-1t-t3020121b>: HTTP status code is not handled or not allowed 2026-01-24 21:56:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-kit-for-electric-pallet-jack-gwk-mpe080vg-lw-xl-fits-yale-model-mpe080vg>: HTTP status code is not handled or not allowed 2026-01-24 21:56:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40056-5-16-id-steel-nozzle-kit-20-cfm-siphon>: HTTP status code is not handled or not allowed 2026-01-24 21:57:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-step-crossover-leg-kit>: HTTP status code is not handled or not allowed 2026-01-24 21:57:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-kit-for-electric-pallet-jack-gwk-epj40-lw-xl-fits-blue-giant-model-epj40>: HTTP status code is not handled or not allowed 2026-01-24 21:57:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polypropylene-bags-14l-x-8w-15-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 21:57:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gondola-free-standing-slider-system-with-28-bins-ivory>: HTTP status code is not handled or not allowed 2026-01-24 21:57:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unsupported-5-step-20-inch-cantilever-ladder-grip-strut>: HTTP status code is not handled or not allowed 2026-01-24 21:57:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/minigrip174-reclosable-poly-bags-12w-x-12l-4-mil-clear-500pack>: HTTP status code is not handled or not allowed 2026-01-24 21:57:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/05070su-f13t5cw-13w-fluorescent-t5-bulb-mini-bipin-cool-white>: HTTP status code is not handled or not allowed 2026-01-24 21:57:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11645-window-underlay-24-x-12-polystyrene>: HTTP status code is not handled or not allowed 2026-01-24 21:57:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wesco-174-triple-truck-gator-grip-drum-handler-attachment-261161-750-lb-cap>: HTTP status code is not handled or not allowed 2026-01-24 21:57:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tevisio-led-39-reach-35d-8d-13w-5000k-dimmable-with-segment-switching>: HTTP status code is not handled or not allowed 2026-01-24 21:57:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4000-psi-quick-connect-nozzle-0-15-25-40-degree-and-6540-soap-nozzle-5-pack-45>: HTTP status code is not handled or not allowed 2026-01-24 21:57:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bearing-thrust-51103--2244plus-50-9000>: HTTP status code is not handled or not allowed 2026-01-24 21:57:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 27671 pages (at 96 pages/min), scraped 13883 items (at 23 items/min) 2026-01-24 21:57:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-deluxe-3-shelf-plastic-utility-service-cart-6-pneumatic-casters>: HTTP status code is not handled or not allowed 2026-01-24 21:57:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/minigrip174-reclosable-poly-bags-14w-x-24l-4-mil-clear-250pack>: HTTP status code is not handled or not allowed 2026-01-24 21:57:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-cal-diamond-grip-rubber-runner-mats-2mm-x-4-x-6-rubber-flooring-rolls-black>: HTTP status code is not handled or not allowed 2026-01-24 21:57:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tarifold-antimicrobial-desktop-organizer-starter-set-10-black-pockets>: HTTP status code is not handled or not allowed 2026-01-24 21:57:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bib1-bench-in-a-box-standard-workbench-60-w-x-30-d-plastic-laminate-top-beige>: HTTP status code is not handled or not allowed 2026-01-24 21:57:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40121-50-x-1-2-i-d-pressure-hose-kit-rubber>: HTTP status code is not handled or not allowed 2026-01-24 21:57:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/resealable-polypropylene-bags-9-12l-x-6-12w-15-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 21:57:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/folding-bed-board-mattress-support-48-x-60-double-size>: HTTP status code is not handled or not allowed 2026-01-24 21:57:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-hardware-kit-for-761221-outdoor-drinking-fountains>: HTTP status code is not handled or not allowed 2026-01-24 21:57:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hugg-a-pillow-hypoallergenic-bed-pillow>: HTTP status code is not handled or not allowed 2026-01-24 21:57:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/knob-708019-104>: HTTP status code is not handled or not allowed 2026-01-24 21:57:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-wood-spoon>: HTTP status code is not handled or not allowed 2026-01-24 21:57:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transmission-shaft-jmd-18pfn-text-291023a>: HTTP status code is not handled or not allowed 2026-01-24 21:57:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-push-to-connect-plug>: HTTP status code is not handled or not allowed 2026-01-24 21:57:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/174-6294905-extension-bed-for-4224b>: HTTP status code is not handled or not allowed 2026-01-24 21:57:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/repair-kit-slt330fh-rk>: HTTP status code is not handled or not allowed 2026-01-24 21:57:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wheel-a-bout-portable-utility-ramp-wabu1230-12l-x-30w-800-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 21:57:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/infeed-direction-label-jjp8bt-88>: HTTP status code is not handled or not allowed 2026-01-24 21:57:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a404sc-screw-cover-type-12-400x400x300-steelgray>: HTTP status code is not handled or not allowed 2026-01-24 21:57:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-conductor-shielded-plenum-cable-6-conductor-22-awg-7-30-bc-white>: HTTP status code is not handled or not allowed 2026-01-24 21:57:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss-piano-wire-whip-epoxy-16>: HTTP status code is not handled or not allowed 2026-01-24 21:57:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/equip-12in-add-on-faucet-for-pre-rinse-unit>: HTTP status code is not handled or not allowed 2026-01-24 21:57:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-power-cable-500-261990-641250-641263-641244-641264-641265-641407-scrubbers>: HTTP status code is not handled or not allowed 2026-01-24 21:57:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bin-nestable-4-1-8-w-x-11-5-8-d-x-6-h-red>: HTTP status code is not handled or not allowed 2026-01-24 21:57:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dandux-fda-approved-plastic-bulk-container-step-wall-16-bushel-natural>: HTTP status code is not handled or not allowed 2026-01-24 21:57:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a242412gsc-galvanized-box-screw-cover-2400x2400x1200-galvanized>: HTTP status code is not handled or not allowed 2026-01-24 21:57:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-polypropylene-bags-8l-x-8w-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 21:57:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-slip-on-shoes-size-8m-white>: HTTP status code is not handled or not allowed 2026-01-24 21:57:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a74h7212lp3pt-2-door-w3-pt-latches-and-floor-stands-type-4-7406x7206x1206>: HTTP status code is not handled or not allowed 2026-01-24 21:57:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/complete-electronic-packing-workbench-esd-safety-edge-72-x-36>: HTTP status code is not handled or not allowed 2026-01-24 21:57:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/17-sand-screen-disc-120-grit-10-per-case>: HTTP status code is not handled or not allowed 2026-01-24 21:57:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powr-flite-replacemnet-paper-bag-upright-vac-pf626-6-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:57:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-gpm-siphon-pump-with-onoff-flow-ctrl>: HTTP status code is not handled or not allowed 2026-01-24 21:57:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-housing-jsg-0719-hd-12001>: HTTP status code is not handled or not allowed 2026-01-24 21:57:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/31-oil-ratio-double-acting-pump-air-operated>: HTTP status code is not handled or not allowed 2026-01-24 21:57:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cam-rod-jtm-1050-b-103>: HTTP status code is not handled or not allowed 2026-01-24 21:57:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavyweight-plastic-soup-spoons-white-100-per-box>: HTTP status code is not handled or not allowed 2026-01-24 21:57:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/packing-jsg1046-07>: HTTP status code is not handled or not allowed 2026-01-24 21:57:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cutlery-keeper-mediumweight-plastic-tableware-180-pieces-pack>: HTTP status code is not handled or not allowed 2026-01-24 21:57:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/optiloc-3-sections-18>: HTTP status code is not handled or not allowed 2026-01-24 21:57:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18vlxt-174-lithium-ion-brushless-cordless-20-articulating-pole-hedge-trimmer-tool-only>: HTTP status code is not handled or not allowed 2026-01-24 21:57:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a24n16alp-medium-type-1-enclosure-2400x1600x662-steelgray>: HTTP status code is not handled or not allowed 2026-01-24 21:57:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-paper-towel-dispenser-garbage-can-unit>: HTTP status code is not handled or not allowed 2026-01-24 21:57:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carry-bag-cloth-black-red-cb1>: HTTP status code is not handled or not allowed 2026-01-24 21:57:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-steel-stack-on-desk-hutch-60w-black>: HTTP status code is not handled or not allowed 2026-01-24 21:57:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-steel-stack-on-desk-hutch-72w-charcoal>: HTTP status code is not handled or not allowed 2026-01-24 21:57:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-36-3-drawer-with-steel-top-workbench>: HTTP status code is not handled or not allowed 2026-01-24 21:57:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-x-30-4-drawer-with-maple-top-workbench>: HTTP status code is not handled or not allowed 2026-01-24 21:57:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/butter-dish-1-ozwhite>: HTTP status code is not handled or not allowed 2026-01-24 21:57:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-male-nptf-connector>: HTTP status code is not handled or not allowed 2026-01-24 21:57:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/memory-foam-backrest-13-1-4w-x-1-3-4d-x-14-1-4h-black>: HTTP status code is not handled or not allowed 2026-01-24 21:57:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dx-top-chest-4-drawers-41w-x-25d-x-22-14h-black-with-red-drawer-pulls>: HTTP status code is not handled or not allowed 2026-01-24 21:57:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c16c16-consolet-sloped-cover-1600x1600x1109-steelgray>: HTTP status code is not handled or not allowed 2026-01-24 21:57:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a24r2410hcr-hinged-cover-medium-type-3r-2400x2400x1000-galvanizedpaint>: HTTP status code is not handled or not allowed 2026-01-24 21:57:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coffee-portion-packs-1-1-2-oz-packs-100-columbian>: HTTP status code is not handled or not allowed 2026-01-24 21:57:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultrane-525-grip-proof-nitrile-3-4-coated-gloves-light-weight-1-pair-size-7-525417>: HTTP status code is not handled or not allowed 2026-01-24 21:57:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a24r2412nk-screw-cover-type-3r-2400x2400x1200>: HTTP status code is not handled or not allowed 2026-01-24 21:57:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a1614ch-continuous-hinge-wclamps-type-12-1600x1400x600-steelgray>: HTTP status code is not handled or not allowed 2026-01-24 21:57:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-enzymatic-chute-dumpster-wash-plus-orange-scent-5-gallon-pail>: HTTP status code is not handled or not allowed 2026-01-24 21:57:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waterhog-fashion-mat-charcoal-3-x-10>: HTTP status code is not handled or not allowed 2026-01-24 21:57:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nl339-chemzoil-neoprene-coated-gloves-14-l-heavy-weight-1-pair-size-10-339420>: HTTP status code is not handled or not allowed 2026-01-24 21:57:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brown-kra-clasp-envelopes-28-lb-12-x-15-1-2-100-box>: HTTP status code is not handled or not allowed 2026-01-24 21:57:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mapa-temp-dex-710-nitrile-palm-coated-thermal-gloves-w-dots-light-weight-1-pair-size-7>: HTTP status code is not handled or not allowed 2026-01-24 21:58:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-12-reducing-tee-x-1-drop>: HTTP status code is not handled or not allowed 2026-01-24 21:58:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a84xm7818ftclp-heavy-duty-fs-disc-encl-wo-panel-2-door-8412x7850x1812-paint>: HTTP status code is not handled or not allowed 2026-01-24 21:58:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fae-6100-refill-cpr-shield-and-nitrile-gloves-1-shield-2-pair-gloves>: HTTP status code is not handled or not allowed 2026-01-24 21:58:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wirebound-pocket-memo-book-3-x-5-narrow-rule-top-bound-50-sheets-book-3-pk>: HTTP status code is not handled or not allowed 2026-01-24 21:58:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-mechanical-turnstile-left-free-exit-stainless>: HTTP status code is not handled or not allowed 2026-01-24 21:58:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ingersoll-rand-308b-die-grinder-033-hp>: HTTP status code is not handled or not allowed 2026-01-24 21:58:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-manifold-12-npt-inlet-x-12-npt-outlet-5-ports>: HTTP status code is not handled or not allowed 2026-01-24 21:58:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-green-wire-shelving-starter-unit-5-tier-42w-x-21d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 21:58:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-5-tier-72w-x-18d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 21:58:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ccs8sjlg-set-up-joint-pendant-arm-fits-80x80mm-castirltgray>: HTTP status code is not handled or not allowed 2026-01-24 21:58:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speedi-boot-90-175-register-vent-boot-w-adj-hangers-for-hvac-duct-work-6-x-12-x-6>: HTTP status code is not handled or not allowed 2026-01-24 21:58:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fabric-reception-chair-black>: HTTP status code is not handled or not allowed 2026-01-24 21:58:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a16r126hcr-hinged-cover-medium-type-3r-1600x1200x600-galvanizedpaint>: HTTP status code is not handled or not allowed 2026-01-24 21:58:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 27771 pages (at 100 pages/min), scraped 13907 items (at 24 items/min) 2026-01-24 21:58:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-solid-galvanized-steel-shelving-starter-unit-5-tier-48w-x-18d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 21:58:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electronically-locking-left-exit-mirror>: HTTP status code is not handled or not allowed 2026-01-24 21:58:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3m-duct-tape-6969-olive-48-mm-x-54-8-m-10-7-mil-24-per-case>: HTTP status code is not handled or not allowed 2026-01-24 21:58:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-tubing-inline-hand-valve>: HTTP status code is not handled or not allowed 2026-01-24 21:58:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scotch-vinyl-electrical-color-coding-tape-35-white-3-4-x-66-100-per-case>: HTTP status code is not handled or not allowed 2026-01-24 21:58:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/file-basket-assembly-kit-for-lx5-laptop-carts>: HTTP status code is not handled or not allowed 2026-01-24 21:58:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scotch-r-box-sealing-tape-dispenser-h180-2-40-per-case>: HTTP status code is not handled or not allowed 2026-01-24 21:58:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-w-x-30-d-production-workbench-esd-laminate-safety-edge-drawer-upright-shelf-tan>: HTTP status code is not handled or not allowed 2026-01-24 21:58:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speedi-grille-hands-free-ceiling-or-wall-vent-grille-vent-register-cover-w-3-way-reflexctor-4-x-12>: HTTP status code is not handled or not allowed 2026-01-24 21:58:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-w-x-36-d-production-workbench-esd-laminate-square-edge-drawer-upright-shelf-tan>: HTTP status code is not handled or not allowed 2026-01-24 21:58:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-coupler-industrial-interchange-h-style-38-mnpt>: HTTP status code is not handled or not allowed 2026-01-24 21:58:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90-176-ss-corner-guard-16-ga-sq-corners-48-lx1-1-2-wx1-1-2-h-standard>: HTTP status code is not handled or not allowed 2026-01-24 21:58:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3m-duct-tape-3900-black-48-mm-x-54-8-m-7-7-mil-24-per-case>: HTTP status code is not handled or not allowed 2026-01-24 21:58:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-series-370-disc-horn-36volt-345hz>: HTTP status code is not handled or not allowed 2026-01-24 21:58:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/block-slide-25a-6295467>: HTTP status code is not handled or not allowed 2026-01-24 21:58:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plate-front-1632-6293408>: HTTP status code is not handled or not allowed 2026-01-24 21:58:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_alarms_signals/audible_alarms already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:58:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speedi-boot-torpedo-end-register-vent-boot-w-adj-hangers-for-hvac-duct-work-4-x-12-x-8>: HTTP status code is not handled or not allowed 2026-01-24 21:58:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/open-sided-wire-exchange-truck-five-wire-shelves-800-lb-cap-5>: HTTP status code is not handled or not allowed 2026-01-24 21:58:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-s1436b-black-epoxy-wire-shelf-36-w-x-14-d-with-clips>: HTTP status code is not handled or not allowed 2026-01-24 21:58:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washer-14-jbos-5-6286953>: HTTP status code is not handled or not allowed 2026-01-24 21:58:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/end-filter-cover-part-replacement-for-kerosene-heater-3221-0029>: HTTP status code is not handled or not allowed 2026-01-24 21:58:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/threaded-male-adapter-1-12-pipe-x-1-12-mnpt-nipple>: HTTP status code is not handled or not allowed 2026-01-24 21:58:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-green-wire-shelving-add-on-unit-5-tier-54w-x-18d-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 21:58:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modine-108000-btu-steam-or-hot-water-unit-heater-with-explosion-proof-motor>: HTTP status code is not handled or not allowed 2026-01-24 21:58:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/34x12x96-solar-star-greenhouse-w-poly-top-and-ends-drop-down-sides>: HTTP status code is not handled or not allowed 2026-01-24 21:58:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-green-wire-shelving-starter-unit-5-tier-48w-x-24d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 21:58:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/power_tools/power_saws_blades/band_saws/band_saw_machines already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:58:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_alarms_signals/audible_alarms>: HTTP status code is not handled or not allowed 2026-01-24 21:58:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delta-black-aluminum-single-lid-low-profile-crossover-fullsize>: HTTP status code is not handled or not allowed 2026-01-24 21:58:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-m6x10-jbos-5-6286487>: HTTP status code is not handled or not allowed 2026-01-24 21:58:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enclosed-bottom-computer-cart-30w-x-24d-x-40h-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:58:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/armorultra-job-on-pallet-2-layer-epoxy-system-1000-sq-ft-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:58:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-computer-cabinet-open-bottom-27w-x-24d-x-43h-gray>: HTTP status code is not handled or not allowed 2026-01-24 21:58:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-foam-silicone-roll-36-l-x-16-w-x-3-8-thick-red>: HTTP status code is not handled or not allowed 2026-01-24 21:58:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelate-silver-epoxy-wire-shelving-add-on-unit-5-tier-72w-x-18d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 21:58:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30x12x48-solar-star-greenhouse-w-solid-polycarbonate>: HTTP status code is not handled or not allowed 2026-01-24 21:58:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bollard-safety-net-4-foot-l-by-28-foot-w-fits-6-3-4-inches-to-7-5-8-inches-dia-brsn-428-8>: HTTP status code is not handled or not allowed 2026-01-24 21:58:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bollard-safety-net-4-foot-l-by-14-foot-w-fits-6-3-4-inches-to-7-5-8-inches-dia-brsn-414-8>: HTTP status code is not handled or not allowed 2026-01-24 21:58:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/power_tools/power_saws_blades/band_saws/band_saw_machines>: HTTP status code is not handled or not allowed 2026-01-24 21:58:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bracket-disc-guard-ds-6295603>: HTTP status code is not handled or not allowed 2026-01-24 21:58:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprayon-711-the-protector-lubricant-5-gallon>: HTTP status code is not handled or not allowed 2026-01-24 21:58:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cap-screw-60-a-jointer-6296031>: HTTP status code is not handled or not allowed 2026-01-24 21:58:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classical-a-v-belt-a48>: HTTP status code is not handled or not allowed 2026-01-24 21:58:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30x12x72-solar-star-greenhouse-w-poly-ends-and-drop-down-sides>: HTTP status code is not handled or not allowed 2026-01-24 21:58:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8090253215-iron-series-two-stage-vertical-air-compressor-10hp-120-gallon>: HTTP status code is not handled or not allowed 2026-01-24 21:58:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/65318-cable-lockout-with-6-l-cable-nylon-1-18w-x-6-12h>: HTTP status code is not handled or not allowed 2026-01-24 21:58:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classical-cogged-bx-v-belt-bx62>: HTTP status code is not handled or not allowed 2026-01-24 21:58:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/65375-steel-lockout-hasp-1-jaw-clearance-12pk-steel-1-34w-x-4-12h>: HTTP status code is not handled or not allowed 2026-01-24 21:58:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-foam-buna-n-pvc-sheet-36-l-x-36-w-x-3-16-thick-black-acrylic-adhesive>: HTTP status code is not handled or not allowed 2026-01-24 21:58:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-view-greenhouse-kit-30w-x-12h-x-60l-natural-gas>: HTTP status code is not handled or not allowed 2026-01-24 21:58:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/krylon-industrial-line-up-sb-pavement-striping-paint-cover-up-black>: HTTP status code is not handled or not allowed 2026-01-24 21:58:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/krylon-industrial-quik-mark-sb-inverted-marking-paint-fluor-neon-green>: HTTP status code is not handled or not allowed 2026-01-24 21:58:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classical-a-v-belt-a23>: HTTP status code is not handled or not allowed 2026-01-24 21:58:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/krylon-industrial-quik-mark-wb-inverted-marking-paint-fluor-safety-green>: HTTP status code is not handled or not allowed 2026-01-24 21:58:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jea-b21-knob-sprng-6285895>: HTTP status code is not handled or not allowed 2026-01-24 21:58:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ready-rack-mobile-gear-storage-rack-locker-single-sided-three-24-sections-chrome>: HTTP status code is not handled or not allowed 2026-01-24 21:58:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/52177-national-fire-protection-association-hazardous-materials-label-5w-x-5h>: HTTP status code is not handled or not allowed 2026-01-24 21:58:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/painting_supplies/spray_paint_primer already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:58:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classical-cogged-cx-v-belt-cx105>: HTTP status code is not handled or not allowed 2026-01-24 21:58:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/krylon-5-ball-interior-exterior-paint-semi-gloss-white>: HTTP status code is not handled or not allowed 2026-01-24 21:58:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/34x12x72-solar-star-greenhouse-w-poly-ends-and-drop-down-sides-prop-heater>: HTTP status code is not handled or not allowed 2026-01-24 21:58:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprng-model-201-plnr-6012291>: HTTP status code is not handled or not allowed 2026-01-24 21:58:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nelson-wrap-dispenser>: HTTP status code is not handled or not allowed 2026-01-24 21:58:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labels-with-do-not-x-ray-print-5l-x-3w-fluorescent-red-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-24 21:58:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trash-can-liner-17-gallon-261530-261530>: HTTP status code is not handled or not allowed 2026-01-24 21:58:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelving-starter-unit-5-tier-36w-x-24d-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 21:58:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/65671-danger-do-not-operate-tag-bilingual-englighspanish-hd-polyester-25pack>: HTTP status code is not handled or not allowed 2026-01-24 21:58:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cover-20-planer-mdl-208-6292787>: HTTP status code is not handled or not allowed 2026-01-24 21:59:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/painting_supplies/spray_paint_primer>: HTTP status code is not handled or not allowed 2026-01-24 21:59:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-foam-buna-n-pvc-sheet-24-l-x-12-w-x-1-thick-black>: HTTP status code is not handled or not allowed 2026-01-24 21:59:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/250-1-16-dia-vc-3-32-dia-7x7-vinyl-coated-galvanized-aircraft-cable>: HTTP status code is not handled or not allowed 2026-01-24 21:59:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/65452-lockout-tag-danger-do-not-operate-2-sided-cardstock-25pack>: HTTP status code is not handled or not allowed 2026-01-24 21:59:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cast-acrylic-sheet-96-l-x-48-w-x-1-8-thick-green-ps-cact-91>: HTTP status code is not handled or not allowed 2026-01-24 21:59:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-x-8-x-16-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 21:59:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_lockout_identification_tags already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:59:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grade-30-proof-coil-chain-3-8-dia-2650-lb-cap-66-pail-bright-zinc>: HTTP status code is not handled or not allowed 2026-01-24 21:59:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cast-acrylic-sheet-12-l-x-12-w-x-1-8-thick-red-ps-cact-65>: HTTP status code is not handled or not allowed 2026-01-24 21:59:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:59:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/99289-lockout-procedure-station-with-binder-polystyrene-14w-x-20h>: HTTP status code is not handled or not allowed 2026-01-24 21:59:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-view-greenhouse-kit-20w-x-107h-x-48l-propane>: HTTP status code is not handled or not allowed 2026-01-24 21:59:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:59:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/skch-bkt-hi-vis-bucket-spill-kitchemical6-12-gallon-bucket>: HTTP status code is not handled or not allowed 2026-01-24 21:59:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/500-5-16-dia-vc-3-8-dia-7x19-vinyl-coated-galvanized-aircraft-cable>: HTTP status code is not handled or not allowed 2026-01-24 21:59:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 27865 pages (at 94 pages/min), scraped 13923 items (at 16 items/min) 2026-01-24 21:59:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cast-acrylic-sheet-48-l-x-12-w-x-3-16-thick-red>: HTTP status code is not handled or not allowed 2026-01-24 21:59:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelving-add-on-unit-5-tier-54w-x-21d-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 21:59:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/hearing_protection/ear_plug_dispensers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:59:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gear-box-20-planer-mdl-208-6292759>: HTTP status code is not handled or not allowed 2026-01-24 21:59:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vital-cluster-box-unit-8-mailboxes-4-parcel-lockers-sandstone>: HTTP status code is not handled or not allowed 2026-01-24 21:59:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_lockout_identification_tags>: HTTP status code is not handled or not allowed 2026-01-24 21:59:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cast-acrylic-bar-24-l-x-4-w-x-1-8-thick-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:59:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-24 21:59:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-x-6-x-4-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 21:59:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-24 21:59:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labels-w-stop-this-package-is-print-8l-x-3w-redwhiteblack-roll-of-250>: HTTP status code is not handled or not allowed 2026-01-24 21:59:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/103748-handicap-reserved-parking-sign-ada-compliant-aluminum-12w-x-18h>: HTTP status code is not handled or not allowed 2026-01-24 21:59:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700256yel-pegboard-floor-stand-w5-c-channel-sliding-16w-x-60h-yl-1-pc>: HTTP status code is not handled or not allowed 2026-01-24 21:59:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/hearing_protection/ear_plug_dispensers>: HTTP status code is not handled or not allowed 2026-01-24 21:59:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/switch-cord-model-201planer-6012109>: HTTP status code is not handled or not allowed 2026-01-24 21:59:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-x-14-x-18-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 21:59:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/104311-toughstripe-floor-marking-tape-polyester-2w-x-100l-white>: HTTP status code is not handled or not allowed 2026-01-24 21:59:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/op15p-op-oil-only-roll15w-x-150lheavy-weight>: HTTP status code is not handled or not allowed 2026-01-24 21:59:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelving-starter-unit-5-tier-60w-x-24d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 21:59:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/liquid-boiler-leak-sealant-1-qt>: HTTP status code is not handled or not allowed 2026-01-24 21:59:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/foam/foam_tubes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:59:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700357clr-pegboard-powerwing-display-16w-x-2025h-clr-1-pc>: HTTP status code is not handled or not allowed 2026-01-24 21:59:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/panel-face-envelopes-packing-list-enclosed-print-4-12l-x-5-12w-orange-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 21:59:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:59:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-x-16-x-19-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 21:59:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-silicone-strip-36-l-x-7-w-x-3-16-thick-blue-detectable-60a-bulk-rs-s60md-64>: HTTP status code is not handled or not allowed 2026-01-24 21:59:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blade-adj-pja-c53c-15-planer-6284832>: HTTP status code is not handled or not allowed 2026-01-24 21:59:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-z-brite-wire-shelving-starter-unit-1-dunnage-shelf-48w-x-18d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 21:59:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_lockout_identification_tags already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 21:59:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/146127-cable-safety-padlock-with-label-8h-clearance-steel-cable-green>: HTTP status code is not handled or not allowed 2026-01-24 21:59:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-plastex-wheel-8x2-12-roller>: HTTP status code is not handled or not allowed 2026-01-24 21:59:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/146121-cable-safety-padlock-with-label-4-316h-clearance-steel-cable-yellow>: HTTP status code is not handled or not allowed 2026-01-24 21:59:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-abs-tube-72-l-x-1-8-id-x-1-4-od-beige>: HTTP status code is not handled or not allowed 2026-01-24 21:59:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/222484-16-compartment-cosmetic-display-12w-x-145h-x-85d-clr-1-pc>: HTTP status code is not handled or not allowed 2026-01-24 21:59:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/foam/foam_tubes>: HTTP status code is not handled or not allowed 2026-01-24 21:59:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700258yel-pegboard-floor-stand-w5-c-channel-sliding-16w-x-66h-yl-1-pc>: HTTP status code is not handled or not allowed 2026-01-24 21:59:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/funnel-king-extra-heavy-duty-144-oz-funnel>: HTTP status code is not handled or not allowed 2026-01-24 21:59:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents>: HTTP status code is not handled or not allowed 2026-01-24 21:59:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-mort-wheel-12x2-1-roller>: HTTP status code is not handled or not allowed 2026-01-24 21:59:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-drainwaterfill-cap-parpacinjs006-for-all-units>: HTTP status code is not handled or not allowed 2026-01-24 21:59:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-fan-assembly-parfanj270f0-for-jetstream-270>: HTTP status code is not handled or not allowed 2026-01-24 21:59:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mdl-pf31-33-41-43locking-handle-6288887>: HTTP status code is not handled or not allowed 2026-01-24 21:59:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-dia-round-paper-labels-with-wed-print-fluorescent-orange-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-24 21:59:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/easy-mount-adapter-w-6-hot-wire-pigtail>: HTTP status code is not handled or not allowed 2026-01-24 21:59:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-x-26-x-26-doublewall-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 21:59:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/terminal-plate-model-201-planer-6012272>: HTTP status code is not handled or not allowed 2026-01-24 21:59:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wesco-hd-appliance-truck-1800-lb-cap-78in-h-auto-rewin-d-ratchet>: HTTP status code is not handled or not allowed 2026-01-24 21:59:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700444blu-pinwheel-pegboard-display-wrb-135w-x-44h-bl-1-pc>: HTTP status code is not handled or not allowed 2026-01-24 21:59:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnetic-mount-base-w-handle>: HTTP status code is not handled or not allowed 2026-01-24 21:59:44 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_lockout_identification_tags landed on page that is not a product page. 2026-01-24 21:59:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ball-bearing-6005-zz-model-201planer-6012105>: HTTP status code is not handled or not allowed 2026-01-24 21:59:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-silicone-strip-36-l-x-5-w-x-1-16-thick-blue-detectable-60a-high-temp-adhesive-bulk-rs-s60md-72>: HTTP status code is not handled or not allowed 2026-01-24 21:59:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700430yel-hshaped-pegboard-display-16w-x-60h-yl-1-pc>: HTTP status code is not handled or not allowed 2026-01-24 21:59:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cooler-backroom-shelving-e-channel-20-inch-60-inch-3-shelves>: HTTP status code is not handled or not allowed 2026-01-24 21:59:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cooler-backroom-shelving-t-bar-style-24-inch-60-inch-3-shelves>: HTTP status code is not handled or not allowed 2026-01-24 21:59:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labels-with-hot-print-2l-x-2w-redwhite-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-24 21:59:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/screw-flat-hd-phillips-632x38-16-32-10-4009-06>: HTTP status code is not handled or not allowed 2026-01-24 21:59:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/duraline-economy-tubular-dunnage-rack-20-inch-w-48-inch-l-8-inch-h>: HTTP status code is not handled or not allowed 2026-01-24 21:59:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/31a-3-seal-oil-32x17x7t-6288969>: HTTP status code is not handled or not allowed 2026-01-24 21:59:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-pvc-90-degree-angle-12-l-x-1-1-2-w-gray-bulk-pa-pvc-56>: HTTP status code is not handled or not allowed 2026-01-24 21:59:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/-light-duty-non-lighted-warning-whip-blue>: HTTP status code is not handled or not allowed 2026-01-24 21:59:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-polypropylene-liner-12x60>: HTTP status code is not handled or not allowed 2026-01-24 21:59:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-foam-epdm-strip-120-l-x-1-w-x-1-thick-black-acrylic-adhesive>: HTTP status code is not handled or not allowed 2026-01-24 21:59:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/160-172-knife-lifting-3438202>: HTTP status code is not handled or not allowed 2026-01-24 21:59:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-amp-12-inch-electric-double-bevel-compound-miter-saw>: HTTP status code is not handled or not allowed 2026-01-24 21:59:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700422gre-pinwheel-pegboard-display-wwheel-rb-16w-x-60h-gn-1-pc>: HTTP status code is not handled or not allowed 2026-01-24 21:59:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-dia-round-paper-labels-with-aug-print-fluorescent-pink-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-24 21:59:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bushing-101380-27spr27-wd-shaper-6095025>: HTTP status code is not handled or not allowed 2026-01-24 21:59:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crowd-control-stanchion-40-in-black-post-blue-belt-10-ft-l>: HTTP status code is not handled or not allowed 2026-01-24 21:59:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/top-braced-wall-mounted-jib-crane-900031-4000-lb-cap-20-span-w-200-deg-rotation>: HTTP status code is not handled or not allowed 2026-01-24 22:00:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700432wht-hshaped-pegboard-display-32w-x-60h-wh-1-pc>: HTTP status code is not handled or not allowed 2026-01-24 22:00:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paper-labels-with-aug-print-3l-x-2w-l-fluorescent-pink-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-24 22:00:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guard-dog-low-profile-5-ch-orange-lidblack-base-ada-ramps>: HTTP status code is not handled or not allowed 2026-01-24 22:00:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/belt-4l320-6077008>: HTTP status code is not handled or not allowed 2026-01-24 22:00:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/670-dual-head-stethoscope-32-25-length-black>: HTTP status code is not handled or not allowed 2026-01-24 22:00:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-dia-round-number-paper-labels-with-7-print-dark-green-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-24 22:00:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honeywell-linear-imaging-scanner-voyager-1400g-with-usb-cable-and-stand>: HTTP status code is not handled or not allowed 2026-01-24 22:00:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honeywell-lithium-ion-battery-for-use-with-1202g-1902-3820-3820i-4820-and-4820i>: HTTP status code is not handled or not allowed 2026-01-24 22:00:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-socket-3-4-drive-6-point-1-3-4-short>: HTTP status code is not handled or not allowed 2026-01-24 22:00:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magliner-174-retractable-underbody-truck-slider-174-ramp-track-24-wx12-l-sr2412>: HTTP status code is not handled or not allowed 2026-01-24 22:00:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wilton-174-20412-bash-4-lb-sledge-hammer-12-handle>: HTTP status code is not handled or not allowed 2026-01-24 22:00:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-yoga-mat-storage-cart-black>: HTTP status code is not handled or not allowed 2026-01-24 22:00:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-x-4-x-3-mailers>: HTTP status code is not handled or not allowed 2026-01-24 22:00:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 27960 pages (at 95 pages/min), scraped 13939 items (at 16 items/min) 2026-01-24 22:00:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shaft-back-up-bearing-lower-3702002>: HTTP status code is not handled or not allowed 2026-01-24 22:00:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rectangular-inventory-paper-labels-3l-x-2w-dark-blue-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-24 22:00:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/work-height-platform-truck-rnh-2448-6py-with-hardboard-top-24-x-48-fixed-height>: HTTP status code is not handled or not allowed 2026-01-24 22:00:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-socket-3-8-drive-6-point-1-deep>: HTTP status code is not handled or not allowed 2026-01-24 22:00:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fiona-salon-nail-table-white>: HTTP status code is not handled or not allowed 2026-01-24 22:00:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-foam-epdm-roll-120-l-x-12-w-x-3-8-thick-black>: HTTP status code is not handled or not allowed 2026-01-24 22:00:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/work-height-platform-truck-rnb2-2460-6pyah-with-lower-shelf-24-x-60-adj-height>: HTTP status code is not handled or not allowed 2026-01-24 22:00:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/work-height-platform-truck-rnb2-2448-6py-with-lower-shelf-24-x-48-fixed-height>: HTTP status code is not handled or not allowed 2026-01-24 22:00:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-outdoor-in-ground-steel-flat-bench-perforated-metal-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:00:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-dolly-pds-4848-6ph-lr-48-x-48-solid-deck-and-load-retainers>: HTTP status code is not handled or not allowed 2026-01-24 22:00:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rectangular-inventory-paper-labels-7l-x-5w-red-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-24 22:00:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-24-l-x-1-1-2-w-x-3-16-thick-black-antistatic>: HTTP status code is not handled or not allowed 2026-01-24 22:00:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-x-6-1-4-x-3-deluxe-literature-mailers>: HTTP status code is not handled or not allowed 2026-01-24 22:00:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-spill-plus-containment-pallet-p2-2-drum-with-drain>: HTTP status code is not handled or not allowed 2026-01-24 22:00:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/impact-socket-1-2-drive-1-1-8-6-point-short>: HTTP status code is not handled or not allowed 2026-01-24 22:00:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guard-mounted-direct-drive-exhaust-fan-24in>: HTTP status code is not handled or not allowed 2026-01-24 22:00:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-12-1-2x19-self-seal-kra-padded-mailers>: HTTP status code is not handled or not allowed 2026-01-24 22:00:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/complete-in-corner-ada-approved-compartment-60in-w-x61-1-4in-d>: HTTP status code is not handled or not allowed 2026-01-24 22:00:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inter-department-envelopes-10w-x-13l-blue-100pack>: HTTP status code is not handled or not allowed 2026-01-24 22:00:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/grill_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:00:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stud-remover-5-16-1-2-drive>: HTTP status code is not handled or not allowed 2026-01-24 22:00:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-non-electric-office-partition-panel-with-raceway-36-1-4w-x-101-1-2h-blue-gl>: HTTP status code is not handled or not allowed 2026-01-24 22:00:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-36-l-x-3-w-x-3-16-thick-black-antistatic>: HTTP status code is not handled or not allowed 2026-01-24 22:00:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nba-houston-rockets-grill-mat-14205>: HTTP status code is not handled or not allowed 2026-01-24 22:00:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labels-with-do-not-bend-print-3l-x-2w-fluorescent-red-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-24 22:00:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-24-l-x-2-w-x-3-16-thick-black-antistatic>: HTTP status code is not handled or not allowed 2026-01-24 22:00:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/econoline-storage-building-26w-x-12h-x-30l-white-freestanding>: HTTP status code is not handled or not allowed 2026-01-24 22:00:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-socket-1-2-drive-6-point-1-2-short>: HTTP status code is not handled or not allowed 2026-01-24 22:00:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-electric-office-partition-panel-with-partial-window-36-1-4w-x-101-1-2h-gray-gl>: HTTP status code is not handled or not allowed 2026-01-24 22:00:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stud-remover-12-mm-1-2-drive>: HTTP status code is not handled or not allowed 2026-01-24 22:00:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leveling-foot-threaded-2700-lbs-capacity-1-57-base-dia-41-40-3-8x16-75-d0-sk>: HTTP status code is not handled or not allowed 2026-01-24 22:00:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-foam-epdm-roll-120-l-x-12-w-x-1-4-thick-black-acrylic-adhesive>: HTTP status code is not handled or not allowed 2026-01-24 22:00:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/floor_mats/grill_mats>: HTTP status code is not handled or not allowed 2026-01-24 22:00:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labels-w-fragile-liquid-handle-with-print-3l-x-2w-redwhite-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-24 22:00:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bi12792-loaded-gear-vx400-tactical-chest-rig-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:00:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labels-w-fragile-liquid-handle-with-print-5l-x-3w-redwhite-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-24 22:00:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fmlrl-14-20840-m4-led-low-profile-round-120v-1600-lumens>: HTTP status code is not handled or not allowed 2026-01-24 22:00:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/circle-mailing-labels-1-12-dia-clear-roll-of-1000>: HTTP status code is not handled or not allowed 2026-01-24 22:00:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-12-l-x-1-2-w-x-1-2-thick-blue-glass-filled>: HTTP status code is not handled or not allowed 2026-01-24 22:00:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-60-l-x-1-w-x-1-8-thick-white>: HTTP status code is not handled or not allowed 2026-01-24 22:00:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/13-x-18-white-stayflat-mailers>: HTTP status code is not handled or not allowed 2026-01-24 22:00:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mgsh904vs-safety-first-sign-report-all-accidents-14w-x-10h-adhesive-vinyl>: HTTP status code is not handled or not allowed 2026-01-24 22:00:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/daddy-long-legs-gable-end-14w-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:00:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labels-with-tested-print-2l-x-1-14w-fluorescent-green-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-24 22:00:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-x-11-80-gauge-pvc-shrink-bag>: HTTP status code is not handled or not allowed 2026-01-24 22:00:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/combination-wrench-7-16>: HTTP status code is not handled or not allowed 2026-01-24 22:00:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-30w-solid-end-panel-green>: HTTP status code is not handled or not allowed 2026-01-24 22:00:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hurricane-mini-360-hand-chain-hoist-1-4t-20-lift>: HTTP status code is not handled or not allowed 2026-01-24 22:00:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/recycled-rubber-tackboard-aluminum-trim-36w-x-24h-black>: HTTP status code is not handled or not allowed 2026-01-24 22:00:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/recycled-rubber-tackboard-aluminum-trim-72-1-2w-x-48-1-2h-black>: HTTP status code is not handled or not allowed 2026-01-24 22:00:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-12-l-x-2-w-x-3-16-thick-black-antistatic>: HTTP status code is not handled or not allowed 2026-01-24 22:00:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machine-table-2-shelves-50-3-8-w-x-26-3-8-d-x-30-h>: HTTP status code is not handled or not allowed 2026-01-24 22:00:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-bit-7-32>: HTTP status code is not handled or not allowed 2026-01-24 22:00:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kss144-lockout-procedure-station-additional-forms-lockout-safety-procedures-paper>: HTTP status code is not handled or not allowed 2026-01-24 22:00:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-24-l-x-4-w-x-3-4-thick-black-antistatic>: HTTP status code is not handled or not allowed 2026-01-24 22:00:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-tilt-truck-36x72-solid-wood-moldon-rubber-wheels-3000>: HTTP status code is not handled or not allowed 2026-01-24 22:00:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-socket-1-2-drive-6-point-20mm-short>: HTTP status code is not handled or not allowed 2026-01-24 22:00:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slide-stop-reversible-non-slip-cushion-rug-pad-for-8-ft-x-10-ft-area-rug-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:00:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/floor-stand-for-stak-a-cab-cabinet-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:00:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-sheet-96-l-x-48-w-x-1-4-thick-black-antistatic>: HTTP status code is not handled or not allowed 2026-01-24 22:00:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mcpg101vs-danger-sign-compressed-gas-10w-x-7h-adhesive-vinyl>: HTTP status code is not handled or not allowed 2026-01-24 22:00:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:00:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oceans-soap-and-hand-sanitizer-dispenser-s890tbk>: HTTP status code is not handled or not allowed 2026-01-24 22:00:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/impact-socket-3-8-drive-10mm-6-point-short>: HTTP status code is not handled or not allowed 2026-01-24 22:00:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-purpose-lubricant-and-corrosion-inhibitor-5-gallon-pail-petroleum>: HTTP status code is not handled or not allowed 2026-01-24 22:00:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22x25-liter-under-the-counter-corrosive-cabinet-fully-lined-left-hinge-23w-x-22d-x-35-1-2h>: HTTP status code is not handled or not allowed 2026-01-24 22:01:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30x48-4-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-24 22:01:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-impact-flex-socket-set-3-8-drive-6-point-short-10-piece>: HTTP status code is not handled or not allowed 2026-01-24 22:01:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-post-63in-h-leveler-installed-pack-of-four>: HTTP status code is not handled or not allowed 2026-01-24 22:01:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-5-8-tube-od-zinc-plated-anti-vibration-cush-a-clamp>: HTTP status code is not handled or not allowed 2026-01-24 22:01:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-foam-epdm-roll-480-l-x-25-w-x-1-thick-black>: HTTP status code is not handled or not allowed 2026-01-24 22:01:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-24-l-x-1-2-w-x-3-8-thick-blue-glass-filled>: HTTP status code is not handled or not allowed 2026-01-24 22:01:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 22:01:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-60-l-x-3-4-w-x-3-8-thick-white>: HTTP status code is not handled or not allowed 2026-01-24 22:01:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-inches-overlap-push-type-seal-for-polyester-5000-per-case>: HTTP status code is not handled or not allowed 2026-01-24 22:01:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-8-t-clamp-assembly-for-high-pressure-hoses-pipe-or-tube>: HTTP status code is not handled or not allowed 2026-01-24 22:01:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-inch-wing-seal-for-stainless-steel-100-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:01:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 28058 pages (at 98 pages/min), scraped 13962 items (at 23 items/min) 2026-01-24 22:01:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reversible-snap-ring-plier-kit-5-piece>: HTTP status code is not handled or not allowed 2026-01-24 22:01:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16x30-4-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-24 22:01:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/madm499vp-authorized-personnel-only-sign-14w-x-10h-plastic>: HTTP status code is not handled or not allowed 2026-01-24 22:01:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-sheet-12-l-x-12-w-x-1-4-thick-clear-scratch-and-uv-resistant>: HTTP status code is not handled or not allowed 2026-01-24 22:01:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fireflex-silicone-coated-fiberglass-sleeve-88-dia-25-red>: HTTP status code is not handled or not allowed 2026-01-24 22:01:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powderfree-textured-exam-grade-nitrile-gloves-black-xl-100box-10-boxcs>: HTTP status code is not handled or not allowed 2026-01-24 22:01:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mt3-heavy-duty-live-centres-880lb-capacity-60-centre-angle-1-1-8-centre-diameter>: HTTP status code is not handled or not allowed 2026-01-24 22:01:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precision-balance-200-g-x-0-01-goph-t202>: HTTP status code is not handled or not allowed 2026-01-24 22:01:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-vertical-railing-base-2>: HTTP status code is not handled or not allowed 2026-01-24 22:01:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-bar-24-l-x-3-4-w-x-3-32-thick-clear>: HTTP status code is not handled or not allowed 2026-01-24 22:01:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insultherm-resin-coated-fiberglass-sleeve-50-dia-50-black>: HTTP status code is not handled or not allowed 2026-01-24 22:01:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/z-line-boltless-shelving-w-steel-industrial-shelves-48-w-x-12-d-x-84-h-add-on-5-shelves>: HTTP status code is not handled or not allowed 2026-01-24 22:01:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-tension-lcd-hanging-scale-50000-lb-x-20-lb>: HTTP status code is not handled or not allowed 2026-01-24 22:01:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3x4-1-5-mil-flat-polypropylene-poly-bags>: HTTP status code is not handled or not allowed 2026-01-24 22:01:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_cabinets_pedestals already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:01:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pm2000t-10-table-saw-with-armorglide--extension-table-50-rip-5hp-1ph-230v>: HTTP status code is not handled or not allowed 2026-01-24 22:01:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polyester-film-300-l-x-36-w-x-0-004-thick-clear>: HTTP status code is not handled or not allowed 2026-01-24 22:01:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cpvc-bar-48-l-x-1-1-2-w-x-1-thick-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:01:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30400-spray-stencil-ink-11-oz-white>: HTTP status code is not handled or not allowed 2026-01-24 22:01:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/msmk407vp-no-smoking-graphic-sign-7w-x-10h-plastic>: HTTP status code is not handled or not allowed 2026-01-24 22:01:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compact-portable-security-box-safe-combo-lock-5-15-16w-x-8d-x-2-5-8h-black>: HTTP status code is not handled or not allowed 2026-01-24 22:01:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1285t-12-parallelogram-jointer-w-armorglide--helical-head-cutter-3hp-3ph-460v>: HTTP status code is not handled or not allowed 2026-01-24 22:01:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-single-socket-tee-1>: HTTP status code is not handled or not allowed 2026-01-24 22:01:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-straight-coupling-3>: HTTP status code is not handled or not allowed 2026-01-24 22:01:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mgsh904vp-safety-first-sign-report-all-accidents-14w-x-10h-plastic>: HTTP status code is not handled or not allowed 2026-01-24 22:01:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mchl170vs-danger-sign-oxygen-14w-x-10h-adhesive-vinyl>: HTTP status code is not handled or not allowed 2026-01-24 22:01:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-plug>: HTTP status code is not handled or not allowed 2026-01-24 22:01:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tss103ctp-ok-this-scaffold-erected-to-meet-tag-pf-cardstock-25pack>: HTTP status code is not handled or not allowed 2026-01-24 22:01:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_cabinets_pedestals>: HTTP status code is not handled or not allowed 2026-01-24 22:01:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mnt101ctp-safety-tag-notice-pf-cardstock-25pack>: HTTP status code is not handled or not allowed 2026-01-24 22:01:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-freestanding-2-panel-corner-room-divider-with-partial-window-48-x-73-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:01:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cpvc-bar-12-l-x-1-w-x-1-thick-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:01:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4flexo-pet-general-purpose-sleeving-dia-75-orange>: HTTP status code is not handled or not allowed 2026-01-24 22:01:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10x14-1-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-24 22:01:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10x16-1-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-24 22:01:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-airpot-racks-univ-2-apr-357280001>: HTTP status code is not handled or not allowed 2026-01-24 22:01:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/madc800va-admittance-security-sign-notice-10w-x-7h-aluminum>: HTTP status code is not handled or not allowed 2026-01-24 22:01:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-oz-glass-jars-straight-side-clear-poly-vinyl-liner-case-of-12>: HTTP status code is not handled or not allowed 2026-01-24 22:01:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26x30-1-5-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-24 22:01:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/liquid-coffee-ambient-dispenser-lca-2-344000001>: HTTP status code is not handled or not allowed 2026-01-24 22:01:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-oz-amber-wide-mouth-packer-bottles-with-cone-lined-caps-case-of-24>: HTTP status code is not handled or not allowed 2026-01-24 22:01:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sentec-concentrate-odor-eliminator-pure-linen-scent-2-l-bottle-2case>: HTTP status code is not handled or not allowed 2026-01-24 22:01:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paper-twist-ties-8-x-532-white-2000-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:01:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mfs1717-exit-w-arrow-floor-sign17-diameter-adhesive-vinyl-1each>: HTTP status code is not handled or not allowed 2026-01-24 22:01:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oroad-off-road-traction-drive-cart-1000-lb-capacity-13-rubber-wheels>: HTTP status code is not handled or not allowed 2026-01-24 22:01:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/upsable-perforated-bubble-wrap-roll-12-x-300-x-316-4-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:01:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/journeyman-2x-2-passengers-personnel-carrier-800-lbs-capacity-white>: HTTP status code is not handled or not allowed 2026-01-24 22:01:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cardboard-corrugated-boxes-24-x-15-x-12-200lb-testect32-20-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:01:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/legacy-desk-shell-42w-x-24d-x-29h-noble-oak>: HTTP status code is not handled or not allowed 2026-01-24 22:01:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:01:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20mm-crimp-seal-open-top-hole-caps-aluminum-green-unlined-case-of-1000>: HTTP status code is not handled or not allowed 2026-01-24 22:01:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/orthopedic-foam-bed-wedge-pillow-7-x-24-x-24-blue-802-8026-0100>: HTTP status code is not handled or not allowed 2026-01-24 22:01:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-bar-48-l-x-1-w-x-1-4-thick-clear-scratch-and-uv-resistant>: HTTP status code is not handled or not allowed 2026-01-24 22:01:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/legacy-executive-desk-with-single-pedestal-drawer-unit-66w-x-30d-x-29h-noble-oak>: HTTP status code is not handled or not allowed 2026-01-24 22:01:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-stretch-film-with-handles-1000l-x-10w-x-80-gauge-4-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:01:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/omnimed-wall-storage-cabinet-ambi-top-2-adjustable-shelves-16w-x-8d-x-26-3-4h-grey>: HTTP status code is not handled or not allowed 2026-01-24 22:01:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/omnimed-stainless-narcotic-cabinet-double-door-ambi-top-4-adj-shelves-16w-x-8d-x-24h>: HTTP status code is not handled or not allowed 2026-01-24 22:01:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wide-mouth-erlenmeyer-flasks-250ml-case-of-48>: HTTP status code is not handled or not allowed 2026-01-24 22:01:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/type-sb-allergen-bags-10-pack-261533>: HTTP status code is not handled or not allowed 2026-01-24 22:01:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-bar-48-l-x-4-w-x-3-32-thick-clear>: HTTP status code is not handled or not allowed 2026-01-24 22:01:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/group-lock-box-with-key-and-side-window-red-s601>: HTTP status code is not handled or not allowed 2026-01-24 22:01:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cpvc-bar-12-l-x-1-w-x-1-16-thick-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:01:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/Sunex-20-ton-shop-press>: HTTP status code is not handled or not allowed 2026-01-24 22:01:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 22:01:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5550102-flat-file-10-second-cut>: HTTP status code is not handled or not allowed 2026-01-24 22:01:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-bench-floor-scale-660-lb-x-0-25-lb>: HTTP status code is not handled or not allowed 2026-01-24 22:01:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:01:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cw-stationary-premium-electric-pressure-washer-3000-psi-8-hp-39-gpm-3-phase-230v>: HTTP status code is not handled or not allowed 2026-01-24 22:01:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caw-stationary-electric-pressure-washer-4000-psi-10-hp-4-gpm-1-phase-230v>: HTTP status code is not handled or not allowed 2026-01-24 22:01:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5550121-flat-file-12-bastard-cut>: HTTP status code is not handled or not allowed 2026-01-24 22:01:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cpvc-bar-12-l-x-1-w-x-1-8-thick-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:02:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-nylon-6-6-bar-24-l-x-3-8-w-x-3-8-thick-black>: HTTP status code is not handled or not allowed 2026-01-24 22:02:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jlh-series-lever-hoist-w-overload-protection-9-ton-15-ft-lift>: HTTP status code is not handled or not allowed 2026-01-24 22:02:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speakman-se-490-traditional-series-eyeface-wash>: HTTP status code is not handled or not allowed 2026-01-24 22:02:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guardair-14-npt-us-industrial-interchange-male>: HTTP status code is not handled or not allowed 2026-01-24 22:02:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cpvc-bar-12-l-x-3-w-x-1-4-thick-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:02:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11-infrared-pocket-thermometer-33-to-220c-27-to-428f-individual-calibration-report>: HTTP status code is not handled or not allowed 2026-01-24 22:02:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-handle-m10-x-1-5-92mm-l-x-64mm-h-die-cast-handle-mh-208>: HTTP status code is not handled or not allowed 2026-01-24 22:02:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 22:02:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slide-check-racks-ck6548a>: HTTP status code is not handled or not allowed 2026-01-24 22:02:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 28151 pages (at 93 pages/min), scraped 13978 items (at 16 items/min) 2026-01-24 22:02:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bridge-sandstone-modular-furniture-1>: HTTP status code is not handled or not allowed 2026-01-24 22:02:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/19-31-degree-baume-shatterproof-plastic-hydrometer>: HTTP status code is not handled or not allowed 2026-01-24 22:02:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plus-astm-like-liquid-in-glass-thermometer-14c-wax-meltingpoint-79mm-immersion>: HTTP status code is not handled or not allowed 2026-01-24 22:02:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/non-perforated-air-foam-rolls-6w-x-250l-x-14-thick-white-12-rolls>: HTTP status code is not handled or not allowed 2026-01-24 22:02:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-nylon-6-6-bar-48-l-x-4-w-x-1-4-thick-black-mds-filled>: HTTP status code is not handled or not allowed 2026-01-24 22:02:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guardair-ultra-xtra-thrust-6-alum-ext-nozzle-special-14-inlet>: HTTP status code is not handled or not allowed 2026-01-24 22:02:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/non-perforated-air-foam-rolls-18w-x-1250l-x-116-thick-white-4-rolls>: HTTP status code is not handled or not allowed 2026-01-24 22:02:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/freestanding-2-panel-corner-room-divider-with-partial-window-60-x-72h-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:02:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:02:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-nylon-6-6-bar-24-l-x-1-w-x-1-2-thick-black-mds-filled>: HTTP status code is not handled or not allowed 2026-01-24 22:02:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pn-30011-privacy-netting-wreinforced-hem-92-x-150-black>: HTTP status code is not handled or not allowed 2026-01-24 22:02:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-x-15-kra-mailing-tubes-with-caps-case>: HTTP status code is not handled or not allowed 2026-01-24 22:02:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0950-1000-specific-gravity-astm-88h-hydrometer-for-light-liquids>: HTTP status code is not handled or not allowed 2026-01-24 22:02:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guardair-36-80lj-alum-extension-alum-nozzle>: HTTP status code is not handled or not allowed 2026-01-24 22:02:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/075lb-weighted-beaker-flask-holder-with-vikem-vinyl-coating-for-250ml-beakers>: HTTP status code is not handled or not allowed 2026-01-24 22:02:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/av-presentation-station-24x18x41>: HTTP status code is not handled or not allowed 2026-01-24 22:02:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-ultimate-tuffy-presentation-station>: HTTP status code is not handled or not allowed 2026-01-24 22:02:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-air-foam-roll-72w-x-2000l-x-132-thick-white-1-roll>: HTTP status code is not handled or not allowed 2026-01-24 22:02:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-ada-partition-door-35-5-8in-w-x-58in-h-almond>: HTTP status code is not handled or not allowed 2026-01-24 22:02:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1200-1425-specific-gravity-and-24-41-degree-baume-dual-scale-hydrometer>: HTTP status code is not handled or not allowed 2026-01-24 22:02:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vertical-sheet-rack-5-bay>: HTTP status code is not handled or not allowed 2026-01-24 22:02:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers>: HTTP status code is not handled or not allowed 2026-01-24 22:02:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-pilaster-with-shoe-4in-w-x-82in-h-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:02:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/newsprint-sheets-24w-x-36l-30-lbs-320pack>: HTTP status code is not handled or not allowed 2026-01-24 22:02:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:02:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gift-grade-tissue-paper-10-lbs-15w-x-20l-scarlet-960-sheets>: HTTP status code is not handled or not allowed 2026-01-24 22:02:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gift-grade-tissue-paper-10-lbs-15w-x-20l-black-960-sheets>: HTTP status code is not handled or not allowed 2026-01-24 22:02:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guardair-lazer-palmswitch-18-alum-ext-alum-nozzle>: HTTP status code is not handled or not allowed 2026-01-24 22:02:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-36-degree-api-astm-27h-hydrometer>: HTTP status code is not handled or not allowed 2026-01-24 22:02:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-shelf-truck-6-wheeled-3600-lbs-2-flush-shelves-30-x-60>: HTTP status code is not handled or not allowed 2026-01-24 22:02:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-ptfe-tube-72-l-x-3-8-id-x-11-16-od-white>: HTTP status code is not handled or not allowed 2026-01-24 22:02:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rt-extra-thick-expandable-foam-bags-60-18w-x-24l-blue-104pack>: HTTP status code is not handled or not allowed 2026-01-24 22:02:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-inch-5-tpi-the-ax-sawzall-blade-5-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:02:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluke-175-true-rms-digital-multimeter-6-000-count>: HTTP status code is not handled or not allowed 2026-01-24 22:02:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silica-gel-packets-78w-x-1-12l-white-5000-packs>: HTTP status code is not handled or not allowed 2026-01-24 22:02:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:02:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock>: HTTP status code is not handled or not allowed 2026-01-24 22:02:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1000-1499-wired-inventory-tag-2-part-carbon-style>: HTTP status code is not handled or not allowed 2026-01-24 22:02:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commercial-faucet-elkay-lk940gn08t6h>: HTTP status code is not handled or not allowed 2026-01-24 22:02:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluke-tp220-suregrip-test-probes-cat-iv-600-v-cat-iii-1000-v-10-a-rating-ul-listed>: HTTP status code is not handled or not allowed 2026-01-24 22:02:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/convoluted-foam-sets-6w-x-6l-x-2-thick-charcoal-64pack>: HTTP status code is not handled or not allowed 2026-01-24 22:02:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25-gallon-combination-drain-evacuator-3639>: HTTP status code is not handled or not allowed 2026-01-24 22:02:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silver-cut-off-wheel-4-1-2-x-045-x-7-8-t27-ceramic-grain>: HTTP status code is not handled or not allowed 2026-01-24 22:02:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2000-2499-inventory-tag-3-part-carbonless>: HTTP status code is not handled or not allowed 2026-01-24 22:02:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-174-chrome-wire-linen-cart-with-nylon-cover-4-shelves-48-l-x-18-w-x-69-h>: HTTP status code is not handled or not allowed 2026-01-24 22:02:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shureshop-bench-w-acc-kit-portable-painted-steel-top-72-x-34-gloss-black>: HTTP status code is not handled or not allowed 2026-01-24 22:02:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-pulley-global-industrial-48-blower-fan>: HTTP status code is not handled or not allowed 2026-01-24 22:02:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roundup-3-4w-8l-endless-poly-roundsling-purple>: HTTP status code is not handled or not allowed 2026-01-24 22:02:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/electrical_power_testing/electrical_power_testing_parts_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:02:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers>: HTTP status code is not handled or not allowed 2026-01-24 22:02:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-edge-4w-8l-1-ply-endless-poly-flat-web-sling>: HTTP status code is not handled or not allowed 2026-01-24 22:02:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crinkle-paper-10-lbs-green>: HTTP status code is not handled or not allowed 2026-01-24 22:02:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-ft-l-lift-out-guard-rails-light-duty>: HTTP status code is not handled or not allowed 2026-01-24 22:02:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-ft-l-lift-out-guard-rails-light-duty>: HTTP status code is not handled or not allowed 2026-01-24 22:02:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crinkle-paper-10-lbs-cognac>: HTTP status code is not handled or not allowed 2026-01-24 22:02:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leviton-cs215-2w-15a-120-277v-toggle-double-pole-ac-quiet-switch-commercial-grade-grounding-white>: HTTP status code is not handled or not allowed 2026-01-24 22:02:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harley-davidson-fl-flh-series-touring-1340cc-motorcycle-battery-1980-1996-23a-12v>: HTTP status code is not handled or not allowed 2026-01-24 22:02:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/orbis-stakpak-modular-straight-wall-container-24l-x-15w-x-5h-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:02:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-can-am-spyder-rs-998cc-motorcycle-battery-2008-2016-23-amps-12v-i-terminals>: HTTP status code is not handled or not allowed 2026-01-24 22:02:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silhouette-banquet-stacking-chair-vinyl-midnight-blue-9300-series>: HTTP status code is not handled or not allowed 2026-01-24 22:02:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-1-4-x-2-5-8-6-yellow-shipping-tag>: HTTP status code is not handled or not allowed 2026-01-24 22:02:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lewisbins-esd-safe-vertical-box-dividers-14-1316l-x-7-1332h>: HTTP status code is not handled or not allowed 2026-01-24 22:02:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/testing_inspection_instruments/electrical_power_testing/electrical_power_testing_parts_accessories landed on page that is not a product page. 2026-01-24 22:02:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powerblanket-30-gallon-insulated-drum-heater-adjustable-thermostat>: HTTP status code is not handled or not allowed 2026-01-24 22:02:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cryo-safe-mini-quick-freeze-cooler-for-05-or-15ml-tubes-12-places-plastic-5-3-4-x-4-x-3>: HTTP status code is not handled or not allowed 2026-01-24 22:02:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-vci-paper-35-lbs-36w-x-200-yd-kraft-1-roll>: HTTP status code is not handled or not allowed 2026-01-24 22:02:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lewisbins-esd-safe-vertical-box-dividers-14-1316l-x-1-2932h>: HTTP status code is not handled or not allowed 2026-01-24 22:02:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/calibrated-electronic-ss-stem-thermometer-50-200c-58-392f-120mm-47-probe>: HTTP status code is not handled or not allowed 2026-01-24 22:02:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-3-4-x-2-3-8-5-white-strung-tag>: HTTP status code is not handled or not allowed 2026-01-24 22:02:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roundup-1w-16l-endless-poly-roundsling-green>: HTTP status code is not handled or not allowed 2026-01-24 22:03:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-hdpe-bar-48-l-x-4-w-x-1-4-thick-black-marine>: HTTP status code is not handled or not allowed 2026-01-24 22:03:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-hdpe-bar-48-l-x-4-w-x-3-4-thick-white-marine>: HTTP status code is not handled or not allowed 2026-01-24 22:03:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-rod-36-l-x-5-16-dia-white>: HTTP status code is not handled or not allowed 2026-01-24 22:03:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h-b-calibrated-electronic-verification-lollipop-stem-thermometer-for-general-applications-32-158f>: HTTP status code is not handled or not allowed 2026-01-24 22:03:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-edge-4w-14l-2-ply-eye-eye-poly-web-sling-flat-eyes>: HTTP status code is not handled or not allowed 2026-01-24 22:03:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-steel-pegboard-18-x-19-blue-796623bl>: HTTP status code is not handled or not allowed 2026-01-24 22:03:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h-b-easy-read-general-purpose-liquid-in-glass-thermometer-35-to-50c-76mm-immersion>: HTTP status code is not handled or not allowed 2026-01-24 22:03:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ivory-plastic-cabinet-drawer-2-3-4w-x-11d-x-2-1-2h>: HTTP status code is not handled or not allowed 2026-01-24 22:03:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/windsor-biodegradable-vacuum-bag-sensorxp12-versamatic-plus-allstar-javelin-12-series>: HTTP status code is not handled or not allowed 2026-01-24 22:03:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-edge-3w-20l-1-ply-endless-poly-flat-web-sling>: HTTP status code is not handled or not allowed 2026-01-24 22:03:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-rod-48-l-x-1-dia-black>: HTTP status code is not handled or not allowed 2026-01-24 22:03:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nobles-paper-vacuum-bag-magna-twin-3000-tennant-model-3280-wide-area-vac>: HTTP status code is not handled or not allowed 2026-01-24 22:03:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-epoxy-wire-shelving-18725gz-72w-x-18d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 22:03:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 28243 pages (at 92 pages/min), scraped 13991 items (at 13 items/min) 2026-01-24 22:03:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-edge-3w-3l-2-ply-eye-eye-poly-web-sling-flat-eyes>: HTTP status code is not handled or not allowed 2026-01-24 22:03:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roundup-1-1-4w-16l-endless-poly-roundsling-yellow>: HTTP status code is not handled or not allowed 2026-01-24 22:03:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roundup-1-1-4w-12l-endless-poly-roundsling-yellow>: HTTP status code is not handled or not allowed 2026-01-24 22:03:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ma-10-lever-pail-pump>: HTTP status code is not handled or not allowed 2026-01-24 22:03:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ghs-labeled-safety-vented-acetone-wash-bottles-1000ml-32oz-polyethylene-w-red-pp-cap-2pk>: HTTP status code is not handled or not allowed 2026-01-24 22:03:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-96-x-36-adj-height-workbench-w-drawer-riser-bk-plastic-laminate-square-top>: HTTP status code is not handled or not allowed 2026-01-24 22:03:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-edge-2w-12l-2-ply-eye-eye-poly-web-sling-twisted-eyes>: HTTP status code is not handled or not allowed 2026-01-24 22:03:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-72-x-30-adj-height-workbench-w-drawer-riser-black-esd-square-edge-top>: HTTP status code is not handled or not allowed 2026-01-24 22:03:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:03:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-drawer-5-5-16w-dividers>: HTTP status code is not handled or not allowed 2026-01-24 22:03:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/euro-drawer-8-3-8w-x-11-5-8d-x4-5-8h-yellow>: HTTP status code is not handled or not allowed 2026-01-24 22:03:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72in-richmond-steel-slat-rectangular-picnic-table-surface-mount-black>: HTTP status code is not handled or not allowed 2026-01-24 22:03:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-wood-luggage-rack-with-shoe-shelf-walnut-finish>: HTTP status code is not handled or not allowed 2026-01-24 22:03:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/used-oil-drain-self-evacuating>: HTTP status code is not handled or not allowed 2026-01-24 22:03:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/small-square-hook-pair-hdpe-with-structural-foam>: HTTP status code is not handled or not allowed 2026-01-24 22:03:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/discharge-nozzle-automatic>: HTTP status code is not handled or not allowed 2026-01-24 22:03:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-alloy-latching-j-hook-style-b-350-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:03:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-strength-plastic-3-tray-shelf-service-utility-cart-44-x-25-1-2-5-rubber-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:03:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-174-galvanized-steel-linen-cart-with-nylon-cover-4-shelves-48-l-x-18-w-x-69-h>: HTTP status code is not handled or not allowed 2026-01-24 22:03:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-piece-outdoor-dining-set-30width-x-41height-table-white-with-white-frame>: HTTP status code is not handled or not allowed 2026-01-24 22:03:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storage-container-with-lid-28-x-22-x-16-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:03:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simplehuman-174-mini-round-step-can-1-1-5-gallon-polished-ss>: HTTP status code is not handled or not allowed 2026-01-24 22:03:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-rod-24-l-x-1-dia-black>: HTTP status code is not handled or not allowed 2026-01-24 22:03:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tempco-channel-strip-240v-t3-42-12l-1500w>: HTTP status code is not handled or not allowed 2026-01-24 22:03:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sw16guu-op-open-linear-bearing-wresin-retainer-and-seals-1id-steel>: HTTP status code is not handled or not allowed 2026-01-24 22:03:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-rod-24-l-x-7-8-dia-black>: HTTP status code is not handled or not allowed 2026-01-24 22:03:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-hdpe-sheet-96-l-x-48-w-x-1-4-thick-black-marine>: HTTP status code is not handled or not allowed 2026-01-24 22:03:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/designer-ribbed-armless-task-chair-low-back-white>: HTTP status code is not handled or not allowed 2026-01-24 22:03:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock landed on page that is not a product page. 2026-01-24 22:03:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/designer-ribbed-armless-task-chair-low-back-black>: HTTP status code is not handled or not allowed 2026-01-24 22:03:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-x-6-orange-packing-list-enclosed-panel-face>: HTTP status code is not handled or not allowed 2026-01-24 22:03:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-1-2-x-5-1-2-blue-packing-list-enclosed-full-face>: HTTP status code is not handled or not allowed 2026-01-24 22:03:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-aluminum-mailbox-without-pole-8w-x-21d-x-11h-mailbox-dim>: HTTP status code is not handled or not allowed 2026-01-24 22:03:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plus-astm-like-liquid-in-glass-thermometer-57f-tag-closed-tester-low-range-4-to-122f>: HTTP status code is not handled or not allowed 2026-01-24 22:03:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tempco-steel-immersion-2-180d-3000w-240v>: HTTP status code is not handled or not allowed 2026-01-24 22:03:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/267300004-hydrantutility-marker-7-long-with-lbracket-redwhite>: HTTP status code is not handled or not allowed 2026-01-24 22:03:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/102236-temporary-pavement-marker-4-x-4-white-1-slide>: HTTP status code is not handled or not allowed 2026-01-24 22:03:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-lounge-ottoman-leather-melrose-white-hercules-alon-series>: HTTP status code is not handled or not allowed 2026-01-24 22:03:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tempco-brasscopper-immersion-2-110d-5000w-480v-t-stat>: HTTP status code is not handled or not allowed 2026-01-24 22:03:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s-gel-retractable-gel-ink-pen-0-5mm-blue-ink>: HTTP status code is not handled or not allowed 2026-01-24 22:03:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:03:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spruce-general-use-spray-paint-16-oz-light-gray-12-cans-case-98-15>: HTTP status code is not handled or not allowed 2026-01-24 22:03:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simplehuman-174-toilet-plunger-black>: HTTP status code is not handled or not allowed 2026-01-24 22:03:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-round-plastic-folding-table-granite-white>: HTTP status code is not handled or not allowed 2026-01-24 22:03:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4x6-4-mil-double-track-reclosable-poly-bags>: HTTP status code is not handled or not allowed 2026-01-24 22:03:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reach-in-refrigerator-horizon-glass-half-glass-doors-26w-hr1-1g>: HTTP status code is not handled or not allowed 2026-01-24 22:03:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-wall-mount-wire-shelving-36w-x-18d-x-14h-1-shelf-add-on-poly-green>: HTTP status code is not handled or not allowed 2026-01-24 22:03:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reach-in-refrigerator-horizon-solid-half-solid-doors-26w-hr1-1hs>: HTTP status code is not handled or not allowed 2026-01-24 22:03:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bsxvl531mm10-hvl531-series-adjustable-arm-executive-chair-black-mesh>: HTTP status code is not handled or not allowed 2026-01-24 22:03:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-wire-shelf-stem-caster-truck-black-ep-18-w-x-48-l-x-69-h>: HTTP status code is not handled or not allowed 2026-01-24 22:03:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/draft-direct-draw-dispensers-dd-79w-dd78>: HTTP status code is not handled or not allowed 2026-01-24 22:03:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/en5631-technopolymer-thandle-wthreaded-stud-mm-diameter-40mm-length-1420>: HTTP status code is not handled or not allowed 2026-01-24 22:03:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/itm-34-17-sds-max>: HTTP status code is not handled or not allowed 2026-01-24 22:03:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-wire-shelving-unit-black-ep-18-w-x-30-l-x-86-h>: HTTP status code is not handled or not allowed 2026-01-24 22:03:45 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers landed on page that is not a product page. 2026-01-24 22:03:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-36-l-x-4-w-x-2-thick-black>: HTTP status code is not handled or not allowed 2026-01-24 22:03:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-48-l-x-6-w-x-1-4-thick-black>: HTTP status code is not handled or not allowed 2026-01-24 22:03:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:03:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/29d-undercounter-refrigerator-food-prep-27w-ucr27a>: HTTP status code is not handled or not allowed 2026-01-24 22:03:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-wire-shelves-utility-cart-silver-ep-21-w-x-42-l-x-39-h>: HTTP status code is not handled or not allowed 2026-01-24 22:03:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gn5981-nylon-revolving-handle-wthreaded-spindle-28mm-diameter-90mm-length-m10x15>: HTTP status code is not handled or not allowed 2026-01-24 22:03:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-wire-shelves-utility-cart-2-casters-w-brakes-chrome-21-w-x-36-l-x-39-h>: HTTP status code is not handled or not allowed 2026-01-24 22:03:51 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-24 22:03:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-wall-mount-wire-shelving-48w-x-14d-x-34h-2-shelf-starter-poly-green>: HTTP status code is not handled or not allowed 2026-01-24 22:03:52 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:14 2026-01-24 22:03:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/school-milk-coolers-sm-58-1-2w-sm58n>: HTTP status code is not handled or not allowed 2026-01-24 22:03:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-wire-shelving-unit-black-ep-24-w-x-48-l-x-86-h>: HTTP status code is not handled or not allowed 2026-01-24 22:03:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reach-in-refrigerator-horizon-glass-half-glass-doors-78w-hrs3-1g>: HTTP status code is not handled or not allowed 2026-01-24 22:03:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kendall-howard-1u-4-point-adjustable-shelf>: HTTP status code is not handled or not allowed 2026-01-24 22:03:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kendall-howard-performance-96-accessory-bar>: HTTP status code is not handled or not allowed 2026-01-24 22:03:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-60-basis-weight-bogus-kra-paper>: HTTP status code is not handled or not allowed 2026-01-24 22:03:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 22:03:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kendall-howard-12u-side-mount-wall-rack>: HTTP status code is not handled or not allowed 2026-01-24 22:03:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-ptfe-bar-12-l-x-4-w-x-3-8-thick-black>: HTTP status code is not handled or not allowed 2026-01-24 22:03:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/packaging/strapping_supplies/strapping_tools already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:03:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyethylene-buchner-table-top-funnel-with-coarse-porosity-removable-plate-18-id-115h>: HTTP status code is not handled or not allowed 2026-01-24 22:03:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-x-24-maple-square-edge-3-drawer-4-drawer>: HTTP status code is not handled or not allowed 2026-01-24 22:04:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kendall-howard-22u-fixed-wall-mount-cabinet>: HTTP status code is not handled or not allowed 2026-01-24 22:04:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/office-partition-gray-60-1-4x60>: HTTP status code is not handled or not allowed 2026-01-24 22:04:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyethylene-buchner-table-top-funnel-with-coarse-porosity-removable-plate-1025-id-8h>: HTTP status code is not handled or not allowed 2026-01-24 22:04:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rugged-case-first-aid-responder-kit-with-aed>: HTTP status code is not handled or not allowed 2026-01-24 22:04:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4ft-x-6ft-bulldog-174-cotton-us-flag-with-sewn-stripes-embroidered-stars>: HTTP status code is not handled or not allowed 2026-01-24 22:04:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dupli-color-automotive-metallic-coating-chrome-11-oz-aerosol>: HTTP status code is not handled or not allowed 2026-01-24 22:04:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vht-extreme-high-temperature-coating-nu-cast-iron-11-oz-aerosol>: HTTP status code is not handled or not allowed 2026-01-24 22:04:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commercial-recess-mount-ceiling-heater-480v-5000w>: HTTP status code is not handled or not allowed 2026-01-24 22:04:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4in-steel-flagpole-bracket>: HTTP status code is not handled or not allowed 2026-01-24 22:04:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-three-in-one-multi-channel-pipette-reservoirs-10pk>: HTTP status code is not handled or not allowed 2026-01-24 22:04:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:04:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/packaging/strapping_supplies/strapping_tools>: HTTP status code is not handled or not allowed 2026-01-24 22:04:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 28338 pages (at 95 pages/min), scraped 14003 items (at 12 items/min) 2026-01-24 22:04:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/geh-17es-2rs-spherical-plain-bearing-metric-heavy-series-sealed>: HTTP status code is not handled or not allowed 2026-01-24 22:04:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ge-40es-spherical-plain-bearing-metric>: HTTP status code is not handled or not allowed 2026-01-24 22:04:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-funnel-holder-for-two-1-to-6-funnels>: HTTP status code is not handled or not allowed 2026-01-24 22:04:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/secador-amber-20-auto-desiccator-cabinet-230v-12-cu-ft>: HTTP status code is not handled or not allowed 2026-01-24 22:04:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-gear-driven-rotary-tiller-implement-w-adjustable-feet-cat-1>: HTTP status code is not handled or not allowed 2026-01-24 22:04:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-washer-for-pipette-pump-yellow-pipettors>: HTTP status code is not handled or not allowed 2026-01-24 22:04:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-medium-duty-landscape-rake-tractor-attachment-category-1>: HTTP status code is not handled or not allowed 2026-01-24 22:04:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-24-l-x-3-w-x-1-2-thick-white>: HTTP status code is not handled or not allowed 2026-01-24 22:04:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/red-500ml-16oz-polyethylene-wash-bottles-polypropylene-cap-28mm-closure-6pk>: HTTP status code is not handled or not allowed 2026-01-24 22:04:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pre-saturated-microscope-optics-cleaning-pads-5x-8-10pk>: HTTP status code is not handled or not allowed 2026-01-24 22:04:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-5-tier-36w-x-30d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 22:04:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/culinary-creations-wooden-play-kitchen-stove>: HTTP status code is not handled or not allowed 2026-01-24 22:04:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 22:04:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commercial-fan-forced-wall-heater-with-tamperproof-thermostat-1500w-120v>: HTTP status code is not handled or not allowed 2026-01-24 22:04:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dehumidifier-commercial-grade-refrigeration-250-pints-a-day-dehumidification-with-water-pump>: HTTP status code is not handled or not allowed 2026-01-24 22:04:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ge-300es-spherical-plain-bearing-metric>: HTTP status code is not handled or not allowed 2026-01-24 22:04:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chem-crest-75a-phosphoric-acid-wash-solution-5-gallon-pail-70075ap>: HTTP status code is not handled or not allowed 2026-01-24 22:04:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lakeside-stand-3-shelves-1-drawer-300>: HTTP status code is not handled or not allowed 2026-01-24 22:04:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poxygrid-aluminum-microliter-pipettor-rack-for-gilson-pipetman-for-desktop-storage>: HTTP status code is not handled or not allowed 2026-01-24 22:04:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4800-watt-commercial-ceiling-heater-14-3-16-l-277v>: HTTP status code is not handled or not allowed 2026-01-24 22:04:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6826-btu-fan-forced-wall-heater-without-summer-fan-switch-277v-1>: HTTP status code is not handled or not allowed 2026-01-24 22:04:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6826-btu-institutional-wall-convector-480v>: HTTP status code is not handled or not allowed 2026-01-24 22:04:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-5-tier-24w-x-24d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 22:04:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-resin-chaise-white>: HTTP status code is not handled or not allowed 2026-01-24 22:04:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/prevention-security-cabinet-48w-18d-84h-single-door-assembled>: HTTP status code is not handled or not allowed 2026-01-24 22:04:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/louvered-wall-panel-without-bins-36x61>: HTTP status code is not handled or not allowed 2026-01-24 22:04:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-42w-x-14d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 22:04:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-labeled-4-color-methanol-wide-mouth-wash-bottles-500ml-16oz-pe-w-green-pp-cap-4pk>: HTTP status code is not handled or not allowed 2026-01-24 22:04:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-labeled-assorted-2-color-wide-mouth-wash-bottles-500ml-16oz-pe-w-pp-cap-6pk>: HTTP status code is not handled or not allowed 2026-01-24 22:04:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-42w-x-21d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 22:04:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ppe-organizer-wall-mountable-60-w-x-9-d-x-25-h-clear>: HTTP status code is not handled or not allowed 2026-01-24 22:04:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/twelve-big-cubby-storage>: HTTP status code is not handled or not allowed 2026-01-24 22:04:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-5-tier-72w-x-21d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 22:04:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/space-wall-system-wire-grid-panel-gray-epoxy-36w-x-48l>: HTTP status code is not handled or not allowed 2026-01-24 22:04:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-vented-labeled-2-color-lyob-wm-wash-bottles-500ml-16oz-pe-w-natural-pp-cap-3pk>: HTTP status code is not handled or not allowed 2026-01-24 22:04:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/abrasives/buffing_polishing/buffing_wheels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:04:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oreck-17-inch-union-mix-scrub-brush>: HTTP status code is not handled or not allowed 2026-01-24 22:04:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-12-l-x-2-1-2-w-x-3-8-thick-white>: HTTP status code is not handled or not allowed 2026-01-24 22:04:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/secador-clear-40-horizontal-profile-gas-purge-desiccator-cabinet-19-cu-ft>: HTTP status code is not handled or not allowed 2026-01-24 22:04:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mounting-kit-with-36-w-louver-48-w-workbench-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:04:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/junior-victorian-greenhouse-12-6l-x-7-9w-x-8-2h>: HTTP status code is not handled or not allowed 2026-01-24 22:04:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cast-acrylic-bar-48-l-x-1-w-x-3-16-thick-clear-antistatic>: HTTP status code is not handled or not allowed 2026-01-24 22:04:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-24w-x-14d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 22:04:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-metal-stapler>: HTTP status code is not handled or not allowed 2026-01-24 22:04:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spinvane-teflon-triangular-magnetic-stirring-bar-56-x-96-x-48mm-fits-1-ml-vials-white>: HTTP status code is not handled or not allowed 2026-01-24 22:04:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-abs-sheet-48-l-x-24-w-x-1-8-thick-black>: HTTP status code is not handled or not allowed 2026-01-24 22:04:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:04:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/abrasives/buffing_polishing/buffing_wheels>: HTTP status code is not handled or not allowed 2026-01-24 22:04:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flexible-truck-step-12-inch-w-5231112>: HTTP status code is not handled or not allowed 2026-01-24 22:04:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lateral-file-cabinet-white-studio-c-series>: HTTP status code is not handled or not allowed 2026-01-24 22:04:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-24w-x-21d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 22:04:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-rectangular-amber-clear-strobe-light-10-30vdc-12-leds-8891602>: HTTP status code is not handled or not allowed 2026-01-24 22:04:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/truck-scissor-dock-li-96-l-x-72-w-5000-lb-cap>: HTTP status code is not handled or not allowed 2026-01-24 22:04:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-42w-x-21d-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 22:04:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-14-gauge-extra-heavy-duty-galvanite-duratough-ventilated-cabinet-48w-x-24d-x-78h>: HTTP status code is not handled or not allowed 2026-01-24 22:04:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-premium-locker-single-tier-18x18x72-3-door-parchment>: HTTP status code is not handled or not allowed 2026-01-24 22:04:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/notched-hook-chrome>: HTTP status code is not handled or not allowed 2026-01-24 22:04:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-underbed-truck-box-18x18x30-gray-1735103>: HTTP status code is not handled or not allowed 2026-01-24 22:04:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-premium-locker-single-tier-18x21x72-3-door-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:04:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/tables/training_tables already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:04:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats>: HTTP status code is not handled or not allowed 2026-01-24 22:04:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-multi-pole-breaker-lockout-7607>: HTTP status code is not handled or not allowed 2026-01-24 22:04:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-double-tier-locker-starter-1-wide-12w-x-15d-x-36h-unassembled-blue-frost>: HTTP status code is not handled or not allowed 2026-01-24 22:04:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/push-button-lockout-electrical-panels-7253>: HTTP status code is not handled or not allowed 2026-01-24 22:04:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:04:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thin-jewel-cases-clear-black-25-per-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:04:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yellow-group-lock-box-7299y-un>: HTTP status code is not handled or not allowed 2026-01-24 22:04:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/recyclockout-lockout-tagout-forklift-propane-tank-recycled-plastic-7293>: HTTP status code is not handled or not allowed 2026-01-24 22:04:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-polypropylene-utility-tray-12-x-16-x-8>: HTTP status code is not handled or not allowed 2026-01-24 22:05:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-polypropylene-utility-tray-18-x-18-x-4>: HTTP status code is not handled or not allowed 2026-01-24 22:05:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-tray-liner-11764290>: HTTP status code is not handled or not allowed 2026-01-24 22:05:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/tables/training_tables>: HTTP status code is not handled or not allowed 2026-01-24 22:05:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/405103-rfid-coded-non-contact-switch-sp-rfid-m-10m>: HTTP status code is not handled or not allowed 2026-01-24 22:05:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-plastic-box-truck>: HTTP status code is not handled or not allowed 2026-01-24 22:05:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-1-2w-x-11h-injection-molded-styrene-clear>: HTTP status code is not handled or not allowed 2026-01-24 22:05:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-buna-n-roll-4320-l-x-36-w-x-1-32-thick-black-high-strength-80a>: HTTP status code is not handled or not allowed 2026-01-24 22:05:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 22:05:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/200008f-kp-tongue-interlock-switch-flat-act-3nc-1no-12npt-pe>: HTTP status code is not handled or not allowed 2026-01-24 22:05:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-sterileware-bent-handle-sterile-scoop-60ml-plastic-individually-wrapped-10pk>: HTTP status code is not handled or not allowed 2026-01-24 22:05:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spproculture-plastic-pestles-for-micro-tube-homogenizer-system-05ml-100pk>: HTTP status code is not handled or not allowed 2026-01-24 22:05:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-double-tier-locker-adder-1-wide-12w-x-15d-x-36h-unassembled-almond>: HTTP status code is not handled or not allowed 2026-01-24 22:05:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 28432 pages (at 94 pages/min), scraped 14021 items (at 18 items/min) 2026-01-24 22:05:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20040dcadc-ultra-quiet-ultra-oil-free-4-0-hp-20-0-gal-air-compressor>: HTTP status code is not handled or not allowed 2026-01-24 22:05:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cuso-wh-h-led-outdoor-emergency-light-w-heater-3000k-nicad-batt-wet-loc-white>: HTTP status code is not handled or not allowed 2026-01-24 22:05:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-silicone-sheet-36-l-x-36-w-x-3-16-thick-red-food-70a>: HTTP status code is not handled or not allowed 2026-01-24 22:05:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trp1-geopak-outdoor-wallpack-w-photocontrol-15w-1500l-4000k-type-3-dist-dk-bronze>: HTTP status code is not handled or not allowed 2026-01-24 22:05:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/143051-glm-rope-pull-switch-3nc-1no-12npt>: HTTP status code is not handled or not allowed 2026-01-24 22:05:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-steel-skirt-hanger-w-loop-hook-chrome>: HTTP status code is not handled or not allowed 2026-01-24 22:05:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-neoprene-roll-360-l-x-36-w-x-3-32-thick-black-high-strength-80a-acrylic-adhesive>: HTTP status code is not handled or not allowed 2026-01-24 22:05:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2w-x-7h-slatgrid-panel-chrome>: HTTP status code is not handled or not allowed 2026-01-24 22:05:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-stirring-rod-8-plastic-6pk>: HTTP status code is not handled or not allowed 2026-01-24 22:05:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kb-1021-extra-wide-desk-clamp-keyboard-tray-with-gel-wrist-rest-black>: HTTP status code is not handled or not allowed 2026-01-24 22:05:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-plastic-twin-wheel-caster-w-5-8-stem-chrome-hood-black>: HTTP status code is not handled or not allowed 2026-01-24 22:05:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/seq-dual-speed-electric-chain-hoist-1-ton-20-lift-17-3-9-ft-min-115v>: HTTP status code is not handled or not allowed 2026-01-24 22:05:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/seq-dual-speed-electric-chain-hoist-1-ton-15-lift-17-3-9-ft-min-115v>: HTTP status code is not handled or not allowed 2026-01-24 22:05:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/truncated-style-acid-solvent-bottle-carrier-holds-one-4-liter-1-gallon-bottle-pp>: HTTP status code is not handled or not allowed 2026-01-24 22:05:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-fiberglass-step-ladder-375-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:05:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cantilever-double-sided-starter-unit-medium-duty-1>: HTTP status code is not handled or not allowed 2026-01-24 22:05:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/max-attach-polymagnet-rare-earth-disc-w-adhesive-1-00-dia-x-0-12-thick>: HTTP status code is not handled or not allowed 2026-01-24 22:05:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wide-mouth-1000ml-32oz-polyethylene-wash-bottles-ylw-polypropylene-cap-53mm-closure-4pk>: HTTP status code is not handled or not allowed 2026-01-24 22:05:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jewel-174-magsnap-adjustable-metal-wrist-strap-09188-12-ft-cord-black>: HTTP status code is not handled or not allowed 2026-01-24 22:05:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-four-tier-locker-3-wide-12w-x-18d-x-18h-assembled-almond>: HTTP status code is not handled or not allowed 2026-01-24 22:05:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/solvent-base-street-utility-marking-paint-20-oz-hi-viz-yellow-20-976-12-case>: HTTP status code is not handled or not allowed 2026-01-24 22:05:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cantilever-single-sided-starter-unit-extra-6>: HTTP status code is not handled or not allowed 2026-01-24 22:05:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-metal-wrist-strap-09085-with-6-ft-coil-cord-black>: HTTP status code is not handled or not allowed 2026-01-24 22:05:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precision-thumb-screw-w-shoulder-1-4-20-3-4-thread-1-2-head-dia-steel-pkg-of-5>: HTTP status code is not handled or not allowed 2026-01-24 22:05:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-diameter-glass-shelf-topper-for-round-racks-clear>: HTTP status code is not handled or not allowed 2026-01-24 22:05:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/statfree-t2-plus-8482-66452-dissipative-dual-layer-rubber-cut-mat-30-d-x-60-w-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:05:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-series-street-utility-marking-paint-20-oz-pink-fluorescent-20-379-12-case>: HTTP status code is not handled or not allowed 2026-01-24 22:05:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/panel-thumb-screw-6-32-1-2-thread-3-8-head-dia-5-16-head-h-stainless-pkg-of-5>: HTTP status code is not handled or not allowed 2026-01-24 22:05:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-x-30-security-cabinet-bench-maple-safety-edge-1>: HTTP status code is not handled or not allowed 2026-01-24 22:05:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/narrow-head-thumb-screw-4-40-1-thread-3-16-head-dia-3-8-head-h-ss-pkg-of-5>: HTTP status code is not handled or not allowed 2026-01-24 22:05:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-grid-hook-semi-gloss-white>: HTTP status code is not handled or not allowed 2026-01-24 22:05:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precision-brass-thumb-screw-4-40-5-16-thread-5-16-head-dia-3-16-head-h-pkg-of-10>: HTTP status code is not handled or not allowed 2026-01-24 22:05:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/infrared-thermometer-compact-12-1-26-752-deg-f-32-400-deg-c>: HTTP status code is not handled or not allowed 2026-01-24 22:05:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-ds06-dialset-low-lead-pressure-regulating-valve-double-union-npt>: HTTP status code is not handled or not allowed 2026-01-24 22:05:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-rack-netting-one-bay-12>: HTTP status code is not handled or not allowed 2026-01-24 22:05:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cm-1714-96-midnight-coffee-condiment-organizer-12-inch-w-x-12-inch-d-x-5-1-2-inch-h>: HTTP status code is not handled or not allowed 2026-01-24 22:05:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-95a-n5-xenon-strobe-with-horn-amber-120v-ac>: HTTP status code is not handled or not allowed 2026-01-24 22:05:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-magnetic-wall-mount--red-danger-entree-interdite-banner>: HTTP status code is not handled or not allowed 2026-01-24 22:05:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cast-acrylic-bar-48-l-x-4-w-x-3-16-thick-yellow>: HTTP status code is not handled or not allowed 2026-01-24 22:05:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-petg-tube-72-l-x-3-4-id-x-1-od-clear>: HTTP status code is not handled or not allowed 2026-01-24 22:05:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/174-j20-251-7425j-153-polypropylene-label-0-5-h-x-1-w-1500-roll-w-second-label>: HTTP status code is not handled or not allowed 2026-01-24 22:05:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-248ledmy240a-48-mm-led-stacklight-module-yellow-240v-ac>: HTTP status code is not handled or not allowed 2026-01-24 22:05:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36l-heavyweight-single-slot-with-1-slots-on-2-centers-satin-zinc>: HTTP status code is not handled or not allowed 2026-01-24 22:05:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mount-anywhere-basket-18-6-8-b>: HTTP status code is not handled or not allowed 2026-01-24 22:05:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30657-diffuser-1-pkg-qty>: HTTP status code is not handled or not allowed 2026-01-24 22:05:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-58c-n5-100wh-rotating-beacon-100wh-clear-120v-ac>: HTTP status code is not handled or not allowed 2026-01-24 22:05:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30647-valve-stem-1-pkg-qty>: HTTP status code is not handled or not allowed 2026-01-24 22:05:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a0068-muffler-insert-1-pkg-qty>: HTTP status code is not handled or not allowed 2026-01-24 22:05:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-125xbrzw120a-xtra-brite-led-random-flash-pattern-white-120vac-0108-amps>: HTTP status code is not handled or not allowed 2026-01-24 22:05:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corporate-desk-trays-metallic-black-2-trays-organizer-pencil-cup-set>: HTTP status code is not handled or not allowed 2026-01-24 22:05:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/174-150504-riptag-153-safety-tag-roll-do-not-operate-3-w-x-5-75-h-250-roll>: HTTP status code is not handled or not allowed 2026-01-24 22:05:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a0019-lower-shaft-balancer-bearing-2-shields-1-pkg-qty>: HTTP status code is not handled or not allowed 2026-01-24 22:05:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ecp4308t-4-40-hp-1190-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:05:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/general_purpose_electric_motors/three_phase_motors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:05:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/174-j20c-4000-2569-153-polyester-label-4-0-w-x-100-l-white>: HTTP status code is not handled or not allowed 2026-01-24 22:05:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/174-m-49-494-or-b-494-color-polyester-labels-1-h-x-1-w-orange-white-260-roll>: HTTP status code is not handled or not allowed 2026-01-24 22:05:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pd35-pd37-red-bearing-assembly>: HTTP status code is not handled or not allowed 2026-01-24 22:05:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interlake-mecalux-pallet-rack-unslotted-beam-48-l-5723-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-24 22:05:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cjl1317a-2-hp-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:05:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insulating-varnish-green>: HTTP status code is not handled or not allowed 2026-01-24 22:05:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/centrifugal-blower-3000-rpm-208-240v-oao-0-28-0-25-fl-amps>: HTTP status code is not handled or not allowed 2026-01-24 22:05:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/draft-inducer-blower-3267-rpm-115v-oao>: HTTP status code is not handled or not allowed 2026-01-24 22:05:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50-gallon-hexagon-trash-container-black>: HTTP status code is not handled or not allowed 2026-01-24 22:05:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cjm3120-11-2-hp-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:05:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ptfe-tape-high-density-1-2-x-520>: HTTP status code is not handled or not allowed 2026-01-24 22:05:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/condenser-fans-motor-1-2-hp-1075-rpm-208-230v-teao-48-frame>: HTTP status code is not handled or not allowed 2026-01-24 22:05:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/general_purpose_electric_motors/three_phase_motors>: HTTP status code is not handled or not allowed 2026-01-24 22:06:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/neatheat-left-end-wall-cap-hot-water-hydronic-baseboard-cover-nhl>: HTTP status code is not handled or not allowed 2026-01-24 22:06:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/neatheat-inside-corner-baseboard-front-cover-nhic>: HTTP status code is not handled or not allowed 2026-01-24 22:06:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-single-phase-tefc-motor-3-4-hp-1725-rpm-115-208-230v-tefc-56c-frame>: HTTP status code is not handled or not allowed 2026-01-24 22:06:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-270ledsg120a-smd-steady-led-module-and-light-source-green-120v-ac>: HTTP status code is not handled or not allowed 2026-01-24 22:06:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-8-thick-3-w-corrugated-rubber-runner-full-roll-75-l>: HTTP status code is not handled or not allowed 2026-01-24 22:06:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sponge-rubber-gasket-tank-to-bowl-3-1-4-o-d-x-2-1-8-i-d-x-9-16-thick>: HTTP status code is not handled or not allowed 2026-01-24 22:06:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poster-hazardous-energy-control-18-x-24>: HTTP status code is not handled or not allowed 2026-01-24 22:06:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-ultra-green-foam-hand-soap-1200ml-refill-6-refills-case>: HTTP status code is not handled or not allowed 2026-01-24 22:06:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ecp3774t-5-10-hp-1760-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:06:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-5540m-24aq-tone-generator-24v-input-24v-acdc-power>: HTTP status code is not handled or not allowed 2026-01-24 22:06:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyester-tape-8991-2-x-72-yds-2-4-mil-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:06:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hardmark-chalk-refill-2-5-lbs-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:06:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 28531 pages (at 99 pages/min), scraped 14042 items (at 21 items/min) 2026-01-24 22:06:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/packaging-tape-pad-3750p-2-x-6-3-1-mil-clear>: HTTP status code is not handled or not allowed 2026-01-24 22:06:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ventilation-motor-1-25-hp-1550-rpm-115v-oao-3-3-frame>: HTTP status code is not handled or not allowed 2026-01-24 22:06:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1010544-flip-out-bin-mobile-parts-cart-single-sided-with-60-bins>: HTTP status code is not handled or not allowed 2026-01-24 22:06:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-boltless-shelving-72-inch-w-x-48-inch-d-x-84-inch-h-3-levels-add-on-w-wire-decking-black>: HTTP status code is not handled or not allowed 2026-01-24 22:06:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rug-ram-carpet-pole-carriage-mounted-144-l-1>: HTTP status code is not handled or not allowed 2026-01-24 22:06:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-boltless-shelving-48-inch-w-x-48-inch-d-x-84-inch-h-3-levels-add-on-w-ez-decking-black>: HTTP status code is not handled or not allowed 2026-01-24 22:06:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-4-x-3-1-2-snap-on-package-qty-25>: HTTP status code is not handled or not allowed 2026-01-24 22:06:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/150-construction-chalk-reel>: HTTP status code is not handled or not allowed 2026-01-24 22:06:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-master-carton-600-pair-hygenx-sanitary-headphone-covers-for-over-ear-headsets>: HTTP status code is not handled or not allowed 2026-01-24 22:06:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anti-static-foam-shipper-12-w-x-12-l-x-2-3-4-d>: HTTP status code is not handled or not allowed 2026-01-24 22:06:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-view-plus-locker-w-digitech-locks-12x12x12-4-person-parchment-assembled>: HTTP status code is not handled or not allowed 2026-01-24 22:06:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-log-book-for-electric-counterbalance-forklift-log-986885>: HTTP status code is not handled or not allowed 2026-01-24 22:06:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-broom-cabinet-48-w-x-24-d-x-75-h-yellow>: HTTP status code is not handled or not allowed 2026-01-24 22:06:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbl-rivet-boltless-shelving-96wx48dx84h-4-levels-add-on-no-decking-620-lbs-shelf-cap-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:06:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-rivet-boltless-shelving-48-inch-w-x-24-inch-d-x-84-inch-h-6-levels-starter-w-o-decking-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:06:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/record-storage-shelving-42-inch-w-x-15-inch-d-x-108-inch-h-5-levels-starter-w-o-decking-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:06:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fan-and-blower-3-4-hp-1725-rpm-115-208-230v-odp>: HTTP status code is not handled or not allowed 2026-01-24 22:06:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-rivet-boltless-shelving-48-inch-w-x-18-inch-d-x-84-inch-h-8-levels-starter-w-o-decking-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:06:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-broom-cabinet-48-w-x-24-d-x-75-h-brown>: HTTP status code is not handled or not allowed 2026-01-24 22:06:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/draft-inducer-blower-3000-rpm-208-230v-oao-0-58-0-60-fl-amps>: HTTP status code is not handled or not allowed 2026-01-24 22:06:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silver-aluminum-aluminum-eggcrate-3>: HTTP status code is not handled or not allowed 2026-01-24 22:06:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-straight-male-3-8-tube-x-1-4-nptf-1-1-3-l>: HTTP status code is not handled or not allowed 2026-01-24 22:06:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-union-5mm-tube-1-2-7-l>: HTTP status code is not handled or not allowed 2026-01-24 22:06:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/contractor-knit-roller-cover-extra-rough-1-1-4-nap>: HTTP status code is not handled or not allowed 2026-01-24 22:06:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbl-rivet-boltless-shelving-72wx48dx84h-4-levels-add-on-no-decking-500-lbs-shelf-cap-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:06:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oem-replacement-motor-1-5-hp-1050-rpm-208-230v-oao>: HTTP status code is not handled or not allowed 2026-01-24 22:06:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vector-duty-motor-3-hp-1755-rpm-230-460v-tenv>: HTTP status code is not handled or not allowed 2026-01-24 22:06:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/men-indus-wrk-shirt-shrt-slv-char-3xl-sp24>: HTTP status code is not handled or not allowed 2026-01-24 22:06:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbl-rivet-boltless-shelving-60wx24dx84h-4-levels-add-on-no-decking-600-lbs-shelf-cap-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:06:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/direct-drive-motor-1-3-hp-1075-rpm-208-230-277v-oao>: HTTP status code is not handled or not allowed 2026-01-24 22:06:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbl-rivet-boltless-shelving-60wx36dx84h-4-levels-starter-no-decking-1200-lbs-shelf-cap-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:06:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/men-shrt-slv-deluxe-western-style-shirt-4xl-sc24-sc24lbss4xl>: HTTP status code is not handled or not allowed 2026-01-24 22:06:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-mesh-deck-36-x36>: HTTP status code is not handled or not allowed 2026-01-24 22:06:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-three-phase-tefc-motor-15-hp-1775-rpm-230-460v-tefc>: HTTP status code is not handled or not allowed 2026-01-24 22:06:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-size-stacking-bin-11x29-7-8x10-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:06:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/toolbox-z300-greenx-jumbo-roll-natural-950-sh-roll-1-roll-case-24310>: HTTP status code is not handled or not allowed 2026-01-24 22:06:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jm-motor-10-hp-1768-rpm-230-460v-odp>: HTTP status code is not handled or not allowed 2026-01-24 22:06:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-4-tier-add-on-unit-30-w-x-24-d-x-63-h>: HTTP status code is not handled or not allowed 2026-01-24 22:06:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pool-pump-motor-1-hp-3450-rpm-230v-odp-48y-frame>: HTTP status code is not handled or not allowed 2026-01-24 22:06:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-trash-container-silver-vein-20-gallon-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:06:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-idnm2334t-20te-hp-4000-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:06:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oem-replacement-motor-1-3-hp-1075-rpm-115v-oao-rolled-steel>: HTTP status code is not handled or not allowed 2026-01-24 22:06:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cabinet-with-keyless-entry-lock-48-w-x-24-d-x-75-h-white>: HTTP status code is not handled or not allowed 2026-01-24 22:06:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jm-motor-3-hp-1770-rpm-230-460v-odp-182jm-frame>: HTTP status code is not handled or not allowed 2026-01-24 22:06:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-1-4-multisocket-x-3-8-male-nptf-2-1-8-l>: HTTP status code is not handled or not allowed 2026-01-24 22:06:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-three-phase-odp-motor-1-3-hp-1725-rpm-230-460v-odp-56-frame>: HTTP status code is not handled or not allowed 2026-01-24 22:06:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-jm3112-3-4-hp-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:06:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-union-elbow-1-4-tube-5-6-l>: HTTP status code is not handled or not allowed 2026-01-24 22:06:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-swivel-elbow-6mm-tube-x-1-4-swift-fit-universal-thread-5-6-l>: HTTP status code is not handled or not allowed 2026-01-24 22:06:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-boltless-shelving-96-inch-w-x-48-inch-d-x-84-inch-h-3-levels-add-on-w-ez-decking-black>: HTTP status code is not handled or not allowed 2026-01-24 22:06:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-deluxe-carry-caddy>: HTTP status code is not handled or not allowed 2026-01-24 22:06:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cabinet-with-keyless-entry-lock-36-w-x-24-d-x-75-h-red>: HTTP status code is not handled or not allowed 2026-01-24 22:06:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-l-x-8-1-4-w-x-7-h-ultra-series-stack-and-hang-bin-red-polyethylene-4-slots>: HTTP status code is not handled or not allowed 2026-01-24 22:06:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbl-rivet-boltless-shelving-60wx36dx84h-4-levels-add-on-no-decking-2000-lbs-shelf-cap-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:06:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v-grip-wire-shelving-spill-tray-48-l-x-18-w-x-2-1-4-d-6-5-gallon-capacity-smooth-office-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:06:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-boltless-shelving-96-inch-w-x-24-inch-d-x-84-inch-h-3-levels-add-on-w-ez-decking-black>: HTTP status code is not handled or not allowed 2026-01-24 22:06:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-swivel-branch-tee-12mm-tube-x-3-8-bspp-2-1-2-l>: HTTP status code is not handled or not allowed 2026-01-24 22:06:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24282-g-max-variable-speed-cordless-blower-40v-150-mph>: HTTP status code is not handled or not allowed 2026-01-24 22:06:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v-grip-pre-configured-wire-shelving-spill-containment-system-36-w-x-24-d-x-84-h-add-on-silver>: HTTP status code is not handled or not allowed 2026-01-24 22:07:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-8-l-x-11-w-x-3-h-ultra-series-stack-and-hang-bin-green-polyethylene-3-slots>: HTTP status code is not handled or not allowed 2026-01-24 22:07:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-straight-male-3-8-tube-x-1-8-nptf-1-1-4-l>: HTTP status code is not handled or not allowed 2026-01-24 22:07:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-l1313-50-1-5-hp-2850-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:07:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-l-x-5-1-2-w-x-5-h-ultra-series-stack-and-hang-bin-blue-polyethylene-4-slots>: HTTP status code is not handled or not allowed 2026-01-24 22:07:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-3kw-208v-lavadvantage-thermostatic-electric-tankless-water-heater>: HTTP status code is not handled or not allowed 2026-01-24 22:07:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/filter-multi-pleat-xl10-20-w-x-35-h-x-2-d-6-ctn>: HTTP status code is not handled or not allowed 2026-01-24 22:07:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-m1705t-1-5-75-hp-1725-850-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:07:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-m1706t-2-1-hp-1725-850-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:07:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-swivel-run-tee-1-4-tube-x-1-4-ml-5-6-l>: HTTP status code is not handled or not allowed 2026-01-24 22:07:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pyramid-surface-cribbing-4-4-30-15203>: HTTP status code is not handled or not allowed 2026-01-24 22:07:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50-and-100-ton-jack-plate-kit-2-piece-kit-includes-14464-and-14465-14479>: HTTP status code is not handled or not allowed 2026-01-24 22:07:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-swivel-run-tee-1-8-tube-x-1-8-swift-fit-universal-thread-2-3-l>: HTTP status code is not handled or not allowed 2026-01-24 22:07:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 28631 pages (at 100 pages/min), scraped 14071 items (at 29 items/min) 2026-01-24 22:07:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/formula-409-cleaner-degreaser-disinfectant-1>: HTTP status code is not handled or not allowed 2026-01-24 22:07:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-ton-locking-jack-stand-single-unit-14410>: HTTP status code is not handled or not allowed 2026-01-24 22:07:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mini-case-cart-29-5-8-w-x-40-1-2-h-x-29-d-1-s-s-wire-pullout-shelf-1-glass-door>: HTTP status code is not handled or not allowed 2026-01-24 22:07:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-deck-for-rivetwell-48-inch-w-x-18-inch-d-x-1-4-inch-h-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:07:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/retractable-feather-duster>: HTTP status code is not handled or not allowed 2026-01-24 22:07:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-x-3-reflective-reboundable-delineator-round-white>: HTTP status code is not handled or not allowed 2026-01-24 22:07:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-1-4-multisocket-x-3-8-female-nptf-2-1-8-l>: HTTP status code is not handled or not allowed 2026-01-24 22:07:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mini-size-binder-pocket-side-loading>: HTTP status code is not handled or not allowed 2026-01-24 22:07:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mri-safe-second-nested-instrument-table-on-casters-stainless-steel-44-l-x-40-w-x-22-h>: HTTP status code is not handled or not allowed 2026-01-24 22:07:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/basket-for-ep30h-s30h-p30h>: HTTP status code is not handled or not allowed 2026-01-24 22:07:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-mesh-premium-chair-black>: HTTP status code is not handled or not allowed 2026-01-24 22:07:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavyweight-cleer-adheer-laminating-film-clear-4-x-5-4-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:07:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hazardous-waste-vinyl-label-non-hazardous-waste-6-inches-by-6-inches-green-white-50pack-121159>: HTTP status code is not handled or not allowed 2026-01-24 22:07:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-flow-control-cylinder-flow-knob-adjustment-1-4-fm-nptf-x-1-4-universal-thread-1-l>: HTTP status code is not handled or not allowed 2026-01-24 22:07:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/maxi-case-cart-52-w-x-40-1-2-h-x-29-d-1-heavy-duty-wire-rollout-shelf-2-solid-doors>: HTTP status code is not handled or not allowed 2026-01-24 22:07:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-adhesive-cd-holder-5-1-3-x-5-2-3-10-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:07:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pwbs-14cs-pow-14-ww-bandsaw-w-stand-2pc>: HTTP status code is not handled or not allowed 2026-01-24 22:07:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-needle-valve-knob-adjustment-1-4-tube-x-10-32-swift-fit-universal-thread-6-7-l>: HTTP status code is not handled or not allowed 2026-01-24 22:07:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/78149-anti-slip-black-yellow-striped-tape-roll-4-inches-by-60-feet>: HTTP status code is not handled or not allowed 2026-01-24 22:07:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wheel-bed-workstation-96-w-x-48-d-x-72-h-starter-section>: HTTP status code is not handled or not allowed 2026-01-24 22:07:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scaf-stsi594-scafftag-caution-tag-insert-100-pkg-3-inches-by-7-inches-yellow-white>: HTTP status code is not handled or not allowed 2026-01-24 22:07:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gpf231sss-0a-general-purpose-stainless-steel-solid-door-freezer-23-cu-ft>: HTTP status code is not handled or not allowed 2026-01-24 22:07:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-crew-shirt-short-sleeve-2xl-charcoal-royal-blue-sy20>: HTTP status code is not handled or not allowed 2026-01-24 22:07:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/knee-boot-size-11-15-h-composite-toe-chevron-plus-outsole-blue-w-red-sole>: HTTP status code is not handled or not allowed 2026-01-24 22:07:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/x-knee-boot-men-s-size-11-15-h-steel-toe-cleated-outsole-white>: HTTP status code is not handled or not allowed 2026-01-24 22:07:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-union-tee-14mm-tube-2-3-8-l>: HTTP status code is not handled or not allowed 2026-01-24 22:07:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-smock-loose-fit-short-sleeve-tan-xl-tp23>: HTTP status code is not handled or not allowed 2026-01-24 22:07:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interlake-mecalux-pallet-rack-tear-drop-welded-upright-frame-36-d-x-144-h-green>: HTTP status code is not handled or not allowed 2026-01-24 22:07:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/knee-boot-men-s-size-3-15-h-plain-toe-cleated-outsole-brown-upper-crepe-sole>: HTTP status code is not handled or not allowed 2026-01-24 22:07:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-flow-control-5mm-tube-x-1-4-swift-fit-universal-thread-1-l>: HTTP status code is not handled or not allowed 2026-01-24 22:07:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b30-indoor-outdoor-industrial-vinyl-labels-2-1-4-w-x-100-l-blueb30c-2250-595-bl>: HTTP status code is not handled or not allowed 2026-01-24 22:07:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bmp21-series-permasleeve-wire-marking-sleeves-1-4-dia-black-white-m21-250-c-342>: HTTP status code is not handled or not allowed 2026-01-24 22:07:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-needle-valve-screw-adjustment-6mm-tube-x-m5-swift-fit-universal-thread-4-5-l>: HTTP status code is not handled or not allowed 2026-01-24 22:07:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-flow-control-cylider-flow-screw-adjustment-10mm-tube-x-1-4-universal-thread-1-l>: HTTP status code is not handled or not allowed 2026-01-24 22:07:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-reel-wire-unit-36-w-x-18-d-x-63-h-chrome-all-welded-4-shelves>: HTTP status code is not handled or not allowed 2026-01-24 22:07:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-em3664t-2-hp-1165-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:07:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-w-shelf-double-cantilever-steel-light-green-proform>: HTTP status code is not handled or not allowed 2026-01-24 22:07:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pbl6-push-button-lockout-cover-30-5-mm-base-clear>: HTTP status code is not handled or not allowed 2026-01-24 22:07:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-w-shelf-single-cantilever-steel-light-green-proform>: HTTP status code is not handled or not allowed 2026-01-24 22:07:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96-x72-heat-shield-canister>: HTTP status code is not handled or not allowed 2026-01-24 22:07:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m73-bracket-tape-dispenser>: HTTP status code is not handled or not allowed 2026-01-24 22:07:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-em3555t-2-hp-3490-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:07:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-gas-heater-infrared-vacuum-u-tube-50000-btu-30l>: HTTP status code is not handled or not allowed 2026-01-24 22:07:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bookcase-29-1-2-w-x-11-5-8-d-x-72-h-5-shelf-bark-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:07:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-diamond-series-laboratory-medical-solid-door-refrigerator-72-cu-ft-nswdr723wws-0>: HTTP status code is not handled or not allowed 2026-01-24 22:07:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-profile-air-mover-carpet-dryer-w-daisy-chain-1050-cfm-1-3-hp-pl-700a>: HTTP status code is not handled or not allowed 2026-01-24 22:07:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/big-d-deo-gard-urinal-mat>: HTTP status code is not handled or not allowed 2026-01-24 22:07:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-gas-heater-infrared-positive-pressure-straight-tube-100000-btu-40l>: HTTP status code is not handled or not allowed 2026-01-24 22:07:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-x-5-x-3-corrugated-boxes-200lb-test-ect-32>: HTTP status code is not handled or not allowed 2026-01-24 22:07:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g-series-perform-02-thirst-quencher-lemon-lime-12-oz-bottle-24-carton>: HTTP status code is not handled or not allowed 2026-01-24 22:07:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-gas-heater-infrared-positive-pressure-u-tube-tough-guy-200000-btu-70l>: HTTP status code is not handled or not allowed 2026-01-24 22:07:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-3-led-alley-lights-3024642>: HTTP status code is not handled or not allowed 2026-01-24 22:07:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4in-clear-surface-rectangular-mount-ultra-thin-strobe-light-with-3-led-8892231>: HTTP status code is not handled or not allowed 2026-01-24 22:07:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/big-tall-mid-back-chair-black-2>: HTTP status code is not handled or not allowed 2026-01-24 22:07:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/x-knee-boot-men-s-size-6-15-h-steel-toe-cleated-outsole-yellow>: HTTP status code is not handled or not allowed 2026-01-24 22:07:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oil-safe-utility-lid-light-green>: HTTP status code is not handled or not allowed 2026-01-24 22:07:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/375in-amber-rectangular-marker-clearance-light-with-2-led-5623822>: HTTP status code is not handled or not allowed 2026-01-24 22:07:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/100-pure-coffee-house-blend-ground-28-oz-bag-3-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:08:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s19-2100n-thermostatic-valve-safety-26-gpm>: HTTP status code is not handled or not allowed 2026-01-24 22:08:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ejmm4114t-50-hp-3540-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:08:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coffee-portion-packs-caf-233-domingo-blend-2-5-oz-frack-pack-18-box>: HTTP status code is not handled or not allowed 2026-01-24 22:08:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/humidity-indicator-card-510-60-range-125-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:08:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-14in-amber-green-surface-mount-ultra-thin-led-strobe-light-8892210>: HTTP status code is not handled or not allowed 2026-01-24 22:08:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-em2513t-15-hp-1765-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:08:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metalin-zipper-bag-18-x-24-100-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:08:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precoiled-strap-on-pipe-marker-natural-gas-4>: HTTP status code is not handled or not allowed 2026-01-24 22:08:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/window-air-conditioner-15000-btu-energy-star-rated-115v-wifi-enabled>: HTTP status code is not handled or not allowed 2026-01-24 22:08:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/repair-kit-for-inswing-1-doors-square-edge>: HTTP status code is not handled or not allowed 2026-01-24 22:08:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/go-specsfoam-lined-spectacle-gg-40c-af-clear-afpc-lens-black-frame>: HTTP status code is not handled or not allowed 2026-01-24 22:08:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-bike-wave-bike-rack-flange-mount>: HTTP status code is not handled or not allowed 2026-01-24 22:08:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ecp844156t-4-150-hp-1190-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:08:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-efm3218t-5-hp-1750-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:08:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 28728 pages (at 97 pages/min), scraped 14096 items (at 25 items/min) 2026-01-24 22:08:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-breathable-jacket-size-men-s-small-attached-hood-black>: HTTP status code is not handled or not allowed 2026-01-24 22:08:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brake-tool-set-prof-8pc-ast7848>: HTTP status code is not handled or not allowed 2026-01-24 22:08:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jacket-size-men-s-medium-polyester-quilted-liner-attached-hood-black>: HTTP status code is not handled or not allowed 2026-01-24 22:08:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stackable-air-mover-4-positions-3-speeds-1-2-hp-p-630>: HTTP status code is not handled or not allowed 2026-01-24 22:08:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4in-clear-round-backup-light-with-10-led-5624350>: HTTP status code is not handled or not allowed 2026-01-24 22:08:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ehfm2543t-50-hp-1775-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:08:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/type-r-class-3-pullover-hoodie-polyester-lime-4xl>: HTTP status code is not handled or not allowed 2026-01-24 22:08:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/400-coverall-zipper-front-hood-elastic-wrist-ankle-stormflap-white-md>: HTTP status code is not handled or not allowed 2026-01-24 22:08:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jack-air-lift-10-ton-truck-trailers-otc1590>: HTTP status code is not handled or not allowed 2026-01-24 22:08:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24in-36-led-strip-light-with-3m-adhesive-back-clear-and-cool-5622537>: HTTP status code is not handled or not allowed 2026-01-24 22:08:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jack-stands-2-ton-pair-kti61201>: HTTP status code is not handled or not allowed 2026-01-24 22:08:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precoiled-strap-on-pipe-marker-hot-water-return-5>: HTTP status code is not handled or not allowed 2026-01-24 22:08:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20814rc-pant-hanger-14l-x-12w-wood-bk-rubberized-pkg-qty-100>: HTTP status code is not handled or not allowed 2026-01-24 22:08:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hemco-ductless-demonstration-hood-35-x-24-x-30>: HTTP status code is not handled or not allowed 2026-01-24 22:08:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tennant-v-bp-6-backpack-replacement-vacuum-bags>: HTTP status code is not handled or not allowed 2026-01-24 22:08:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/500-dress-hanger-heavy-weight-17l-plastic-cl-pkg-qty-100>: HTTP status code is not handled or not allowed 2026-01-24 22:08:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precoiled-strap-on-pipe-marker-steam-supply-5>: HTTP status code is not handled or not allowed 2026-01-24 22:08:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precoiled-strap-on-pipe-marker-storm-sewer-6>: HTTP status code is not handled or not allowed 2026-01-24 22:08:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precoiled-strap-on-pipe-marker-waste-water-4>: HTTP status code is not handled or not allowed 2026-01-24 22:08:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-to-3-1-4-28-degree-wire-weld-framing-nailer>: HTTP status code is not handled or not allowed 2026-01-24 22:08:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sc-125-8-qfm-double-faced-wall-blanket-w-1-lb-barrier-septum-4-w-x-8-h-x-2-thick>: HTTP status code is not handled or not allowed 2026-01-24 22:08:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2505-dress-hanger-wchrome-hook-medium-heavy-weight-17l-plastic-bk-pkg-qty-100>: HTTP status code is not handled or not allowed 2026-01-24 22:08:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precoiled-strap-on-pipe-marker-acid-2>: HTTP status code is not handled or not allowed 2026-01-24 22:08:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/plumbing_parts_accessories/pipe_hose_tubing_accessories/pipe_markers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:08:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hardware-base-36in-square-picnic-table>: HTTP status code is not handled or not allowed 2026-01-24 22:08:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-wide-welded-steel-lockers-six-tier-18x18x72-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:08:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-x-131-21-degree-ring-galvanized-framing-nails-2000-qty>: HTTP status code is not handled or not allowed 2026-01-24 22:08:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-diopter-led-magnifying-lamp-with-covered-metal-arm-black-277493>: HTTP status code is not handled or not allowed 2026-01-24 22:08:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/radiant-safety-screen-100-000-to-120-000-btu-ceramic-heaters>: HTTP status code is not handled or not allowed 2026-01-24 22:08:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-rsp2451a-33-hp-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:08:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3400l-unlined-pigskin-driver-gloves-cream-large-12-pairs>: HTTP status code is not handled or not allowed 2026-01-24 22:08:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coil-siding-nail-316-stainless-steel-1-3-4-x-090-ring-4200-qty>: HTTP status code is not handled or not allowed 2026-01-24 22:08:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-or-post-mount-bracket-j-d-poly-chiller-black-vrsbrd36b>: HTTP status code is not handled or not allowed 2026-01-24 22:08:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-vem3770t-5-7-5-hp-1770-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:08:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-x-3-1-2-brass-pipe-nipple-schedule-80>: HTTP status code is not handled or not allowed 2026-01-24 22:08:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/21-degree-framing-nails-3-x-131-smooth-bright-2000-qty>: HTTP status code is not handled or not allowed 2026-01-24 22:08:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tray-camtread-23x29-oval-black>: HTTP status code is not handled or not allowed 2026-01-24 22:08:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/plumbing/plumbing_parts_accessories/pipe_hose_tubing_accessories/pipe_markers>: HTTP status code is not handled or not allowed 2026-01-24 22:08:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/orbis-lid-folding-bulk-shipping-container-40x48>: HTTP status code is not handled or not allowed 2026-01-24 22:08:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4725-rectangle-plastic-height-adjustable-activity-table-set-with-4-chairs-green>: HTTP status code is not handled or not allowed 2026-01-24 22:08:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-3mm-insert-bit-x-1-s2-modified-steel-gunmetal-grey>: HTTP status code is not handled or not allowed 2026-01-24 22:08:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-vefwdm3546t-1-hp-1760-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:08:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-vecp3581-4-1-hp-1765-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:08:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/four-way-stretch-mobile-stacking-chair-fabric-black>: HTTP status code is not handled or not allowed 2026-01-24 22:08:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-tamper-5-16-insert-bit-x-1-s2-modified-steel-gunmetal-grey>: HTTP status code is not handled or not allowed 2026-01-24 22:08:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/protective-bag-for-novax-rubber-insulating-gloves-14-l-black-one-size>: HTTP status code is not handled or not allowed 2026-01-24 22:08:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-economy-manual-chain-hoist-1-2-ton-capacity-20-lift>: HTTP status code is not handled or not allowed 2026-01-24 22:08:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/cubicle_partition_panels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:08:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-vl1313a-1-5-hp-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:08:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/peapod-mini-8-15-locking-bike-shelter-21-10-l-x-7-5-w-15-bike-capacity-hip-roof>: HTTP status code is not handled or not allowed 2026-01-24 22:08:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-port-cap-with-plugs-for-glass-bottles-with-gl45-closure>: HTTP status code is not handled or not allowed 2026-01-24 22:08:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/square-2-insert-bit-x-1-s2-modified-steel-gunmetal-grey>: HTTP status code is not handled or not allowed 2026-01-24 22:08:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roady-polywoven-airbag-36-x-48>: HTTP status code is not handled or not allowed 2026-01-24 22:08:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torq-set-1-4-power-bit-x-1-1-4-s2-modified-steel-gunmetal-grey>: HTTP status code is not handled or not allowed 2026-01-24 22:08:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spanish-plastic-sign-peligro-alto-voltaje-mantengase-afuera>: HTTP status code is not handled or not allowed 2026-01-24 22:08:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-m3560-5-hp-850-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:08:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:08:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/101-t12-eg-all-plastic-maintenance-free-type-i-traffic-barricade-white-foldable>: HTTP status code is not handled or not allowed 2026-01-24 22:08:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esp-kit-1-pair-black-esp-glove-1-pair-goat-class-0-size-12>: HTTP status code is not handled or not allowed 2026-01-24 22:08:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/phillips-3-power-bit-x-6-s2-modified-steel-gunmetal-grey>: HTTP status code is not handled or not allowed 2026-01-24 22:08:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lithonia-elb-0607-replacement-battery-lead-calcium-6v-6-5ah-12v-units-requir>: HTTP status code is not handled or not allowed 2026-01-24 22:08:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/partitions_dividers/cubicle_partition_panels>: HTTP status code is not handled or not allowed 2026-01-24 22:08:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-2-tier-wire-shelving-starter-unit-54w-x-24d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 22:08:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/appliances/refrigerators_freezers/refrigerators/undercounter_refrigerators already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:08:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-2-tier-wire-shelving-starter-unit-42w-x-14d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 22:08:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torx-174-30-insert-bit-x-1-gunmetal-bronze-s2-modified-steel>: HTTP status code is not handled or not allowed 2026-01-24 22:08:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interior-semi-gloss-off-white-gallon-pail-1-case-21600-1>: HTTP status code is not handled or not allowed 2026-01-24 22:08:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oil-enamel-silver-gray-32-oz-pail-1-case-31200-4>: HTTP status code is not handled or not allowed 2026-01-24 22:08:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks>: HTTP status code is not handled or not allowed 2026-01-24 22:09:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-2-tier-wire-shelving-starter-unit-60w-x-24d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 22:09:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-zdnm2238t-10te-hp-5000-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:09:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-zdm3587t-2te-hp-6000-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:09:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buyers-35-gallon-upright-aluminum-hydraulic-reservoir>: HTTP status code is not handled or not allowed 2026-01-24 22:09:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-2-tier-wire-shelving-add-on-unit-72w-x-24d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 22:09:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-latch-container-lid-clear-lid1915-cl>: HTTP status code is not handled or not allowed 2026-01-24 22:09:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-plastic-sign-notice-eye-protection-required-beyond-this-point>: HTTP status code is not handled or not allowed 2026-01-24 22:09:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/appliances/refrigerators_freezers/refrigerators/undercounter_refrigerators>: HTTP status code is not handled or not allowed 2026-01-24 22:09:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bg-boring-gimlet-steel>: HTTP status code is not handled or not allowed 2026-01-24 22:09:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1234087-trigger-x-adjustable-rope-positioning-lanyard-10-foot-long-single-mode-310-cap-lbs>: HTTP status code is not handled or not allowed 2026-01-24 22:09:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1234090-trigger-x-adjustable-rope-positioning-lanyard-6-foot-long-single-mode-310-cap-lbs>: HTTP status code is not handled or not allowed 2026-01-24 22:09:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smart-lock-self-retracting-lifeline-20-foot-galvanized-cable-blue-3503803>: HTTP status code is not handled or not allowed 2026-01-24 22:09:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-jumper-cables-1-0-awg-15-ft-black-red>: HTTP status code is not handled or not allowed 2026-01-24 22:09:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/l-3hdb-3-heavy-duty-saw-blade-w-l-connector>: HTTP status code is not handled or not allowed 2026-01-24 22:09:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 28822 pages (at 94 pages/min), scraped 14111 items (at 15 items/min) 2026-01-24 22:09:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/l-shd-spear-head-w-l-connector>: HTTP status code is not handled or not allowed 2026-01-24 22:09:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rtr-2-large-corkscrew-retrieving-tool>: HTTP status code is not handled or not allowed 2026-01-24 22:09:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-vinyl-sign-caution-keep-aisles-clear-1>: HTTP status code is not handled or not allowed 2026-01-24 22:09:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thumb-dressing-forceps-5-l-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 22:09:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-vinyl-sign-caution-hard-hat-area-2>: HTTP status code is not handled or not allowed 2026-01-24 22:09:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kelly-hemostatic-forceps-straight-5-1-2-l-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 22:09:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-vinyl-sign-caution-no-smoking-beyond-this-point>: HTTP status code is not handled or not allowed 2026-01-24 22:09:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-3-tier-esd-14w-x-36l-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 22:09:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/660s-12-light-duty-f-clamp-12>: HTTP status code is not handled or not allowed 2026-01-24 22:09:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sk2000-boltless-pallet-rack-8-inch-wall-ties>: HTTP status code is not handled or not allowed 2026-01-24 22:09:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sk3000-structural-channel-pallet-rack-3-inch-x-48-inch-x-120-inch-upright>: HTTP status code is not handled or not allowed 2026-01-24 22:09:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-battery-cable-with-5-16-ring-terminals-2-awg-1-ft-red>: HTTP status code is not handled or not allowed 2026-01-24 22:09:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caution-sign-10x7-rigid-plastic-safety-shoes-required>: HTTP status code is not handled or not allowed 2026-01-24 22:09:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/danger-sign-14x10-rigid-plastic-do-not-watch-the-arc>: HTTP status code is not handled or not allowed 2026-01-24 22:09:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tdb3108-terreon-8-person-54-semi-circular-8-1-2-deep-bowl-washfountain-off-line-vent>: HTTP status code is not handled or not allowed 2026-01-24 22:09:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-step-heavy-duty-mobile-work-platform-handrails-24-x-60-platform>: HTTP status code is not handled or not allowed 2026-01-24 22:09:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oty-38530-quadtro-washing-machine-outlet-box-14-turn-brass-ball-valve-copper-sweat>: HTTP status code is not handled or not allowed 2026-01-24 22:09:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flexbollard-52h-x-7-14w-w99h-signpost-asphalt-blackwhite-xl-b528-a-blackwhite>: HTTP status code is not handled or not allowed 2026-01-24 22:09:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-battery-cable-with-5-16-ring-terminals-4-0-awg-1-ft-black>: HTTP status code is not handled or not allowed 2026-01-24 22:09:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/divided-quad-glove-box-dispenser-5-5-8-w-x-3-13-16-d-x-9-15-16-h-white>: HTTP status code is not handled or not allowed 2026-01-24 22:09:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-stacking-chair-black>: HTTP status code is not handled or not allowed 2026-01-24 22:09:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oty-30919-abs-extra-special-black-cement-32-oz>: HTTP status code is not handled or not allowed 2026-01-24 22:09:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-trash-container-garbage-can-55-gallon-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:09:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/danger-sign-14x10-vinyl-ear-protection-required>: HTTP status code is not handled or not allowed 2026-01-24 22:09:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-trash-container-garbage-can-55-gallon-white>: HTTP status code is not handled or not allowed 2026-01-24 22:09:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-glove-box-dispenser-22-1-4-w-x-5-d-x-21-15-16-h-quartz>: HTTP status code is not handled or not allowed 2026-01-24 22:09:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ch-271063-6-long-test-ball-plug-13-psi-30-ft>: HTTP status code is not handled or not allowed 2026-01-24 22:09:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vital-cluster-box-unit-16-mailboxes-2-parcel-lockers-black>: HTTP status code is not handled or not allowed 2026-01-24 22:09:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oty-31193-heavy-duty-wax-bowl-ring-with-sleeve>: HTTP status code is not handled or not allowed 2026-01-24 22:09:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/protective-wear-organizer-22-5-16-w-x-3-13-16-d-x-10-h>: HTTP status code is not handled or not allowed 2026-01-24 22:09:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vital-cluster-box-unit-w-vogue-traditional-accessories-8-unit-2-parcel-lockers-dark-bronze>: HTTP status code is not handled or not allowed 2026-01-24 22:09:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/notice-sign-14x10-rigid-plastic-no-parking>: HTTP status code is not handled or not allowed 2026-01-24 22:09:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cover-your-cough-compliance-kit-9-1-16-w-x-4-1-16-d-x-7-1-2-h>: HTTP status code is not handled or not allowed 2026-01-24 22:09:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oty-30601-pipegasket-lubricant-1-gallon>: HTTP status code is not handled or not allowed 2026-01-24 22:09:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40170-s-cond-hd-qc-satin-disc-type-r-2-aluminum-oxide-medium>: HTTP status code is not handled or not allowed 2026-01-24 22:09:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/phaseii-pht-980-shore-d-durometer>: HTTP status code is not handled or not allowed 2026-01-24 22:09:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/parking-lot-starter-kit-stencil-1-8-thick-polytough-plastic-white>: HTTP status code is not handled or not allowed 2026-01-24 22:09:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-shelf-truck-esd-3-shelf-21w-x-54l-x-40h-polyurethane-4-swivel-2-brake-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:09:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/28967-flap-mop-mandrel-2-x-1-x-1-4-aluminum-oxide-fine>: HTTP status code is not handled or not allowed 2026-01-24 22:09:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8x12x4-1-2-stackable-bin-boxes>: HTTP status code is not handled or not allowed 2026-01-24 22:09:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-shelf-truck-esd-3-shelf-18w-x-60l-x-40h-polyurethane-4-swivel-2-brake-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:09:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-shelf-cart-esd-2-shelf-24w-x-42l-polyurethane-4-swivel-2-brake-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:09:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-inch-x-4-inch-x-4-inch-long-corrugated-boxes>: HTTP status code is not handled or not allowed 2026-01-24 22:09:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-inch-x-11-inch-x-8-inch-corrugated-cartons>: HTTP status code is not handled or not allowed 2026-01-24 22:09:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-esd-24w-x-30l-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 22:09:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-2-tier-esd-12w-x-36l-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 22:09:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-esd-21w-x-60l-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 22:09:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss102-sureseal-fire-waterproof-chest-12-3-16w-x-9-13-16d-x-7-5-16h-015-cuft>: HTTP status code is not handled or not allowed 2026-01-24 22:09:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-4-tier-esd-12w-x-48l-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 22:09:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-esd-21w-x-54l-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 22:09:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cloth-belt-82574-1-2-24-60-grit-ceramic>: HTTP status code is not handled or not allowed 2026-01-24 22:09:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/goffs-welding-screen-6w-x-6h-grey>: HTTP status code is not handled or not allowed 2026-01-24 22:09:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plant-marking-stencil-20x20-electrical-panel>: HTTP status code is not handled or not allowed 2026-01-24 22:09:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-3-tier-esd-18w-x-48l-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 22:09:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-flexible-expandable-conveyor-steel-skate-wheels-175-lbs-per-foot>: HTTP status code is not handled or not allowed 2026-01-24 22:09:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-4-tier-esd-24w-x-60l-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 22:09:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mw-48406-display-base-pallet-solid-top-end-cap-48w-x-40d-x-6h-black>: HTTP status code is not handled or not allowed 2026-01-24 22:09:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/martin-yale-1711-electronic-autofolder-paper-folder>: HTTP status code is not handled or not allowed 2026-01-24 22:09:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ring-aramid-flange-gasket-for-4-pipe-116-thick-class-150>: HTTP status code is not handled or not allowed 2026-01-24 22:09:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/individual-character-stencil-24-number-5>: HTTP status code is not handled or not allowed 2026-01-24 22:09:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-40-x-military-pan-head-machine-screws-MS51957-12>: HTTP status code is not handled or not allowed 2026-01-24 22:09:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-freestanding-office-partition-panel-wpartial-window-36-14-w-x-73-12-h-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:09:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rust-oleum-industrial-choice-1600-sys-ge-purp-enamel-aero-ultr-flat-bk>: HTTP status code is not handled or not allowed 2026-01-24 22:10:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clean-room-fda-viton-sanitary-gasket-for-4-tube>: HTTP status code is not handled or not allowed 2026-01-24 22:10:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hp-everyday-pigment-ink-photo-paper-roll-q8916a-24-x-100-glossy-white-1-roll>: HTTP status code is not handled or not allowed 2026-01-24 22:10:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/individual-character-stencil-36-letter-j>: HTTP status code is not handled or not allowed 2026-01-24 22:10:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/neenah-paper-classic-laid-stationery-writing-paper-6571-8-12-x-11-solar-white-500-sheetsream>: HTTP status code is not handled or not allowed 2026-01-24 22:10:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-4-tier-esd-36w-x-72l-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 22:10:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-esd-36w-x-36l-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 22:10:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-link-t-link-hard-hat-tychem-2000-hood-hx5-backpack-papr>: HTTP status code is not handled or not allowed 2026-01-24 22:10:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/neoprene-foam-no-adhesive-14-thick-x-12-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 22:10:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rust-oleum-srra-perf-metlmx-0-voc-dtm-acryl-enam-semi-gls-dp-bse-5-gal>: HTTP status code is not handled or not allowed 2026-01-24 22:10:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-visi-sign-fire-hose-3>: HTTP status code is not handled or not allowed 2026-01-24 22:10:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 28923 pages (at 101 pages/min), scraped 14139 items (at 28 items/min) 2026-01-24 22:10:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-mobile-office-partition-panel-36-14-w-x-96-h-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:10:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-4-tier-esd-36w-x-36l-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 22:10:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/notebooks_pads already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:10:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-safety-sign-fire-alarm-plastic-4>: HTTP status code is not handled or not allowed 2026-01-24 22:10:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-esd-21w-x-60l-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 22:10:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tow-drop-spiker-seeder-spreader-100-lbs-36-inch-l-x-42-inch-w-x-20-inch-h-black>: HTTP status code is not handled or not allowed 2026-01-24 22:10:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tun5001-tire-tube-replacement-size-22-x-11-12-tr6>: HTTP status code is not handled or not allowed 2026-01-24 22:10:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-safety-sign-fire-extinguisher-plastic-5>: HTTP status code is not handled or not allowed 2026-01-24 22:10:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/entrance-mat-3-8-thick-2-l-x-3-w-black-smooth-backing>: HTTP status code is not handled or not allowed 2026-01-24 22:10:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jdc-510-jet-1500-cfm-industrial-dust-collector-3hp-220v-3phase>: HTTP status code is not handled or not allowed 2026-01-24 22:10:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/easel-pads-self-stick-easel-pads-560vad4pk-25-x-30-white-30-sheets-4pack>: HTTP status code is not handled or not allowed 2026-01-24 22:10:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rule-index-cards-7321gre-3-x-5-green-100pack>: HTTP status code is not handled or not allowed 2026-01-24 22:10:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10x1-1-2-flat-head-self-drilling-screws-1024KPU>: HTTP status code is not handled or not allowed 2026-01-24 22:10:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/paper_stationery/notebooks_pads>: HTTP status code is not handled or not allowed 2026-01-24 22:10:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bedford-tufted-upholstered-queen-size-headboard-in-camel>: HTTP status code is not handled or not allowed 2026-01-24 22:10:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tribeca-king-size-tufted-upholstered-platform-bed-in-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:10:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/webbed-barrier-tape-magenta-yellow-2-w-1>: HTTP status code is not handled or not allowed 2026-01-24 22:10:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4x3-8-3hd-pan-head-high-low-screws-0406HTP>: HTTP status code is not handled or not allowed 2026-01-24 22:10:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/full-face-viton-flange-gasket-for-2-pipe-18-thick-class-150>: HTTP status code is not handled or not allowed 2026-01-24 22:10:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/calendars_planners/planners_appointment_books already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:10:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-evaporator-filter-for-portable-air-conditioner-with-heat-293164>: HTTP status code is not handled or not allowed 2026-01-24 22:10:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rm40-p-industrial-dehumidifier>: HTTP status code is not handled or not allowed 2026-01-24 22:10:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/riverdale-king-tufted-upholstered-platform-bed-light-gray-memory-foam-mattress>: HTTP status code is not handled or not allowed 2026-01-24 22:10:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/soft-silicone-o-ring-dash-119-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-24 22:10:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8x3-4-6hd-pan-head-high-low-screws-0812HPPBZ>: HTTP status code is not handled or not allowed 2026-01-24 22:10:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/comfortable-sleep-twin-12-foam-pocket-spring-mattress-2-gel-memory-foam-topper>: HTTP status code is not handled or not allowed 2026-01-24 22:10:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buna-n-o-ring-dash-356-pack-of-5>: HTTP status code is not handled or not allowed 2026-01-24 22:10:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m5-0-8x5-metric-socket-set-screws-cup-point-M5005SSC>: HTTP status code is not handled or not allowed 2026-01-24 22:10:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stacking-ventilation-tray-with-drop-sides-30-3-8-x-15-7-8-x-3-5-8-white>: HTTP status code is not handled or not allowed 2026-01-24 22:10:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ingersoll-rand-air-compressor-2545e10v-value-230v-3ph-60hz>: HTTP status code is not handled or not allowed 2026-01-24 22:10:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-24x5-8-button-head-socket-cap-screws-1010CSB>: HTTP status code is not handled or not allowed 2026-01-24 22:10:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/printed-barricade-tape-danger-peligro-2>: HTTP status code is not handled or not allowed 2026-01-24 22:10:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/calendars_planners/planners_appointment_books>: HTTP status code is not handled or not allowed 2026-01-24 22:10:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/printed-barricade-tape-caution-do-not-enter-2>: HTTP status code is not handled or not allowed 2026-01-24 22:10:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-20x2-1-2-hanger-bolt-fully-threaded-1440BH188>: HTTP status code is not handled or not allowed 2026-01-24 22:10:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-message-dater-4-messages-red-blue-1-3-4-x-15-16>: HTTP status code is not handled or not allowed 2026-01-24 22:10:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-fuel-ventless-gas-fireplace-w-mantel-26000-btu-remote-control-walnut>: HTTP status code is not handled or not allowed 2026-01-24 22:10:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-32x2-3-8-round-head-machine-screws-0838MSR>: HTTP status code is not handled or not allowed 2026-01-24 22:10:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lux-lspp3b-o5x-ls-wick-b-eye-polo-org-5x>: HTTP status code is not handled or not allowed 2026-01-24 22:10:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-28x3-8-socket-set-screws-half-dog-point-1506SSD>: HTTP status code is not handled or not allowed 2026-01-24 22:10:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triple-wall-mount-soap-pumps-640266>: HTTP status code is not handled or not allowed 2026-01-24 22:10:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-corner-desk-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:10:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/divider-kit-for-10-h-drawer-of-modular-drawer-cabinet-30-wx27-d-black>: HTTP status code is not handled or not allowed 2026-01-24 22:10:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-2-11110-30-psi-18-lm-plastic>: HTTP status code is not handled or not allowed 2026-01-24 22:10:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-retardant-silicone-foam-with-high-temp-adhesive-14-thick-x-1w-x-10l>: HTTP status code is not handled or not allowed 2026-01-24 22:10:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/activity-table-w-adjustable-legs-24-x48-black-frame-with-oak-top>: HTTP status code is not handled or not allowed 2026-01-24 22:10:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-15-11116-160-psi-18cbm-with-u-plastic>: HTTP status code is not handled or not allowed 2026-01-24 22:10:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quick-adjust-stabilizing-trailer-jack-stand-100000-lb-static-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:10:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-x-36-mobile-plastic-square-edge-work-bench-adjustable-height-1-5-8-top>: HTTP status code is not handled or not allowed 2026-01-24 22:10:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-2-11110-60-psikpa-14-lm-plastic>: HTTP status code is not handled or not allowed 2026-01-24 22:10:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3d-glow-sign-acrylic-first-aid>: HTTP status code is not handled or not allowed 2026-01-24 22:10:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-8-extra-hollow-wall-anchors-1805AWHXL>: HTTP status code is not handled or not allowed 2026-01-24 22:10:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glow-danger-rigid-plastic-no-smoking>: HTTP status code is not handled or not allowed 2026-01-24 22:10:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glow-nyc-sign-forward>: HTTP status code is not handled or not allowed 2026-01-24 22:10:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8x1-5-8-bugle-head-drywall-screws-0826HPG>: HTTP status code is not handled or not allowed 2026-01-24 22:10:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sharp-mechanical-pencil-1-3-mm-hb-2-5-black-lead-blue-barrel>: HTTP status code is not handled or not allowed 2026-01-24 22:10:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glow-nyc-directional-sign-le>: HTTP status code is not handled or not allowed 2026-01-24 22:10:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-40x9-16-hex-standoffs-140904HFN>: HTTP status code is not handled or not allowed 2026-01-24 22:10:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96-x-48-flat-top-pallet-4-way-entry>: HTTP status code is not handled or not allowed 2026-01-24 22:10:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grade-f1-felt-sheet-14-thick-x-12w-x-12l>: HTTP status code is not handled or not allowed 2026-01-24 22:10:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-x-42-office-partition-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:10:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/non-illuminated-selector-22mm-black-d-cam-p9m-smd5n>: HTTP status code is not handled or not allowed 2026-01-24 22:10:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/self_luminous_emergency_exit_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:10:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/icy-mechanical-pencil-0-7-mm-hb-2-5-black-lead-blue-red-white-barrel-dozen>: HTTP status code is not handled or not allowed 2026-01-24 22:10:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/protect-all-surface-protectant-1-gallon-4-bottle>: HTTP status code is not handled or not allowed 2026-01-24 22:10:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/finito-stick-porous-point-pen-extra-fine-0-4mm-blue-ink-blue-silver-barrel-dozen>: HTTP status code is not handled or not allowed 2026-01-24 22:11:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/non-illuminated-push-button-operator-22mm-black-extended-style>: HTTP status code is not handled or not allowed 2026-01-24 22:11:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g2-mechanical-pencil-0-7-mm-hb-2-5-black-lead-assorted-barrel-colors-5-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:11:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-vaccine-storage-cart-with-24-cu-ft-refrigerator-3-drawer>: HTTP status code is not handled or not allowed 2026-01-24 22:11:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/highlighters_markers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:11:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-six-drawer-anesthesia-cart-with-md30-ans3-accessory-package-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:11:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glow-floor-sign-emergency-shower>: HTTP status code is not handled or not allowed 2026-01-24 22:11:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6900-2rs-double-sealed-10mm-bore-22mm-od>: HTTP status code is not handled or not allowed 2026-01-24 22:11:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-epoxy-wire-shelving-14487b-48w-x-14d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 22:11:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/self_luminous_emergency_exit_signs>: HTTP status code is not handled or not allowed 2026-01-24 22:11:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/color_coding_labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:11:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glow-sign-vinyl-fire-exit>: HTTP status code is not handled or not allowed 2026-01-24 22:11:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/partition-t25-torx-bit-steel-hw101030>: HTTP status code is not handled or not allowed 2026-01-24 22:11:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/long-sleeve-t-with-perimeter-insect-guard-ansi-3-yellow-l-uhv867-l>: HTTP status code is not handled or not allowed 2026-01-24 22:11:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 29019 pages (at 96 pages/min), scraped 14161 items (at 22 items/min) 2026-01-24 22:11:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-four-drawer-procedure-cart-with-standard-key-lock-beige>: HTTP status code is not handled or not allowed 2026-01-24 22:11:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6306zz-double-shielded-30mm-bore-72mm-od>: HTTP status code is not handled or not allowed 2026-01-24 22:11:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-x-14-working-length-j-hook-style-b>: HTTP status code is not handled or not allowed 2026-01-24 22:11:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/highlighters_markers>: HTTP status code is not handled or not allowed 2026-01-24 22:11:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-six-drawer-anesthesia-cart-with-md30-ans3-accessory-package-purple>: HTTP status code is not handled or not allowed 2026-01-24 22:11:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-four-drawer-procedure-cart-with-standard-key-lock-yellow>: HTTP status code is not handled or not allowed 2026-01-24 22:11:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11-pocket-verticle-literature-rack-black>: HTTP status code is not handled or not allowed 2026-01-24 22:11:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/champion-fmps5411mc-fmpro-stainless-steel-54-wx20-1-4-dx43-1-8-h-11-drawer-roller-cabinet>: HTTP status code is not handled or not allowed 2026-01-24 22:11:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/self_luminous_emergency_exit_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:11:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-four-drawer-procedure-cart-with-pushbutton-e-lock-red>: HTTP status code is not handled or not allowed 2026-01-24 22:11:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6303zznr-double-shielded-wsnap-ring-17mm-bore-47mm-od>: HTTP status code is not handled or not allowed 2026-01-24 22:11:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/color_coding_labels landed on page that is not a product page. 2026-01-24 22:11:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14000-lbs-capacity-workbench-36x48x34>: HTTP status code is not handled or not allowed 2026-01-24 22:11:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6208-open-40mm-bore-80mm-od>: HTTP status code is not handled or not allowed 2026-01-24 22:11:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-red-extra-plastic-forkli-able-tilt-truck-1-cu-yard>: HTTP status code is not handled or not allowed 2026-01-24 22:11:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hooded-sweatshirt-wliner-ansi-class-3-2-slash-pockets-polar-fleece-orangeblack-xl>: HTTP status code is not handled or not allowed 2026-01-24 22:11:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-paper-wiper-192136>: HTTP status code is not handled or not allowed 2026-01-24 22:11:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/champion-fmp4111rc-rd-fmpro-41-w-x-20-d-x-42-1-2-h-11-drawer-red-roller-cabinet>: HTTP status code is not handled or not allowed 2026-01-24 22:11:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-profile-pourover-coffee-brewer-1-warmer-vp17-1-sst>: HTTP status code is not handled or not allowed 2026-01-24 22:11:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/champion-fmp4110tc-bl-fmpro-41-w-x-20-d-x-23-5-8-h-10-drawer-blue-chest>: HTTP status code is not handled or not allowed 2026-01-24 22:11:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/credenzas already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:11:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/self_luminous_emergency_exit_signs>: HTTP status code is not handled or not allowed 2026-01-24 22:11:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-short-emergency-crash-cart-md30-emg1-std-width-4-drawers-breakaway-lock-pink>: HTTP status code is not handled or not allowed 2026-01-24 22:11:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/first-aid-cabinet-3-shelf-15x5-9-16x16-5-32>: HTTP status code is not handled or not allowed 2026-01-24 22:11:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rmt-709-02-110-piece-accessory-kit-for-dremel>: HTTP status code is not handled or not allowed 2026-01-24 22:11:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dino-lite-am4515ztl-edge-microscope-1-3mp-10x140x>: HTTP status code is not handled or not allowed 2026-01-24 22:11:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-tall-emergency-crash-cart-6-drawers-36-3-4-w-x-26-l-x-43-h-teal>: HTTP status code is not handled or not allowed 2026-01-24 22:11:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-tall-emergency-crash-cart-6-drawers-36-3-4-w-x-26-l-x-43-h-taupe>: HTTP status code is not handled or not allowed 2026-01-24 22:11:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6212-2rsnr-double-sealed-wsnap-ring-60mm-bore-110mm-od>: HTTP status code is not handled or not allowed 2026-01-24 22:11:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/depository-safe-mb2720icxil-electronic-lock-24-1-2w-x-19d-x-27h-black>: HTTP status code is not handled or not allowed 2026-01-24 22:11:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-dia-x-14-working-length-j-hook-style-a>: HTTP status code is not handled or not allowed 2026-01-24 22:11:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-tall-emergency-crash-cart-6-drawers-36-3-4-w-x-26-l-x-43-h-pink>: HTTP status code is not handled or not allowed 2026-01-24 22:11:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/110-125-volt-start-capacitor-130-156-mfd-130b>: HTTP status code is not handled or not allowed 2026-01-24 22:11:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cold-protection-insulated-hi-vis-terry-glove-latex-coated-41-1420-l>: HTTP status code is not handled or not allowed 2026-01-24 22:11:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/330-volt-start-capacitor-72-88-mfd-72c>: HTTP status code is not handled or not allowed 2026-01-24 22:11:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/credenzas landed on page that is not a product page. 2026-01-24 22:11:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/parking-permit-red-rearview-401-500>: HTTP status code is not handled or not allowed 2026-01-24 22:11:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-voltage-370-440-oval-run-capacitor-25-mfd-25dv>: HTTP status code is not handled or not allowed 2026-01-24 22:11:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dot-flip-placard-combination-liquid-carrying-vehicles>: HTTP status code is not handled or not allowed 2026-01-24 22:11:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-feet-x-20-feet-sf-14-90-oz-flame-resist-canvas-tarp-olive-drab>: HTTP status code is not handled or not allowed 2026-01-24 22:11:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dot-placard-flammable-solid-2>: HTTP status code is not handled or not allowed 2026-01-24 22:11:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n-ferno-6466-thermal-jacket-black-3xl-41117>: HTTP status code is not handled or not allowed 2026-01-24 22:11:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8377-type-r-class-3-quilted-bomber-jacket-orange-xl-25615>: HTTP status code is not handled or not allowed 2026-01-24 22:11:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8377-type-r-class-3-quilted-bomber-jacket-orange-l-25614>: HTTP status code is not handled or not allowed 2026-01-24 22:11:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-spread-basket-lifting-beam-yellow-2000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:11:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-feet-x-10-feet-sf-14-90-oz-flame-resist-canvas-tarp-olive-drab>: HTTP status code is not handled or not allowed 2026-01-24 22:11:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cob-socks-312-x-3-25-gallon-capacity-40pack>: HTTP status code is not handled or not allowed 2026-01-24 22:11:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/squids-3118fx-tool-lanyard-dual-locking-carabiner-15-lbs-capacity-19828>: HTTP status code is not handled or not allowed 2026-01-24 22:11:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-tall-isolation-cart-4-drawers-key-lock-18-w-x-18-l-x-40-3-4-h-purple>: HTTP status code is not handled or not allowed 2026-01-24 22:11:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6315zznr-double-shielded-wsnap-ring-75mm-bore-160mm-od>: HTTP status code is not handled or not allowed 2026-01-24 22:11:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6015-2rsnr-double-sealed-wsnap-ring-75mm-bore-115mm-od>: HTTP status code is not handled or not allowed 2026-01-24 22:11:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6315-2rs-double-sealed-75mm-bore-160mm-od>: HTTP status code is not handled or not allowed 2026-01-24 22:11:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swf20-114-id-round-flange-type-linear-bearing-steel>: HTTP status code is not handled or not allowed 2026-01-24 22:11:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xeno-wet-56-inch-black-with-black-blades-and-led>: HTTP status code is not handled or not allowed 2026-01-24 22:11:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dot-placard-flammable-3-blank-2>: HTTP status code is not handled or not allowed 2026-01-24 22:11:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6211-2rs-double-sealed-55mm-bore-100mm-od>: HTTP status code is not handled or not allowed 2026-01-24 22:11:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6228-open-140mm-bore-250mm-od>: HTTP status code is not handled or not allowed 2026-01-24 22:11:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edge-protectors-2-x-2-x-48-160-thick-white-2240-skid>: HTTP status code is not handled or not allowed 2026-01-24 22:11:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-column-medical-storage-cart-glass-door-22-1-2-w-x-27-d-x-75-3-4-h-yellow>: HTTP status code is not handled or not allowed 2026-01-24 22:11:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tall-savary-dilator-drying-cart-hepa-filter-23-89-100-wx22-lx51-89-100-h-sand>: HTTP status code is not handled or not allowed 2026-01-24 22:11:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tall-savary-dilator-drying-cart-hepa-filter-23-89-100-wx22-lx51-89-100-h-white>: HTTP status code is not handled or not allowed 2026-01-24 22:11:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-dia-x-14-working-length-j-hook-style-b>: HTTP status code is not handled or not allowed 2026-01-24 22:11:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrosive-label-worded-paper-500-roll>: HTTP status code is not handled or not allowed 2026-01-24 22:11:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6011-2rs-double-sealed-55mm-bore-90mm-od>: HTTP status code is not handled or not allowed 2026-01-24 22:11:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/120-240v-5a-control-board-circuit-breaker>: HTTP status code is not handled or not allowed 2026-01-24 22:11:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-series-aluminium-isolation-cart-6-drawers-30-w-x-22-l-x-43-3-4-h-hammertone-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:11:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-hazcom-this-label-4-inch-x-4-inch-paper-500-roll>: HTTP status code is not handled or not allowed 2026-01-24 22:11:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/d181610chscfgwp-wifi-cab-type-4x-window-door-1800x1600x1000-fiberglass>: HTTP status code is not handled or not allowed 2026-01-24 22:11:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6926-open-130mm-bore-180mm-od>: HTTP status code is not handled or not allowed 2026-01-24 22:11:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blade-for-jet-horizontal-band-saws>: HTTP status code is not handled or not allowed 2026-01-24 22:11:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6912zz-double-shielded-60mm-bore-85mm-od>: HTTP status code is not handled or not allowed 2026-01-24 22:11:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ecwtd8b-velcroblack-velcrocable-wrap-w-m5-fastener-qty-10-8-in-velcro>: HTTP status code is not handled or not allowed 2026-01-24 22:12:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/111-optics-laser-infrared-thrmo-wthermocouple>: HTTP status code is not handled or not allowed 2026-01-24 22:12:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/miscellaneous-dangerous-goods-placard-un-3082-removable-vinyl-25-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:12:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-short-isolation-cart-5-drawers-e-lock-30-w-x-22-l-x-37-1-4-h-forest-green>: HTTP status code is not handled or not allowed 2026-01-24 22:12:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pthw162415g2-protek-dh-window-1709x2362x1496-type-nema-12>: HTTP status code is not handled or not allowed 2026-01-24 22:12:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drylin-n-17mm-miniature-guide-carriage>: HTTP status code is not handled or not allowed 2026-01-24 22:12:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/light-sounder-combination-flashing-24vdc-amber-av1-024a>: HTTP status code is not handled or not allowed 2026-01-24 22:12:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vynckier-vm433-vm-4-inch-x-3-inch-x-3-inch-non-metallic-enclosure-opaque-cover>: HTTP status code is not handled or not allowed 2026-01-24 22:12:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/strobe-light-male-pipe-mount-12-48vdc-amber-lp3m-012-048a>: HTTP status code is not handled or not allowed 2026-01-24 22:12:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-column-medical-storage-cart-glass-door-22-1-2-w-x-27-d-x-75-3-4-h-sand>: HTTP status code is not handled or not allowed 2026-01-24 22:12:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/utility-scrub-brush-20in-40501c05>: HTTP status code is not handled or not allowed 2026-01-24 22:12:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lsxr-610-fixture-mount-interchangeable-lens-sensor-low-high-bay>: HTTP status code is not handled or not allowed 2026-01-24 22:12:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-narrow-procedure-cart-6-drawers-key-lock-18-w-x-18-l-x-40-3-4-h-light-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:12:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-x-1-4-iglide-g300-polymer-flange-bearing>: HTTP status code is not handled or not allowed 2026-01-24 22:12:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-x-1-4-iglide-g300-polymer-sleeve-bearing>: HTTP status code is not handled or not allowed 2026-01-24 22:12:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 29117 pages (at 98 pages/min), scraped 14174 items (at 13 items/min) 2026-01-24 22:12:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/q251815abe-qline-j-box-screw-cover-type-4x-250x175x149mm-abs>: HTTP status code is not handled or not allowed 2026-01-24 22:12:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oil-transfer-pump-extension-tube-30-with-diameter-of-42mm>: HTTP status code is not handled or not allowed 2026-01-24 22:12:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/credenzas already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:12:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-shipper-ata-case-919-wheeled-case-foam-filled-28l-x-25w-x-17h-silver>: HTTP status code is not handled or not allowed 2026-01-24 22:12:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-shipper-ata-case-919-wheeled-case-30l-x-19w-x-15h-silver>: HTTP status code is not handled or not allowed 2026-01-24 22:12:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/300-181-l-low-retention-pipette-tip-reload-system-non-sterile-1920-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:12:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/boss-reception-desk-71w-x-30d-x-42h-mahogany>: HTTP status code is not handled or not allowed 2026-01-24 22:12:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cap-insert-for-cf-cryogenic-vials-assorted-colors-non-sterile-polypropylene-500-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:12:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/500mm-drylin-n-27mm-miniature-guide-rail>: HTTP status code is not handled or not allowed 2026-01-24 22:12:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/facility-flange-sign-first-aid>: HTTP status code is not handled or not allowed 2026-01-24 22:12:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-spill-containment-cabinet-large-12w-x-14d-x-28h-aggressive>: HTTP status code is not handled or not allowed 2026-01-24 22:12:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-spill-containment-cabinet-large-12w-x-14d-x-28h-oil-only>: HTTP status code is not handled or not allowed 2026-01-24 22:12:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cell-lifter-double-end-flat-blade-j-hook-sterile-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:12:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labor-law-poster-new-york-spanish>: HTTP status code is not handled or not allowed 2026-01-24 22:12:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pack-cylinder-rack>: HTTP status code is not handled or not allowed 2026-01-24 22:12:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/credenzas>: HTTP status code is not handled or not allowed 2026-01-24 22:12:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labor-law-poster-west-virginia>: HTTP status code is not handled or not allowed 2026-01-24 22:12:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-16-carbon-steel-heavy-wall-insert-for-metal-335-6>: HTTP status code is not handled or not allowed 2026-01-24 22:12:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rugged-transit-case-foam-lined-852-carrying-case-28l-x-22w-x-14h-black>: HTTP status code is not handled or not allowed 2026-01-24 22:12:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/permeable-cell-culture-insert-pkd-in-24-well-plate-hanging-pet-3-0-181-m-24pk>: HTTP status code is not handled or not allowed 2026-01-24 22:12:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-tall-procedure-cart-6-drawers-e-lock-36-3-4-w-x-22-l-x-43-3-4-h-hammertone-black>: HTTP status code is not handled or not allowed 2026-01-24 22:12:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/55ml-reagent-reservoir-polystyrene-white-individually-wrapped-sterile-80-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:12:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/boltless-heavy-duty-die-rack-72-w-x-36-d-x-96-h-5-shelves-medium-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:12:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1161430-comfort-vest-style-climbing-harness-tongue-buckle-pass-through-m-l>: HTTP status code is not handled or not allowed 2026-01-24 22:12:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1161441-comfort-vest-style-positioning-climbing-harness-tongue-buckle-quick-connect-xl>: HTTP status code is not handled or not allowed 2026-01-24 22:12:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1161440-comfort-vest-style-positioning-climbing-harness-tongue-buckle-quick-connect-m-l>: HTTP status code is not handled or not allowed 2026-01-24 22:12:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15ml-centrifuge-tube-plastic-rack-sterile-polypropylene-300-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:12:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-tall-isolation-cart-4-drawers-37-1-2-w-x-22-l-x-42-63-100-h-mauve>: HTTP status code is not handled or not allowed 2026-01-24 22:12:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m6-10-insert-for-soft-wood-flush-800610-13>: HTTP status code is not handled or not allowed 2026-01-24 22:12:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/278-expo-ii-telescoping-shipping-case-trade-show-case-51-1-2l-x-31w-x-9-1-2h-black>: HTTP status code is not handled or not allowed 2026-01-24 22:12:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/850ml-suspension-culture-flask-vent-cap-sterile-18-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:12:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/engraved-occupancy-sign-conference-room-use-vacant-red>: HTTP status code is not handled or not allowed 2026-01-24 22:12:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-16-18-carbon-steel-insert-for-metal-329-918>: HTTP status code is not handled or not allowed 2026-01-24 22:12:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1161513-vest-style-positioning-climbing-retrieval-harness-pass-through-s>: HTTP status code is not handled or not allowed 2026-01-24 22:12:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-height-triple-column-medical-storage-cabinet-60-2-5-wx27-4-5-dx55-3-5-h-brushed-silver>: HTTP status code is not handled or not allowed 2026-01-24 22:12:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-poly-drum-funnel>: HTTP status code is not handled or not allowed 2026-01-24 22:12:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/envirosalv-95-gallon-poly-overpack>: HTTP status code is not handled or not allowed 2026-01-24 22:12:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/telescoping-case-276-carrying-case-42l-x-20w-x-12h-black>: HTTP status code is not handled or not allowed 2026-01-24 22:12:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-32-tapered-ultrasonic-insert-single-vane-th-008-sv>: HTTP status code is not handled or not allowed 2026-01-24 22:12:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/floor-sign-caution-chemical-spill-keep-out>: HTTP status code is not handled or not allowed 2026-01-24 22:12:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1161553-vest-style-climbing-harness-pass-through-s>: HTTP status code is not handled or not allowed 2026-01-24 22:12:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m10-15-thin-wall-insert-for-metal-319-m10>: HTTP status code is not handled or not allowed 2026-01-24 22:12:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-cart-standard-width-tall-cabinet-6-drawers-41-2-5-wx22-lx64-1-10-h-hammertone-black>: HTTP status code is not handled or not allowed 2026-01-24 22:12:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-32-tapered-ultrasonic-insert-double-vane-dv-132-th>: HTTP status code is not handled or not allowed 2026-01-24 22:12:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/suredry-high-volume-16-scope-cabinet-dri-scope-aid-key-lock-44-wx24-dx93-h-navy>: HTTP status code is not handled or not allowed 2026-01-24 22:12:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quantum-divider-dus970-hulk-stacking-bins-qus970-8-1-4-x-29-7-8-x-7-price-per-package-of-6>: HTTP status code is not handled or not allowed 2026-01-24 22:12:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-56-flush-press-insert-brass-240-002-br>: HTTP status code is not handled or not allowed 2026-01-24 22:12:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_repair_maintenance/horns_horn_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:12:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/engraved-sign-women-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:12:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/engraved-sign-women-brown>: HTTP status code is not handled or not allowed 2026-01-24 22:12:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15300-dynafile-iii-abrasive-belt-tool-7hp-7-degree-offset-20000-rpm-front-exhaust>: HTTP status code is not handled or not allowed 2026-01-24 22:12:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mini-warning-light-super-bright-leds-amber-lens-3035-mp-a>: HTTP status code is not handled or not allowed 2026-01-24 22:12:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-profile-48-light-bar-12-volt-clear-lens-blue-leds-7825-b>: HTTP status code is not handled or not allowed 2026-01-24 22:12:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-glass-door-vaccine-refrigerator-nsf-certified-23-cf>: HTTP status code is not handled or not allowed 2026-01-24 22:12:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thermoplastic-tape-2-width-x-9-ft-length-black-1-roll>: HTTP status code is not handled or not allowed 2026-01-24 22:12:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/think-osha-7x10-vinyl-be-alert-don-t-get-hurt>: HTTP status code is not handled or not allowed 2026-01-24 22:12:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-7-8-pneumatic-stick-stapler>: HTTP status code is not handled or not allowed 2026-01-24 22:12:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ws201608ss-watershedwall-mt-encl-type-4x-2000x1600x800in>: HTTP status code is not handled or not allowed 2026-01-24 22:12:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-heavy-duty-14-ga-communications-locker-42-w-x-24-d-x-78-h-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:12:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-18-ga-ventilated-cabinet-w-3-shelves-dual-locking-48-w-x-24-d-x-72-h>: HTTP status code is not handled or not allowed 2026-01-24 22:12:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-20-316-stainless-insert-for-metal-316-4>: HTTP status code is not handled or not allowed 2026-01-24 22:12:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_repair_maintenance/horns_horn_accessories>: HTTP status code is not handled or not allowed 2026-01-24 22:12:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precut-strips-thin-18-x-1-5-x-1-12-gold-10-pieces-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:12:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/think-osha-10x14-plastic-don-t-try-to-li-more>: HTTP status code is not handled or not allowed 2026-01-24 22:12:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thhn-12-gauge-building-wire-stranded-type-black-500-ft>: HTTP status code is not handled or not allowed 2026-01-24 22:12:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylon-plastic-adjustable-lever-steel-38-16-157-stud-307l>: HTTP status code is not handled or not allowed 2026-01-24 22:12:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-acting-pneumatic-actuator-851-in-lbs-80psi>: HTTP status code is not handled or not allowed 2026-01-24 22:12:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/graphic-machine-labels-danger-p-points>: HTTP status code is not handled or not allowed 2026-01-24 22:12:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbi-sala-lad-saf-50-ft-vertical-safety-system-stainless-steel-cable-6119050>: HTTP status code is not handled or not allowed 2026-01-24 22:12:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/surface-mounted-end-wall-90-degrees-corners-2-ft-ft-wings-12-ft-h-w-caps-6-inch-w-chinchilla>: HTTP status code is not handled or not allowed 2026-01-24 22:13:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-condenser-for-nexel-models-243035-243036>: HTTP status code is not handled or not allowed 2026-01-24 22:13:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-54-watertight-limit-switch-18-1-gear-ratio-w-3-contact-blocks>: HTTP status code is not handled or not allowed 2026-01-24 22:13:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-evaporator-for-nexel-model-243006>: HTTP status code is not handled or not allowed 2026-01-24 22:13:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-cantilever-rack-double-sided-starter-md-48w-x54d-x-8-h-w-lip>: HTTP status code is not handled or not allowed 2026-01-24 22:13:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-front-grille-for-nexel-models-243035-243036>: HTTP status code is not handled or not allowed 2026-01-24 22:13:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lithonia-ibe-15lm-mvolt-40k-led-linear-high-bay-107w-13966-lumens-4000k-dimmable-dlc-premium>: HTTP status code is not handled or not allowed 2026-01-24 22:13:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/soft-epdm-foam-strip-no-adhesive-14-thick-x-1-wide-x-10-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 22:13:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/halseytaylor-bi-level-hydroboost-water-refilling-station-w-filter-stainless-hthb-hac8bl-wf-ss>: HTTP status code is not handled or not allowed 2026-01-24 22:13:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pictorial-osha-sign-plastic-eye-wash-fountain>: HTTP status code is not handled or not allowed 2026-01-24 22:13:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pictorial-osha-sign-vinyl-no-smoking>: HTTP status code is not handled or not allowed 2026-01-24 22:13:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/complete-key-lock-device-kit>: HTTP status code is not handled or not allowed 2026-01-24 22:13:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trl10266cf-nitro-elite-cf-carbon-fiber-rollator-rolling-walker-black>: HTTP status code is not handled or not allowed 2026-01-24 22:13:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vertical-lifting-locking-screw-clamp-orange-steel-3000-lbs-capacity-1-1-4-opening>: HTTP status code is not handled or not allowed 2026-01-24 22:13:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 29213 pages (at 96 pages/min), scraped 14190 items (at 16 items/min) 2026-01-24 22:13:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/american-lock-solid-steel-rectangular-padlock-no-a5102>: HTTP status code is not handled or not allowed 2026-01-24 22:13:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/american-lock-solid-steel-round-body-padlock-no-a700>: HTTP status code is not handled or not allowed 2026-01-24 22:13:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-342-dn-7-4-nipple-w-1-4-male-npt>: HTTP status code is not handled or not allowed 2026-01-24 22:13:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-2-pc-ss-npt-ball-valve-w-da-pneumatic>: HTTP status code is not handled or not allowed 2026-01-24 22:13:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/american-lock-solid-aluminum-rectangular-padlock-red-no-a1267red>: HTTP status code is not handled or not allowed 2026-01-24 22:13:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-lab-coat-white-polycombed-cotton-tall-44>: HTTP status code is not handled or not allowed 2026-01-24 22:13:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-467-dn6-coupling-w-valve-2-41-100-l>: HTTP status code is not handled or not allowed 2026-01-24 22:13:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-3-pc-ss-sanitary-clamp-end-ball-valve-w-sr-pneumatic>: HTTP status code is not handled or not allowed 2026-01-24 22:13:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/restricted-area-plastic-authorized-employees-only>: HTTP status code is not handled or not allowed 2026-01-24 22:13:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vertical-pallet-lifter-fixed-fork-1-ton-capacity-42-fork-length-yellow>: HTTP status code is not handled or not allowed 2026-01-24 22:13:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/american-lock-solid-steel-rectangular-padlock-no-a5201>: HTTP status code is not handled or not allowed 2026-01-24 22:13:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unisex-esdanti-static-counter-jacket-white-polyesternylon-m>: HTTP status code is not handled or not allowed 2026-01-24 22:13:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-sheet-332-thick-x-18-wide-x-36-long>: HTTP status code is not handled or not allowed 2026-01-24 22:13:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unistrut-1-58-flat-plate-fitting-p1064eg-1-hole-electro-galvanized-12>: HTTP status code is not handled or not allowed 2026-01-24 22:13:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-light-13-flush-mount-alabaster-glass-brushed-nickel>: HTTP status code is not handled or not allowed 2026-01-24 22:13:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/packaging/strapping_supplies/strapping_tools already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:13:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/euro-rackable-plastic-pallet-with-3-bottom-skids-48x32-2200-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-24 22:13:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unistrut-1-58-u-shape-fitting-p1377eg-4-hole>: HTTP status code is not handled or not allowed 2026-01-24 22:13:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/composites/composite_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:13:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unistrut-1-58-general-fitting-p1036eg-3-hole-electro-galvanized>: HTTP status code is not handled or not allowed 2026-01-24 22:13:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-12-thick-x-34-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-24 22:13:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:13:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unistrut-1-58-wing-shape-fitting-p2227eg-electro-galvanized>: HTTP status code is not handled or not allowed 2026-01-24 22:13:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-for-pmwr-30-walkie-rider-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-24 22:13:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-tape-4-w-x-108-l-5-mil-solid-red-1-roll>: HTTP status code is not handled or not allowed 2026-01-24 22:13:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/frost-standard-toilet-tissue-holder-with-hood-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 22:13:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fork-riser-8000-lbs-cap-for-8300-8400-8500-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-24 22:13:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-ton-pintle-hook-ph30>: HTTP status code is not handled or not allowed 2026-01-24 22:13:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24x36x74-chrome-wire-shelving-with-13-24-d-hopper-bins-yellow>: HTTP status code is not handled or not allowed 2026-01-24 22:13:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30x36x74-chrome-wire-shelving-with-13-30-d-hopper-bins-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:13:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/packaging/strapping_supplies/strapping_tools>: HTTP status code is not handled or not allowed 2026-01-24 22:13:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/frost-toilet-seat-cover-dispenser-white>: HTTP status code is not handled or not allowed 2026-01-24 22:13:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-uhmw-polyethylene-plastic-sheet-18-thick-x-36-long-x-48-long>: HTTP status code is not handled or not allowed 2026-01-24 22:13:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/composites/composite_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 22:13:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/designer-stacking-arm-chair-desk-w-left-handed-tablet-charcoal-seat-back>: HTTP status code is not handled or not allowed 2026-01-24 22:13:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/handle-wiring-harness-for-mpb-040-e-b827-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-24 22:13:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:13:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits>: HTTP status code is not handled or not allowed 2026-01-24 22:13:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x36x74-chrome-wire-shelving-with-10-hopper-bins-red>: HTTP status code is not handled or not allowed 2026-01-24 22:13:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-x-96-rub-rail-040-thick-eggshell>: HTTP status code is not handled or not allowed 2026-01-24 22:13:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caster-wheel-assembly-for-6hbw30-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-24 22:13:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-high-security-combo-padlock-with-key-control>: HTTP status code is not handled or not allowed 2026-01-24 22:13:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/contact-adhesive-for-installation-of-wall-sheet-and-vinyl-corner-guards-1-gal-container>: HTTP status code is not handled or not allowed 2026-01-24 22:13:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ez-squeeze-153-40-sheet-hole-punch-silver-black-3-case>: HTTP status code is not handled or not allowed 2026-01-24 22:13:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24x36x74-chrome-wire-shelving-with-64-6-h-shelf-bins-yellow>: HTTP status code is not handled or not allowed 2026-01-24 22:13:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stac-a-shelf-locker-shelf-extra-tall-blue-6-case>: HTTP status code is not handled or not allowed 2026-01-24 22:13:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24x36x74-chrome-wire-shelving-with-32-6-h-shelf-bins-red>: HTTP status code is not handled or not allowed 2026-01-24 22:13:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-sheet-2-thick-x-8-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-24 22:13:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x36x74-chrome-wire-shelving-with-24-6-h-shelf-bins-red>: HTTP status code is not handled or not allowed 2026-01-24 22:13:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits>: HTTP status code is not handled or not allowed 2026-01-24 22:13:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tandem-pivot-weldment-for-8300-8400-8500-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-24 22:13:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acorn-washfountain-circular-8-stations-foot-operated>: HTTP status code is not handled or not allowed 2026-01-24 22:13:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/konnect-letter-tray-24-case-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:13:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-sheets-40-x-40-200-ect-32-white-5-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:13:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x36x74-chrome-wire-shelving-with-20-giant-stacking-bins-yellow>: HTTP status code is not handled or not allowed 2026-01-24 22:13:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/control-key-for-built-in-combo-locks>: HTTP status code is not handled or not allowed 2026-01-24 22:13:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/konnect-wide-storage-bin-24-case-white>: HTTP status code is not handled or not allowed 2026-01-24 22:13:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-x-96-counter-height-pedestal-table-grey-nebula-top-round-silver-base>: HTTP status code is not handled or not allowed 2026-01-24 22:13:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-general-security-combo-padlock-key-control-black-dial>: HTTP status code is not handled or not allowed 2026-01-24 22:13:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-masking-tape-1-1-2-x-60-yards-3-core>: HTTP status code is not handled or not allowed 2026-01-24 22:13:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-power-easy-gel-control-014-oz-clear>: HTTP status code is not handled or not allowed 2026-01-24 22:13:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-built-in-key-operated-manual-deadbolt-locking-lock-keyed-alike>: HTTP status code is not handled or not allowed 2026-01-24 22:13:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bushing-for-wp-2000-pallet-trucks-cr-055084-008>: HTTP status code is not handled or not allowed 2026-01-24 22:13:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-built-in-combination-lock-red-metal-dial-right-hinged>: HTTP status code is not handled or not allowed 2026-01-24 22:13:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-containment-berm-rapid-rise-model-6-x-6>: HTTP status code is not handled or not allowed 2026-01-24 22:13:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-containment-berm-agriculture-economy-model-8-x-8-x-20>: HTTP status code is not handled or not allowed 2026-01-24 22:13:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/solenoid-for-101-series-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-24 22:13:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wood-desk-tray-legal-size-walnut>: HTTP status code is not handled or not allowed 2026-01-24 22:13:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergonomic-adj-height-pallet-stand-with-handle-42-x-48-solid-deck>: HTTP status code is not handled or not allowed 2026-01-24 22:14:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-portable-sink-hose-in-tank-waste-22-gauge-type-304-stainless-steel-4-satin-finish>: HTTP status code is not handled or not allowed 2026-01-24 22:14:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tap-10v-1-8-kw-point-of-use-tankless-electric-water-heater>: HTTP status code is not handled or not allowed 2026-01-24 22:14:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/recpro-half-rd-sat-alum-2-14-1-2-gal-rec-waste-mw2499sa-sa-b-c>: HTTP status code is not handled or not allowed 2026-01-24 22:14:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/work-height-platform-truck-with-6-deep-deck-24-x-48-adj-height-polyurethane>: HTTP status code is not handled or not allowed 2026-01-24 22:14:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3m-e-a-r-classic-earplugs-uncorded>: HTTP status code is not handled or not allowed 2026-01-24 22:14:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-w-lse-acrylic-adhesive-12-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 22:14:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/karcher-hard-main-roller-brush-km105-sweeper-4-762-529-0>: HTTP status code is not handled or not allowed 2026-01-24 22:14:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fr-clothing-insect-repellent-14036>: HTTP status code is not handled or not allowed 2026-01-24 22:14:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-ergonomic-utility-cart-large-black-4-shelf-700-capacity-lbs>: HTTP status code is not handled or not allowed 2026-01-24 22:14:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-hole-punch-9-32-inch-punch-size-with-12-sheet-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:14:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tumbler-dm-16oz-light-amber>: HTTP status code is not handled or not allowed 2026-01-24 22:14:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plate-dw-salad-7-1-4-slate-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:14:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/control-head-for-mpb-040-e-b827-mpw-045-e-b802-pallet-trucks-yl-524338448>: HTTP status code is not handled or not allowed 2026-01-24 22:14:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/koch-filter-merv-11-high-capacity-xl11-pleated-panel-extended-surface-24-inch-w-x-24-inch-h-x-1-inch-d>: HTTP status code is not handled or not allowed 2026-01-24 22:14:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 29309 pages (at 96 pages/min), scraped 14207 items (at 17 items/min) 2026-01-24 22:14:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/koch-filter-merv-8-standard-capacity-xl8-pleated-panel-ext-surface-25-inch-w-x-25-inch-h-x-2-inch-d>: HTTP status code is not handled or not allowed 2026-01-24 22:14:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-oz-coffee-cup-navy-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:14:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storage-tote-clear-t181-4-16l-x-12w-x-4h>: HTTP status code is not handled or not allowed 2026-01-24 22:14:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-ner-electric-chain-hoist-push-trolley-14-10-36>: HTTP status code is not handled or not allowed 2026-01-24 22:14:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-lb-lever-hoist-3-15-foot-lift>: HTTP status code is not handled or not allowed 2026-01-24 22:14:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/koch-filter-merv-11-standard-capacity-xl11-pleated-panel-ext-surface-16-inch-w-x-20-inch-h-x-1-inch-d>: HTTP status code is not handled or not allowed 2026-01-24 22:14:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/koch-filter-merv-8-standard-capacity-xl8-pleated-panel-ext-surface-16-inch-w-x-24-inch-h-x-2-inch-d>: HTTP status code is not handled or not allowed 2026-01-24 22:14:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xx-garolite-rod-516-diameter-x-1-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 22:14:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-ner-electric-chain-hoist-push-trolley-14-15-36>: HTTP status code is not handled or not allowed 2026-01-24 22:14:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caster-wheel-assembly-for-8300-8400-8500-pallet-trucks-ra-671-041-a>: HTTP status code is not handled or not allowed 2026-01-24 22:14:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:14:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/koch-filter-merv-8-high-capacity-xl8-pleated-extended-surface-20-inch-w-x-25-inch-h-x-1-inch-d>: HTTP status code is not handled or not allowed 2026-01-24 22:14:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hcp-50-humidity-chamber-115-volt-56-liters>: HTTP status code is not handled or not allowed 2026-01-24 22:14:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/tool_storage/chests_roller_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:14:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-sner-electric-chain-hoist-hook-suspension-14-20-14>: HTTP status code is not handled or not allowed 2026-01-24 22:14:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/koch-filter-merv-8-high-capacity-xl8-pleated-extended-surface-18-inch-w-x-24-inch-h-x-2-inch-d>: HTTP status code is not handled or not allowed 2026-01-24 22:14:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x42x75-chrome-shelving-with-24-magnum-giant-hopper-bins-black>: HTTP status code is not handled or not allowed 2026-01-24 22:14:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/37u-linier-server-cabinet-glass-glass-doors-36-depth>: HTTP status code is not handled or not allowed 2026-01-24 22:14:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-4-thick-x-6-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 22:14:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hygienic-hi-flex-wall-bracket-system-purple-polypropylene-tpe-rubber-polyamide>: HTTP status code is not handled or not allowed 2026-01-24 22:14:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nps-6ft-mobile-convertible-bench-unit-with-ganging-mdf-core-top-protect-edge-oak>: HTTP status code is not handled or not allowed 2026-01-24 22:14:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zurn-z825a2-deck-mounted-gooseneck-lab-faucet>: HTTP status code is not handled or not allowed 2026-01-24 22:14:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 22:14:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30w-solar-panel-led-flood-light-black>: HTTP status code is not handled or not allowed 2026-01-24 22:14:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/tool_storage/chests_roller_cabinets>: HTTP status code is not handled or not allowed 2026-01-24 22:14:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-2-12-thick-x-6-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 22:14:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-5215zz-double-row-angular-contact-bearing-double-shielded-75mm-bore-x-130mm-od-x-41-3mm-w>: HTTP status code is not handled or not allowed 2026-01-24 22:14:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/material-handling-basket-16l-x-10w-x-3-15-16h-plain-steel-price-each-for-qty-5-25-wire>: HTTP status code is not handled or not allowed 2026-01-24 22:14:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:14:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-5203zz-double-row-angular-contact-bearing-double-shielded-17mm-bore-x-40mm-od-x-17-5mm-w>: HTTP status code is not handled or not allowed 2026-01-24 22:14:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/antistatic-square-weighing-boat-41-x-41-x-8mm-20ml-polystyrene-case-of-500>: HTTP status code is not handled or not allowed 2026-01-24 22:14:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:14:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-5204zz-double-row-angular-contact-bearing-double-shielded-20mm-bore-x-47mm-od-x-20-6mm-w>: HTTP status code is not handled or not allowed 2026-01-24 22:14:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9000cd-stainless-steel-automatic-professional-dental-steam-cleaner-5l-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:14:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tension-bar-end-for-pe-3000-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-24 22:14:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caster-wheel-for-pe-3000-pallet-trucks-cr-83179-xl>: HTTP status code is not handled or not allowed 2026-01-24 22:14:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_repair_maintenance/tires_wheels/tire_chains already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:14:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ballymore-11-step-perforated-cantilever-ladder-14-overhang>: HTTP status code is not handled or not allowed 2026-01-24 22:14:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/infecon-liquid-tight-zipper-bag-13-x-18-pocket-and-printed-biohazard-symbol-1000-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:14:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caster-wheel-assembly-for-pe-pw-pc-series-pallet-trucks-cr-083179-a-sg>: HTTP status code is not handled or not allowed 2026-01-24 22:14:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-plastic-bar-1-12-thick-x-4-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 22:14:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-24 22:14:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ballymore-13-step-perforated-cantilever-ladder-28-overhang>: HTTP status code is not handled or not allowed 2026-01-24 22:14:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-drawer-wood-top-cast-vertical-storage-dresser-white-light-gray-fabric-drawers>: HTTP status code is not handled or not allowed 2026-01-24 22:14:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-24 22:14:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/infrared-bar-heater-48>: HTTP status code is not handled or not allowed 2026-01-24 22:14:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roller-grill-18-hot-dogs-gripsit>: HTTP status code is not handled or not allowed 2026-01-24 22:14:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/collarless-butcher-wrap-wexterior-pockets-navy-polyestercombed-cotton-3xl>: HTTP status code is not handled or not allowed 2026-01-24 22:14:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bearing-cone-for-pe-4000-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-24 22:14:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bushing-for-pe-3000-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-24 22:14:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_repair_maintenance/tires_wheels/tire_chains>: HTTP status code is not handled or not allowed 2026-01-24 22:14:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/variable-speed-controller>: HTTP status code is not handled or not allowed 2026-01-24 22:14:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/easy-frykutter-6-sec-wedge>: HTTP status code is not handled or not allowed 2026-01-24 22:14:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-recycled-white-cut-rags-25-lb-box>: HTTP status code is not handled or not allowed 2026-01-24 22:14:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-pocket-acrylic-oak-wall-display-medium-oak>: HTTP status code is not handled or not allowed 2026-01-24 22:14:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-sheet-34-thick-x-48-wide-x-96-long>: HTTP status code is not handled or not allowed 2026-01-24 22:14:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cook-shirt-black-polyestercotton-4xl>: HTTP status code is not handled or not allowed 2026-01-24 22:14:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-magazine-8-brochure-oak-acrylic-wall-display-light-oak>: HTTP status code is not handled or not allowed 2026-01-24 22:14:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-spacer-for-7400-reach-pallet-trucks-5-x-2-88-wheels-open-toe>: HTTP status code is not handled or not allowed 2026-01-24 22:14:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-magazine-12-brochure-oak-acrylic-wall-display-light-oak>: HTTP status code is not handled or not allowed 2026-01-24 22:14:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-sheet-1-thick-x-39-wide-x-78-long>: HTTP status code is not handled or not allowed 2026-01-24 22:14:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-axle-for-npp-40-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-24 22:14:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stratus-1-4-perforated-plastic-panel-ceiling-t-grid-pk2>: HTTP status code is not handled or not allowed 2026-01-24 22:14:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-alligator-clips-w-5-ft-7x19-stranded-flex-steel-cable>: HTTP status code is not handled or not allowed 2026-01-24 22:14:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-pocket-4h-acrylic-oak-wall-display-mahogany>: HTTP status code is not handled or not allowed 2026-01-24 22:14:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wheelbarrow-ladder-8-steps-aluminum-350-lb-capacity-14-d-top-step-24-w-step>: HTTP status code is not handled or not allowed 2026-01-24 22:14:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/complete-riser-load-wheel-assembly-for-w-40z-b218-45z-c215-pallet-trucks-hy-2108789>: HTTP status code is not handled or not allowed 2026-01-24 22:14:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unisex-butcher-wrap-white-polyestercombed-cotton-l>: HTTP status code is not handled or not allowed 2026-01-24 22:14:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-for-w-40z-b218-45z-c215-pallet-trucks-hy-2046054>: HTTP status code is not handled or not allowed 2026-01-24 22:14:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-sheet-1-14-thick-x-6-wide-x-6-long>: HTTP status code is not handled or not allowed 2026-01-24 22:14:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/button-front-short-sleeve-cook-shirt-white-polyestercotton-3xl>: HTTP status code is not handled or not allowed 2026-01-24 22:15:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-assembly-for-mpe-060-mpe-080-pallet-trucks-yl-524140950-a-sg>: HTTP status code is not handled or not allowed 2026-01-24 22:15:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-grinders-buffer-330b-24m-2p-buffer>: HTTP status code is not handled or not allowed 2026-01-24 22:15:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-14-thick-x-1-14-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-24 22:15:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-sitex-hunter-green-3-piece-suit-pvc-size-small>: HTTP status code is not handled or not allowed 2026-01-24 22:15:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-grinders-buffer-8107wd-24c-2p-grndr>: HTTP status code is not handled or not allowed 2026-01-24 22:15:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-grinders-buffer-8250w-24c-2p-grndr-buffer>: HTTP status code is not handled or not allowed 2026-01-24 22:15:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-acetal-plastic-bar-12-thick-x-1-14-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 22:15:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-kit-for-electric-pallet-jack-truck-gwk-pr4500-lw-xl-fits-crown-model-pr-4500>: HTTP status code is not handled or not allowed 2026-01-24 22:15:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-tier-mesh-file-organizer-black>: HTTP status code is not handled or not allowed 2026-01-24 22:15:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x36x74-chrome-wire-truck-with-48-4-h-shelf-bins-yellow>: HTTP status code is not handled or not allowed 2026-01-24 22:15:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-for-7400-reach-pallet-trucks-ra-632-052-007-xl>: HTTP status code is not handled or not allowed 2026-01-24 22:15:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-qwik-cap>: HTTP status code is not handled or not allowed 2026-01-24 22:15:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-kit-for-electric-pallet-jack-gwk-mpe060e-lw-fits-yale-model-mpe-060e-a896-series>: HTTP status code is not handled or not allowed 2026-01-24 22:15:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/snap-n-rack-tube-rack-for-20mm-and-21mm-tubes-40-place-polypropylene-yellow>: HTTP status code is not handled or not allowed 2026-01-24 22:15:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 29402 pages (at 93 pages/min), scraped 14219 items (at 12 items/min) 2026-01-24 22:15:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24x48x74-chrome-wire-shelving-with-110-shelf-bins-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:15:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tube-holder-foam-for-use-w-gvm-series-48-x-6mm-tubes-must-use-w-vm-as-plate-gvm-as-rod>: HTTP status code is not handled or not allowed 2026-01-24 22:15:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-pedestal-post-cd-premium-fan-292651>: HTTP status code is not handled or not allowed 2026-01-24 22:15:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ridgeline-cap-style-hard-hat-white-6-point-ratchet-suspension>: HTTP status code is not handled or not allowed 2026-01-24 22:15:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-corded-earplugs-with-iron-ball-embedded-plug>: HTTP status code is not handled or not allowed 2026-01-24 22:15:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-chemtex-green-bib-overall-plain-front-pvc-on-polyester-size-small>: HTTP status code is not handled or not allowed 2026-01-24 22:15:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyurethane-dipped-latex-gloves-gl401-series-size-small-pkg-qty-12>: HTTP status code is not handled or not allowed 2026-01-24 22:15:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-profile-foldaway-earmuffs-hi-vis-lime-nrr-26db-individually-packaged>: HTTP status code is not handled or not allowed 2026-01-24 22:15:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/female-upper-plastic-form-on-faux-crystal-base-black>: HTTP status code is not handled or not allowed 2026-01-24 22:15:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x48x74-chrome-wire-shelving-with-24-8-h-grid-container-red>: HTTP status code is not handled or not allowed 2026-01-24 22:15:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ridgeline-full-brim-hard-hat-mate-camo-pattern-4-point-ratchet-suspension>: HTTP status code is not handled or not allowed 2026-01-24 22:15:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-stacking-bin-5-1-2x14-3-4x5-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:15:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-uhmw-polyethylene-plastic-bar-1-12-thick-x-3-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 22:15:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extruded-aluminum-hose-cable-bridge-yellow-36-x-718-x-1116>: HTTP status code is not handled or not allowed 2026-01-24 22:15:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-webtex-olive-green-3-piece-suit-pvc-size-extra-large>: HTTP status code is not handled or not allowed 2026-01-24 22:15:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-bar-38-thick-x-2-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 22:15:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fiberglass-fabric-reinforced-silicone-rubber-sheet-no-adhesive-70a-116-thick-x-12w-x-12l>: HTTP status code is not handled or not allowed 2026-01-24 22:15:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sunshade-hard-hat-accessory-yellow-green-10098032>: HTTP status code is not handled or not allowed 2026-01-24 22:15:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-men-s-17-slicker-yellow-overboot-w-strap-cleated-outsole-pvc-size-14>: HTTP status code is not handled or not allowed 2026-01-24 22:15:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ridgeline-cap-style-hard-hat-matte-camo-pattern-4-point-ratchet-suspension>: HTTP status code is not handled or not allowed 2026-01-24 22:15:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ridgeline-cap-style-hard-hat-matte-red-pattern-4-point-ratchet-suspension>: HTTP status code is not handled or not allowed 2026-01-24 22:15:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/snap-n-rack-tube-rack-for-12mm-and-13mm-tubes-90-place-polypropylene-red>: HTTP status code is not handled or not allowed 2026-01-24 22:15:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5hp-3-phase-total-enclosed-canister-dust-collector-ufo-dc105>: HTTP status code is not handled or not allowed 2026-01-24 22:15:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latchways-personal-fall-limiter-10-foot-web-single-leg-rebar-hook-63013-00a>: HTTP status code is not handled or not allowed 2026-01-24 22:15:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-sheet-332-thick-x-8-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 22:15:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-men-s-boot-6-monarch-black-steel-toe-w-cleated-outsole-pvc-size-10>: HTTP status code is not handled or not allowed 2026-01-24 22:15:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/red-hinged-lid-1-7-cu-yd-self-dumping-plastic-hopper>: HTTP status code is not handled or not allowed 2026-01-24 22:15:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-men-s-boot-16-goliath-black-steel-toe-w-power-lug-outsole-pvc-size-5>: HTTP status code is not handled or not allowed 2026-01-24 22:15:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4841-w-cape-cod-window-box-white-polyethylene-4-gallon-capacity-48-x-108-x-11>: HTTP status code is not handled or not allowed 2026-01-24 22:15:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5823-w-fairfield-window-box-white-polyethylene-4-gallon-capacity-48-x-108-x-11>: HTTP status code is not handled or not allowed 2026-01-24 22:15:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4829-w-nantucket-window-box-white-polyethylene-16-gallon-capacity-24-x-10-x-115>: HTTP status code is not handled or not allowed 2026-01-24 22:15:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/full-body-harness-univ-3>: HTTP status code is not handled or not allowed 2026-01-24 22:15:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/fans_blowers/exhaust_fans/exhaust_fan_accessories/dampers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:15:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-washer-m6-hardened-steel-zinc-yellow-class-10-9-din-125a-pkg-of-100-abk06>: HTTP status code is not handled or not allowed 2026-01-24 22:15:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vis-i-rack-8482-high-capacity-blueprint-storage-rack-13-openings>: HTTP status code is not handled or not allowed 2026-01-24 22:15:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cap-plug-12-13mm-santoprene-for-vacuum-and-test-tubes-green-1000-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:15:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-head-cap-screw-m12-x-1-5-x-90mm-steel-zinc-clear-class-8-8-din-960-pkg-of-25>: HTTP status code is not handled or not allowed 2026-01-24 22:15:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/560-fissured-ceiling-panels-mineral-fiber-white-24-x-24>: HTTP status code is not handled or not allowed 2026-01-24 22:15:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-head-cap-screw-m8-x-1-25-x-25mm-steel-zinc-yellow-class-10-9-din-933-pkg-of-100-aap08025>: HTTP status code is not handled or not allowed 2026-01-24 22:15:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elkay-lk4405bf-single-level-wall-mount-outdoor-pedestal-fountain-with-water-refilling-station>: HTTP status code is not handled or not allowed 2026-01-24 22:15:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bar-rod-truck-with-rubber-casters-84-l-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:15:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-cabinet-30-w-black>: HTTP status code is not handled or not allowed 2026-01-24 22:15:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2010-pencil-clip-orange>: HTTP status code is not handled or not allowed 2026-01-24 22:15:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-solenoid-valve-assembly-641264-641244-641265-641245-641750-641746-641747>: HTTP status code is not handled or not allowed 2026-01-24 22:15:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-suction-hose-d-38x800-641245>: HTTP status code is not handled or not allowed 2026-01-24 22:15:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/fans_blowers/exhaust_fans/exhaust_fan_accessories/dampers>: HTTP status code is not handled or not allowed 2026-01-24 22:15:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tach-it-e-8-50-uv-black-intermediate-ul-recognized-cable-tie>: HTTP status code is not handled or not allowed 2026-01-24 22:15:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microscope-cover-glass-24mm-x-24mm--1-thickness-1-oz-vacuum-pack-10-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:15:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-flat-shelf-portable-bin-cart-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:15:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-flat-shelf-portable-bin-cart-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:15:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-quilted-bomber-jacket-ansi-class-4-navy-blue-l-nvbj-s1-l>: HTTP status code is not handled or not allowed 2026-01-24 22:15:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/americana-vented-full-brim-with-4-point-slide-lock-suspension-red>: HTTP status code is not handled or not allowed 2026-01-24 22:15:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tach-it-650-heavy-duty-nylon-tagging-fasteners>: HTTP status code is not handled or not allowed 2026-01-24 22:15:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-quilted-bomber-jacket-ansi-class-7-navy-blue-3x-nvbj-s1-3xl>: HTTP status code is not handled or not allowed 2026-01-24 22:15:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rotor-for-use-with-gcm-24-series-15000rpm-max-18-place-for-5ml-tubes>: HTTP status code is not handled or not allowed 2026-01-24 22:15:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/46in-round-metal-mesh-picnic-table-black>: HTTP status code is not handled or not allowed 2026-01-24 22:15:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h-stand-support-for-36-bf-roller-conveyors-adjustable-19-1-2-to-31h>: HTTP status code is not handled or not allowed 2026-01-24 22:15:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-bar-18-thick-x-14-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 22:15:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/omega-ii-cap-with-accessory-slots-and-6-point-slide-lock-suspension-black>: HTTP status code is not handled or not allowed 2026-01-24 22:15:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transfer-pipet-150ml-narrow-stem-large-bulb-155mm-2500-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:15:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storage-bin-cabinet-264-red-hook-on-bins-72w-24d-84h>: HTTP status code is not handled or not allowed 2026-01-24 22:15:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-w-lse-acrylic-adhesive-18-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 22:15:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-drawer-bench-high-24-w-cabinet-avalanche-blue-5>: HTTP status code is not handled or not allowed 2026-01-24 22:15:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/105k-btu-indoor-kerosene-convection-heater-246635>: HTTP status code is not handled or not allowed 2026-01-24 22:15:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/l4-new-hire-kit-with-liberty-18450-white>: HTTP status code is not handled or not allowed 2026-01-24 22:15:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30k-lp-tank-top-heater-csa-246651>: HTTP status code is not handled or not allowed 2026-01-24 22:15:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-drawer-counter-high-48-w-cabinet-beige-3>: HTTP status code is not handled or not allowed 2026-01-24 22:15:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xx-garolite-rod-14-diameter-x-2-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 22:15:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storage-bin-cabinet-4-adj-shelves-and-14-door-shelves-48w-24d-72h>: HTTP status code is not handled or not allowed 2026-01-24 22:15:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-welded-12-gauage-singel-sided-bin-rack-with-210-bins-51051-br-210-1s>: HTTP status code is not handled or not allowed 2026-01-24 22:16:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-drawer-full-height-36-w-cabinet-beige-1>: HTTP status code is not handled or not allowed 2026-01-24 22:16:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-inch-npt-brass-thermowell-6-inch-stem>: HTTP status code is not handled or not allowed 2026-01-24 22:16:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-lens-cleaning-wipes-100-per-box>: HTTP status code is not handled or not allowed 2026-01-24 22:16:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e-1236vs-with-collet-closer>: HTTP status code is not handled or not allowed 2026-01-24 22:16:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/undercounter-refrigerator-33-38f-48-38w-30-18d-tuc-48d-4-ada>: HTTP status code is not handled or not allowed 2026-01-24 22:16:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medium-flat-panel-turntable-black>: HTTP status code is not handled or not allowed 2026-01-24 22:16:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quicksleeve-handheld-sleeve-applicator-qs>: HTTP status code is not handled or not allowed 2026-01-24 22:16:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deep-work-top-refrigerator-2-section-67-14w-32-58d-33-38h-twt-67>: HTTP status code is not handled or not allowed 2026-01-24 22:16:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deep-work-top-refrigerator-2-section-67-14w-32-38d-33-38h-twt-67d-2>: HTTP status code is not handled or not allowed 2026-01-24 22:16:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=jtm-1254rvs-with-x-y-powerfeeds-air-power-drawbar already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:16:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/end-cap-for-wg-4-silver-gray-left-return>: HTTP status code is not handled or not allowed 2026-01-24 22:16:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 29498 pages (at 96 pages/min), scraped 14237 items (at 18 items/min) 2026-01-24 22:16:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nps-wood-science-lab-table-24-x-48-x-30-whiteboard-top>: HTTP status code is not handled or not allowed 2026-01-24 22:16:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/364827-v5-gold-banana-plugs>: HTTP status code is not handled or not allowed 2026-01-24 22:16:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tube-rocker-double-tier-120v-60hz-us-plug-99-x-67-platforms-24-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:16:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-pro-undercounter-refrigerator-33-38f-48-38w-30-18d-tuc-48-lp>: HTTP status code is not handled or not allowed 2026-01-24 22:16:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/centrifuge-clinical-enhanced-120-240v-50-60hz-w-12-place-rotor-sleeves-risers>: HTTP status code is not handled or not allowed 2026-01-24 22:16:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-drawer-bench-high-30-w-cabinet-everest-blue-3>: HTTP status code is not handled or not allowed 2026-01-24 22:16:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72in-yellow-plus-ii-quarter-in-hose-21072>: HTTP status code is not handled or not allowed 2026-01-24 22:16:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/refrigerator-reach-in-1-section-27w-29-34d-78-38h-t-23-1-g-1>: HTTP status code is not handled or not allowed 2026-01-24 22:16:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/titan-4-valve-manifold-49963>: HTTP status code is not handled or not allowed 2026-01-24 22:16:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tru-ray-construction-paper-9-x-12-purple50-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:16:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jtm-1254rvs-with-newall-dp700-3x-k-dro-x-powerfeed-air-power-drawbar>: HTTP status code is not handled or not allowed 2026-01-24 22:16:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=jtm-1254rvs-with-x-y-powerfeeds-air-power-drawbar>: HTTP status code is not handled or not allowed 2026-01-24 22:16:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/welding-cable-lead-reel-fixed-base-double>: HTTP status code is not handled or not allowed 2026-01-24 22:16:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mig-nozzle-dip-gel-16-oz>: HTTP status code is not handled or not allowed 2026-01-24 22:16:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-tabs-repositionable-tabs-1-x-1-1-2-neon-green-pink-yellow-orange-40-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:16:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-camwear-spoon-red>: HTTP status code is not handled or not allowed 2026-01-24 22:16:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chemical-waterproof-uv-resistent-labels-laser-2-x-2-600box>: HTTP status code is not handled or not allowed 2026-01-24 22:16:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chemical-waterproof-uv-resistent-labels-laser-4-x-4-200box>: HTTP status code is not handled or not allowed 2026-01-24 22:16:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-swivel-shackle-dss-m-24>: HTTP status code is not handled or not allowed 2026-01-24 22:16:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24in-tilt-heavy-duty-roll-around-tilt-air-circulator-bf24tfyel-4000-cfm>: HTTP status code is not handled or not allowed 2026-01-24 22:16:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jwds-2244-drum-sander-w-open-stand>: HTTP status code is not handled or not allowed 2026-01-24 22:16:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forklift-maintenance-platform-48x36-cal-osha>: HTTP status code is not handled or not allowed 2026-01-24 22:16:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-light-duty-black-trash-bags-33-gal-0-43-mil-500-bags-case>: HTTP status code is not handled or not allowed 2026-01-24 22:16:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hbs-56s-5-inch-x-6-inch-variable-speed-mitering-horizontal-mitering-bandsaw>: HTTP status code is not handled or not allowed 2026-01-24 22:16:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kraft-easy-fold-mailer-9-1-2-x-6-1-2-x-2>: HTTP status code is not handled or not allowed 2026-01-24 22:16:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-milk-cooler-8-crts-34w-33-38d-41-12h-tmc-34>: HTTP status code is not handled or not allowed 2026-01-24 22:16:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-purpose-industrial-footwear-traction-aid-with-8-tungsten-carbide-spikes-small>: HTTP status code is not handled or not allowed 2026-01-24 22:16:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-welded-gear-locker-with-door-foot-locker-legs-24x18x72-red>: HTTP status code is not handled or not allowed 2026-01-24 22:16:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kraft-double-wall-corrugated-sheets-40-x-48>: HTTP status code is not handled or not allowed 2026-01-24 22:16:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/etm-949evs-mill-with-2-axis-acu-rite-millpwr-cnc>: HTTP status code is not handled or not allowed 2026-01-24 22:16:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/convenience-store-cooler-2-section-47-18w-21d-59-78h-gdm-41sl-60>: HTTP status code is not handled or not allowed 2026-01-24 22:16:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hanging-tailpipe-rack-96x18x120>: HTTP status code is not handled or not allowed 2026-01-24 22:16:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triton-lbs18k-slv-silver-key-pegboard-kit>: HTTP status code is not handled or not allowed 2026-01-24 22:16:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bactizyme-drain-cleaner-maintainer-fresh-clean-50-gallon-nl044-d55>: HTTP status code is not handled or not allowed 2026-01-24 22:16:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evs-949-mill-with-x-y-axis-jet-powerfeeds>: HTTP status code is not handled or not allowed 2026-01-24 22:16:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insight-variable-adf-welding-helmet-shade-9-13-stars-scars-hlx-100-series>: HTTP status code is not handled or not allowed 2026-01-24 22:16:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96-x-30-mobile-plastic-square-edge-work-bench-fixed-height-1-5-8-top>: HTTP status code is not handled or not allowed 2026-01-24 22:16:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-corrugated-sheets-48-x-48>: HTTP status code is not handled or not allowed 2026-01-24 22:16:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milwaukee-electrical-combo-kit-2220-20>: HTTP status code is not handled or not allowed 2026-01-24 22:16:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/summit-laboratory-chest-freezer-capable-of-30-c-22-f-operation-replaces-fcl88>: HTTP status code is not handled or not allowed 2026-01-24 22:16:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/advantage-full-brim-hard-hat-non-vented-4-pt-ratchet-suspension-yellow>: HTTP status code is not handled or not allowed 2026-01-24 22:16:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/market-tray-pens-9x26x1-5-white>: HTTP status code is not handled or not allowed 2026-01-24 22:16:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-nesting-dunnage-rack-60x20x12>: HTTP status code is not handled or not allowed 2026-01-24 22:16:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/telescoping-outer-boxes-6-1-2-x-6-1-2-x-72>: HTTP status code is not handled or not allowed 2026-01-24 22:16:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-corrugated-mailer-6-x-5-x-2->: HTTP status code is not handled or not allowed 2026-01-24 22:16:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kfi-3315b-wl-black-metal-frame-cafe-chair-with-wood-seat-and-back-walnut>: HTTP status code is not handled or not allowed 2026-01-24 22:16:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-corrugated-mailer-6-x-5-x-4->: HTTP status code is not handled or not allowed 2026-01-24 22:16:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-carpet-bonnets-mini-floor-scrubber-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:16:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1847-sc-6-5-8-x-1-4-standard-cut-cylindrical-ball-nose-bur>: HTTP status code is not handled or not allowed 2026-01-24 22:16:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stackbin-heavy-duty-steel-square-edge-48-w-x-36-d-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:16:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cylinder-measuring-diamond-essentials-class-b-pmp-molded-graduations-tall-form-100ml>: HTTP status code is not handled or not allowed 2026-01-24 22:16:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conveyor-magnet-30in-l>: HTTP status code is not handled or not allowed 2026-01-24 22:16:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-glamour-bubble-mailer-9-x-11-1-2>: HTTP status code is not handled or not allowed 2026-01-24 22:16:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pipette-tip-01-20ul-certified-universal-natural-45mm-extended-length-sterile-960-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:16:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cylinder-measuring-diamond-essentials-class-b-pmp-molded-graduations-tall-form-2000ml>: HTTP status code is not handled or not allowed 2026-01-24 22:16:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-x-30-mobile-plastic-square-lab-bench-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:16:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stackbin-3505-series-hardboard-over-stainless-48-w-x-36-d-black>: HTTP status code is not handled or not allowed 2026-01-24 22:16:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5370-95-cast-iron-suction-type-coolant-pump-56gpm-carbon-ceramic-seal-1-inch-npt-discharge>: HTTP status code is not handled or not allowed 2026-01-24 22:16:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5390-95-cast-iron-immersion-type-coolant-pump-56gpm-seal-less-design-1-2-inch-npt-discharge>: HTTP status code is not handled or not allowed 2026-01-24 22:16:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camrack-base-rack-4-1-4-inch-inside-stack-height-black-nsf>: HTTP status code is not handled or not allowed 2026-01-24 22:16:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-boxes-8-x-5-x-3->: HTTP status code is not handled or not allowed 2026-01-24 22:16:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pipette-diamond-fixed-volume-1000ul-1ml-light-green>: HTTP status code is not handled or not allowed 2026-01-24 22:16:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/no-5-shear>: HTTP status code is not handled or not allowed 2026-01-24 22:16:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-step-stock-picking-ladder-abrasive-tread-spl-9-r>: HTTP status code is not handled or not allowed 2026-01-24 22:16:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/282b-95-1-5-inch-cast-iron-self-priming-centrifugal-pump-buna-n-seal-2hp-tefc-3-phase-motor>: HTTP status code is not handled or not allowed 2026-01-24 22:16:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wireless-tube-rack-20mm-full-size-40-place-nylon-magenta>: HTTP status code is not handled or not allowed 2026-01-24 22:17:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4250-98-3x2-inch-316-ss-heavy-duty-straight-centrifugal-pump-viton-seal-10hp-215jm-3-phase-motor>: HTTP status code is not handled or not allowed 2026-01-24 22:17:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beaker-diamond-essentials-griffin-style-low-form-printed-graduations-pmp-50ml-12-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:17:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rv-pette-dispenser-tip-for-repeat-volume-pipettors-10ml-100-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:17:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-industrial-duty-exhaust-fan-3-phase-3-hp>: HTTP status code is not handled or not allowed 2026-01-24 22:17:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pipette-diamond-jr-25ul-green-autoclavable>: HTTP status code is not handled or not allowed 2026-01-24 22:17:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-easy-fold-mailer-9-5-8-x-6-5-8-x-2-1-2>: HTTP status code is not handled or not allowed 2026-01-24 22:17:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cylinder-measuring-diamond-essentials-class-b-polypropylene-molded-graduations-250ml>: HTTP status code is not handled or not allowed 2026-01-24 22:17:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beaker-diamond-essentials-griffin-style-low-form-printed-graduations-pmp-5000ml>: HTTP status code is not handled or not allowed 2026-01-24 22:17:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/through-the-wall-air-conditioner-12-000-btu-cool-with-heat-208-230v>: HTTP status code is not handled or not allowed 2026-01-24 22:17:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/berries-plastic-chair-with-chrome-plated-legs-12-ht-orange>: HTTP status code is not handled or not allowed 2026-01-24 22:17:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beaker-with-handle-diamond-essentials-low-form-printed-graduations-pmp-2000ml>: HTTP status code is not handled or not allowed 2026-01-24 22:17:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-corrugated-mailer-10-x-6-x-4->: HTTP status code is not handled or not allowed 2026-01-24 22:17:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lt191g01-lavatory-undercounter-2012-x-1238-wsanagloss-cotton-white>: HTTP status code is not handled or not allowed 2026-01-24 22:17:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 29598 pages (at 100 pages/min), scraped 14258 items (at 21 items/min) 2026-01-24 22:17:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1853-sd-42-3-00mm-x-3mm-standard-cut-ball-shaped-bur>: HTTP status code is not handled or not allowed 2026-01-24 22:17:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1848-sf-5l6-1-2-x-1-4-double-cut-round-nose-tree-bur>: HTTP status code is not handled or not allowed 2026-01-24 22:17:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kraft-triple-wall-corrugated-sheets-48-x-96>: HTTP status code is not handled or not allowed 2026-01-24 22:17:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-corrugated-mailer-9-x-6-x-5->: HTTP status code is not handled or not allowed 2026-01-24 22:17:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergo-computer-transport-stand-with-cord-reel>: HTTP status code is not handled or not allowed 2026-01-24 22:17:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1854-sh-2l6-5-16-x-1-4-standard-cut-flame-shaped-bur>: HTTP status code is not handled or not allowed 2026-01-24 22:17:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1846-sb-5-1-2-x-1-4-standard-cut-cylindrical-bur-w-end-cut>: HTTP status code is not handled or not allowed 2026-01-24 22:17:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microcentrifuge-tube-05ml-polypropylene-blue-stand-up-zipbag-mg-500-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:17:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poly-overbed-clipboard-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:17:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-step-aluminum-rolling-ladder-16w-grip-tread-14d-top-step>: HTTP status code is not handled or not allowed 2026-01-24 22:17:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/74xl037-standard-timing-belt-xl-3-8-x-7-3-8-t37-trapezoidal>: HTTP status code is not handled or not allowed 2026-01-24 22:17:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/600-pound-single-door-magnetic-lock>: HTTP status code is not handled or not allowed 2026-01-24 22:17:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fan-shutter-mounting-box-30-exhaust-fans>: HTTP status code is not handled or not allowed 2026-01-24 22:17:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/floor-finish-stripper-41-concentrate-gallon-bottle-4-case-fs-3500>: HTTP status code is not handled or not allowed 2026-01-24 22:17:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-step-aluminum-rolling-ladder-24w-grip-tread-21d-top-step>: HTTP status code is not handled or not allowed 2026-01-24 22:17:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transport-basket>: HTTP status code is not handled or not allowed 2026-01-24 22:17:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stackbin-16-long-drawer-cabinet-pedestal-workbench-plastic-laminate-safety-edge-72w-x-30d-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:17:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/138xl050-standard-timing-belt-xl-1-2-x-13-13-16-t69-trapezoidal>: HTTP status code is not handled or not allowed 2026-01-24 22:17:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-1-4-x-43-1-2-h-deluxe-office-partition-panel-blue-277525bl>: HTTP status code is not handled or not allowed 2026-01-24 22:17:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unex-additional-level-for-gravity-flow-roller-rack-with-knuckled-span-track-wheel-bed-48w-x-96d>: HTTP status code is not handled or not allowed 2026-01-24 22:17:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-1-4-x-47-1-2-h-deluxe-office-partition-panel-electric-gray-277543gy>: HTTP status code is not handled or not allowed 2026-01-24 22:17:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/supco-defrost-control-temperature-or-pressure-terminated-s814100>: HTTP status code is not handled or not allowed 2026-01-24 22:17:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-strip-for-09028-elastic>: HTTP status code is not handled or not allowed 2026-01-24 22:17:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-1-4-x-47-1-2-h-deluxe-office-partition-panel-with-electric-gray-277561gy>: HTTP status code is not handled or not allowed 2026-01-24 22:17:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/supco-pressure-switch-600-psi-open-475-psi-closed>: HTTP status code is not handled or not allowed 2026-01-24 22:17:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-1-4-x-77-1-2-h-deluxe-office-partition-panel-with-electric-blue-277553bl>: HTTP status code is not handled or not allowed 2026-01-24 22:17:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-x-24-inch-heavy-duty-adjustable-height-steel-work-platform-orange-5-h-to-8-h>: HTTP status code is not handled or not allowed 2026-01-24 22:17:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-mil-acm-cutter-for-5-mil-alumacorr>: HTTP status code is not handled or not allowed 2026-01-24 22:17:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/penco-vanguard-box-over-locker-18x21x72-champagne>: HTTP status code is not handled or not allowed 2026-01-24 22:17:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/saw-trax-1000-series-panel-saw-package-52-crosscut>: HTTP status code is not handled or not allowed 2026-01-24 22:17:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/supco-thermostat-guard-3-12x3x6-38>: HTTP status code is not handled or not allowed 2026-01-24 22:17:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-174-medium-blue-looped-mop-head-narrow-band>: HTTP status code is not handled or not allowed 2026-01-24 22:17:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/supco-therm-o-disc-sequencer-q105>: HTTP status code is not handled or not allowed 2026-01-24 22:17:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-174-medium-blue-looped-mop-head-wide-band>: HTTP status code is not handled or not allowed 2026-01-24 22:17:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quick-attach-pole-pusher-for-motorized-hand-trucks>: HTTP status code is not handled or not allowed 2026-01-24 22:17:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vornado-530b-small-whole-room-air-circulator>: HTTP status code is not handled or not allowed 2026-01-24 22:17:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machine-table-w-2-shelves-30-w-x-18-d-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:17:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/schedule-40-pvc-tee-fitting-12dia-white-121-u>: HTTP status code is not handled or not allowed 2026-01-24 22:17:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ventureclad-4inch-x-15-yards-white>: HTTP status code is not handled or not allowed 2026-01-24 22:17:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ec510-waterproof-exstikand174-ii-phconductivity-meter-kit-electrode>: HTTP status code is not handled or not allowed 2026-01-24 22:17:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/snap-clamp-fittings-4l-1-14dia-furniture-grade-abs-white>: HTTP status code is not handled or not allowed 2026-01-24 22:17:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/furniture-pipe-fittings-5l-34dia-furniture-grade-pvc-white>: HTTP status code is not handled or not allowed 2026-01-24 22:17:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lifting-magnet-800-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:17:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1325h150-standard-timing-belt-h-1-1-2-x-132-1-2-t265-trapezoidal>: HTTP status code is not handled or not allowed 2026-01-24 22:17:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/380941nist-mini-clamp-meter-greenorange-nist-certified>: HTTP status code is not handled or not allowed 2026-01-24 22:17:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-duct-tape-2-inch-x-60-yards-white>: HTTP status code is not handled or not allowed 2026-01-24 22:17:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/table-top-model-food-bar-33x71-green>: HTTP status code is not handled or not allowed 2026-01-24 22:17:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-tier-carton-rack>: HTTP status code is not handled or not allowed 2026-01-24 22:17:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/445702-hygrothermometer-clock-white-wall-mount-aaa-battery>: HTTP status code is not handled or not allowed 2026-01-24 22:17:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multiple-roll-stand-60-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:17:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-40btl60-taper-bushed-12-pitch-60-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:17:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rf12-portable-brix-refractometer-018-brix>: HTTP status code is not handled or not allowed 2026-01-24 22:17:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-spunbond-oil-only-socks-15-case-8-x-3>: HTTP status code is not handled or not allowed 2026-01-24 22:17:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/graduated-cylinder-polypropylene-printed-graduations-10ml-10-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:17:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kac-111nwi-10-24-red>: HTTP status code is not handled or not allowed 2026-01-24 22:17:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/biobox-81-for-30ml-40ml-and-50ml-cryoclear-vials-polycarbonate-holds-81-place-red-4-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:17:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/standard_duty_boltless_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:17:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-angle-beam-72-w-x-24-d-x-84-h-4-level-add-on-no-deck-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:17:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-angle-beam-60-w-x-36-d-x-84-h-4-level-starter-no-deck-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:17:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mid-size-w-post-mix-system-bag-box-syrup-black>: HTTP status code is not handled or not allowed 2026-01-24 22:17:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/airvantage-ad-061236-6-central-vacuum-random-orbital-sander-3-32-orbit>: HTTP status code is not handled or not allowed 2026-01-24 22:17:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pneumatic-engraving-pen-kit-with-3-stylus-2-5-cfm-18750-bpm>: HTTP status code is not handled or not allowed 2026-01-24 22:17:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grid-style-satin-aluminum-umbrella-stand-for-up-to-24-full-and-tote-size-umbrellas>: HTTP status code is not handled or not allowed 2026-01-24 22:17:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nitrogen-gas-cylinder-1066-99999-pure-103l-j>: HTTP status code is not handled or not allowed 2026-01-24 22:17:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lbx-30-latitude-compact-bench-scale-65-lb-x-0-01-lb>: HTTP status code is not handled or not allowed 2026-01-24 22:17:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lens-cover-ff-400-15-25-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:17:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tune-up-kit-for-mfr-no-2940-series>: HTTP status code is not handled or not allowed 2026-01-24 22:17:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60bs24hx138-34-pitch-1-38-finished-bore-24-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:18:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bottle-with-screwcap-wide-mouth-polypropylene-graduated-500ml-5-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:18:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-inch-drive-size-impact-wrench-2000-ft-lb-maxtorque>: HTTP status code is not handled or not allowed 2026-01-24 22:18:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/growt-hgbb6-6-bamboo-stakes-natural-color-25-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:18:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/standard_duty_boltless_shelves landed on page that is not a product page. 2026-01-24 22:18:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securall-double-door-b600-through-b1200>: HTTP status code is not handled or not allowed 2026-01-24 22:18:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-duty-3-8-inch-square-drive-size-impact-wrench-25-to-230-ft-lb-torque>: HTTP status code is not handled or not allowed 2026-01-24 22:18:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-o-smith-up-rated-pool-filter-motor-115-230-volts-3450-rpm-4>: HTTP status code is not handled or not allowed 2026-01-24 22:18:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/funnel-analytical-polypropylene-120mm-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:18:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securall-galvanized-steel-floor-grating-b8000>: HTTP status code is not handled or not allowed 2026-01-24 22:18:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/funnel-analytical-polypropylene-80mm-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:18:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelf-computer-lanstation-workstation-keyboard-tray-34-hx24-wx48-l-black-3-shelf>: HTTP status code is not handled or not allowed 2026-01-24 22:18:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 29696 pages (at 98 pages/min), scraped 14277 items (at 19 items/min) 2026-01-24 22:18:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-o-smith-pool-filter-motor-115-230-volts-3450-rpm-3-4hp-2>: HTTP status code is not handled or not allowed 2026-01-24 22:18:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securall-primer-coat-saltwater-agb1200>: HTTP status code is not handled or not allowed 2026-01-24 22:18:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compact-pro-high-bay-led-18000-lumens-120v-277v-4000k>: HTTP status code is not handled or not allowed 2026-01-24 22:18:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-o-smith-5-5-8-direct-drive-blower-motor-208-230-volts-1075-rpm-1>: HTTP status code is not handled or not allowed 2026-01-24 22:18:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beaker-pmp-tpx-printed-graduations-5000ml-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:18:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microcentrifuge-tube-20ml-polypropylene-attached-snap-cap-graduated-natural-1000-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:18:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spark-plug-standard-denso-auto-parts-6022>: HTTP status code is not handled or not allowed 2026-01-24 22:18:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cryoclear-vials-3ml-external-threads-self-standing-polypropylene-50-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:18:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/structural-plastic-solid-shelf-66-x-36>: HTTP status code is not handled or not allowed 2026-01-24 22:18:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/structural-plastic-dunnage-cube-24-x-12-x-6>: HTTP status code is not handled or not allowed 2026-01-24 22:18:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700-series-padlock-1-and-half-inch-shackle-blue-keyed-different>: HTTP status code is not handled or not allowed 2026-01-24 22:18:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leviton-16242-e-15a-125v-decora-plus-duplex-receptacle-commercial-grade-black>: HTTP status code is not handled or not allowed 2026-01-24 22:18:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-40sds60-qd-bushed-12-pitch-60-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:18:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-care-cobra-6-air-duct-cleaning-machine>: HTTP status code is not handled or not allowed 2026-01-24 22:18:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-o-smith-3-phase-square-flange-pump-motor-208-230-460-volts-2hp>: HTTP status code is not handled or not allowed 2026-01-24 22:18:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-bags-12-x-15-2-mil-blue-1000-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:18:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/boost-max-12v-5250a-jump-starter>: HTTP status code is not handled or not allowed 2026-01-24 22:18:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-20a15h-metric-a-plate-1-14-pitch-25mm-bore-15-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:18:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-bank-5a-onboard-battery-charger>: HTTP status code is not handled or not allowed 2026-01-24 22:18:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-3-3-double-sha-motor-115-volts-1550-rpm-1>: HTTP status code is not handled or not allowed 2026-01-24 22:18:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-care-20-x25-x5-wide-frame-electrostatic-air-filter-merv-8>: HTTP status code is not handled or not allowed 2026-01-24 22:18:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-care-20-x20-x4-wide-frame-electrostatic-air-filter-merv-8>: HTTP status code is not handled or not allowed 2026-01-24 22:18:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/half-round-side-open-trash-can-9-gallon-matte-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 22:18:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/02ml-96-well-pcr-plate-no-skirt-flat-top-clear-10-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:18:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-12bs23hx35-metric-34-pitch-35mm-finished-bore-23-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:18:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/armor-wrap-vci-paper-30r-9-x-9-sheets-1000-sheets-pkg>: HTTP status code is not handled or not allowed 2026-01-24 22:18:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/packard-c-frame-nutone-replacement-motor-120-volts-3000-rpm-6>: HTTP status code is not handled or not allowed 2026-01-24 22:18:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35bs18hx34-38-pitch-34-finished-bore-18-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:18:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/folding-door-gate-48-w-x-81-h-81>: HTTP status code is not handled or not allowed 2026-01-24 22:18:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50bs35hx1316-58-pitch-1-316-finished-bore-35-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:18:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-the-basic-one-body-foam-roller-14-1273>: HTTP status code is not handled or not allowed 2026-01-24 22:18:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-12bs24hx30-metric-34-pitch-30mm-finished-bore-24-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:18:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/384-well-pcr-plate-a24-p24-two-notch-design-roche-style-white-10-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:18:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honeycomb-pallet-cushion-sheets-40-x-48-x-2-20-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:18:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triple-wall-gaylord-bottoms-40-x-40-x-40-5-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:18:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shake-well-2-dia-fluorescent-red-black>: HTTP status code is not handled or not allowed 2026-01-24 22:18:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paper-twist-ties-5-x-5-32-yellow-2000-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:18:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/number-1-adjustable-height-complete-packing-workbench-laminate-safety-edge-60-x-30-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:18:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/01ml-96-well-pcr-plate-low-profile-fast-half-skirt-ridged-rim-clear-10-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:18:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anti-static-bubble-roll-with-dispenser-12-x-50-x-1-2>: HTTP status code is not handled or not allowed 2026-01-24 22:18:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cotton-twine-8-ply-20-lb-6300>: HTTP status code is not handled or not allowed 2026-01-24 22:18:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-100sk14h-taper-bushed-1-14-pitch-14-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:18:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/02ml-8-strip-dome-caps-no-tubes-natural-125-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:18:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-gauge-20-ft-600-amp-parrot-clamp-booster-cables>: HTTP status code is not handled or not allowed 2026-01-24 22:18:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-twist-ties-10-x-5-32-green-2000-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:18:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ntep-bench-scale-led-display-100-lb-x-0-2-lb>: HTTP status code is not handled or not allowed 2026-01-24 22:18:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-ton-king-pin-pusher-set>: HTTP status code is not handled or not allowed 2026-01-24 22:18:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-3-3-shaded-pole-open-motor-115-volts-1500-rpm-4>: HTTP status code is not handled or not allowed 2026-01-24 22:18:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-tier-3-door-premium-steel-locker-12-w-x-15-d-x-72-h-black-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 22:18:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/morrill-cast-iron-unit-bearing-fan-motor-9-watts-115-volts-1>: HTTP status code is not handled or not allowed 2026-01-24 22:18:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/job-sight-class-1-short-sleeve-t-shirt-black-with-fluorescent-yellow-green-4xl>: HTTP status code is not handled or not allowed 2026-01-24 22:18:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/to-go-rack-silver-ep-24w-x-54l-x-63h-4-wire-shelves-dividers-ledges>: HTTP status code is not handled or not allowed 2026-01-24 22:18:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60bs18hx112-34-pitch-1-12-finished-bore-18-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:18:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-5-split-capacitor-unit-heater-motor-115-volts-1075-rpm-2>: HTTP status code is not handled or not allowed 2026-01-24 22:18:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/job-sight-class-1-short-sleeve-t-shirt-black-with-fluorescent-yellow-green-tape-s>: HTTP status code is not handled or not allowed 2026-01-24 22:18:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluorescent-green-3-5>: HTTP status code is not handled or not allowed 2026-01-24 22:18:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commercial-80lb-broadcast-13-stud-type-tires-semi-assembled>: HTTP status code is not handled or not allowed 2026-01-24 22:18:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-capacity-broadcast-spreader-with-8in-wheels>: HTTP status code is not handled or not allowed 2026-01-24 22:18:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-container-4011-2-3-8-x-7-8>: HTTP status code is not handled or not allowed 2026-01-24 22:18:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dont-break-down-skid-2-3-fluorescent-red-black>: HTTP status code is not handled or not allowed 2026-01-24 22:18:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-3-3-permanent-split-capacitor-motor-208-230-volts-3000-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:18:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-aluminum-camlock-fitting-male-barb-x-male-coupler-thread>: HTTP status code is not handled or not allowed 2026-01-24 22:18:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-trash-can-lid-55-gallon-brown>: HTTP status code is not handled or not allowed 2026-01-24 22:18:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-16bs12hx40-metric-1-pitch-40mm-finished-bore-12-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:18:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-100bs11hx2316-1-14-pitch-2-316-finished-bore-11-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:18:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fragile-2-1-2-2-1-2-white-red>: HTTP status code is not handled or not allowed 2026-01-24 22:18:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-2-aluminum-camlock-fitting-male-coupler-x-fpt-thread>: HTTP status code is not handled or not allowed 2026-01-24 22:19:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-reflective-plastic-barrier-chain-2-x-25-ft-evergreen>: HTTP status code is not handled or not allowed 2026-01-24 22:19:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-1-1-2-mr-clip-safety-pink-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-24 22:19:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/curbside-cart-silver-ep-21w-x-30l-x-40h-3-wire-shelves-polyurethane-2-rigid-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:19:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-two-tone-poly-filled-winter-insulated-bibs-w-multi-pockets-lg-xl>: HTTP status code is not handled or not allowed 2026-01-24 22:19:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-2-mr-clip-white-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-24 22:19:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/to-go-rack-chrome-21w-x-48l-x-63h-4-wire-shelves-dividers-ledges>: HTTP status code is not handled or not allowed 2026-01-24 22:19:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-2-master-link-traffic-blue-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 22:19:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forkli-container-ramp-60-w-x-72-l-15-000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:19:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-wheel-riser-wheel-ramp-84-l-x-24-w-x-8-1-4-h-20-000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:19:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rb4310-men-s-athletic-work-shoe-black-grey-size-9-5-m>: HTTP status code is not handled or not allowed 2026-01-24 22:19:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bubble-lined-poly-mailers-7-14-1-4-x-20-white-50-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:19:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ready-to-ship-3-5-bright-yellow-black>: HTTP status code is not handled or not allowed 2026-01-24 22:19:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-e-premium-two-tone-poly-filled-winter-insulated-bibs-w-multi-pockets-lg-xl>: HTTP status code is not handled or not allowed 2026-01-24 22:19:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 29796 pages (at 100 pages/min), scraped 14298 items (at 21 items/min) 2026-01-24 22:19:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inspect-for-damage-1-3-8-2-fluorescent-red-black>: HTTP status code is not handled or not allowed 2026-01-24 22:19:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-multi-color-vest-red-4xl-5xl>: HTTP status code is not handled or not allowed 2026-01-24 22:19:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-multi-color-vest-black-s-m>: HTTP status code is not handled or not allowed 2026-01-24 22:19:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thu-1-dia-fluorescent-green-black>: HTTP status code is not handled or not allowed 2026-01-24 22:19:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-20bs14hx42-metric-1-14-pitch-42mm-finished-bore-14-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:19:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-slotted-pallet-rack-starter-no-deck-96-inches-w-x-42-inches-d-x-96-inches-h>: HTTP status code is not handled or not allowed 2026-01-24 22:19:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/no-m1kalfsts-magnum-laminated-steel-padlock-keyed-alike>: HTTP status code is not handled or not allowed 2026-01-24 22:19:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-tip-out-bin-unit-double-sided-48-h-beige>: HTTP status code is not handled or not allowed 2026-01-24 22:19:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ula-choke-15w-120v-3a-jengyu>: HTTP status code is not handled or not allowed 2026-01-24 22:19:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-100bs17hx212-1-14-pitch-2-12-finished-bore-17-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:19:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/accepted-by-3-5-fluorescent-green-black>: HTTP status code is not handled or not allowed 2026-01-24 22:19:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-100bs17hx11516-1-14-pitch-1-1516-finished-bore-17-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:19:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mantis-1x2-discretion-fly-light-stainless-steel-3-case>: HTTP status code is not handled or not allowed 2026-01-24 22:19:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-piece-phillips-slotted-dual-material-screwdriver-set>: HTTP status code is not handled or not allowed 2026-01-24 22:19:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/csi-kit-replacement-parts-flash-light-with-charger>: HTTP status code is not handled or not allowed 2026-01-24 22:19:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-piece-phillips-slottedpozidriv-dual-material-screwdriver-set>: HTTP status code is not handled or not allowed 2026-01-24 22:19:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1627-250-250-1-4-id-push-on-and-lock-hose>: HTTP status code is not handled or not allowed 2026-01-24 22:19:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/food_storage/food_storage_containers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:19:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60bs27hx112-34-pitch-1-12-finished-bore-27-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:19:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/geometric-style-bohemian-shag-area-rug-5-ft-x-7-ft-ivory>: HTTP status code is not handled or not allowed 2026-01-24 22:19:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80bs24x114-1-pitch-1-14-finished-bore-24-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:19:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80bs24x178-1-pitch-1-78-finished-bore-24-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:19:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mini-airheads-80-count-gr>: HTTP status code is not handled or not allowed 2026-01-24 22:19:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-drawer-mobile-cabinet-36-wx24-dx35-1-2-h-red>: HTTP status code is not handled or not allowed 2026-01-24 22:19:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sunburst-discreet-fly-light-black>: HTTP status code is not handled or not allowed 2026-01-24 22:19:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exq-17-drawer-pro-triple-bank-roller-72-w-x-30-d-x-h-44-3-4-green-w-black-drawer-pulls>: HTTP status code is not handled or not allowed 2026-01-24 22:19:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-9034-1-6-hp-1-phase-1075-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:19:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-wdp2s2ahc-2-hp-3-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:19:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/puller-dent-81500>: HTTP status code is not handled or not allowed 2026-01-24 22:19:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/food_storage/food_storage_containers>: HTTP status code is not handled or not allowed 2026-01-24 22:19:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ring-pop-individually-wrapped-lollipop-candy-40-count-bulk-tub-gr>: HTTP status code is not handled or not allowed 2026-01-24 22:19:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-wdp5e2ac-5-hp-3-phase-1755-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:19:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/skittles-fun-size-packs-4-lb-gr>: HTTP status code is not handled or not allowed 2026-01-24 22:19:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-drawer-mobile-multi-drawer-cabinet-60-wx27-dx37-1-2-h-everest-blue-1>: HTTP status code is not handled or not allowed 2026-01-24 22:19:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-ff15e1gv-15-hp-3-phase-3495-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:19:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-4125-1-30-hp-1-phase-1100-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:19:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-5454-1-5-hp-1-phase-1075-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:19:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=us-motor-9034-1-6-hp-1-phase-1075-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:19:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-ceiling-mount-ventilation-fan-434-cfm-2-3-sones-l400>: HTTP status code is not handled or not allowed 2026-01-24 22:19:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-step-24-wx70-d-stainless-steel-rolling-safety-ladder-serrated-grating>: HTTP status code is not handled or not allowed 2026-01-24 22:19:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-step-24-wx70-d-stainless-steel-rolling-safety-ladder-perforated-tread>: HTTP status code is not handled or not allowed 2026-01-24 22:19:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-step-24-wx46-d-stainless-steel-rolling-safety-ladder-perforated-tread>: HTTP status code is not handled or not allowed 2026-01-24 22:19:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/three-panel-mobile-medical-privacy-screen-81-w-x-69-h-green-vinyl-panels>: HTTP status code is not handled or not allowed 2026-01-24 22:19:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-sugar-box-gr>: HTTP status code is not handled or not allowed 2026-01-24 22:19:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/44-h-cal-osha-handrail-kit-extra-rolling-safety-ladders>: HTTP status code is not handled or not allowed 2026-01-24 22:19:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slim-jim-beef-and-cheese-1-5-oz-18-count-gr>: HTTP status code is not handled or not allowed 2026-01-24 22:19:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-t2c2jhc-2-hp-1-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:19:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-t34ca2j-3-4-hp-1-phase-1725-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:19:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20v-max-xr-versa-clutch-adjustable-torque-screwgun-kit-dcf622m2>: HTTP status code is not handled or not allowed 2026-01-24 22:19:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-inch-high-performance-paddle-switch-grinder-dwe43144>: HTTP status code is not handled or not allowed 2026-01-24 22:19:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20v-max-xr-versa-clutch-adjustable-torque-screwgun-bare-tool-only-dcf622b>: HTTP status code is not handled or not allowed 2026-01-24 22:19:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20v-max-xr-7-inch-cordless-variable-speed-rotary-polisher-bare-tool-only-dcm849b>: HTTP status code is not handled or not allowed 2026-01-24 22:19:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-step-extra-rolling-safety-ladder-serrated-grating>: HTTP status code is not handled or not allowed 2026-01-24 22:19:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/atomic-20v-max-cordless-one-handed-reciprocating-saw-bare-tool-only-dcs369b>: HTTP status code is not handled or not allowed 2026-01-24 22:19:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cp3-pallet-jack-truck-27-x-48-forks-5500-lb-capacity-orange>: HTTP status code is not handled or not allowed 2026-01-24 22:19:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20v-max-xr-compact-reciprocating-saw-kit-dcs367p1>: HTTP status code is not handled or not allowed 2026-01-24 22:19:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-h5v1bc-5-hp-3-phase-3520-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:19:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20v-max-xr-brushless-high-torque-3-4-inch-impact-wrench-hog-ring-bare-tool-only-dcf897b>: HTTP status code is not handled or not allowed 2026-01-24 22:20:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=us-motor-t34ca2j-3-4-hp-1-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:20:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ax-series-42-shutter-mounted-fan-1-hp-850-rpm-230v>: HTTP status code is not handled or not allowed 2026-01-24 22:20:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-d10v1b-10-hp-3-phase-3505-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:20:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-in-9-outlet-power-strip-surge-protector-15-ft-cord>: HTTP status code is not handled or not allowed 2026-01-24 22:20:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-eh741-3-hp-3-phase-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:20:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-infection-control-station-with-3-dispensers-double-sided-poster-frame-silver-lav-2228s-3d>: HTTP status code is not handled or not allowed 2026-01-24 22:20:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lbr-6-inch-round-retrofit-downlight-1000-2000-lumens-3000-5000k-silver>: HTTP status code is not handled or not allowed 2026-01-24 22:20:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-u5p1g-5-hp-3-phase-3525-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:20:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lbr-4-inch-round-retrofit-downlight-1000-2000-lumens-3000-5000k-silver>: HTTP status code is not handled or not allowed 2026-01-24 22:20:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/technical-workbench-w-3-4-drawer-cabinets-plastic-laminate-top-blue-1>: HTTP status code is not handled or not allowed 2026-01-24 22:20:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wwe-khn47-60-1-h-56c-cast-iron-helical-bevel-speed-reducer-56c-input-flange-60-1-foot-mt>: HTTP status code is not handled or not allowed 2026-01-24 22:20:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 29890 pages (at 94 pages/min), scraped 14326 items (at 28 items/min) 2026-01-24 22:20:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-butcher-block-24-w-shallow-depth-mobile-cabinet-top>: HTTP status code is not handled or not allowed 2026-01-24 22:20:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drop-spreader-6-0-cubic-feet-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 22:20:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-d32c2p18z-1-5-hp-1-phase-1740-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:20:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wwe-khn47-7-5-1-h-182-4tc-cast-iron-helical-bevel-speed-reducer-182-4tc-input-flange-7-5-1-ft-mt>: HTTP status code is not handled or not allowed 2026-01-24 22:20:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-ec3002-3-hp-1-phase-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:20:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-assembled-electronic-access-locker-single-tier-1-wide-12x15x78>: HTTP status code is not handled or not allowed 2026-01-24 22:20:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beaker-globe-glass-low-form-griffin-style-dual-graduations-astm-e960-600ml-6-box>: HTTP status code is not handled or not allowed 2026-01-24 22:20:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbi-sala-174-6100564-fixed-ladder-srl-anchor-6-galvanized-1-ea>: HTTP status code is not handled or not allowed 2026-01-24 22:20:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-assembled-antimicrobial-locker-double-tier-1-wide-15x18x78>: HTTP status code is not handled or not allowed 2026-01-24 22:20:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-t2p2ahz-2-hp-3-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:20:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-assembled-corrosion-resistant-locker-single-tier-3-wide-12x12x78>: HTTP status code is not handled or not allowed 2026-01-24 22:20:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securitymax-high-security-solid-welded-locker-24x22x72>: HTTP status code is not handled or not allowed 2026-01-24 22:20:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-assembled-corrosion-resistant-locker-double-tier-1-wide-12x12x78>: HTTP status code is not handled or not allowed 2026-01-24 22:20:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-u32p2d-1-5-hp-3-phase-1755-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:20:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-u2p2h-2-hp-3-phase-1740-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:20:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/strobe-120vac-pipe-mount-red-131st-120r>: HTTP status code is not handled or not allowed 2026-01-24 22:20:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-yc2p1b-2-hp-3-phase-3500-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:20:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steady-burn-light-120vac-red-lp2-120r>: HTTP status code is not handled or not allowed 2026-01-24 22:20:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-rivet-steel-shelving-with-wood-deck-72-w-x-24-d-x-84-h-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:20:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lau-3-blade-condenser-propeller-20-diameter-ccw-rotation-1>: HTTP status code is not handled or not allowed 2026-01-24 22:20:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-xs1ca1j-1-hp-1-phase-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:20:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lau-double-inlet-blower-wheel-11-1-8-diameter-1-2-bore-12>: HTTP status code is not handled or not allowed 2026-01-24 22:20:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-c200p2cs-200-hp-3-phase-1785-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:20:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lau-4-blade-condenser-propeller-26-diameter-cw-rotation-3>: HTTP status code is not handled or not allowed 2026-01-24 22:20:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-d7p3g-7-5-hp-3-phase-1185-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:20:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gussetted-poly-bag-rolls-15x9x32-1-5-mil-500-rl>: HTTP status code is not handled or not allowed 2026-01-24 22:20:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rs15i-a145-200-3-60-base-mount-145-psi>: HTTP status code is not handled or not allowed 2026-01-24 22:20:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-conductive-bags-12x18-4-mil-100-ctn>: HTTP status code is not handled or not allowed 2026-01-24 22:20:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-4742-1-6-hp-1-phase-1650-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:20:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-kit-camshelving-24x72-vented-shelves-1>: HTTP status code is not handled or not allowed 2026-01-24 22:20:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-ring-3-4-tool-holder>: HTTP status code is not handled or not allowed 2026-01-24 22:20:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-5482h-3-4-1-4-hp-1-phase-1075-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:20:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/46-piece-hook-assortment>: HTTP status code is not handled or not allowed 2026-01-24 22:20:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-3135-1-3-hp-1-phase-1075-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:20:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/threaded-stem-so-tread-caster-with-brake-75mm-dia-1-2-13-x-1-stem-black>: HTTP status code is not handled or not allowed 2026-01-24 22:20:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wwe-kan37-40-1-h-56c-cast-iron-helical-bevel-speed-reducer-56c-input-flange-40-1>: HTTP status code is not handled or not allowed 2026-01-24 22:20:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wwe-kan47-85-1-h-56c-cast-iron-helical-bevel-speed-reducer-56c-input-flange-85-1>: HTTP status code is not handled or not allowed 2026-01-24 22:20:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camtainer-soup-carrier-no-spigot-insulated-plastic-3-3-8-gallon-16-1-2x9x18-3-8-coffee-beige>: HTTP status code is not handled or not allowed 2026-01-24 22:20:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camwear-food-pan-1-3-size-6-deep-polycarbonate-clear-nsf>: HTTP status code is not handled or not allowed 2026-01-24 22:20:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/office-partition-panel-gray-60-1-4-x-96-h>: HTTP status code is not handled or not allowed 2026-01-24 22:20:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/versa-food-bars-work-table-cold-food-72-x-29-low-6-swivel-casters-with-brakes-black>: HTTP status code is not handled or not allowed 2026-01-24 22:20:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interion-freestanding-office-partition-panel-with-partial-window-48-1-4-w-x-96-h-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:20:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grip-strut-aluminum-walk-ramp-16-l-x-38-w>: HTTP status code is not handled or not allowed 2026-01-24 22:20:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interion-deluxe-mobile-office-partition-panel-with-partial-window-36-1-4-w-x-96-h-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:20:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-retrofit-door-for-upch-400-series-heated-ultra-pancarrier-220v-granite-green>: HTTP status code is not handled or not allowed 2026-01-24 22:20:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lid-for-10cw-bowl-clear>: HTTP status code is not handled or not allowed 2026-01-24 22:20:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-compartment-steel-literature-organizer-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:20:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/versa-well-cover-black-nsf>: HTTP status code is not handled or not allowed 2026-01-24 22:20:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-pocket-onyx-magazine-rack>: HTTP status code is not handled or not allowed 2026-01-24 22:20:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-6-magazine-12-pamphlet-display>: HTTP status code is not handled or not allowed 2026-01-24 22:20:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/organizer-rack-with-8-bins-black>: HTTP status code is not handled or not allowed 2026-01-24 22:20:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pet-waste-station-with-rolled-bags-green>: HTTP status code is not handled or not allowed 2026-01-24 22:20:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-locker-single-tier-1-wide-12-x-15-x-60-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:20:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-storage-case-medim-21-1-2-x-16-1-4-x-18-1-2>: HTTP status code is not handled or not allowed 2026-01-24 22:20:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cherry-counter-blue-panel-l-shaped-reception-station-3>: HTTP status code is not handled or not allowed 2026-01-24 22:20:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/square-food-container-6-quart-8-3-8-x-8-3-8-x-7-1-4-natural-white-orange-graduation>: HTTP status code is not handled or not allowed 2026-01-24 22:20:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manual-18-diameter-pedestal-turntable-21-to-32-h-300-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:20:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manual-24-diameter-turntable-7-8-h-1000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:20:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reference-shelf-30-wx24-d-shelf-with-hardware>: HTTP status code is not handled or not allowed 2026-01-24 22:20:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medium-back-caresso-8482-guest-chair-black-1>: HTTP status code is not handled or not allowed 2026-01-24 22:20:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-flat-top-cabinet-hinged-doors-galv-30-x-96>: HTTP status code is not handled or not allowed 2026-01-24 22:20:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/meal-delivery-cart-tall-profile-3-doors-60x29-1-4x63-5-8-granite-gray-with-cream-color-door>: HTTP status code is not handled or not allowed 2026-01-24 22:20:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/handwashing-station-w-paper-towel-and-soap-dispenser-use-w-2-1-2-or-5-gallon-camtainer-dark-brown>: HTTP status code is not handled or not allowed 2026-01-24 22:20:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-angle-frame-mirror-with-shelf-48-wx30-h>: HTTP status code is not handled or not allowed 2026-01-24 22:20:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camshelving-elements-shelf-plate-kit-14-w-x-72-l-solid-brushed-graphite>: HTTP status code is not handled or not allowed 2026-01-24 22:21:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reducer-5-16-male-stud-x-5-32-tube>: HTTP status code is not handled or not allowed 2026-01-24 22:21:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camshelving-add-on-unit-18-w-x-60-l-x-64-h-brushed-graphite>: HTTP status code is not handled or not allowed 2026-01-24 22:21:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-flat-top-workbench-72-w-x-24-d-2>: HTTP status code is not handled or not allowed 2026-01-24 22:21:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wood-storage-cabinet-48x22x84>: HTTP status code is not handled or not allowed 2026-01-24 22:21:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cast-iron-jaw-coupling-hub-style-l075-5-16-inch-bore-diameter-no-keyway>: HTTP status code is not handled or not allowed 2026-01-24 22:21:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-male-metal-release-collet-branch-tee-4mm-tube-x-m5-bspp>: HTTP status code is not handled or not allowed 2026-01-24 22:21:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/female-bulkhead-connector-metal-release-collet-1-2-tube-x-1-2-nptf>: HTTP status code is not handled or not allowed 2026-01-24 22:21:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-male-metal-release-collet-6mm-tube-x-1-8-swift-fit>: HTTP status code is not handled or not allowed 2026-01-24 22:21:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reducer-10mm-stud-x-6mm-tube>: HTTP status code is not handled or not allowed 2026-01-24 22:21:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/union-y-manifold-6mm-tube-x-8mm-tube>: HTTP status code is not handled or not allowed 2026-01-24 22:21:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forced-air-circulation-incubator-programmable-256-liters-115-volt>: HTTP status code is not handled or not allowed 2026-01-24 22:21:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 29988 pages (at 98 pages/min), scraped 14348 items (at 22 items/min) 2026-01-24 22:21:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-level-48-inch-l>: HTTP status code is not handled or not allowed 2026-01-24 22:21:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-electronic-left-hand-caliper-0-8-inch-0-200mm-range>: HTTP status code is not handled or not allowed 2026-01-24 22:21:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v-block-set-2-inch-l-x-11-16-inch-w-x-7-8-inch-h-pair>: HTTP status code is not handled or not allowed 2026-01-24 22:21:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-sidedshield-24-inch-reach-w-o-interlock-16-inch-l-x-14-inch-w-x-12-inch-h>: HTTP status code is not handled or not allowed 2026-01-24 22:21:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shield-octagonal-24-inch-reach-w-interlock-led-24-inch-l-x-10-inch-w-24-lb>: HTTP status code is not handled or not allowed 2026-01-24 22:21:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shield-octagonal-24-inch-reach-w-interlock-led-28-inch-l-x-10-inch-w-30-lb>: HTTP status code is not handled or not allowed 2026-01-24 22:21:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shield-octagonal-direct-mount-w-interlock-6-inch-l-x-8-inch-w-14-inch-offset>: HTTP status code is not handled or not allowed 2026-01-24 22:21:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shield-octagonal-direct-mount-w-o-interlock-20-inch-l-x-10-inch-w>: HTTP status code is not handled or not allowed 2026-01-24 22:21:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shield-curved-direct-mount-w-interlock-led-28-inch-l-x-10-inch-w>: HTTP status code is not handled or not allowed 2026-01-24 22:21:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-grade-looped-end-mop-w-acme-thread-connection-blue-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-24 22:21:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ecp-single-face-foam-dispenser-36-inch-l>: HTTP status code is not handled or not allowed 2026-01-24 22:21:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ecp-bi-support-stand-w-automatic-slide-cutter-48-inch-l>: HTTP status code is not handled or not allowed 2026-01-24 22:21:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/birdwire-stainless-steel-500l>: HTTP status code is not handled or not allowed 2026-01-24 22:21:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extension-poles-w-nonslip-safety-grip-12h-aluminum>: HTTP status code is not handled or not allowed 2026-01-24 22:21:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tree-shock-bird-deterrent-for-trees-15l>: HTTP status code is not handled or not allowed 2026-01-24 22:21:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/synthetic-winch-line-w-integrated-shackle-916-dia-x-200l>: HTTP status code is not handled or not allowed 2026-01-24 22:21:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/synthetic-winch-line-w-integrated-shackle-38-dia-x-200l>: HTTP status code is not handled or not allowed 2026-01-24 22:21:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sj-24-inch-snow-blower-kit-w-batteries--quad-charger-96v-12-amp-hours>: HTTP status code is not handled or not allowed 2026-01-24 22:21:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbw-antimicrobial-vinyl-basket-truck-w-bumper-18-bushel-capacity-navy>: HTTP status code is not handled or not allowed 2026-01-24 22:21:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbw-vinyl-bumper-truck-all-swivel-20-bushel-capacity-yellow>: HTTP status code is not handled or not allowed 2026-01-24 22:21:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbw-antimicrobial-vinyl-basket-truck-all-swivel-20-bushel-capacity-navy>: HTTP status code is not handled or not allowed 2026-01-24 22:21:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbw-poly-bumper-truck-10-bushel-capacity-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:21:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aquavantage-aquaflush-exp-urinal-fv-3-0-gpf>: HTTP status code is not handled or not allowed 2026-01-24 22:21:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/intercept-micro-filter-bag-fits-proteam-gofit-3-backpack-vacuum-open-collar-10-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:21:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/weldon-6-backless-picnic-bench-coastal-teak>: HTTP status code is not handled or not allowed 2026-01-24 22:21:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bistro-36-dining-height-table-round-black>: HTTP status code is not handled or not allowed 2026-01-24 22:21:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/outdoor_structures_storage/gazebos_pavilions_pergolas already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:21:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/muskoka-adirondack-3-piece-bistro-dining-set-30h-table-whitewash>: HTTP status code is not handled or not allowed 2026-01-24 22:21:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/muskoka-adirondack-3-piece-bistro-dining-set-30h-table-black>: HTTP status code is not handled or not allowed 2026-01-24 22:21:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:21:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/largo-silver-lines-slab-door-wood-30w-x-80h-white>: HTTP status code is not handled or not allowed 2026-01-24 22:21:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/naples-night-lines-slab-door-wood-36w-x-80h-whiskey-oak>: HTTP status code is not handled or not allowed 2026-01-24 22:21:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:21:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sarasota-slab-door-wood-36w-x-80h-white>: HTTP status code is not handled or not allowed 2026-01-24 22:21:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/palmdale-slab-door-wood-24w-x-80h-white>: HTTP status code is not handled or not allowed 2026-01-24 22:21:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:21:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cape-coral-night-lines-slab-door-wood-28w-x-80h-white>: HTTP status code is not handled or not allowed 2026-01-24 22:21:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/largo-gold-lines-slab-door-wood-30w-x-80h-ice-maple>: HTTP status code is not handled or not allowed 2026-01-24 22:21:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:21:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/naples-gold-lines-slab-door-wood-30w-x-80h-white>: HTTP status code is not handled or not allowed 2026-01-24 22:21:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/orlando-night-lines-slab-door-wood-30w-x-80h-white>: HTTP status code is not handled or not allowed 2026-01-24 22:21:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:21:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/outdoor_structures_storage/gazebos_pavilions_pergolas>: HTTP status code is not handled or not allowed 2026-01-24 22:21:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/excalibur-handle-for-european-lockset-chrome>: HTTP status code is not handled or not allowed 2026-01-24 22:21:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors>: HTTP status code is not handled or not allowed 2026-01-24 22:21:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-shower-commode-chair-slideout-footrest-310-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:21:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors>: HTTP status code is not handled or not allowed 2026-01-24 22:21:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hog-heavenanti-fatigue-mat-58-thick-20l-x-5w-black-striped-border>: HTTP status code is not handled or not allowed 2026-01-24 22:21:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors>: HTTP status code is not handled or not allowed 2026-01-24 22:21:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hog-heavenanti-fatigue-mat-78-thick-15l-x-5w-black-striped-border>: HTTP status code is not handled or not allowed 2026-01-24 22:21:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors>: HTTP status code is not handled or not allowed 2026-01-24 22:21:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polarforce-hiker-boots-size-6-black>: HTTP status code is not handled or not allowed 2026-01-24 22:21:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors>: HTTP status code is not handled or not allowed 2026-01-24 22:21:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extreme-hiker-boots-size-9-black>: HTTP status code is not handled or not allowed 2026-01-24 22:21:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polarforce-mens-sweatshirt-3xl-black>: HTTP status code is not handled or not allowed 2026-01-24 22:21:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wego-polystyrene-fork-18-l-black-pack-of-1000>: HTTP status code is not handled or not allowed 2026-01-24 22:21:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/renewable-compostable-sugarcane-plates-6-dia-natural-white-pack-of-1000>: HTTP status code is not handled or not allowed 2026-01-24 22:21:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gen-heavy-duty-aluminum-foil-roll-500-l-x-12-w-silver-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-24 22:21:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durable-packaging-heavy-duty-aluminum-foil-roll-1000-l-x-24-w-silver>: HTTP status code is not handled or not allowed 2026-01-24 22:21:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metro-pop-up-aluminum-foil-sheets-10-3-4-l-x-12-w-silver-pack-of-3000>: HTTP status code is not handled or not allowed 2026-01-24 22:21:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/newspring-delitainer-microwavable-container-2-l-x-2-w-x-2-h-pack-of-240>: HTTP status code is not handled or not allowed 2026-01-24 22:21:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microgourmet-deli-container-4-11-16-dia-x-5-11-16-h-clear-pack-of-500>: HTTP status code is not handled or not allowed 2026-01-24 22:21:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-heat-cook-s-scraper-9-1-2-l-red-white>: HTTP status code is not handled or not allowed 2026-01-24 22:21:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durable-packaging-aluminum-round-container-3-dia-x-1-9-16-h-silver-pack-of-500>: HTTP status code is not handled or not allowed 2026-01-24 22:21:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/bar_restaurant_equipment/dispensers_organizers/condiment_caddies_sugar_holders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:21:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guest-check-book-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-24 22:21:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bun-rack-covers-high-density-80-l-x-52-w-natural-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-24 22:21:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paper-food-container-white-pack-of-250>: HTTP status code is not handled or not allowed 2026-01-24 22:21:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caterline-casuals-plastic-platters-12-dia-black-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-24 22:22:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iodized-salt-packets-0-75-g-pack-of-3000>: HTTP status code is not handled or not allowed 2026-01-24 22:22:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paper-hot-drink-cups-10-oz-pack-of-1k>: HTTP status code is not handled or not allowed 2026-01-24 22:22:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hard-candy-mints-pep-o-mint-44-93-oz-bag>: HTTP status code is not handled or not allowed 2026-01-24 22:22:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lifting-synthetic-shackle-5-16-dia-x-5-l-2-5-8-ton-wll>: HTTP status code is not handled or not allowed 2026-01-24 22:22:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/colormaxx-paint-primer-12-oz-black-primer-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-24 22:22:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-boltless-steel-shelf-60-in-w-x-24-in-d>: HTTP status code is not handled or not allowed 2026-01-24 22:22:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/bar_restaurant_equipment/dispensers_organizers/condiment_caddies_sugar_holders>: HTTP status code is not handled or not allowed 2026-01-24 22:22:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-cooled-chiller-w-closed-loop-water-application-1-ton-capacity-9373-btus-per-hour>: HTTP status code is not handled or not allowed 2026-01-24 22:22:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-visibility-windbreaker-water-resistant-jacket-type-r-class-3-orange-medium>: HTTP status code is not handled or not allowed 2026-01-24 22:22:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/terry-adult-bib-w-hook-loop-strap-21l-x-33w-blue-12-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:22:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 30078 pages (at 90 pages/min), scraped 14366 items (at 18 items/min) 2026-01-24 22:22:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/basic-steel-rollator-w-8-casters-burgundy>: HTTP status code is not handled or not allowed 2026-01-24 22:22:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simplicity-2-rollator-w-8-casters-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:22:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-instant-cold-pack-9l-x-5-34w-pack-of-24>: HTTP status code is not handled or not allowed 2026-01-24 22:22:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/padded-steel-drop-arm-commode-350-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:22:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pathway-wheelchair-ramp-kit-l-shaped-aluminum-4-turn-platform-16l-x-36w>: HTTP status code is not handled or not allowed 2026-01-24 22:22:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-pharmacy-upright-refrigerator-16-cu-ft-capacity-glass-door>: HTTP status code is not handled or not allowed 2026-01-24 22:22:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torch-flux-welding-gloves-large-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:22:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5th-wheel-garden-truck-steel-1200-lb-capacity-64-1-2l-x-36w-x-36-3-4h>: HTTP status code is not handled or not allowed 2026-01-24 22:22:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-panel-moving-truck-steel-3600-lb-cap-39-5-16l-x-24w-x-36h>: HTTP status code is not handled or not allowed 2026-01-24 22:22:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mesh-truck-w-1-adjustable-shelf-steel-2000-lb-cap-66-1-2l-x-26w-x-56-7-16h>: HTTP status code is not handled or not allowed 2026-01-24 22:22:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flexible-stainless-steel-ballcock-connector-16l-38-x-78-od-x-bc-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-24 22:22:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flexible-stainless-steel-faucet-connector-12l-12-x-12-fip-x-fip-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-24 22:22:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delta-seat-spring-kit-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-24 22:22:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-foot-operated-valve>: HTTP status code is not handled or not allowed 2026-01-24 22:22:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-faucet-w-12-spout--soap-dish>: HTTP status code is not handled or not allowed 2026-01-24 22:22:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brass-shower-head-2-12-dia-18-gpm>: HTTP status code is not handled or not allowed 2026-01-24 22:22:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-lever-kitchen-faucet-w-side-spray-lever-handle-satin-nickel>: HTTP status code is not handled or not allowed 2026-01-24 22:22:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/copper-corrugated-water-heater-connector-34-fip-x-15l-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-24 22:22:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delta-pressure-balance-cartridge-for-multi-choice-17-series-valves-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-24 22:22:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lavatory-kitchen-handles-for-price-pfister-new-style-1-pair-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-24 22:22:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delta-pressure-balance-cartridge-for-for-pre-2006-1314-series-valves-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-24 22:22:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vertical-dumbbell-rack-28l-x-23w-x-50-12h-black>: HTTP status code is not handled or not allowed 2026-01-24 22:22:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/business-furniture-hybrid-bookcase-w-2-shelves-36w-x-16d-x-30h-white>: HTTP status code is not handled or not allowed 2026-01-24 22:22:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/releasable-trailer-door-limit-strap-37l-yellow-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 22:22:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beverage-center-1-glass-door-31-cuft-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:22:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-tap-keg-cooler-1-glass-door-54-cuft-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:22:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wine-cooler-94-bottles-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:22:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/apartment-sized-refrigeratorfreezer-combo-unit-101-cuft-capacity-black>: HTTP status code is not handled or not allowed 2026-01-24 22:22:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/top-mount-refrigeratorfreezer-combo-unit-74-cuft-capacity-silver>: HTTP status code is not handled or not allowed 2026-01-24 22:22:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/appliances/cooking_equipment/microwaves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:22:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scentiva-disinfecting-wipes-pacific-breeze-coconut-75-wipescanister-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-24 22:22:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drive-ratchet-tool-w-38--14-interchangeable-anvils>: HTTP status code is not handled or not allowed 2026-01-24 22:22:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fs-48mag2-bl-cabinet-w-fixed-shelves-assembled-48w-x-18d-x-72h-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:22:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cabinet-w-fixed-shelves-assembled-48w-x-24d-x-72h-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:22:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/organizer-cabinet-assembled-24w-x-18d-x-72h-walnut>: HTTP status code is not handled or not allowed 2026-01-24 22:22:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cabinet-w-fixed-shelves-assembled-36w-x-18d-x-78h-red>: HTTP status code is not handled or not allowed 2026-01-24 22:22:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cabinet-w-fixed-shelves-assembled-36w-x-18d-x-78h-walnut>: HTTP status code is not handled or not allowed 2026-01-24 22:22:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bookcase-assembled-36w-x-18d-x-60h-black>: HTTP status code is not handled or not allowed 2026-01-24 22:22:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-wardrobe-cabinet-assembled-36w-x-24d-x-72h-charcoal>: HTTP status code is not handled or not allowed 2026-01-24 22:22:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bookcase-assembled-36w-x-18d-x-60h-leaf-green>: HTTP status code is not handled or not allowed 2026-01-24 22:22:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-wardrobe-cabinet-assembled-36w-x-24d-x-72h-pastel-green>: HTTP status code is not handled or not allowed 2026-01-24 22:22:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bookcase-assembled-36w-x-18d-x-55h-tropic-sand>: HTTP status code is not handled or not allowed 2026-01-24 22:22:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bookcase-assembled-36w-x-18d-x-60h-red>: HTTP status code is not handled or not allowed 2026-01-24 22:22:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/appliances/cooking_equipment/microwaves>: HTTP status code is not handled or not allowed 2026-01-24 22:22:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microtube-w-locktop-style-cap-15-ml-capacity-pack-of-500>: HTTP status code is not handled or not allowed 2026-01-24 22:22:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sterile-sampling-bag-18-oz-capacity-9l-x-4-12w-pack-of-500>: HTTP status code is not handled or not allowed 2026-01-24 22:22:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-premium-heavy-duty-vest-w-multi-pockets-class-2-4xl5xl-lime>: HTTP status code is not handled or not allowed 2026-01-24 22:22:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-waterproof-parka-jacket-w-fleece-lining-nylon-navy-5xl>: HTTP status code is not handled or not allowed 2026-01-24 22:22:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-diamond-quilted-jacket-w-removable-hood-blue-3xl>: HTTP status code is not handled or not allowed 2026-01-24 22:22:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-tone-short-sleeve-polo-shirt-w-1-pocket-polyester-black-3xl>: HTTP status code is not handled or not allowed 2026-01-24 22:22:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onyx-enhanced-visibility-full-zip-hoodie-polyester-fleece-navy-medium>: HTTP status code is not handled or not allowed 2026-01-24 22:22:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-tone-short-sleeve-polo-shirt-polyester-black-4xl>: HTTP status code is not handled or not allowed 2026-01-24 22:22:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/friedrich--thru-the-wall-air-conditioner-1490-watts-230-v-14000-btu>: HTTP status code is not handled or not allowed 2026-01-24 22:22:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/diy-4th-generation-e-star-heat-pump-condenser-12000-btu-225-seer-115v>: HTTP status code is not handled or not allowed 2026-01-24 22:22:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/diy-4th-generation-multi-zone-2-zone-condenser-27000-btu-22-seer-230v>: HTTP status code is not handled or not allowed 2026-01-24 22:22:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-bar--pipe-truck-w-2-rigid-casters-3600-lb-cap-60l-x-30w-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:22:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-shelf-truck-2-handle-3600-lb-capacity-71l-x-30w-x-34h-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:22:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-class-2-hi-vis-safety-vest-2-reflective-strips-poly-mesh-orange-s-m>: HTTP status code is not handled or not allowed 2026-01-24 22:22:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chill-its-6667-wet-evaporative-cooling-vest-pva-zipper-5xl-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:22:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8381-hi-vis-4-in-1-winter-bomber-jacket-3xl-orange>: HTTP status code is not handled or not allowed 2026-01-24 22:22:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8384-hi-vis-quilted-parka-winter-jacket-2xl-orange>: HTTP status code is not handled or not allowed 2026-01-24 22:22:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/squids-5541-handheld-barcode-scanner-holster-w-belt-loop-small-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:22:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/squids-5541-handheld-barcode-scanner-holster-w-belt-loop-large-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:22:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8289bk-hi-vis-t-shirt-black-bottom-class-2-xl-lime>: HTTP status code is not handled or not allowed 2026-01-24 22:22:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-leg-gaiters-two-tone-mesh-lime-1-pair>: HTTP status code is not handled or not allowed 2026-01-24 22:22:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8280bk-hi-vis-performance-t-shirt-black-bottom-class-2-m-orange>: HTTP status code is not handled or not allowed 2026-01-24 22:22:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-7001-abrasion-resistant-gloves-nitrile-coated-s-black-1-pair>: HTTP status code is not handled or not allowed 2026-01-24 22:22:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8289bk-hi-vis-t-shirt-black-bottom-class-2-3xl-orange>: HTTP status code is not handled or not allowed 2026-01-24 22:23:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-7022-cut-resistant-gloves-dsx-coated-ansi-a2-2xl-lime-1-pair>: HTTP status code is not handled or not allowed 2026-01-24 22:23:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-7000-nitrile-coated-gloves-w-microfoam-palm-xl-gray-12-pairs>: HTTP status code is not handled or not allowed 2026-01-24 22:23:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-7030-cut-resistant-gloves-polyurethane-coated-ansi-a3-m-gray-1-pair>: HTTP status code is not handled or not allowed 2026-01-24 22:23:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8005-hi-vis-hard-hat-neck-shade-orange>: HTTP status code is not handled or not allowed 2026-01-24 22:23:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-922cr-cut-resistant-gloves-nitrile-coated-ansi-a3-2xl-gray-96-pairs>: HTTP status code is not handled or not allowed 2026-01-24 22:23:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-bag-dispenser-10-116w-x-3-78d-x-3-12h-quartz-beige-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-24 22:23:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/34-air-impact-wrench-1100-ft-lbs-torque-pistol-grip-6-extended-anvil>: HTTP status code is not handled or not allowed 2026-01-24 22:23:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/no-5-air-impact-wrench-1770-ft-lbs-torque-d-handle-6-extended-anvil>: HTTP status code is not handled or not allowed 2026-01-24 22:23:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/367-x-675-random-orbital-air-sander-paper-clamp-pad-8000-rpm-03-hp>: HTTP status code is not handled or not allowed 2026-01-24 22:23:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extended-air-die-grinder-14-and-6mm-collets-7500-rpm-1-hp-rear-exhaust>: HTTP status code is not handled or not allowed 2026-01-24 22:23:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 30173 pages (at 95 pages/min), scraped 14384 items (at 18 items/min) 2026-01-24 22:23:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/air_tools_accessories/specialty_tool_air_tools_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:23:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-drill-14-keyless-chuck-500-rpm-05-hp-pistol-grip>: HTTP status code is not handled or not allowed 2026-01-24 22:23:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chipping-hammer-58-hex-shank-4-stroke-1600-bpm>: HTTP status code is not handled or not allowed 2026-01-24 22:23:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1516-hex-sae-deep-powersocket-for-ingersoll-rand-12in-drive-tool>: HTTP status code is not handled or not allowed 2026-01-24 22:23:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-tier-2-door-digital-locker-12inwx-18ind-x-78inh-tan-assembled>: HTTP status code is not handled or not allowed 2026-01-24 22:23:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compact-all-freezer-14-cu-ft-capacity-white>: HTTP status code is not handled or not allowed 2026-01-24 22:23:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ez-care-heavy-duty-scrub-pad-17-dia-redgray-pack-of-5>: HTTP status code is not handled or not allowed 2026-01-24 22:23:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-kit-for-w-650ic>: HTTP status code is not handled or not allowed 2026-01-24 22:23:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pc-board-for-w-series-long-hand-sealers>: HTTP status code is not handled or not allowed 2026-01-24 22:23:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-kit-for-kf-450f>: HTTP status code is not handled or not allowed 2026-01-24 22:23:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pc-board-for-yc-300hl--yc-450hl-l-bar-sealers>: HTTP status code is not handled or not allowed 2026-01-24 22:23:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/air_tools_accessories/specialty_tool_air_tools_kits>: HTTP status code is not handled or not allowed 2026-01-24 22:23:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silicone-rubber-pad-for-w-650i-w-650ic>: HTTP status code is not handled or not allowed 2026-01-24 22:23:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-convex-mirror-w-stainless-steel-backer-outdoor-34-dia>: HTTP status code is not handled or not allowed 2026-01-24 22:23:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shatter-resistant-roundtangular-convex-mirror-glass-outdoor-12l-x-18w>: HTTP status code is not handled or not allowed 2026-01-24 22:23:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lithium-ion-battery-12v-max-2-ah-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:23:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dril-driving-set-pack-of-20>: HTTP status code is not handled or not allowed 2026-01-24 22:23:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/radius-cone-shaped-bur-for-stainless-steel-2l-x-14-shank-dia>: HTTP status code is not handled or not allowed 2026-01-24 22:23:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/radius-cylindrical-bur-for-stainless-steel-2l-x-14-shank-dia>: HTTP status code is not handled or not allowed 2026-01-24 22:23:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/radius-tree-shaped-bur-double-cut-2-34l-x-14-shank-dia-34-cut-dia>: HTTP status code is not handled or not allowed 2026-01-24 22:23:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ball-shaped-bur-double-cut-6-38l-x-14-shank-dia>: HTTP status code is not handled or not allowed 2026-01-24 22:23:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bur-box-aluma-cut-sa1-sa3-sc1-sc3-sd1-sd3-sf1-sf3-set-of-8>: HTTP status code is not handled or not allowed 2026-01-24 22:23:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bur-box-single-cut-sa3-sa5-sc3-sc5-sd3-sd5-sf3-sf5-set-of-8>: HTTP status code is not handled or not allowed 2026-01-24 22:23:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t5-led-tube-light-bulb-miniature-bi-pin-base-12w-1500-lumens-5000k-super-white>: HTTP status code is not handled or not allowed 2026-01-24 22:23:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-wall-sconce-light-fixture-23w-1200-lumens-80-cri-24-size-white>: HTTP status code is not handled or not allowed 2026-01-24 22:23:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blender-bag-400mlx70m-w-full-nylon-filter-300x190mm-pack-of-500>: HTTP status code is not handled or not allowed 2026-01-24 22:23:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/block-for-12-x-15ml-tubes-200-750-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:23:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/block-w-1-standard-microplate>: HTTP status code is not handled or not allowed 2026-01-24 22:23:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tube-holder-for-4-x-10-15ml-2-per-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:23:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/voyager-dc-adaptor-for-20v-dewalt174-batteries>: HTTP status code is not handled or not allowed 2026-01-24 22:23:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pin-style-pull-cord-alarm-2l-x-2w-x-2h>: HTTP status code is not handled or not allowed 2026-01-24 22:23:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_components/pallet_rack_upright_frames already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:23:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-king-sk2000-pallet-load-stop-beam-3-or-4-setback-w-same-clip-96w-yellow>: HTTP status code is not handled or not allowed 2026-01-24 22:23:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/customizable-mobile-heavy-duty-cabinet-empty-14-gauge-48w-x-24d-x-76h>: HTTP status code is not handled or not allowed 2026-01-24 22:23:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/richmond-rectangular-picnic-table-recycled-plastic-brown-72l>: HTTP status code is not handled or not allowed 2026-01-24 22:23:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ball-end-mill-power-a-2-flute-4l-x-1-dia-1-12-cut-length>: HTTP status code is not handled or not allowed 2026-01-24 22:23:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbi-sala-nano-lok-personal-self-retracting-lifeline-w-rebar-hook-2726-lb-wt-72l>: HTTP status code is not handled or not allowed 2026-01-24 22:23:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-m36-multi-trolley-esd-w-4-adj-shelves-37-18w-x-29-1516d-x-70-12h-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:23:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/box-sentinel174-front-loader-cabinet-26w-x-26dx-43h-black>: HTTP status code is not handled or not allowed 2026-01-24 22:23:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-sided-starter-36l-x-14w-x-54h-antique-white>: HTTP status code is not handled or not allowed 2026-01-24 22:23:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-sided-starter-36l-x-14w-x-60h-antique-white>: HTTP status code is not handled or not allowed 2026-01-24 22:23:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drum-liner-4-mil-thick-55-gal-capacity-pack-of-150>: HTTP status code is not handled or not allowed 2026-01-24 22:23:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/end-unit-w-t-joiner-support-clip-36l-x-18w-x-96h-black>: HTTP status code is not handled or not allowed 2026-01-24 22:23:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-starter-36l-x-28w-x-96h-black>: HTTP status code is not handled or not allowed 2026-01-24 22:23:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/end-unit-w-t-joiner-support-clip-36l-x-12w-x-96h-black>: HTTP status code is not handled or not allowed 2026-01-24 22:23:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/end-unit-w-t-joiner-support-clip-48l-x-12w-x-96h-antique-white>: HTTP status code is not handled or not allowed 2026-01-24 22:23:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-starter-36l-x-24w-x-84h-antique-white>: HTTP status code is not handled or not allowed 2026-01-24 22:23:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-sided-starter-36l-x-20w-x-96h-antique-white>: HTTP status code is not handled or not allowed 2026-01-24 22:23:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-starter-48l-x-40w-x-54h-antique-white>: HTTP status code is not handled or not allowed 2026-01-24 22:23:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-starter-48l-x-28w-x-96h-antique-white>: HTTP status code is not handled or not allowed 2026-01-24 22:23:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v-gard-h1pro-safety-helmet-trivent-fas-trac-iii-pivot-ratchet-suspension-green>: HTTP status code is not handled or not allowed 2026-01-24 22:23:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/solopol-gfx-hand-cleaner-citrus-scent-325-l-refill-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-24 22:23:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tp1-staircase-platform-330-lb-load-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:23:51 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_components/pallet_rack_upright_frames landed on page that is not a product page. 2026-01-24 22:23:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/marque-quad-unit-reception-curved-desk-workstation-142w-x-103d-x-45-12h-mahogany>: HTTP status code is not handled or not allowed 2026-01-24 22:23:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bottle-media-globe-glass-gl45-screw-cap-dual-graduations-500ml-10-box>: HTTP status code is not handled or not allowed 2026-01-24 22:23:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-flex-ii-vinyl-strip-door-36w-x-84h-dark-amber>: HTTP status code is not handled or not allowed 2026-01-24 22:23:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mmb-yb14l-a2-12v-12ah-210-cca-battery>: HTTP status code is not handled or not allowed 2026-01-24 22:23:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mmb-ytz14sgel-12v-11-2ah-230cca-gel-battery>: HTTP status code is not handled or not allowed 2026-01-24 22:23:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mmb-ytx16-bs-12v-14ah-230cca-battery>: HTTP status code is not handled or not allowed 2026-01-24 22:23:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mmb-ytx30l-bs-12v-30ah-385cca-battery>: HTTP status code is not handled or not allowed 2026-01-24 22:23:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/utility-cap-dark-green-34439>: HTTP status code is not handled or not allowed 2026-01-24 22:23:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/medical_devices_instruments/vital_signs_monitors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:23:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gel-hand-sanitizer-for-cs2-dispensers-fragrance-free-1000-ml-capacity-pack-of-8>: HTTP status code is not handled or not allowed 2026-01-24 22:23:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adv-foam-hand-sanitizer-for-adx-7-dispensers-fragrance-free-700-ml-cap-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-24 22:24:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h1-2-bar-steel-ladder-rack-for-ram-promaster-cargo-van-2013-on-white>: HTTP status code is not handled or not allowed 2026-01-24 22:24:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h1-3-bar-steel-ladder-roof-rack-for-nissan-nv200-2013-on-black>: HTTP status code is not handled or not allowed 2026-01-24 22:24:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/light-duty-conveyor-netting-w-debris-liner-1146-lb-capacity-300l-x-108w-black>: HTTP status code is not handled or not allowed 2026-01-24 22:24:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/secure-click-organic-vapor-acid-gas-cartridge-filter-p100-d80923>: HTTP status code is not handled or not allowed 2026-01-24 22:24:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lr4-led-signal-tower-light-w-l-bracket-no-flashingbuzzer-240v-redambergreen>: HTTP status code is not handled or not allowed 2026-01-24 22:24:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80bs20hx2-1-pitch-2-finished-bore-20-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:24:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/patch-seal-spray-16-oz-capacity-black-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-24 22:24:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ceramic-simple-touch-heater-1500w-black>: HTTP status code is not handled or not allowed 2026-01-24 22:24:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/medical_devices_instruments/vital_signs_monitors>: HTTP status code is not handled or not allowed 2026-01-24 22:24:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waterproof-caulk-seal-sealant-10-oz-capacity-clear-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-24 22:24:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/upright-vaccine-refrigerator-8-cuft-capacity-glass-door>: HTTP status code is not handled or not allowed 2026-01-24 22:24:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 30270 pages (at 97 pages/min), scraped 14408 items (at 24 items/min) 2026-01-24 22:24:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/upright-vaccine-refrigerator-18-cuft-capacity-solid-door>: HTTP status code is not handled or not allowed 2026-01-24 22:24:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/upright-vaccine-refrigerator-8-cuft-capacity-solid-door>: HTTP status code is not handled or not allowed 2026-01-24 22:24:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gssid-post-decon-kit-youth>: HTTP status code is not handled or not allowed 2026-01-24 22:24:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-safety-crafted-waterproof-wellington-work-boots-steel-toe-size-6w-10-inch-h-brown>: HTTP status code is not handled or not allowed 2026-01-24 22:24:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-safety-crafted-waterproof-work-boots-steel-toe-size-10m-6-inch-h-dark-brown>: HTTP status code is not handled or not allowed 2026-01-24 22:24:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-20bs15hx45-metric-1-14-pitch-45mm-finished-bore-15-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:24:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/truflex-fly-skylar-work-zip-bootie-composite-toe-size-9m-4-inch-h-black>: HTTP status code is not handled or not allowed 2026-01-24 22:24:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-safety-crafted-waterproof-logger-work-boots-steel-toe-size-10-5m-8-inch-h-black>: HTTP status code is not handled or not allowed 2026-01-24 22:24:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-certified-advanced-refreshing-gel-hand-sanitizer-for-ltx-7-700-ml-3-ctn>: HTTP status code is not handled or not allowed 2026-01-24 22:24:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-100bs25hx21516-1-14-pitch-2-1516-finished-bore-25-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:24:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80bs09hx1316-1-pitch-1-316-finished-bore-9-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:24:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-blank-out-sign-x-down-arrow-24-h-x-24-w-x-6-d>: HTTP status code is not handled or not allowed 2026-01-24 22:24:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80sf33-1-pitch-qd-bushed-33-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:24:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-safety-crafted-waterproof-logger-work-boots-steel-toe-size-12-5w-8-inch-h-black>: HTTP status code is not handled or not allowed 2026-01-24 22:24:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-safety-crafted-waterproof-logger-work-boots-steel-toe-size-15m-8-inch-h-dark-brown>: HTTP status code is not handled or not allowed 2026-01-24 22:24:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lk4405bf-outdoor-wall-mount-bottle-filling-station-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:24:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-100bs24hx1716-1-14-pitch-1-716-finished-bore-24-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:24:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steri-drape-adhesive-split-sheet-9196-90-inch-x-102-inch-10-each-carton-2-carton-case>: HTTP status code is not handled or not allowed 2026-01-24 22:24:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35bs36hx58-38-pitch-58-finished-bore-36-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:24:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/futuro-compression-stabilizing-wrist-brace-right-hand-small-medium-2-pk-6-pk-case>: HTTP status code is not handled or not allowed 2026-01-24 22:24:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80bs28x11516-1-pitch-1-1516-finished-bore-28-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:24:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steri-drape-aperture-pouch-drape-1084-34-inch-x-42-inch-30-each-case>: HTTP status code is not handled or not allowed 2026-01-24 22:24:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zig-elusion-heritage-work-high-top-sneaker-composite-toe-size-10w-black-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:24:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:24:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/attest-biological-indicator-1261-for-steam-sterilization-100-bx-4-bx-ctn>: HTTP status code is not handled or not allowed 2026-01-24 22:24:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-100bs21hx11516-1-14-pitch-1-1516-finished-bore-21-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:24:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-16bs13hx40-metric-1-pitch-40mm-finished-bore-13-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:24:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/groundbreaker-work-boots-w-external-met-guard-steel-toe-size-8w-8-inch-h-brown>: HTTP status code is not handled or not allowed 2026-01-24 22:24:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-16bs24x38-metric-1-pitch-38mm-finished-bore-24-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:24:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-16bs24x42-metric-1-pitch-42mm-finished-bore-24-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:24:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/groundbreaker-work-boots-w-external-met-guard-steel-toe-size-8-5w-8-inch-h-brown>: HTTP status code is not handled or not allowed 2026-01-24 22:24:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zig-elusion-heritage-work-men-s-low-cut-sneaker-composite-toe-size-16m-black-red>: HTTP status code is not handled or not allowed 2026-01-24 22:24:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-boltless-shelving-48-in-w-x-18-in-d-x-96-in-h-5-shelves-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:24:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-boltless-shelving-48-in-w-x-24-in-d-x-72-in-h-5-shelves-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:24:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zig-elusion-heritage-work-men-s-low-cut-sneaker-composite-toe-size-4-5w-black>: HTTP status code is not handled or not allowed 2026-01-24 22:24:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-100bs19hx2-1-14-pitch-2-finished-bore-19-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:24:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots>: HTTP status code is not handled or not allowed 2026-01-24 22:24:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zig-elusion-heritage-work-women-s-low-cut-sneaker-composite-toe-size-5m-black>: HTTP status code is not handled or not allowed 2026-01-24 22:24:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-propette-le-single-channel-pipette-5-50ul>: HTTP status code is not handled or not allowed 2026-01-24 22:24:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zig-elusion-heritage-work-men-s-low-cut-sneaker-composite-toe-size-11m-black-red>: HTTP status code is not handled or not allowed 2026-01-24 22:24:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/truflex-fly-skylar-sneaker-composite-toe-size-7-5w-black>: HTTP status code is not handled or not allowed 2026-01-24 22:24:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-aspir-8-reservoir-for-8-channel-pipettes-non-sterile-10-ml-300-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:24:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-transfer-pipette-with-extended-tip-non-sterile-5-ml-500-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:24:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-short-stem-transfer-pipette-non-sterile-4-ml-250-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:24:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35ja30h-38-pitch-qd-bushed-30-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:24:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-gsx-800-793cc-snowmobile-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:24:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/truflex-fly-blucher-sneaker-composite-toe-size-10w-black-gold>: HTTP status code is not handled or not allowed 2026-01-24 22:24:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/truflex-fly-blucher-sneaker-composite-toe-size-12m-black-gold>: HTTP status code is not handled or not allowed 2026-01-24 22:24:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pgo-comet-25cc-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:24:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-41b19hx58mpb-12-pitch-58-plain-bore-19-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:24:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-12bs12hx24-metric-34-pitch-24mm-finished-bore-12-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:24:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-crafted-western-work-boots-w-steel-toe-and-internal-met-guard-size-13-5w-brown>: HTTP status code is not handled or not allowed 2026-01-24 22:24:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stone-skate-inspired-work-shoes-composite-toe-size-8w-black-red>: HTTP status code is not handled or not allowed 2026-01-24 22:24:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-40bs18hx138-12-pitch-1-38-finished-bore-18-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:24:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35bs28hx1-38-pitch-1-finished-bore-28-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:24:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-crafted-western-work-boots-steel-toe-size-9m-brown-burnt-orange>: HTTP status code is not handled or not allowed 2026-01-24 22:24:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nellcor-npb-3930-medical-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:24:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80bs38x114-1-pitch-1-14-finished-bore-38-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:24:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/savaria-k2-stairlift-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:24:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-10bs18hx42-metric-58-pitch-42mm-finished-bore-18-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:24:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paper-twist-ties-6-x-5-32-green-2000-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:24:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harmar-pinnacle-stairlift-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:24:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-10bs23hx30-metric-58-pitch-30mm-finished-bore-23-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:24:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rolls-18-x-200-yards>: HTTP status code is not handled or not allowed 2026-01-24 22:24:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bubble-bags-36-x-36-30-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:24:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nanoflex-tr-athletic-work-shoes-composite-toe-size-8-5w-black-seafoam-green-white>: HTTP status code is not handled or not allowed 2026-01-24 22:24:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-16a14h-metric-1-pitch-14mm-bore-14-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:24:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-seal-bubble-bags-6-x-15-1-2-250-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:24:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cool-shield-thermal-bubble-mailers-8-x-11-100-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:24:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caterpillar-920-industrial-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:24:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60bs17hx112-34-pitch-1-12-finished-bore-17-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:25:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-bags-6-x-9-2-mil-red-1000-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:25:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galion-3500-industrial-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:25:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-10bs20hx28-metric-58-pitch-28mm-finished-bore-20-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:25:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premier-work-casual-oxford-shoes-steel-toe-11-5eee-black>: HTTP status code is not handled or not allowed 2026-01-24 22:25:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galion-125fa-industrial-replacement-battery-18ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:25:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caterpillar-613-industrial-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:25:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60sds19h-qd-bushed-34-pitch-19-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:25:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-100sds11h-qd-bushed-1-14-pitch-11-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:25:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/construction-harness-tongue-buckle-legs-medium-large>: HTTP status code is not handled or not allowed 2026-01-24 22:25:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-10bs21hx40-metric-58-pitch-40mm-finished-bore-21-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:25:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acme-rbat6-alarm-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:25:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-12bs13hx32-metric-34-pitch-32mm-finished-bore-13-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:25:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 30370 pages (at 100 pages/min), scraped 14425 items (at 17 items/min) 2026-01-24 22:25:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ledsl27wh-led-security-light-wall-pack-27w-4000-cct-2240-lumen-82-cri-white>: HTTP status code is not handled or not allowed 2026-01-24 22:25:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nanoflex-tr-athletic-work-shoes-composite-toe-size-9-5m-black-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:25:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nanoflex-tr-athletic-work-shoes-composite-toe-size-10w-navy-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:25:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96-inch-w-x-30-inch-d-extra-long-production-workbench-shop-top-square-edge-gray-318913>: HTTP status code is not handled or not allowed 2026-01-24 22:25:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nanoflex-tr-athletic-work-shoes-composite-toe-size-9w-black>: HTTP status code is not handled or not allowed 2026-01-24 22:25:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-08bs25hx30-metric-12-pitch-30mm-finished-bore-25-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:25:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-08bs12hx24-metric-12-pitch-24mm-finished-bore-12-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:25:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35a23hx12-a-plate-38-pitch-12-bore-23-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:25:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merits-sp24-wheelchair-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:25:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nano-x1-adventure-athletic-work-shoes-composite-toe-size-7m-gray-black>: HTTP status code is not handled or not allowed 2026-01-24 22:25:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lavante-trail-2-running-work-shoes-composite-toe-size-9m-gray-black>: HTTP status code is not handled or not allowed 2026-01-24 22:25:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lavante-trail-2-running-work-shoes-composite-toe-size-8-5w-gray-red-black>: HTTP status code is not handled or not allowed 2026-01-24 22:25:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ag10-series-4-x-24-solid-alum-fixed-bar-supply-return-air-vent-grille-black-matte>: HTTP status code is not handled or not allowed 2026-01-24 22:25:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bruno-32-wheelchair-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:25:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merits-mp3c-jr-wheelchair-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:25:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bruno-35-wheelchair-replacement-battery-18ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:25:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-25bs18hx34-14-pitch-34-finished-bore-18-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:25:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merits-p710-wheelchair-replacement-battery-75ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:25:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lavante-trail-2-running-work-shoes-composite-toe-size-8-5m-gray-black>: HTTP status code is not handled or not allowed 2026-01-24 22:25:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hook-n-loop-tape-fasteners-orange-3-8-inch-x-75-feet>: HTTP status code is not handled or not allowed 2026-01-24 22:25:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nano-x1-adventure-athletic-work-shoes-composite-toe-size-7w-silver-gray-clay-black>: HTTP status code is not handled or not allowed 2026-01-24 22:25:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hook-n-loop-tape-fasteners-orange-half-inch-x-75-feet>: HTTP status code is not handled or not allowed 2026-01-24 22:25:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rascal-245-wheelchair-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:25:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hook-n-loop-tape-fasteners-yellow-1-inch-x-75-feet>: HTTP status code is not handled or not allowed 2026-01-24 22:25:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-hook-with-acrylic-adhesive-3-4-inch-x-75-feet>: HTTP status code is not handled or not allowed 2026-01-24 22:25:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-40a22hx58-a-plate-12-pitch-58-bore-22-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:25:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nano-x1-adventure-athletic-work-shoes-composite-toe-size-7-5m-silver-gray-clay-black>: HTTP status code is not handled or not allowed 2026-01-24 22:25:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-20bs13hx38-metric-1-14-pitch-38mm-finished-bore-13-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:25:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nano-x1-adventure-athletic-work-shoes-composite-toe-size-12w-black>: HTTP status code is not handled or not allowed 2026-01-24 22:25:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/satin-wall-mounting-black-wet-umbrella-bag-holder>: HTTP status code is not handled or not allowed 2026-01-24 22:25:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nano-x1-adventure-athletic-work-shoes-composite-toe-size-15w-silver-gray-clay-black>: HTTP status code is not handled or not allowed 2026-01-24 22:25:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-40bs14hx34-12-pitch-34-finished-bore-14-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:25:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hybrid-skate-inspired-work-shoes-composite-toe-size-9w-navy-celestial-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:25:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35a27hx12-a-plate-38-pitch-12-bore-27-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:25:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/food_storage/food_storage_containers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:25:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/toro-11-44-lawn-and-garden-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:25:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/murray-31704-lawn-and-garden-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:25:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-40ja15h-qd-bushed-12-pitch-15-teeth>: HTTP status code is not handled or not allowed 2026-01-24 22:25:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rack-protector-with-cutout-3-x-3-opening-18h-yellow>: HTTP status code is not handled or not allowed 2026-01-24 22:25:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elsar-410-emergency-light-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:25:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hybrid-skate-inspired-work-shoes-composite-toe-size-8-5m-navy-celestial-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:25:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extreme-softshell-jacket-black-60f-comfort-rating-xl>: HTTP status code is not handled or not allowed 2026-01-24 22:25:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rotary-screw-air-compressor-230v-125-psi-28-cfm-75hp-1ph>: HTTP status code is not handled or not allowed 2026-01-24 22:25:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:25:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rotary-screw-air-compressor-wdryer-200v-140-psi-34-cfm-10hp-3ph>: HTTP status code is not handled or not allowed 2026-01-24 22:25:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-tapered-roller-bearing-e32007j-35mm-bore-62mm-od>: HTTP status code is not handled or not allowed 2026-01-24 22:25:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/food_storage/food_storage_containers>: HTTP status code is not handled or not allowed 2026-01-24 22:25:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evolve-skate-inspired-work-shoes-composite-toe-size-10-5m-navy>: HTTP status code is not handled or not allowed 2026-01-24 22:25:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-tapered-roller-bearing-e30212j-60mm-bore-110mm-od>: HTTP status code is not handled or not allowed 2026-01-24 22:25:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-tapered-roller-bearing-e30310j-50mm-bore-110mm-od>: HTTP status code is not handled or not allowed 2026-01-24 22:25:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-tapered-roller-bearing-e30214j-70mm-bore-125mm-od>: HTTP status code is not handled or not allowed 2026-01-24 22:25:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/epdm-synthetic-rubber-tarp-tie-2231epdm-31-50-pieces-per-box>: HTTP status code is not handled or not allowed 2026-01-24 22:25:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-stage-electric-air-compressor-l001124-230v-10hp-3ph-120-gal>: HTTP status code is not handled or not allowed 2026-01-24 22:25:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-quest-50cc-atv-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:25:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-24 22:25:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evolve-skate-inspired-work-shoes-composite-toe-size-7m-triple-black>: HTTP status code is not handled or not allowed 2026-01-24 22:25:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-rbc-bk-drawer-16w-x-14d-x-5h-black>: HTTP status code is not handled or not allowed 2026-01-24 22:25:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-bcp-gy-cabinet-pedestal-16w-x-30d-x-30-14h-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:25:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/binders_covers/reports_covers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:25:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evolve-skate-inspired-work-shoes-composite-toe-size-7w-olive-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:25:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/presstex-report-cover-prong-clip-letter-3-capacity-light-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:25:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evolve-skate-inspired-high-top-work-shoes-composite-toe-size-7w-black-yellow>: HTTP status code is not handled or not allowed 2026-01-24 22:25:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/atk-125cc-motorcycle-replacement-battery-2001-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:25:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honeywell-pressuretrol-main-scale-l404f1367-less-siphon-18-psi-main>: HTTP status code is not handled or not allowed 2026-01-24 22:25:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cc-pump-motor-wwe2-36-145jp-tefc-rigid-c-3-ph-145jp-2-hp-3600-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:26:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/comfortie-casual-work-shoes-steel-toe-size-5-5m-black-white>: HTTP status code is not handled or not allowed 2026-01-24 22:26:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bb4500-high-top-work-sneaker-composite-toe-size-7m-black-white>: HTTP status code is not handled or not allowed 2026-01-24 22:26:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extreme-xta14ah-bs-powersports-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:26:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-frame-motor-wwht60-18-364tc-gp-tefc-rigid-c-3-ph-364tc-705-fla>: HTTP status code is not handled or not allowed 2026-01-24 22:26:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/can-am-outlander-450cc-atv-replacement-battery-2016-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:26:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interstate-cytx4l-bs-powersports-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:26:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-pewwe5-36-184t-prem-eff-tefc-rigid-3-ph-184t-208-230-460v-6-fla>: HTTP status code is not handled or not allowed 2026-01-24 22:26:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/daystart-slip-on-work-shoes-steel-toe-size-10-5m-black>: HTTP status code is not handled or not allowed 2026-01-24 22:26:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lottus-49cc-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:26:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/prang-colored-pencils-assorted-lead-assorted-barrel-12-set>: HTTP status code is not handled or not allowed 2026-01-24 22:26:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-frame-motor-wwht5-18-184t-gp-tefc-rigid-3-ph-184t-208-230-460v-69-fla>: HTTP status code is not handled or not allowed 2026-01-24 22:26:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/retractable-gel-pens-black-ink-black-barrel-1-each-bl407aa>: HTTP status code is not handled or not allowed 2026-01-24 22:26:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/d-flange-kit-w400td-404t-405t-8-pole>: HTTP status code is not handled or not allowed 2026-01-24 22:26:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cc-pump-motor-wwe75-36-213jp-tefc-rigid-c-3-ph-213jp-75-hp-3600-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:26:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polaris-trail-wide-trak-snowmobile-replacement-battery-12v-i>: HTTP status code is not handled or not allowed 2026-01-24 22:26:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/binders_covers/reports_covers landed on page that is not a product page. 2026-01-24 22:26:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 30465 pages (at 95 pages/min), scraped 14443 items (at 18 items/min) 2026-01-24 22:26:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-touring-500-fan-snowmobile-replacement-battery-12v-i>: HTTP status code is not handled or not allowed 2026-01-24 22:26:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-mx-z-rev-800-800cc-snowmobile-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:26:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-frame-motor-wwht150-12-445-7t-f2-gp-tefc-rigid-3-ph-f2-445-7t-rb>: HTTP status code is not handled or not allowed 2026-01-24 22:26:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-frame-motor-wwht150-12-445-7tsbb-gp-tefc-rigid-3-ph-445-7ts-171-fla>: HTTP status code is not handled or not allowed 2026-01-24 22:26:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-slide-base-w215t>: HTTP status code is not handled or not allowed 2026-01-24 22:26:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pgo-b2-150-efi-150cc-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:26:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-frame-motor-wwht100-18-405tc-f2-gp-tefc-rigid-3-ph-f2-405tc-115-fla>: HTTP status code is not handled or not allowed 2026-01-24 22:26:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/general_purpose_electric_motors/three_phase_motors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:26:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-mx-z-550-550cc-snowmobile-replacement-battery-2003-2012-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:26:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/170-industrial-carton-sealing-tape-2-x-110-yds-tan-pack-of-36>: HTTP status code is not handled or not allowed 2026-01-24 22:26:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-gsx-550f-550cc-snowmobile-replacement-battery-2005-2008-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:26:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/200cc-colored-carton-sealing-tape-2-x-110-yds-clear-pack-of-36>: HTTP status code is not handled or not allowed 2026-01-24 22:26:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/measuring-cup-for-mr-100-steam-cleaner-261553>: HTTP status code is not handled or not allowed 2026-01-24 22:26:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manual-drum-truck-w-brake-mold-on-rubber-wheels-aluminum-1000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:26:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steam-outlet-cleaning-tool-for-mr-50-steam-vacuum-261576>: HTTP status code is not handled or not allowed 2026-01-24 22:26:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-skandic-wide-track-500cc-snowmobile-replacement-battery-12v-i>: HTTP status code is not handled or not allowed 2026-01-24 22:26:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-gasket-for-dust-collector-261575>: HTTP status code is not handled or not allowed 2026-01-24 22:26:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/apc-smart-ups-sc-620va-sc620-ups-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 22:26:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/general_purpose_electric_motors/three_phase_motors>: HTTP status code is not handled or not allowed 2026-01-24 22:26:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rais-n-rol-machinery-mover-w-5-polyurethane-casters-2000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:26:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/computers_electronics/cables_connectors/cable_connectors_adapters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:26:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carton-sealing-tape-stopalto-3-x-110-yds-redwhite-pack-of-24>: HTTP status code is not handled or not allowed 2026-01-24 22:26:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/apc-smartups-750va-usb-sua750-ups-replacement-battery-7ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 22:26:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-port-vga-svga-video-splitter-350mhz-hd15-male-2-x-female>: HTTP status code is not handled or not allowed 2026-01-24 22:26:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/greenline-626-hand-truck-w-pin-handle-mold-on-rubber-wheels-600-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:26:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-port-vga-over-cat5-cat6-extender-splitter-transmitter-edid>: HTTP status code is not handled or not allowed 2026-01-24 22:26:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-extinguisher-cabinet-solid-door-fully-recessed-55d-steel>: HTTP status code is not handled or not allowed 2026-01-24 22:26:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbi-sala174-nano-lok-xl-self-retracting-lifeline-w-steel-carabiner-snap-hook-11l>: HTTP status code is not handled or not allowed 2026-01-24 22:26:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-visibility-class-3-waterproof-quilt-lined-bomber-jacket-lime-black-3xl>: HTTP status code is not handled or not allowed 2026-01-24 22:26:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-visibility-class-3-3-in-1-waterproof-bomber-jacket-w-fleece-lining-lime-black-3xl>: HTTP status code is not handled or not allowed 2026-01-24 22:26:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbi-sala-nano-lok-xl-twin-leg-self-retracting-lifeline-w-quick-connect-carabiner-11l>: HTTP status code is not handled or not allowed 2026-01-24 22:26:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbi-sala174-nano-lok-xl-order-picker-self-retracting-lifeline-w-steel-snap-hook-11l>: HTTP status code is not handled or not allowed 2026-01-24 22:26:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/upg-ub1270-sealed-lead-acid-replacement-battery-9ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 22:26:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/computers_electronics/cables_connectors/cable_connectors_adapters>: HTTP status code is not handled or not allowed 2026-01-24 22:26:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/greenline-626-hand-truck-w-pin-handle-pe-pneumatic-wheels-600-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:26:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ajc-prescolite-em-exit-6v-4-5ah-emergency-light-battery>: HTTP status code is not handled or not allowed 2026-01-24 22:26:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/snap-2-in-reflector-dot-star-white-6-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:26:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ready-built-ii153-1-tier-1-door-locker-12w-x-15d-x-83h-light-gray-assembled>: HTTP status code is not handled or not allowed 2026-01-24 22:26:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7175-nonslip-coating-durable-antislip-paint-1-gal-red-7175-re-gl>: HTTP status code is not handled or not allowed 2026-01-24 22:26:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/painting_supplies/specialty_coatings_paint already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:26:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quick-vault-in-wall-pistol-safe-41214-s-ivy-2-gun-capacity-12w-x-4d-x-14h-ivory>: HTTP status code is not handled or not allowed 2026-01-24 22:26:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ready-built-ii-6-tier-18-doors-locker-36w-x-18d-x-84h-light-gray-assembled>: HTTP status code is not handled or not allowed 2026-01-24 22:26:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-2-tier-6-doors-locker-54w-x-18d-x-78h-dark-gray-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 22:26:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scheduled-maintenance-kit-for-air-cooled-generators-530cc-engine-10-watt>: HTTP status code is not handled or not allowed 2026-01-24 22:26:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caterpillar-970f-industrial-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:26:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-waterfall-deck-shelving-5-shelves-48w-x-24d-x-96h-midnight-ebony>: HTTP status code is not handled or not allowed 2026-01-24 22:26:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/topin-tp12-7-sealed-lead-acid-replacement-battery-9ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 22:26:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silhouette-fabric-padded-stack-chair-ebony-black-fabric-and-black-santex-frame>: HTTP status code is not handled or not allowed 2026-01-24 22:26:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-waterfall-deck-shelving-5-shelves-96w-x-24d-x-96h-midnight-ebony>: HTTP status code is not handled or not allowed 2026-01-24 22:26:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ritar-rt640s-sealed-lead-acid-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:26:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/apc-back-ups-es500-es-500-va-8-outlet-be500r-12v-8ah-ups-battery>: HTTP status code is not handled or not allowed 2026-01-24 22:26:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powered-drum-truck-f89503f-fiber-chime-hook-10-wheels>: HTTP status code is not handled or not allowed 2026-01-24 22:26:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/painting_supplies/specialty_coatings_paint>: HTTP status code is not handled or not allowed 2026-01-24 22:26:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ready-built153-6-tier-6-doors-locker-12w-x-15d-x-78h-light-gray-assembled>: HTTP status code is not handled or not allowed 2026-01-24 22:26:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/calm90-opc-output-cover-for-calm-series-90mm-aluminum-worm-gear-reducer>: HTTP status code is not handled or not allowed 2026-01-24 22:26:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/see-thru-toddler-book-browser>: HTTP status code is not handled or not allowed 2026-01-24 22:26:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/infant-coordination-mirror>: HTTP status code is not handled or not allowed 2026-01-24 22:26:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/calm40-101-56c-aluminum-worm-gear-reducer-40mm-101-56c-nema-frame>: HTTP status code is not handled or not allowed 2026-01-24 22:26:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-tech-closed-stand-alone-shelving-8-shelves-500-lb-cap-36w-x-12d-x-87h-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:26:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-tech-closed-style-metal-shelving-8-shelves-800-lb-cap-36w-x-18d-x-87h-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:26:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-chairries-premium-tray-5-seat-height>: HTTP status code is not handled or not allowed 2026-01-24 22:26:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/calm90-1001-56c-aluminum-worm-gear-reducer-90mm-1001-56c-nema-frame>: HTTP status code is not handled or not allowed 2026-01-24 22:26:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-tech-open-shelving-8-shelves-starter-unit-450-lb-cap-48w-x-18d-x-87h-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:27:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/977s04-catwalk-96-high-unit-walkway-48-x-48>: HTTP status code is not handled or not allowed 2026-01-24 22:27:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eco-flo-ecd75w-submersible-sump-pump-cast-iron>: HTTP status code is not handled or not allowed 2026-01-24 22:27:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mighty-products-mn-dn-1416-debris-safety-netting-14-x-16>: HTTP status code is not handled or not allowed 2026-01-24 22:27:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-in-x-16-in-aluminum-door-louver>: HTTP status code is not handled or not allowed 2026-01-24 22:27:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mighty-products-mn-dn-0820-debris-safety-netting-8-x-20>: HTTP status code is not handled or not allowed 2026-01-24 22:27:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cove-base-pinnacle-rubber-4x18x48-smoke>: HTTP status code is not handled or not allowed 2026-01-24 22:27:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leoch-lpx12-24-sealed-lead-acid-replacement-battery-26ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:27:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/csb-gp12120f2-gp-12120-f2-12v-12ah-ups-battery>: HTTP status code is not handled or not allowed 2026-01-24 22:27:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powertron-pt10-6-sealed-lead-acid-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:27:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e-track-weld-on-single-strap-anchor-zinc-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-24 22:27:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 30561 pages (at 96 pages/min), scraped 14468 items (at 25 items/min) 2026-01-24 22:27:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/work-height-platform-truck-rnh2-2448-6mr-ah-with-lower-shelf-24-x-48-adj-height>: HTTP status code is not handled or not allowed 2026-01-24 22:27:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/barbell-long-shackle-coupler-lock-for-ufp-couplers-3-12>: HTTP status code is not handled or not allowed 2026-01-24 22:27:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/upsonic-ih-5000-sealed-lead-acid-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:27:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-in-black-malleable-union-150-psi-lead-free>: HTTP status code is not handled or not allowed 2026-01-24 22:27:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extinguisher-cabinet-full-tempered-safety-glass-ss-surface-mt-11-1-2-inch-l-x-26-1-2-inch-h-x-5-3-4-inch-d>: HTTP status code is not handled or not allowed 2026-01-24 22:27:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eveleen-indoor-outdoor-rocking-chair-black>: HTTP status code is not handled or not allowed 2026-01-24 22:27:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-lever-hoist-6-ton-capacity-15-lift>: HTTP status code is not handled or not allowed 2026-01-24 22:27:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n246060-4-equipment-stands-4-shelf-shelving-unit-24w-x-60h-x-60l-aluminum>: HTTP status code is not handled or not allowed 2026-01-24 22:27:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-in-5-1-2-in-black-steel-pipe-nipple-150-psi-lead-free>: HTTP status code is not handled or not allowed 2026-01-24 22:27:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/re3018-7-half-size-pan-rack-kd-7-pan-20-116w-x-31-12h-x-24-12d-aluminum>: HTTP status code is not handled or not allowed 2026-01-24 22:27:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chill2-chillhand-chill-tray-dollies-double-handle-aluminum>: HTTP status code is not handled or not allowed 2026-01-24 22:27:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cm-lodestar-vs-electric-chain-hoist-12-ton-10-lift-26-16-fpm-115-1-60-2-step-iv-control>: HTTP status code is not handled or not allowed 2026-01-24 22:27:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gravity-fed-eyewash-insulated-jacket-for-30-gallon-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:27:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pillowcases-100bx-poly-tissue-bl>: HTTP status code is not handled or not allowed 2026-01-24 22:27:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pressure-adhesive-bandage-waterproof-275l-x-1w-100bx-wh-green>: HTTP status code is not handled or not allowed 2026-01-24 22:27:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carboy-autoclavable-pp-20-liter-capacity-white>: HTTP status code is not handled or not allowed 2026-01-24 22:27:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/small-submersible-pump-3e12nt-dual-purpose-115v-500-gph-at-1>: HTTP status code is not handled or not allowed 2026-01-24 22:27:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnetic-drive-pump-te3mdhc-highly-corrosive-115v-590-at-1-107-watts>: HTTP status code is not handled or not allowed 2026-01-24 22:27:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fortress-655-22nf-wheelchair-replacement-battery-55ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:27:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reagent-bottle-wide-mouth-pp-60ml-capacity-clear-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-24 22:27:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drench-shower-floor-mount-stainless-steel-pipe-nylon-showerhead>: HTTP status code is not handled or not allowed 2026-01-24 22:27:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-shim-coil-006-6-50>: HTTP status code is not handled or not allowed 2026-01-24 22:27:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/test-tube-rack-for-25mm-tubes-wet-dry-pp-24-places-white-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-24 22:27:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reagent-bottle-narrow-mouth-hdpe-60ml-capacity-amber-pack-of-72>: HTTP status code is not handled or not allowed 2026-01-24 22:27:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/62132rs-deep-groove-ball-bearing-double-sealed-65mm-bore-120mm-od>: HTTP status code is not handled or not allowed 2026-01-24 22:27:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shoprider-4dxd-wheelchair-replacement-battery-3-2ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:27:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merits-p328-2s-c-wheelchair-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:27:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merits-mp3-junior-wheelchair-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:27:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/630032rs-deep-groove-ball-bearing-wide-width-double-sealed-17mm-bore-35mm-od>: HTTP status code is not handled or not allowed 2026-01-24 22:27:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6208zznr-deep-groove-ball-bearing-double-shielded-snap-ring-40mm-bore-80mm-od>: HTTP status code is not handled or not allowed 2026-01-24 22:27:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6308zz-deep-groove-ball-bearing-double-shielded-40mm-bore-90mm-od>: HTTP status code is not handled or not allowed 2026-01-24 22:27:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/32210-tapered-roller-bearing-metric-50mm-bore-90mm-od>: HTTP status code is not handled or not allowed 2026-01-24 22:27:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/calibration-weight-astm-class-1-nvlap-certificate-stainless-steel-1kg>: HTTP status code is not handled or not allowed 2026-01-24 22:27:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60032rs-p53-deep-groove-ball-bearing-double-sealed-abec-5-17mm-bore-35mm-od>: HTTP status code is not handled or not allowed 2026-01-24 22:27:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wardrobe-locker-w-padlock-hasp-3-tier-assembled-12w-x-12d-x-60h-beige>: HTTP status code is not handled or not allowed 2026-01-24 22:27:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6315zz-deep-groove-ball-bearing-double-shielded-75mm-bore-160mm-od>: HTTP status code is not handled or not allowed 2026-01-24 22:27:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/63122rs-deep-groove-ball-bearing-double-sealed-60mm-bore-130mm-od>: HTTP status code is not handled or not allowed 2026-01-24 22:27:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quantum-rehab-salsa-m2-wheelchair-replacement-battery-7ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 22:27:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shoprider-te889dx2-4-wheelchair-replacement-battery-55ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:27:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6305zznr-deep-groove-ball-bearing-double-shielded-snap-ring-25mm-bore-62mm-od>: HTTP status code is not handled or not allowed 2026-01-24 22:27:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/damaco-childs-elite-wheelchair-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:27:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/62092rs-deep-groove-ball-bearing-double-sealed-45mm-bore-85mm-od>: HTTP status code is not handled or not allowed 2026-01-24 22:27:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rally-16-hp-lawn-and-garden-replacement-battery-18ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:27:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelf-truck-54l-x-24w-x-60h-polyurethane-brake-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:27:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hiliter-desk-style-highlighter-chisel-tip-fluorescent-pink-ink-dozen>: HTTP status code is not handled or not allowed 2026-01-24 22:27:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60062rsnr-deep-groove-ball-bearing-double-sealed-snap-ring-30mm-bore-55mm-od>: HTTP status code is not handled or not allowed 2026-01-24 22:27:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelf-truck-54l-x-21w-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:27:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/marksalot-permanent-marker-large-chisel-tip-orange-ink>: HTTP status code is not handled or not allowed 2026-01-24 22:27:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60440212128-cloth-belt-340d-37w-x-75l-p150-grit>: HTTP status code is not handled or not allowed 2026-01-24 22:27:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jacobsen-53020-lawn-and-garden-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:27:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wheelhorse-308-lawn-and-garden-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:27:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ingersol-6020-lawn-and-garden-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:27:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/john-deere-650-lawn-and-garden-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:27:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelf-truck-48l-x-24w-x-80h-polyurethane-brake-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:27:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/61500084142-low-stretch-belt-1w-x-42l-med-grit>: HTTP status code is not handled or not allowed 2026-01-24 22:27:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60440201188-341d-3-4dia-x-1w-80-grit-100-case>: HTTP status code is not handled or not allowed 2026-01-24 22:27:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelf-truck-72l-x-14w-x-69h-polyurethane-brake-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:27:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/toro-titan-zx5400-lawn-and-garden-replacement-battery-9ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 22:27:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lightalarms-rsqg-emergency-light-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:28:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sunworks-construction-paper-12x18-yellow-50-sheets>: HTTP status code is not handled or not allowed 2026-01-24 22:28:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8020-15-series-1-12-x-4-12-eight-t-slotted-extrusion-profile-120l-stock-bar>: HTTP status code is not handled or not allowed 2026-01-24 22:28:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sonnenschein-mg2mf-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:28:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sonnenschein-cr610-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:28:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bisley-womens-short-sleeve-t-shirt-class-2-s-hi-vis-yellow>: HTTP status code is not handled or not allowed 2026-01-24 22:28:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/begreen-b2p-retractable-gel-pen-refillable-fine-blue-ink-1-each>: HTTP status code is not handled or not allowed 2026-01-24 22:28:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/columbia-mtd-397-lawn-and-garden-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:28:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/frixion-ball-erasable-gel-pen-fine-07mm-blue-barrelink>: HTTP status code is not handled or not allowed 2026-01-24 22:28:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pen-style-permanent-marker-fine-point-assorted-ink-8set>: HTTP status code is not handled or not allowed 2026-01-24 22:28:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/accent-highlighter-narrow-chisel-tip-nontoxic-fluorescent-yellow-ink-dozen>: HTTP status code is not handled or not allowed 2026-01-24 22:28:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enviroalert-professional174-wireless-temperature-sensor-12vdc>: HTTP status code is not handled or not allowed 2026-01-24 22:28:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-speed-low-profile-diesel-pump-9005lp-27gpm>: HTTP status code is not handled or not allowed 2026-01-24 22:28:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 30656 pages (at 95 pages/min), scraped 14493 items (at 25 items/min) 2026-01-24 22:28:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/power_tools/power_grinder/die_grinders/corded_die_grinders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:28:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foot-pedal-pump-steel-fpp-chrm>: HTTP status code is not handled or not allowed 2026-01-24 22:28:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elsar-16238-emergency-light-replacement-battery-1-3ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:28:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elsar-23064-emergency-light-replacement-battery-7ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 22:28:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-12-709-emergency-light-replacement-battery-26ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:28:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crossover-ladder-36l-x-28-12w-x-94-716h>: HTTP status code is not handled or not allowed 2026-01-24 22:28:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trestle-brackets-dark-horseconquest-aluminium-300-lb-capacity-silver>: HTTP status code is not handled or not allowed 2026-01-24 22:28:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/red-kap-womens-straight-fit-jean-prewashed-denim-12-x-34u-pd63>: HTTP status code is not handled or not allowed 2026-01-24 22:28:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sd-t18-air-sample-tube-18>: HTTP status code is not handled or not allowed 2026-01-24 22:28:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/els-eds12600-emergency-light-replacement-battery-75ah-12v-it>: HTTP status code is not handled or not allowed 2026-01-24 22:28:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hyperlite-extension-ladder-w-cable-hooks-v-rung-24-type-iaa-375-lb-cap>: HTTP status code is not handled or not allowed 2026-01-24 22:28:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-ez-2i-emergency-light-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:28:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/438d-6n5-vibrating-fire-alarm-bell-6-120-v-60-hz-diode>: HTTP status code is not handled or not allowed 2026-01-24 22:28:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hyperlite-extension-ladder-w-ground-cue-cable-hooks-claw-24-type-iaa-375-lb-cap>: HTTP status code is not handled or not allowed 2026-01-24 22:28:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dyna-ray-s18191-emergency-light-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:28:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/power_tools/power_grinder/die_grinders/corded_die_grinders>: HTTP status code is not handled or not allowed 2026-01-24 22:28:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2459-smb-w-surface-mount-box-white>: HTTP status code is not handled or not allowed 2026-01-24 22:28:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eg1r-cvm-genesis-chimestrobe-selectable-output>: HTTP status code is not handled or not allowed 2026-01-24 22:28:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sure-lites-263-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:28:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/revolution-20-articulated-extendable-ladder-aluminum-6-type-ia-300-lb-cap>: HTTP status code is not handled or not allowed 2026-01-24 22:28:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electrostatic-air-filter-merv-8-18w-x-1d-x-30h>: HTTP status code is not handled or not allowed 2026-01-24 22:28:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ajc-12v-72ah-sealed-lead-acid-agm-vrla-battery>: HTTP status code is not handled or not allowed 2026-01-24 22:28:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ajc-6v-45ah-sealed-lead-acid-agm-vrla-battery>: HTTP status code is not handled or not allowed 2026-01-24 22:28:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sumostance-extension-ladder-w-hyperlite-tech-fiberglass-20-type-ia-300-lb-cap>: HTTP status code is not handled or not allowed 2026-01-24 22:28:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-trouble-light-sl426a-50l-cord-163-ga-yellow-6pk>: HTTP status code is not handled or not allowed 2026-01-24 22:28:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cord-storage-reel-kw130-150l-cord-storage-reel-6pk>: HTTP status code is not handled or not allowed 2026-01-24 22:28:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-1799112-emergency-light-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:28:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-storage-cabinet-turn-handle-4-adj-shelves-36-inch-w-x-18-inch-d-x-78-inch-h-black>: HTTP status code is not handled or not allowed 2026-01-24 22:28:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clip-bin-front-42-inch-w-x-3-inch-h-marine-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:28:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-low-temp-scratch-resistant-strip-for-strip-curtain-doors-12w-x-12h-clear>: HTTP status code is not handled or not allowed 2026-01-24 22:28:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sigmas-sp1235-nb-12v-35ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-24 22:28:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qp-max-two-stage-air-compressor-vertical-5-hp-80-gallon-capacity-1-phase-230v>: HTTP status code is not handled or not allowed 2026-01-24 22:28:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simplex-429115-emergency-light-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:28:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interlake-bulk-storage-rack-beam-zs-60-inch-l-for-metal-shelves>: HTTP status code is not handled or not allowed 2026-01-24 22:28:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/k1800-8-drawer-roller-cabinet-27w-x-18d-x-39h-black>: HTTP status code is not handled or not allowed 2026-01-24 22:28:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/panasonic-12v-18ah-12v-18ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-24 22:28:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/technacell-ep666-emergency-light-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:28:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/k2000-5-drawer-roller-cabinet-29w-x-20d-x-34-1516h-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:28:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-lm130-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:28:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slide-door-5w-x-8h>: HTTP status code is not handled or not allowed 2026-01-24 22:28:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/k1800-7-drawer-roller-cabinet-27w-x-18d-x-34-1516h-black>: HTTP status code is not handled or not allowed 2026-01-24 22:28:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leoch-djw645-sealed-lead-acid-agm-vrla-battery>: HTTP status code is not handled or not allowed 2026-01-24 22:28:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/panasonic-lcr127r2p-sealed-lead-acid-agm-vrla-battery>: HTTP status code is not handled or not allowed 2026-01-24 22:28:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/tables/conference_tables already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:28:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/k2000-5-drawer-hang-on-side-cabinet-13-58w-x-20d-x-29-116h-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:28:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/astral-square-umbrella-w-centerpost-137w-x-137d-x-110h-mediterranean-blue-tweed>: HTTP status code is not handled or not allowed 2026-01-24 22:28:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-rack-placard-label-holder-3-x-100-ft-rollwhite-797046>: HTTP status code is not handled or not allowed 2026-01-24 22:28:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simplex-ub12220-emergency-light-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:28:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/umbrellas_awnings_shades/outdoor_umbrellas_bases/umbrellas already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:28:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/johnlite-9266ns-emergency-light-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:28:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wave-lever-keyed-entry-aged-bronze>: HTTP status code is not handled or not allowed 2026-01-24 22:28:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergi-lite-em1-emergency-light-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:28:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interstate-6v-14ah-6v-12ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-24 22:28:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leoch-lp612-sealed-lead-acid-agm-vrla-battery>: HTTP status code is not handled or not allowed 2026-01-24 22:28:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1515lite97-112-x-112-lite-tslotted-profile-97-stock-bar>: HTTP status code is not handled or not allowed 2026-01-24 22:28:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c-channel-class-c-8l-white>: HTTP status code is not handled or not allowed 2026-01-24 22:28:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/tables/conference_tables>: HTTP status code is not handled or not allowed 2026-01-24 22:28:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drop-ceiling-replacement-anchor-tile>: HTTP status code is not handled or not allowed 2026-01-24 22:28:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/framed-view-binder-with-one-touch-ezd-rings-3inch-capacity-navy-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:28:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dyna-ray-dr7477s-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:28:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nerm020l-l-fg-10-ner-food-grade-hoist-with-motorized-trolley-2-ton-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:28:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-12-828-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:28:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/outdoor_furniture/umbrellas_awnings_shades/outdoor_umbrellas_bases/umbrellas>: HTTP status code is not handled or not allowed 2026-01-24 22:28:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2617-expanded-pvc-panel-black>: HTTP status code is not handled or not allowed 2026-01-24 22:28:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esd-full-length-lab-coat-snap-cuff-light-blue-4xl>: HTTP status code is not handled or not allowed 2026-01-24 22:28:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4107-2-hole-joining-strip>: HTTP status code is not handled or not allowed 2026-01-24 22:29:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergi-lite-tsm27-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:29:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/biometric-wall-safe-3-34w-x-15-716d-x-20-34h-white>: HTTP status code is not handled or not allowed 2026-01-24 22:29:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/special_purpose_carts_trucks/medical_supply_carts/medical_storage_carts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:29:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dyna-ray-70930s-emergency-light-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:29:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6530-highcycle-single-flange-linear-bearing-brake-kit-ready>: HTTP status code is not handled or not allowed 2026-01-24 22:29:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8000-watt-power-inverter-pwrinv8kw12v>: HTTP status code is not handled or not allowed 2026-01-24 22:29:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2500-watt-24-volt-power-inverter-pwrinv250024w>: HTTP status code is not handled or not allowed 2026-01-24 22:29:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6873-single-unibearing-bearing-assembly-long>: HTTP status code is not handled or not allowed 2026-01-24 22:29:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-lz35i--emergency-light-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:29:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/teledyne-d2br12s7-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:29:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96-w-x-36-d-production-workbench-w-drawer-upright-shelf-maple-butcher-block-square-edge-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:29:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 30748 pages (at 92 pages/min), scraped 14512 items (at 19 items/min) 2026-01-24 22:29:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2414-economy-floor-mount-base-plate-1w>: HTTP status code is not handled or not allowed 2026-01-24 22:29:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ge-concord-4-12v-45ah-alarm-battery>: HTTP status code is not handled or not allowed 2026-01-24 22:29:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machinists-level-w-ground-graduated-vial-12l-x-2w-x-3h-wood-case>: HTTP status code is not handled or not allowed 2026-01-24 22:29:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ion-audio-party-rocker-speaker-replacement-battery-9ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 22:29:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caution-move-on-green-only-truck-delivery-sign-mirror-image-vinyl-10-x-14-yellow>: HTTP status code is not handled or not allowed 2026-01-24 22:29:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/special_purpose_carts_trucks/medical_supply_carts/medical_storage_carts landed on page that is not a product page. 2026-01-24 22:29:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/if-you-cant-see-my-mirrors-i-cant-see-you-sign-adhesive-dura-vinyl-7-x-10-yellow>: HTTP status code is not handled or not allowed 2026-01-24 22:29:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2335-deluxe-flange-mount-caster-rigid-without-brake>: HTTP status code is not handled or not allowed 2026-01-24 22:29:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2472-pvc-coated-wire-mesh-panel>: HTTP status code is not handled or not allowed 2026-01-24 22:29:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2605-acrylic-panel-black>: HTTP status code is not handled or not allowed 2026-01-24 22:29:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/warning-if-you-cant-see-my-mirrors-i-cant-see-you-sign-dura-vinyl-10-x-14-red>: HTTP status code is not handled or not allowed 2026-01-24 22:29:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arctic-cat-550-550cc-atv-replacement-battery-2010-2014-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:29:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3410-flat-head-socket-cap-screw-58l>: HTTP status code is not handled or not allowed 2026-01-24 22:29:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nerp030cd-20-ner-electric-hoist-push-trolley-3-ton-20-lift-173-ftmin-230v>: HTTP status code is not handled or not allowed 2026-01-24 22:29:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nerp001h-15-ner-electric-hoist-push-trolley-18-ton-15-lift-55-ftmin-230v>: HTTP status code is not handled or not allowed 2026-01-24 22:29:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enter-only-traffic-sign-egp-reflective-aluminum-18-x-18-blackwhite>: HTTP status code is not handled or not allowed 2026-01-24 22:29:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oversize-load-hinged-transportation-sign-aluminum-12-x-72-blackyellow>: HTTP status code is not handled or not allowed 2026-01-24 22:29:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5920-double-horizontal-base-1>: HTTP status code is not handled or not allowed 2026-01-24 22:29:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/notice-drivers-are-required-to-sign-in--out-sign-aluminum-10-x-14-blackbluewhite>: HTTP status code is not handled or not allowed 2026-01-24 22:29:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/parking_traffic_signs/parking_traffic_signs_1 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:29:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ups-pick-up-delivery-location-sign-aluminum-10-x-14-blackwhite>: HTTP status code is not handled or not allowed 2026-01-24 22:29:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yamaha-tw125-125cc-motorcycle-replacement-battery-1998-2004-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:29:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aprilia-rally-50cc-motorcycle-replacement-battery-2004-2005-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:29:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ner003sd-20-ner-dual-speed-electric-chain-hoist-14-ton-20-lift-366-ftmin-230v>: HTTP status code is not handled or not allowed 2026-01-24 22:29:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-hot-surface-ignition-module-with-variable-timings-50e47-843>: HTTP status code is not handled or not allowed 2026-01-24 22:29:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ner005s-20-ner-electric-hoist-hook-suspension-12-ton-20-lift-29-ftmin-230v>: HTTP status code is not handled or not allowed 2026-01-24 22:29:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/truck-route-traffic-sign-egp-reflective-aluminum-18-x-24-blackwhite>: HTTP status code is not handled or not allowed 2026-01-24 22:29:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/betamotor-eikon-150cc-motorcycle-replacement-battery-2000-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:29:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ner005l-fg-10-ner-food-grade-hoist-12-ton-capacity-230v>: HTTP status code is not handled or not allowed 2026-01-24 22:29:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ner010ld-20-ner-dual-speed-electric-chain-hoist-1-ton-20-lift-1425-ftmin-208v>: HTTP status code is not handled or not allowed 2026-01-24 22:29:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/parking_traffic_signs/parking_traffic_signs_1>: HTTP status code is not handled or not allowed 2026-01-24 22:29:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/retaining-magnet-assembly-threaded-bushing-126-dia-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 22:29:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/retaining-magnet-assembly-threaded-stud-and-rubber-jacket-346-dia-zinc-plated-steel>: HTTP status code is not handled or not allowed 2026-01-24 22:29:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/husqvarna-txc-310cc-motorcycle-replacement-battery-2010-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:29:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-only-label-w-recycle-sign-plastic-5-x-7>: HTTP status code is not handled or not allowed 2026-01-24 22:29:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/import-1-2-inch-extra-long-drill-bit-12-inch-long>: HTTP status code is not handled or not allowed 2026-01-24 22:29:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sner005l-20-sner-electric-hoist-hook-suspension-12-ton-20-lift-7-ftmin-230v>: HTTP status code is not handled or not allowed 2026-01-24 22:29:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pgo-tre-50-50cc-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:29:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forklift-karrier-55-gallon-12v-dc-battery-power-tilt-capacity-1500-lbs>: HTTP status code is not handled or not allowed 2026-01-24 22:29:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/notice-please-turn-out-lights-when-not-in-use-label-adhesive-vinyl-10-x-14>: HTTP status code is not handled or not allowed 2026-01-24 22:29:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-mobile-instrument-table-30-x-16-undershelf-304-grade>: HTTP status code is not handled or not allowed 2026-01-24 22:29:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wesco-stainless-steel-drum-dolly-30-gallon-zinc-rigs-hard-rubber-987294>: HTTP status code is not handled or not allowed 2026-01-24 22:29:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-channel-fixed-volume-pipettor-71212122-50ul-volume>: HTTP status code is not handled or not allowed 2026-01-24 22:29:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shaker-fixing-clip-18900032-200-250ml-for-round-flasks-use-w-sk-o180-sk-o330>: HTTP status code is not handled or not allowed 2026-01-24 22:29:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-channel-pipettor-71211206-5-50ul-volume-range>: HTTP status code is not handled or not allowed 2026-01-24 22:29:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60090002027-roll-211k-1w-x-50-yd-120>: HTTP status code is not handled or not allowed 2026-01-24 22:29:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/neodymium-telescoping-pickup-magnet-tool-07256-with-locking-hinge>: HTTP status code is not handled or not allowed 2026-01-24 22:29:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mx-s-vortex-mixer-82120004-adjustable-speed-110v-60hz-0-1500rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:29:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-cover-rc-rb80-for-round-base-magnetic-cups-rb80-3187-dia-375-hole>: HTTP status code is not handled or not allowed 2026-01-24 22:29:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-channel-pipettor-71211212-50-300ul-volume-range>: HTTP status code is not handled or not allowed 2026-01-24 22:29:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lpw-2-forced-air-wall-heater-12502750w-120v-white-dove>: HTTP status code is not handled or not allowed 2026-01-24 22:29:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g-tek-polykor-seamless-knit-blended-cr-gloves-nitrile-coated-ansi-a6-xl-black-1-pair>: HTTP status code is not handled or not allowed 2026-01-24 22:29:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anti-scratch-semi-rimless-safety-glasses-green-ir-shade-30-lens-black-frame-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-24 22:29:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pukka-gx400c-electric-scooter-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:29:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/fall_protection/anchorage_connectors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:29:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/array-jumbo-card-stock-paper-8-1-2-inch-x-11-inch-65-lb-assorted-250-sheets-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:29:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/astrobrights-colored-paper-8-1-2-inch-x-11-inch-24-lb-lunar-blue-500-sheets-ream>: HTTP status code is not handled or not allowed 2026-01-24 22:29:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/izip-i-750-24v-electric-scooter-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 22:29:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durapro-xcp-knee-boots-chevron-outsole-steel-toe-size-8-15h-brown>: HTTP status code is not handled or not allowed 2026-01-24 22:29:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/linen-business-paper-8-1-2-inch-x-11-inch-24-lb-linen-white-100-sheets-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:29:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durapro-knee-boots-safety-loc-outsole-steel-toe-size-9-15h-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:29:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/laser-inkjet-post-card-5-1-2-inch-x-4-1-4-inch-matte-white-200-cards-box>: HTTP status code is not handled or not allowed 2026-01-24 22:29:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96-x-48-slatwall-panel-brushed-aluminum-with-aluminum-inserts-pkg-qty-4>: HTTP status code is not handled or not allowed 2026-01-24 22:29:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vaccine-freezer-rhd-solid-door-388-cu-ft-capacity-artic-white>: HTTP status code is not handled or not allowed 2026-01-24 22:29:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-expedition-550f-sport-550cc-snowmobile-replacement-battery-2006-2012-12v>: HTTP status code is not handled or not allowed 2026-01-24 22:30:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/fall_protection/anchorage_connectors>: HTTP status code is not handled or not allowed 2026-01-24 22:30:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sticky-mat-refill-sheets-white-30-sheets-pad-10-case-sm-white>: HTTP status code is not handled or not allowed 2026-01-24 22:30:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pic-a-watt-compact-wall-heater-5002250w-240208v-stainless-steel-grill>: HTTP status code is not handled or not allowed 2026-01-24 22:30:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acifort-metmax-full-safety-boots-cleated-outsole-steel-toe-size-12-15h-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:30:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/laerdal-artstart-training-battery-medical-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:30:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durapro-xcp-knee-boots-chevron-outsole-steel-toe-size-15-16h-brown>: HTTP status code is not handled or not allowed 2026-01-24 22:30:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/snugboot-workpro-full-safety-knee-boots-cleated-outsole-composite-toe-size-8-16h-black>: HTTP status code is not handled or not allowed 2026-01-24 22:30:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-grey-nebula-round-table-w-4-chair-set-in-espresso-finish>: HTTP status code is not handled or not allowed 2026-01-24 22:30:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durapro-xcp-knee-boots-chevron-outsole-steel-toe-size-16-16h-brown>: HTTP status code is not handled or not allowed 2026-01-24 22:30:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/Ultra-Bariatric-Cushion-22-x-18-x-3-77004>: HTTP status code is not handled or not allowed 2026-01-24 22:30:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/loop-tape-individual-strips-with-adhesive75l-x-1w-white>: HTTP status code is not handled or not allowed 2026-01-24 22:30:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/physio-control-life-stat-1600-printer-medical-replacement-battery-1-3ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:30:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/invitations_cards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:30:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medical-industries-zappy-3-standard-medical-replacement-battery-9ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 22:30:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/index-maker-clear-label-divider-easy-apply-85x11-3-tabs-25-sets-white-white>: HTTP status code is not handled or not allowed 2026-01-24 22:30:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/loop-tape-individual-dots-with-adhesive-34l-x-34w-white-pack-of-1028>: HTTP status code is not handled or not allowed 2026-01-24 22:30:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 30845 pages (at 97 pages/min), scraped 14531 items (at 19 items/min) 2026-01-24 22:30:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onestep-printable-toc-divider-printed-a-to-z-9x11-26-tabs-white-white>: HTTP status code is not handled or not allowed 2026-01-24 22:30:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/index-maker-clear-label-divider-blank-85x11-5-tabs-25-sets-white-white>: HTTP status code is not handled or not allowed 2026-01-24 22:30:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/patriot-solar-100w-portable-generator-solar-replacement-battery-100ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:30:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/index-maker-punched-clear-label-tab-divider-blank-85x11-12-tabs-5-sets-white-multi>: HTTP status code is not handled or not allowed 2026-01-24 22:30:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-source-wp20-12-91-219-sealed-lead-acid-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:30:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fixed-height-steel-gantry-crane-2000-lb-capacity-20w-x-10-78h>: HTTP status code is not handled or not allowed 2026-01-24 22:30:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dock-seal-36-oz-capacity-10-projection-10w-x-10-black>: HTTP status code is not handled or not allowed 2026-01-24 22:30:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/recycled-write-on-tab-divider-85x11-8-tabs-24-sets-multicolor-multicolor>: HTTP status code is not handled or not allowed 2026-01-24 22:30:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/schumacher-electric-ip-1825fl-jump-starter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:30:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/paper_stationery/invitations_cards>: HTTP status code is not handled or not allowed 2026-01-24 22:30:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stackable-shelf-bin-8-1-4w-x-18d-x-7h-black>: HTTP status code is not handled or not allowed 2026-01-24 22:30:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stackable-shelf-bin-8-1-4w-x-20-1-2d-x-7h-black>: HTTP status code is not handled or not allowed 2026-01-24 22:30:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-step-mate-stand-with-handrail-22-78l-x-25-1516w-x-43-18h-yellow>: HTTP status code is not handled or not allowed 2026-01-24 22:30:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spectra-art-tissue-10-lbs-12-x-18-10-assorted-colors-50-sheets-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:30:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-newsprint-30-lbs-18-x-24-white-500-sheets-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:30:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cream-manila-drawing-paper-50-lbs-12-x-18-500-sheets-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:30:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spin-top-ratchet-trailer-stabilizing-jack-hard-rubber-wheels-40000-lb-lifting-cap>: HTTP status code is not handled or not allowed 2026-01-24 22:30:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelf-unit-36w-x-21d-x-74h-with-12-ssb483-and-12-ssb485-stackable-shelf-bins-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:30:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nanuk-330-0011-330-series-nano-case-clear>: HTTP status code is not handled or not allowed 2026-01-24 22:30:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nanuk-990-1006-990-series-rifle-case-w-foam-olive>: HTTP status code is not handled or not allowed 2026-01-24 22:30:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pivot-steel-mezzanine-pallet-gate-standard-72w-x-60d-x-78h-clearance>: HTTP status code is not handled or not allowed 2026-01-24 22:30:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-pocket-drawer-organizer-plastic-tray-10l-x-3w-x-14h>: HTTP status code is not handled or not allowed 2026-01-24 22:30:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/optiway-technology-2000-mini-u1-wheelchair-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:30:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/opus-38-thick-entrance-floor-mat-2x3-brown>: HTTP status code is not handled or not allowed 2026-01-24 22:30:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pride-mobility-victory-es-9-s93-wheelchair-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:30:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paramount-wardrobe-cabinet-easy-assembly-36x24x72-gray-270033GY>: HTTP status code is not handled or not allowed 2026-01-24 22:30:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emt-plastic-bushing-polyethylene-1-size>: HTTP status code is not handled or not allowed 2026-01-24 22:30:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-clear-quartz-lamp-631-38579-for-462-463-and-mm-electric-infrared-heaters-3800w-600v>: HTTP status code is not handled or not allowed 2026-01-24 22:30:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-clear-quartz-lamp-630-16249-for-222-223-and-mm-electric-infrared-heaters-1600w-240v>: HTTP status code is not handled or not allowed 2026-01-24 22:30:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-power-group-ub1270-40800-sealed-lead-acid-replacement-battery-7ah-12v>: HTTP status code is not handled or not allowed 2026-01-24 22:30:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electric-mobility-rascal-500t-wheelchair-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:30:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/45-degree-liquid-tight-connector-alloy-steel-1-size-5-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:30:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-pvc-liquid-tight-connector-34-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:30:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-person-stage-12-inch-w-x-24-ft-l-2324>: HTTP status code is not handled or not allowed 2026-01-24 22:30:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rigid-conduit-sealing-locknut-carbon-steel-1-14-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:30:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-nut-silicon-bronze-38-16-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:30:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dwa-dwe402-412-small-angle-grinder-no-lockon>: HTTP status code is not handled or not allowed 2026-01-24 22:30:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-ft-type-1a-aluminum-d-rung-straight-ladder-d1516-1>: HTTP status code is not handled or not allowed 2026-01-24 22:30:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2361-20-portable-18v-led-flood-light-1100-lumens>: HTTP status code is not handled or not allowed 2026-01-24 22:30:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c-style-threaded-conduit-body-w-cover-and-gasket-4-size>: HTTP status code is not handled or not allowed 2026-01-24 22:30:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/orthofab-lifestyles-vip-wheelchair-22nf-wheelchair-replacement-battery-55ah-12v>: HTTP status code is not handled or not allowed 2026-01-24 22:30:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lr-threaded-conduit-body-w-cover-and-gasket-1-12-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:30:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ef-ct-fr-dress-uniform-shirt-slu2-khaki-size-3xl-regular>: HTTP status code is not handled or not allowed 2026-01-24 22:30:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/golden-technologies-gp600-cc-22nf-wheelchair-replacement-battery-55ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:30:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ef-flame-resistant-classic-coverall-cec2-navy-size-52-regular>: HTTP status code is not handled or not allowed 2026-01-24 22:30:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/permobil-chairman-entra-stander-jr-wheelchair-replacement-battery-75ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:30:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-profile-air-curtain-36-inch-w-door-208v-unheated-3-ph-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 22:30:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ef-ct-fr-dress-uniform-shirt-slu2-silver-gray-size-m-regular>: HTTP status code is not handled or not allowed 2026-01-24 22:30:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ef-ct-fr-dress-uniform-shirt-slu2-silver-gray-size-s-regular>: HTTP status code is not handled or not allowed 2026-01-24 22:30:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milwaukee-2729-20-m18-fuel-deep-cut-band-saw-bare-tool>: HTTP status code is not handled or not allowed 2026-01-24 22:30:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/no-thread-coupling-12-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:30:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/invacare-cat-basic-14-inch-or-less-wheelchair-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:30:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dalton-primechair-midwheel-drive-pc-mp3cm-ar-bl-wheelchair-battery-35ah-12v>: HTTP status code is not handled or not allowed 2026-01-24 22:30:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pride-mobility-elite-traveller-lr-sc40lr-wheelchair-replacement-battery-22ah-12v>: HTTP status code is not handled or not allowed 2026-01-24 22:30:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kpex-95-62-190-sba-wire-rope-cutters-comfort-grip-7-12-oal>: HTTP status code is not handled or not allowed 2026-01-24 22:30:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-pulling-hook-316-size>: HTTP status code is not handled or not allowed 2026-01-24 22:30:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kpex-71-21-200-sba-angeled-high-leverage-cobolt-bolt-cutters-8-oal>: HTTP status code is not handled or not allowed 2026-01-24 22:30:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pull-rope-polypropylene-12-x-600-size>: HTTP status code is not handled or not allowed 2026-01-24 22:30:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slotted-flat-head-machine-screw-14-20-x-2-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:30:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eagle-paintink-safety-cabinet-with-manual-close30-gallon-yellow-270055>: HTTP status code is not handled or not allowed 2026-01-24 22:30:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5506-class-3-standard-moisture-wicking-tshirt-with-chest-pocket-orange-xl>: HTTP status code is not handled or not allowed 2026-01-24 22:30:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-precision-bearing-7214cydup4-universal-ground-duplex-70mm-bore-125mm-od>: HTTP status code is not handled or not allowed 2026-01-24 22:30:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylon-walldriller-anchor-kit-w-phillips-wafer-head-screw-8-size-50-pieces>: HTTP status code is not handled or not allowed 2026-01-24 22:30:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triple-grip-anchor-kit-w-combo-pan-head-screw-12-size-tuff-pack-101-pieces>: HTTP status code is not handled or not allowed 2026-01-24 22:30:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-precision-bearing-7012cydup4-universal-ground-duplex-60mm-bore-95mm-od>: HTTP status code is not handled or not allowed 2026-01-24 22:30:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1801-standard-class-2-5point-breakaway-vest-lime-3xl>: HTTP status code is not handled or not allowed 2026-01-24 22:30:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pacesaver-plus-iii-titan-plus-3-titan-wheelchair-replacement-battery-75ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:31:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/phillips-and-square-flat-head-machine-screw-kit-6-32-size-assorted>: HTTP status code is not handled or not allowed 2026-01-24 22:31:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibration-mount-1-tapped-hole-1-threaded-stud-100-inch-dia-50-inch-h-1-4-20-thread>: HTTP status code is not handled or not allowed 2026-01-24 22:31:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slotted-flat-head-machine-screw-8-32-x-1-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:31:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1003-standard-class-2-mesh-hook-loop-safety-vest-lime-xl>: HTTP status code is not handled or not allowed 2026-01-24 22:31:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6001-class-3-rain-coat-with-2-patch-pockets-lime-4xl5xl>: HTTP status code is not handled or not allowed 2026-01-24 22:31:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sonnenschein-powerware-personal-emergency-light-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:31:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-fish-stick-rod-w-eye-and-hook-fibreglass-14-x-12-size>: HTTP status code is not handled or not allowed 2026-01-24 22:31:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6002-class-3-rain-coat-with-2-patch-pockets-orange-sm>: HTTP status code is not handled or not allowed 2026-01-24 22:31:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slotted-flat-head-machine-screw-10-24-x-2-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:31:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mongoose-2004-procosmic-electric-bicycle-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 22:31:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 30941 pages (at 96 pages/min), scraped 14550 items (at 19 items/min) 2026-01-24 22:31:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perf-24-w-11-step-stl-roll-lad-21-d-top-stp-hndrl-lck-orng-fs113221po>: HTTP status code is not handled or not allowed 2026-01-24 22:31:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wallpro-300-black-wall-mount-retracting-barrier-75-yellowblack-out-of-service-belt>: HTTP status code is not handled or not allowed 2026-01-24 22:31:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-back-executive-chair-25-inchw-x-23-5-8-inchd-x-53-inchh-black-leather-seat-mesh-back>: HTTP status code is not handled or not allowed 2026-01-24 22:31:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/can-am-outlander-max-500-efi-500cc-atv-replacement-battery-2011-2014-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:31:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-head-wood-screw-square-carbon-steel-10-x-2-12-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:31:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inventory-cone-lime-3-sided>: HTTP status code is not handled or not allowed 2026-01-24 22:31:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12mn81d-cast-iron-spoked-handwheel-wrevolving-handle-315-dia-x-12mm-bore>: HTTP status code is not handled or not allowed 2026-01-24 22:31:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ehvm1hg-mp3-smart-alert-user-programmable-off-white-dc12v-to-dc24v>: HTTP status code is not handled or not allowed 2026-01-24 22:31:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manual-anti-tilt-switch-thp-atme-heaters-from-2009-or-newer>: HTTP status code is not handled or not allowed 2026-01-24 22:31:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sheet-metal-screw-square-pan-head-carbon-steel-10-x-12-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:31:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-degree-easyjet-mc101a-motorcaddy-and-golf-caddy-replacement-battery-26ah-12v>: HTTP status code is not handled or not allowed 2026-01-24 22:31:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/duplicate-auto-park-tags-numbered-1-to-500-4-3-4-x-2-3-8-manila-500-tags-box>: HTTP status code is not handled or not allowed 2026-01-24 22:31:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/525m-adjustable-5-hole-punch-40-sheet-capacity-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:31:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/peg-perego-cam-am-traxter-igor0019cam-ride-on-toys-replacement-battery-35ah-12v>: HTTP status code is not handled or not allowed 2026-01-24 22:31:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drywall-screw-phillips-bugle-head-fine-thread-6-x-1-14-size-tuff-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:31:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrs262401r-cast-iron-right-angle-worm-gear-reducer-401-ratio>: HTTP status code is not handled or not allowed 2026-01-24 22:31:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/record-book-record-ruled-8-3-4-x-14-1-4-black-cover-500-pages-pad>: HTTP status code is not handled or not allowed 2026-01-24 22:31:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/saunders-redi-rite-aluminum-storage-clipboard-8-1-2-x-12-silver>: HTTP status code is not handled or not allowed 2026-01-24 22:31:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-head-wood-screw-phillips-carbon-steel-10-x-1-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:31:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-storage-clipboard-top-opening-8-1-2-x-12-silver>: HTTP status code is not handled or not allowed 2026-01-24 22:31:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hanger-bolt-14-x-3-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:31:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrf175101l56c-cast-iron-right-angle-worm-gear-reducer-101-ratio-56c-frame>: HTTP status code is not handled or not allowed 2026-01-24 22:31:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/through-hardened-flat-washer-steel-12-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:31:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-staples-3-8-leg-length-125-per-strip-2500-box>: HTTP status code is not handled or not allowed 2026-01-24 22:31:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrf262201r56c-cast-iron-right-angle-worm-gear-reducer-201-ratio-56c-frame>: HTTP status code is not handled or not allowed 2026-01-24 22:31:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sheet-metal-screw-phillipsslotted-hex-washer-head-12-x-1-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:31:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrf325501de145tc-cast-iron-right-angle-worm-gear-reducer-501-ratio-145t-frame>: HTTP status code is not handled or not allowed 2026-01-24 22:31:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/panel-wall-clips-40-sheet-capacity-black-20-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:31:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-heavy-duty-staples-3-8-leg-length-100-per-strip-1000-box>: HTTP status code is not handled or not allowed 2026-01-24 22:31:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-bands-7-x-1-8-red-12-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:31:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-washer-stainless-steel-0-14-id-12-od-10-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:31:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pop-up-notes-dispenser-3-in-x-3-in-black-dispenser-50-sheets-dispenser-1-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:31:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harley-davidson-cvo-dyna-fat-bob-fxdfse2-1584cc-motorcycle-battery-2010-12v>: HTTP status code is not handled or not allowed 2026-01-24 22:31:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/twinleg-self-retracting-lifeline-steel-snap-hook-6>: HTTP status code is not handled or not allowed 2026-01-24 22:31:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/norton-850-commando-commando-850cc-motorcycle-replacement-battery-1975-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:31:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/titanium-bonded-scissors-8l-straight-gray-yellow-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:31:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/automatic-label-rewinder-for-up-to-6-1-2-w-x-12-diameter-3-core-rolls>: HTTP status code is not handled or not allowed 2026-01-24 22:31:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waterproof-breathable-pants-wsilver-reflective-tape-black-2xl>: HTTP status code is not handled or not allowed 2026-01-24 22:31:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3230st-electric-3-hole-punch-stapler-24-sheet-capacity-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:31:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-strut-strap-carbon-steel-2-size-50-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:31:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pistol-case-16l-x-13w-x-5h-silver-dot>: HTTP status code is not handled or not allowed 2026-01-24 22:31:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/closed-add-on-cab73-2442-7x-42-inch-w-x-24-inch-d-x-73-inch-h-7-shelves-18-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:31:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/closed-starter-csb73-2448-6x-48-inch-w-x-24-inch-d-x-73-inch-h-6-shelves-18-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:31:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/closed-starter-csb73-1542-5x-42-inch-w-x-15-inch-d-x-73-inch-h-5-shelves-18-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:31:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-head-tap-bolt-stainless-steel-14-20-x-2-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:31:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-head-tap-bolt-stainless-steel-38-16-x-1-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:31:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/victory-arlen-ness-vision-1731cc-motorcycle-replacement-battery-2010-2012-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:31:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/closed-starter-csb73-1236-6x-36-inch-w-x-12-inch-d-x-73-inch-h-6-shelves-18-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:31:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pr-fleece-work-jacket-w-2-zipped-slash-pockets-elastic-cuffs-black-size-xl>: HTTP status code is not handled or not allowed 2026-01-24 22:31:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pr-48-two-tone-waterproof-rain-coat-ansi-class-3-lime-black-size-5x>: HTTP status code is not handled or not allowed 2026-01-24 22:31:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pr-48-two-tone-waterproof-rain-coat-ansi-class-3-lime-black-size-xl>: HTTP status code is not handled or not allowed 2026-01-24 22:31:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harley-davidson-low-rider-fxs-1200cc-motorcycle-replacement-battery-1977-1978-12v>: HTTP status code is not handled or not allowed 2026-01-24 22:31:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-style-warmer-dispenser-with-pump-7w-7d-12-3-4h>: HTTP status code is not handled or not allowed 2026-01-24 22:31:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-popcorn-scoop>: HTTP status code is not handled or not allowed 2026-01-24 22:31:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/combo-led-emergency-exit-sign-red-letters-battery-backup>: HTTP status code is not handled or not allowed 2026-01-24 22:31:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-deluxe-warmer-ladle-unit-9-1-4w-11-3-4d-9h-model-a>: HTTP status code is not handled or not allowed 2026-01-24 22:31:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/twinkle-stainless-steel-cleaner-polish-17oz-aerosol-1-case-dvo91224ea>: HTTP status code is not handled or not allowed 2026-01-24 22:31:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fixture-carrier-in-wall-floor-mount>: HTTP status code is not handled or not allowed 2026-01-24 22:31:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fixture-carrier-with-custom-fit-side-drain-port-in-wall-floor-mount>: HTTP status code is not handled or not allowed 2026-01-24 22:31:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polaris-sl-900-900cc-personal-watercraft-replacement-battery-1996-1997-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:31:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-hcflu208-24-flange-mount-bearing-2-bolt-flange-eccentric-locking-collar-bore-38-1-mm>: HTTP status code is not handled or not allowed 2026-01-24 22:31:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-ucf210-30-flange-mount-bearing-4-bolt-flange-set-screw-locking-bore-47-625-mm>: HTTP status code is not handled or not allowed 2026-01-24 22:31:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kawasaki-jet-ski-jh750-xi-750cc-personal-watercraft-battery-1992-1999-12v>: HTTP status code is not handled or not allowed 2026-01-24 22:31:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-hcflu207-20-flange-mount-bearing-2-bolt-flange-eccentric-locking-collar-bore-31-75-mm>: HTTP status code is not handled or not allowed 2026-01-24 22:31:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zap-sx-electric-bicycle-electric-scooter-replacement-battery-18ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:31:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-row-spherical-roller-bearing-22319exw33c3-95mm-bore-200mm-od>: HTTP status code is not handled or not allowed 2026-01-24 22:31:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/imc-heartway-escape-hp8-mobility-scooter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:32:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-ucfl208-24-flange-mount-bearing-2-bolt-flange-set-screw-locking-bore-38-1-mm>: HTTP status code is not handled or not allowed 2026-01-24 22:32:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jenoptik-125-st4-leader-125cc-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:32:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1505-lowprofile-weight-lifters-back-support-black-xl>: HTTP status code is not handled or not allowed 2026-01-24 22:32:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zapworld-classic-scooter-electric-bicycle-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:32:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6717fr-evaporative-cooling-hard-hat-pad-with-neck-shade-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:32:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/golden-technologies-sterling-mobility-scooter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:32:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/piaggio-vespa-skipper-125-125cc-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:32:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flammable-cabinet-self-close-single-door-30-gallon-capacity-43w-x-18d-x-44h>: HTTP status code is not handled or not allowed 2026-01-24 22:32:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8091-noncertified-long-sleeve-tshirt-orange-xl>: HTTP status code is not handled or not allowed 2026-01-24 22:32:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thoratec-laboratories-cs600x-mobility-scooter-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:32:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/golden-technologies-agm1248-mobility-scooter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:32:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/odin-pz-safety-glasses-polarized-g15-lens-black-frame>: HTTP status code is not handled or not allowed 2026-01-24 22:32:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 31039 pages (at 98 pages/min), scraped 14569 items (at 19 items/min) 2026-01-24 22:32:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nferno6820-fr-knit-cap-black>: HTTP status code is not handled or not allowed 2026-01-24 22:32:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-row-cylindrical-roller-bearing-n308-40mm-bore-90mm-od>: HTTP status code is not handled or not allowed 2026-01-24 22:32:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-power-group-ub12180-replaces-17ah-mobility-scooter-battery-18ah-12v>: HTTP status code is not handled or not allowed 2026-01-24 22:32:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6602-evaporative-cooling-towel-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:32:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/klip-it-open-starter-osk73-1542-5-42-inch-w-x-15-inch-d-x-73-inch-h-5-shelves-20-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:32:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drive-medical-wildcat-450-wildcat450bk24ss-mobility-scooter-battery-55ah-12v>: HTTP status code is not handled or not allowed 2026-01-24 22:32:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/headlight-rh-led-for-global-industrial-utility-vehicle-615162>: HTTP status code is not handled or not allowed 2026-01-24 22:32:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/klip-it-closed-starter-csk97-3042-8-42-inch-w-x-30-inch-d-x-97-inch-h-8-shelves-20-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:32:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/klip-it-closed-add-on-cak73-2448-5-48-inch-w-x-24-inch-d-x-73-inch-h-5-shelves-20-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:32:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelite-plastic-mat-21-x-18-end-panel-PME2118>: HTTP status code is not handled or not allowed 2026-01-24 22:32:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merits-travel-ease-commuter-p200-mobility-scooter-replacement-battery-55ah-12v>: HTTP status code is not handled or not allowed 2026-01-24 22:32:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-bolt-open-starter-ost75-1836-5-36-inch-w-x-18-inch-d-x-75-inch-h-5-shelves-20-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:32:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/razor-15128030-mx-400-dirt-rocket-electric-scooter-replacement-battery-8ah-12v>: HTTP status code is not handled or not allowed 2026-01-24 22:32:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-in-x-4-1-2-in-lead-free-seamless-red-brass-pipe-nipple-140-psi-sch-40-import>: HTTP status code is not handled or not allowed 2026-01-24 22:32:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bravo-b4-lightning-electricscooter-electric-scooter-replacement-battery-5ah-12v>: HTTP status code is not handled or not allowed 2026-01-24 22:32:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-chrome-esd-wire-shelving-unit-starter-36w-x-24d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 22:32:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-in-x-1-1-8-in-stainless-steel-pipe-nipple-16168-psi-sch-40-domestic>: HTTP status code is not handled or not allowed 2026-01-24 22:32:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/golden-technologies-buzz-around-xls-3-wheel-gb117-gb117s-scooter-battery-12ah-12v>: HTTP status code is not handled or not allowed 2026-01-24 22:32:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-bolt-closed-add-on-cat75-1836-6-36-inch-w-x-18-inch-d-x-75-inch-h-6-shelves-20-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:32:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-chrome-wire-shelving-unit-starter-42w-x-14d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 22:32:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-gsx-limited-1200-1171cc-snowmobile-replacement-battery-2009-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:32:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grip-recycled-correction-tape-1-5-in-x-335-in-white-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:32:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-chrome-wire-shelving-unit-add-on-60w-x-21d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 22:32:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-chrome-wire-shelving-unit-add-on-72w-x-12d-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 22:32:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n499f-snow-safety-sign-notice-ice-and-snow-take-it-slow-32-x-24-white-blue-black>: HTTP status code is not handled or not allowed 2026-01-24 22:32:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bombardier-mx-z-sdi-adrenaline-and-x-593cc-snowmobile-battery-2005-2006-12v>: HTTP status code is not handled or not allowed 2026-01-24 22:32:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-chrome-wire-shelving-unit-add-on-60w-x-24d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 22:32:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wxtp-50v-14-gallon-expansion-tank-1-in-fnpt>: HTTP status code is not handled or not allowed 2026-01-24 22:32:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-inking-message-date-stamp-paid-date-blue-red>: HTTP status code is not handled or not allowed 2026-01-24 22:32:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-ski-doo-gtx-550f-550cc-snowmobile-replacement-battery-2005-2008-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:32:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blank-award-certificate-golden-seal-25-x-8-1-2-gold-25-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:32:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-formula-500-deluxe-500cc-snowmobile-replacement-battery-1997-1998-12v-i>: HTTP status code is not handled or not allowed 2026-01-24 22:32:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slide-seal-reclosable-poly-bags-22-x-22-3-mil-clear-100-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:32:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pre-inked-message-stamp-for-deposit-only-1-2-x-1-5-8-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:32:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pre-inked-message-stamp-received-1-2-x-1-5-8-red>: HTTP status code is not handled or not allowed 2026-01-24 22:32:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-stainless-steel-wire-shelving-unit-add-on-24w-x-21d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 22:32:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arctic-cat-m1000-efi-1000cc-snowmobile-replacement-battery-2007-2008-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:32:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yamaha-fx10rtra-fx-nytro-rtx-40th-anniv-1049cc-snowmobile-battery-2008-12v>: HTTP status code is not handled or not allowed 2026-01-24 22:32:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yamaha-rs-venture-gt-1049cc-snowmobile-replacement-battery-2009-2012-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:32:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bill-of-lading-snap-a-way-3-part-carbonless-8-1-2-x-7-250-sets-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:32:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-chrome-esd-wire-shelving-unit-starter-60w-x-12d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 22:32:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-chrome-esd-wire-shelving-unit-starter-60w-x-12d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 22:32:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-ski-doo-mx-z-500ss-trail-adrenaline-593cc-snowmobile-battery-2004-2005-12v>: HTTP status code is not handled or not allowed 2026-01-24 22:32:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cool-shield-thermal-bubble-rolls-16-x-125-x-3-16-thickness>: HTTP status code is not handled or not allowed 2026-01-24 22:32:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-gsx-sport-500ss-597cc-snowmobile-replacement-battery-2008-2009-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:32:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/packing-list-enclosed-2-x-3-labels-red-black-500-per-roll>: HTTP status code is not handled or not allowed 2026-01-24 22:32:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sl-03663-su-mh400-u-mog-400-watt-metal-halide-light-bulb-mogul-base>: HTTP status code is not handled or not allowed 2026-01-24 22:32:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polaris-indy-frontier-classic-built-after-1-01-02-784cc-snowmobile-battery-12v>: HTTP status code is not handled or not allowed 2026-01-24 22:32:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-poly-green-wire-shelving-unit-starter-60w-x-14d-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 22:32:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-mx-z-tnt-ace-900-899cc-snowmobile-replacement-battery-2014-2018-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:32:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-poly-green-wire-shelving-unit-starter-30w-x-18d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 22:32:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-poly-green-wire-shelving-unit-add-on-60w-x-12d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 22:32:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-8274-containment-berm-economy-12-x-12-copolymer-2000>: HTTP status code is not handled or not allowed 2026-01-24 22:32:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/miller-turbolite15343-personal-fall-limiters-snap-hook-carabiner-9l>: HTTP status code is not handled or not allowed 2026-01-24 22:32:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bopp-film-packaging-tape-ap-201-acrylic-production-grade-2-x-110yd-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:32:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fiberglass-reinforced-strapping-tape-gs-500-utility-grade-2-x-60yd-white>: HTTP status code is not handled or not allowed 2026-01-24 22:32:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-mx-z-x-e-tec-850-850cc-snowmobile-replacement-battery-2017-2018-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:32:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chemical-resistant-gloves-rough-grip-butyl-13-mil-thick-size-8-black>: HTTP status code is not handled or not allowed 2026-01-24 22:32:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/door-frame-with-drywall-afterset-curries-hinge-location-left-hand-36w-x-80h>: HTTP status code is not handled or not allowed 2026-01-24 22:32:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-plastic-locker-11-921212600-double-tier-12-x-12-x-30-1-wide-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:32:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m18-redlithium-xc-battery-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:33:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/87-in-h-heavy-duty-patio-heater-cover-hvd-sgtcv-p-for-36-in-square-glass-tube-paprika>: HTTP status code is not handled or not allowed 2026-01-24 22:33:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v6011tx-combination-flat-elbow-tee-ivory-4-34l>: HTTP status code is not handled or not allowed 2026-01-24 22:33:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-plus-phenolic-locker-11-811218721-single-tier-12-x-18-x-72-dove-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:33:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/freshaire-make-up-air-inverter-ptac-unit-with-heat-pump-11800-btu-cool-208230v>: HTTP status code is not handled or not allowed 2026-01-24 22:33:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2600wc-wire-clip-34l>: HTTP status code is not handled or not allowed 2026-01-24 22:33:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-ski-doo-mx-z-600-ho-sdi-adrenaline-and-x-593cc-snowmobile-battery-2005-2006-12v>: HTTP status code is not handled or not allowed 2026-01-24 22:33:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yamaha-cs340-ovation-deluxe-le-340cc-snowmobile-replacement-battery-1989-1998-12v>: HTTP status code is not handled or not allowed 2026-01-24 22:33:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-1002nl-full-port-ball-valve-no-lead-forged-brass-14-threaded>: HTTP status code is not handled or not allowed 2026-01-24 22:33:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-renegade-800-backcountry-800cc-snowmobile-replacement-battery-2010-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:33:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/styluspen-retractable-ballpoint-pen-07mm-black-ink-silver-barrel-1-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:33:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-heavyweight-reference-sheet-protectors-8-1-2w-x-11h-nonglare-50-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:33:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-ski-doo-mx-z-renegade-800r-800cc-snowmobile-replacement-battery-2004-2013-12v>: HTTP status code is not handled or not allowed 2026-01-24 22:33:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/contractor-3-inch-stiff-scraper-angled-140900430>: HTTP status code is not handled or not allowed 2026-01-24 22:33:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xl-glide-2-inch-144152320>: HTTP status code is not handled or not allowed 2026-01-24 22:33:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 31138 pages (at 99 pages/min), scraped 14593 items (at 24 items/min) 2026-01-24 22:33:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-summit-highmark-700-700cc-snowmobile-replacement-battery-2000-2003-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:33:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-ski-doo-grand-touring-700-697cc-snowmobile-replacement-battery-2001-2003-12v>: HTTP status code is not handled or not allowed 2026-01-24 22:33:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/photoluminescent-egress-tape-glow-in-the-dark-1-x-60>: HTTP status code is not handled or not allowed 2026-01-24 22:33:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sl-toolbox-z600-big-grip-bucket-wh-rags-300-sheets-2-bucket-60320>: HTTP status code is not handled or not allowed 2026-01-24 22:33:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/piling-bumper-6-pvc-white-1-case-1020-f>: HTTP status code is not handled or not allowed 2026-01-24 22:33:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bombardier-summit-1000-sdi-highmark-and-highmark-x-995cc-snowmobile-battery-2005-12v>: HTTP status code is not handled or not allowed 2026-01-24 22:33:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_supplies_dispensers/restroom_urinal_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:33:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lucy-3-piece-outdoor-bistro-set-27-1-2-w-x-29-1-2-h-table-taupe>: HTTP status code is not handled or not allowed 2026-01-24 22:33:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-725-2-piece-large-port-ball-valve-carbon-steel-12-threaded>: HTTP status code is not handled or not allowed 2026-01-24 22:33:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-408nl-non-rising-stem-gate-valve-no-lead-brass-2-threaded>: HTTP status code is not handled or not allowed 2026-01-24 22:33:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-x-5-inventory-rectangle-labels-in-5-fluorescent-colors-5000-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:33:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/severe-duty-motor-e617-184ttfs6526-5hp-460v-1800rpm-3ph-184t-fr-tefc>: HTTP status code is not handled or not allowed 2026-01-24 22:33:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-365ss-butterfly-valve-with-epdm-seals--10-position-handle-4-lug>: HTTP status code is not handled or not allowed 2026-01-24 22:33:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-710-1-piece-conventional-port-ball-valve-stainless-steel-34-threaded>: HTTP status code is not handled or not allowed 2026-01-24 22:33:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sky-3-piece-outdoor-bistro-set-31-1-2-w-x-29-1-2-h-table-black>: HTTP status code is not handled or not allowed 2026-01-24 22:33:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/building_construction_tape/gaffer_tape already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:33:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4611-double-sided-vhb-acrylic-foam-tape-1-x-5-yds-45-mil-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:33:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inverter-duty-motor-y572-326thfs8028-50hp-230-460v-1800rpm-3ph-326t-tefc>: HTTP status code is not handled or not allowed 2026-01-24 22:33:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_supplies_dispensers/restroom_urinal_mats>: HTTP status code is not handled or not allowed 2026-01-24 22:33:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fao-physicianscare-antacid-50-x-2-box>: HTTP status code is not handled or not allowed 2026-01-24 22:33:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-712-2-piece-full-port-316-ball-valve-stainless-steel-12-threaded>: HTTP status code is not handled or not allowed 2026-01-24 22:33:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fao-sterile-gauze-pads-3-x-3-10-box>: HTTP status code is not handled or not allowed 2026-01-24 22:33:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/small-room-fan-forced-wall-heater-sra1512dsf-1500w-120v>: HTTP status code is not handled or not allowed 2026-01-24 22:33:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/prescription-drop-box-rx-164-6-5-8w-x-14-1-8d-x-15-3-4h-beige>: HTTP status code is not handled or not allowed 2026-01-24 22:33:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/centrifugal-pump-motor-j057-1hp-115-208-230v-3600rpm-1ph-56j-fr-dp>: HTTP status code is not handled or not allowed 2026-01-24 22:33:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oxygen-response-bag-orange>: HTTP status code is not handled or not allowed 2026-01-24 22:33:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/building_construction_tape/gaffer_tape>: HTTP status code is not handled or not allowed 2026-01-24 22:33:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-response-complete-navy>: HTTP status code is not handled or not allowed 2026-01-24 22:33:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/para-systems-minuteman-b00009-ups-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 22:33:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/argosrack-gun-open-rack-awr83-18r2b-holds-18-rifles-with-2-bins-42w-x-15d-x-83h-battleship-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:33:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storage-argos-half-shelf-ws-6421-ws23-battleship-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:33:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-365di-butterfly-valve-with-epdm-seals--10-position-handle-2-lug>: HTTP status code is not handled or not allowed 2026-01-24 22:33:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-365ab-butterfly-valve-with-epdm-seals--10-position-handle-12-lug>: HTTP status code is not handled or not allowed 2026-01-24 22:33:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pressure-washer-motor-k615-56b34f5312-2hp-115-230v-3600rpm-1ph-56hc-fr>: HTTP status code is not handled or not allowed 2026-01-24 22:33:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inverter-duty-motor-y1372-184thfw7726-5hp-230-460v-1800rpm-3ph-184tc-tefc>: HTTP status code is not handled or not allowed 2026-01-24 22:33:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/severe-duty-motor-w537-215thfs9001-10hp-460v-3600rpm-3ph-215t-fr-tefc>: HTTP status code is not handled or not allowed 2026-01-24 22:33:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-335ab-butterfly-valve-with-epdm-seals--10-position-handle-3-wafer>: HTTP status code is not handled or not allowed 2026-01-24 22:33:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enersys-np17-12i-sealed-lead-acid-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:33:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vision-cg12-17xa-sealed-lead-acid-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:33:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/250-lbs-concrete-base-with-6-u-channel-post-18-square-base-white>: HTTP status code is not handled or not allowed 2026-01-24 22:33:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hyperpure-pe-rt-tubing-12id-x-58od-x-100-coil-red>: HTTP status code is not handled or not allowed 2026-01-24 22:33:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/expertpower-exp650-sealed-lead-acid-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:33:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e261-5kcr46mn0090x-1-2hp-1725-1425rpm-100-120-200-240v-1ph-56c-fr-dp>: HTTP status code is not handled or not allowed 2026-01-24 22:33:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/panasonic-lcl-24p-sealed-lead-acid-replacement-battery-26ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:33:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u901-284tstdp4004-30hp-3600rpm-200v-3ph-284ts-fr-dp>: HTTP status code is not handled or not allowed 2026-01-24 22:33:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yuasa-npx25-250fr-sealed-lead-acid-replacement-battery-5ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 22:33:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/amstron-ap-1270f2-sealed-lead-acid-replacement-battery-8ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 22:33:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/900-147-fish-tape-50-steel>: HTTP status code is not handled or not allowed 2026-01-24 22:33:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/defender174-spdt-explosion-proof-hd-thermostat-heatcool-only-22a>: HTTP status code is not handled or not allowed 2026-01-24 22:33:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hyperpure-pe-rt-tubing-38id-x-12od-x-500-coil-white>: HTTP status code is not handled or not allowed 2026-01-24 22:33:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simpson-95003-mobile-trailer-4200psi-390cc-4-0gpm-gas-pressure-washer-honda-gx390-engine-elec--start>: HTTP status code is not handled or not allowed 2026-01-24 22:33:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hyperpure-pe-rt-tubing-1id-x-1-18od-x-100-coil-red>: HTTP status code is not handled or not allowed 2026-01-24 22:33:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-312-ductile-iron-check-valve-with-epdm-seats-aluminum-bronze-disc-10-wafer>: HTTP status code is not handled or not allowed 2026-01-24 22:33:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/refrigeration-duty-motor-x525-2hp-1-1-2hp-1200rpm-208-230-460v-3ph-56hz-opao>: HTTP status code is not handled or not allowed 2026-01-24 22:33:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/refrigeration-duty-motor-x528-1hp-3-4hp-900rpm-208-230-460v-3ph-56y-opao>: HTTP status code is not handled or not allowed 2026-01-24 22:33:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/li-ion-battery-for-r2-u-r2-v-r2-ua-r2-va-portable-radios-2300-mah>: HTTP status code is not handled or not allowed 2026-01-24 22:33:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sunnyway-swe12140-sealed-lead-acid-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 22:33:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hvac-motor-g141-056t17d5346-3-4hp-1800rpm-208-230-460v-3-ph-56-fr>: HTTP status code is not handled or not allowed 2026-01-24 22:33:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-pipe-fitting-rail-flange-1-dia>: HTTP status code is not handled or not allowed 2026-01-24 22:33:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-pipe-fitting-two-socket-cross-1-1-4-dia>: HTTP status code is not handled or not allowed 2026-01-24 22:33:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-pipe-fitting-crossover-1-1-2-dia>: HTTP status code is not handled or not allowed 2026-01-24 22:33:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fullriver-hghl1235w-sealed-lead-acid-replacement-battery-9ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 22:33:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60455033252-stikitgold-disc-roll-df-6dia-p220-grit>: HTTP status code is not handled or not allowed 2026-01-24 22:34:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/skytron-3500b-table-sealed-lead-acid-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:34:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rockford-rlsw-70-3-lint-separators-25-l-x-25-w-x-20-h-steel>: HTTP status code is not handled or not allowed 2026-01-24 22:34:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coopower-cp12-4-5-sealed-lead-acid-replacement-battery-4-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:34:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mighty-max-ml18-12-sealed-lead-acid-replacement-battery-18ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:34:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-nexelon-blue-wire-shelving-unit-add-on-42w-x-14d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 22:34:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/volcano-kb12240-nb-sealed-lead-acid-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:34:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v2048-2-switch-receptacle-box-2-gang-125v-15a-4-34l>: HTTP status code is not handled or not allowed 2026-01-24 22:34:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/skytron-6500-elite-sealed-lead-acid-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:34:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-shelf-nexelon-blue-wire-shelving-unit-add-on-42w-x-18d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 22:34:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-shelf-nexelon-blue-wire-shelving-unit-add-on-36w-x-21d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 22:34:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-shelf-nexelon-blue-wire-shelving-unit-add-on-60w-x-21d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 22:34:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 31234 pages (at 96 pages/min), scraped 14617 items (at 24 items/min) 2026-01-24 22:34:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ul215bc-multi-outlet-power-unit-125v-15a-13l-4-outlets-6-cord>: HTTP status code is not handled or not allowed 2026-01-24 22:34:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ulm4-15-multi-outlet-power-unit-10-34l-15-cord>: HTTP status code is not handled or not allowed 2026-01-24 22:34:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/firstpower-lfp1280a-sealed-lead-acid-replacement-battery-75ah-12v-it>: HTTP status code is not handled or not allowed 2026-01-24 22:34:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/duracell-dura12-7f2-sealed-lead-acid-replacement-battery-7ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 22:34:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/designer-pvc-ceiling-tile-75600-waterproof-washable-border-fill-2l-x-2w-white>: HTTP status code is not handled or not allowed 2026-01-24 22:34:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unicell-tla12180-nb-sealed-lead-acid-replacement-battery-9ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 22:34:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-top-hanger-10900-white>: HTTP status code is not handled or not allowed 2026-01-24 22:34:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-kingdom-pk40-12-sealed-lead-acid-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:34:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-cal-neoprene-commercial-grade-60a-rubber-sheet-3-16-thick-x-36-wx-6-l>: HTTP status code is not handled or not allowed 2026-01-24 22:34:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-cal-neoprene-commercial-grade-60a-rubber-sheet-1-16-thick-x-36-wx-14-l>: HTTP status code is not handled or not allowed 2026-01-24 22:34:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-nexelon-blue-wire-shelving-unit-starter-48w-x-24d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 22:34:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-sonic-ps-6100f1-sealed-lead-acid-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:34:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-shelf-nexelon-blue-wire-shelving-unit-starter-54w-x-18d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 22:34:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/firstpower-fp12280l-sealed-lead-acid-replacement-battery-26ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:34:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-black-epoxy-wire-shelving-unit-starter-72w-x-21d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 22:34:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pp-heavyweight-55-gal-cylindrical-tank-w-cover-22dia-x-36h-1-4wall-white>: HTTP status code is not handled or not allowed 2026-01-24 22:34:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-shelf-nexelate-silver-epoxy-wire-shelving-unit-add-on-24w-x-14d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 22:34:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-patrol-sla2608-sealed-lead-acid-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:34:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-gallon-1-door-manual-right-hinge-fume-hood-acid-cabinet-24x21-5-8x35-3-4-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:34:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-shelf-nexelate-silver-epoxy-wire-shelving-unit-add-on-36w-x-36d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 22:34:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ironguard-laminated-dock-bumper-24-3488-16-w-x-9-d-x-10-h>: HTTP status code is not handled or not allowed 2026-01-24 22:34:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-1-2-gallon-funnel-for-non-flammables>: HTTP status code is not handled or not allowed 2026-01-24 22:34:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brass-horizontal-safety-drum-vent-for-petroleum-based-applications>: HTTP status code is not handled or not allowed 2026-01-24 22:34:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-nexelate-silver-epoxy-wire-shelving-unit-add-on-60w-x-24d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 22:34:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pride-mobility-go-chair-wheelchair-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 22:34:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/invacare-pte-14-inch-or-less-wheelchair-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:34:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-55-gal-drum-manual-flammable-cabinet-incl-drum-support-34w-x-34d-x-65h-yellow>: HTTP status code is not handled or not allowed 2026-01-24 22:34:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-nexelate-silver-epoxy-wire-shelving-unit-starter-54w-x-24d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 22:34:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ibc-outdoor-shed-w-pallet>: HTTP status code is not handled or not allowed 2026-01-24 22:34:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-nexelate-silver-epoxy-wire-shelving-unit-starter-24w-x-18d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 22:34:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-55-gal-drum-2-door-manual-acid-drum-cabinet-w-drum-rollers-43w-x-34d-x-65h-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:34:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drive-medical-gladiator-wheelchair-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:34:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-nexelate-silver-epoxy-wire-shelving-unit-starter-48w-x-14d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 22:34:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fallte-8240-elastech-4-1-2-to-6-shock-absorbing-lanyard-with-2-snap-hooks>: HTTP status code is not handled or not allowed 2026-01-24 22:34:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hose-assembly-98398244-100r2at-cpld-4000-psi-38-mnpt-38-hose-id-x-72l>: HTTP status code is not handled or not allowed 2026-01-24 22:34:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-nexelate-silver-epoxy-wire-shelving-unit-starter-36w-x-30d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 22:34:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adapter-39004276-38-male-pipe-x-38-female-pipe-swivel-132-restricted>: HTTP status code is not handled or not allowed 2026-01-24 22:34:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/amigo-mobility-fiesta-se-wheelchair-replacement-battery-18ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:34:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-d-e-cylinder-vertical-medical-fire-lined-gas-cabinet-43w-x-18d-x-46h>: HTTP status code is not handled or not allowed 2026-01-24 22:34:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-d-e-cylinder-vertical-medical-gas-cabinet-14w-x-9d-x-44h>: HTTP status code is not handled or not allowed 2026-01-24 22:34:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quantum-rehab-s646-power-wheelchair-replacement-battery-7ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 22:34:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/invacare-lynx-sx-3p-patriot-wheelchair-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:34:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/w-01233-1-2-in-x-1-4-in-wrot-copper-female-adapter-copper-x-fpt>: HTTP status code is not handled or not allowed 2026-01-24 22:34:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-filter-for-all-cleantecpaper-shredders>: HTTP status code is not handled or not allowed 2026-01-24 22:34:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stand-for-848>: HTTP status code is not handled or not allowed 2026-01-24 22:34:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-poly-z-brite-wire-shelving-unit-starter-36w-x-14d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 22:34:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-inlet-blower-wheel-5-3-4-inch-dia-ccw-3450-rpm-1-2-inch-bore-8-1-2-inch-w-aluminium>: HTTP status code is not handled or not allowed 2026-01-24 22:34:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-curbers-s700-b-lawn-and-garden-replacement-battery-7ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 22:34:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acl-f-60x24-combination-louver-damper-with-flange-60wx24h>: HTTP status code is not handled or not allowed 2026-01-24 22:34:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-inlet-blower-wheel-12-5-8-inch-dia-cw-or-ccw-6-inch-w-1-inch-bore>: HTTP status code is not handled or not allowed 2026-01-24 22:34:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-poly-z-brite-wire-shelving-unit-add-on-54w-x-14d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 22:34:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-magnetic-clips-for-psz-slm-portable-safety-zone-4-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:34:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-inlet-blower-wheel-7-1-2-inch-dia-cw-2500-rpm-1-4-inch-bore-5-inch-w-aluminium>: HTTP status code is not handled or not allowed 2026-01-24 22:34:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-shipping-poly-container-mte-2h5hl-with-lid-and-casters-41-l-x-28-1-4-w-x-32-1-2-h-yellow>: HTTP status code is not handled or not allowed 2026-01-24 22:34:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-stainless-steel-solid-shelving-unit-starter-48w-x-18d-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 22:34:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/technacell-tc12240-emergency-light-replacement-battery-26ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:34:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-inlet-blower-wheel-5-3-4-inch-dia-cw-3450-rpm-1-2-inch-bore-5-7-8-inch-w-aluminium>: HTTP status code is not handled or not allowed 2026-01-24 22:34:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vinyl-numbers-letters-2h-white>: HTTP status code is not handled or not allowed 2026-01-24 22:34:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/astralite-20-0002-emergency-light-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:34:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergi-lite-8800006-emergency-light-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:34:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ma-020-22x-handheld-led-light-magnifier>: HTTP status code is not handled or not allowed 2026-01-24 22:34:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-inlet-blower-wheel-7-3-32-inch-dia-ccw-2500-rpm-1-2-inch-bore-3-5-32-inch-w-galvanized>: HTTP status code is not handled or not allowed 2026-01-24 22:34:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portalac-lac-cf18-emergency-light-replacement-battery-18ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:34:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/talkabout-emergency-two-way-radio-22-channels-15-watt-redwhite-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-24 22:34:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/teledyne-h2lt6s20-emergency-light-replacement-battery-26ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:34:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torque-multiplier-pipe-reaction-anchor-34-drive-1000-ftlb-2-716l>: HTTP status code is not handled or not allowed 2026-01-24 22:34:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/902-332-lunar-crimper-for-insulated-flag-terminals-awg-12-10-and-14-16>: HTTP status code is not handled or not allowed 2026-01-24 22:35:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/light-weight-bottles-and-containers-accessory-for-lab01>: HTTP status code is not handled or not allowed 2026-01-24 22:35:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/p-touch-tc-tape-cartridges-1-2w-black-on-clear-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:35:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rotor-adapter-plugs-19300218-12pcs-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:35:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portalac-lac-pe6-5f1-emergency-light-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:35:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hubbell-12-830-12-832-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:35:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/continuous-length-shipping-label-tape-for-ql-1050-4in-x-100ft-roll-white>: HTTP status code is not handled or not allowed 2026-01-24 22:35:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portalac-lac-pe72f1-emergency-light-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:35:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-add-on-48-in-w-x-24-in-d-x-84-in-h-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 22:35:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-wide-boltless-shelving-add-on-60-in-w-x-48-in-d-x-60-in-h-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 22:35:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-extra-hd-boltless-shelving-add-on-48-in-w-x-18-in-d-x-60-in-h-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 22:35:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-high-cap-boltless-shelving-add-on-72-in-w-x-24-in-d-x-96-in-h-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 22:35:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-flange-shelf-18-inch-l-x-42-inch-w-20-gauge-300-lb-capacity-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:35:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 31332 pages (at 98 pages/min), scraped 14636 items (at 19 items/min) 2026-01-24 22:35:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-high-cap-boltless-shelving-steel-deck-add-on-36-in-w-x-24-in-d-x-96-in-h>: HTTP status code is not handled or not allowed 2026-01-24 22:35:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-high-cap-boltless-shelving-steel-deck-add-on-48-in-w-x-18-in-d-x-84-in-h>: HTTP status code is not handled or not allowed 2026-01-24 22:35:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-yellow-chart-tape>: HTTP status code is not handled or not allowed 2026-01-24 22:35:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-wide-boltless-shelving-add-on-48-in-w-x-36-in-d-x-60-in-h>: HTTP status code is not handled or not allowed 2026-01-24 22:35:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rc3shtcbs-poke-thru-non-metallic-slide-holder-assembly-brass-colored>: HTTP status code is not handled or not allowed 2026-01-24 22:35:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/818tcal-floor-box-1-gang-combination-carpet-tile-flange-aluminum>: HTTP status code is not handled or not allowed 2026-01-24 22:35:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trio-lighting-tl930096-emergency-light-replacement-battery-9ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 22:35:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-extra-heavy-duty-boltless-shelving-add-on-48-in-w-x-12-in-d-x-96-in-h>: HTTP status code is not handled or not allowed 2026-01-24 22:35:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/national-power-gt024p6-emergency-light-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:35:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speedcell-kit-with-small-columns-for-96w-x-42d-bay-white>: HTTP status code is not handled or not allowed 2026-01-24 22:35:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stl-3600-1-undermount-stainless-steel-large-single-bowl-kitchen-sink>: HTTP status code is not handled or not allowed 2026-01-24 22:35:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simplex-18ah-retrofit-emergency-light-replacement-battery-18ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:35:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sonnenschein-153302003-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:35:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speedcell-ready-bay-pick-rack-starter-medium-96w-x-42d-x-96h-bluewhite>: HTTP status code is not handled or not allowed 2026-01-24 22:35:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ballistic-blanket-level-iii-a-jumbo-48l-x-72w-black>: HTTP status code is not handled or not allowed 2026-01-24 22:35:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bomb-suppression-blanket-jumbo-63l-x-63w-black>: HTTP status code is not handled or not allowed 2026-01-24 22:35:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chloride-power-1001145-emergency-light-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:35:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/surface-bumper-guard-type-c-164-1-916-black-yellow-60-6720>: HTTP status code is not handled or not allowed 2026-01-24 22:35:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sonnenschein-ps12260-emergency-light-replacement-battery-26ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:35:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/concrete-reinforcing-chairs3-mesh-up-for-0-gauge-wire100-box-qty>: HTTP status code is not handled or not allowed 2026-01-24 22:35:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elite-bookcase-assembled-36w-x-18d-x-36h-putty>: HTTP status code is not handled or not allowed 2026-01-24 22:35:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/99829-10-pc-set-double-hookangled-end2>: HTTP status code is not handled or not allowed 2026-01-24 22:35:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/playmate-volley-tennis-ball-machine-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:35:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sonnenschein-a412-12-sr-emergency-light-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:35:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ps-80-f-cloud-gray-privacy-screen>: HTTP status code is not handled or not allowed 2026-01-24 22:35:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ibc-tank-overfill-alert-gauge-aluminum-bushing-galvanized-rod-pp-float-1-1-2-fitting-8-depth>: HTTP status code is not handled or not allowed 2026-01-24 22:35:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/three-shelf-wide-boltless-shelving-add-on-72inw-x-48ind-x-96inh-wood-deck-usa>: HTTP status code is not handled or not allowed 2026-01-24 22:35:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drum-overfill-alert-gauge-stainless-bushing-rod-float-2-fitting-6-depth>: HTTP status code is not handled or not allowed 2026-01-24 22:35:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/four-shelf-extra-hd-boltless-shelving-add-on-48inw-x-24ind-x-60inh-wood-deck-usa>: HTTP status code is not handled or not allowed 2026-01-24 22:35:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/awsw-partition-80x7-f-latte>: HTTP status code is not handled or not allowed 2026-01-24 22:35:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ion-audio-pathfinder-2-red-speaker-replacement-battery-7ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 22:35:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ps-70-pc-blue-privacy-screen>: HTTP status code is not handled or not allowed 2026-01-24 22:35:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/peg-perego-four-runner-ride-on-toys-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 22:35:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/five-shelf-heavy-duty-boltless-shelving-add-on-36inw-x-24ind-x-60inh-wire-deck-usa>: HTTP status code is not handled or not allowed 2026-01-24 22:35:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/five-shelf-extra-hd-boltless-shelving-add-on-36in-w-x-18in-d-x-84in-h-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 22:35:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smoking-shelter-52wsvrdkb-4sided-l-r-open-fr-12l-x-5w-vented-standing-seam-roof-dk-brz>: HTTP status code is not handled or not allowed 2026-01-24 22:35:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/awsw-partition-80x7-f-sand>: HTTP status code is not handled or not allowed 2026-01-24 22:35:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/can-am-outlander-800-max-800cc-atv-replacement-battery-2006-2013-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:35:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/can-am-outlander-max-500-500cc-atv-replacement-battery-2007-2013-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:35:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/peg-perego-john-deere-e-tractor-iged1061-ride-on-toys-battery-35ah-12v>: HTTP status code is not handled or not allowed 2026-01-24 22:35:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/four-shelf-heavy-duty-boltless-shelving-add-on-48in-w-x-12in-d-x-60in-h>: HTTP status code is not handled or not allowed 2026-01-24 22:35:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smoking-shelter-64wsvrdkb-3sided-l-r-open-fr-15l-x-10w-vented-standing-seam-roof-dk-brz>: HTTP status code is not handled or not allowed 2026-01-24 22:35:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/match-mate-p1-value-tennis-ball-machine-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 22:35:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cylinder-bottle-lifter-16307-2-cylinders-26-w-x-64-1-2-h-750-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:35:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motion-madness-mallard-hen-peck-and-sweep-feeder-decoy-battery-7ah-12v>: HTTP status code is not handled or not allowed 2026-01-24 22:35:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd18-2-two-18-skates>: HTTP status code is not handled or not allowed 2026-01-24 22:35:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/truck-cargo-net-grommet-fastening-system-55-inch-x-55-inch-ttgs>: HTTP status code is not handled or not allowed 2026-01-24 22:35:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd30-2-two-30-skates>: HTTP status code is not handled or not allowed 2026-01-24 22:35:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd21-3-three-21-skates-wing-kit>: HTTP status code is not handled or not allowed 2026-01-24 22:35:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26256-radial-flow-gac-8-gpm-micron>: HTTP status code is not handled or not allowed 2026-01-24 22:35:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mt244830-2k195-machine-table-with-one-shelf-48w-x-24d-x-30h>: HTTP status code is not handled or not allowed 2026-01-24 22:35:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hercules-sachs-gipsy-50cc-motorcycle-replacement-battery-1995-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:35:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/imported-swivel-angle-plate-10-inch-x-6-1-2-inch-x-8-inch>: HTTP status code is not handled or not allowed 2026-01-24 22:35:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m8-metric-eyebolt-shoulder-pattern-m8-x-125>: HTTP status code is not handled or not allowed 2026-01-24 22:35:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/k2025-316ss-316-stainless-eyebolt-shoulder-pattern-1-2-13-1-1-2-shank>: HTTP status code is not handled or not allowed 2026-01-24 22:35:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aprilia-shiver-900-900cc-motorcycle-replacement-battery-2017-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:35:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/915-case-w-foam-15-3-8l-x-12-1-8w-x-6-13-16h-graphite>: HTTP status code is not handled or not allowed 2026-01-24 22:35:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multithotic-orthotic-anti-fatigue-insole-womens-567>: HTTP status code is not handled or not allowed 2026-01-24 22:35:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bridgestone-350-gto-350cc-motorcycle-replacement-battery-1969-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:35:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yamaha-yfm400fwn-big-bear-400cc-atv-replacement-battery-2000-2003-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:35:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ironhorse-outlaw-1820cc-motorcycle-replacement-battery-2000-2007-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:35:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pam-anti-fatigue-insole-mens-1415>: HTTP status code is not handled or not allowed 2026-01-24 22:35:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-electric-heater-catchers-mask-1500w-3-speed-with-infrared-convection-dual-heat>: HTTP status code is not handled or not allowed 2026-01-24 22:35:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/905-case-w-padded-divider-12-1-2l-x-10w-x-6h-silver>: HTTP status code is not handled or not allowed 2026-01-24 22:35:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-gauge-mobile-bench-cabinet-2211a-lu-95-4-drawers>: HTTP status code is not handled or not allowed 2026-01-24 22:35:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cagiva-elefant-2-cyl-350cc-motorcycle-replacement-battery-1986-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:35:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stationary-enclosed-shop-desk-with-locking-door--drawer-24w-x-22d-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:35:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/945-case-w-foam-25-1-8l-x-19-7-8w-x-8-13-16h-yellow>: HTTP status code is not handled or not allowed 2026-01-24 22:36:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-steel-mobile-storage-bin-msb6-2036-95-6-openings-36w-x-24d-x-49-1-4h>: HTTP status code is not handled or not allowed 2026-01-24 22:36:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/161791-bi-pass-closet-track-kit-48-2-door-hardware-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:36:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pic-a-watt-unit-heater-kbp1230-2850w-max-120v-1-phase-pearl-white>: HTTP status code is not handled or not allowed 2026-01-24 22:36:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harley-davidson-fxr4-1450cc-motorcycle-replacement-battery-2000-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:36:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd-welded-workbench-48-x-24-adj-height-louvered-panel-steel-square-edge>: HTTP status code is not handled or not allowed 2026-01-24 22:36:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bsm-inc-ada-ss-hand-sink-1-station-with-lever-faucet-wall-mount-20l-x-19w-x-5-12d-sink>: HTTP status code is not handled or not allowed 2026-01-24 22:36:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harley-davidson-fat-boy-flstfi-1450cc-motorcycle-battery-2001-2006-12v>: HTTP status code is not handled or not allowed 2026-01-24 22:36:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/183070003-lead-ring-vinyl-coated-round-36lb-fits-5-10ml-cylinders-red>: HTTP status code is not handled or not allowed 2026-01-24 22:36:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/indian-roadmaster-1819cc-motorcycle-replacement-battery-2014-2017-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:36:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-cabinet-flush-door-with-185-yellow-bins-16-ga-all-welded-cabinet-60-w-x-24-d-x-84-h-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:36:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-cabinet-deep-door-with-132-yellow-bins-16-ga-all-welded-cabinet-36-w-x-24-d-x-72-h-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:36:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bsm-inc-ada-ss-sink-4-station-with-electronic-faucet-round-tubes-132l-x-20w-x-6d-sink>: HTTP status code is not handled or not allowed 2026-01-24 22:36:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 31434 pages (at 102 pages/min), scraped 14658 items (at 22 items/min) 2026-01-24 22:36:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/371110002-round-spinbar-magnetic-stir-bar-teflon-ptfe-2l-x-5-16dia-white>: HTTP status code is not handled or not allowed 2026-01-24 22:36:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bsm-inc-ada-ss-sink-4-station-with-electronic-faucet-wall-mount-96l-x-20w-x-6d-sink>: HTTP status code is not handled or not allowed 2026-01-24 22:36:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gssafe-5008-class-3-hi-viz-moisture-wicking-birdeyes-short-sleeve-t-shirt-orange-3xl>: HTTP status code is not handled or not allowed 2026-01-24 22:36:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gssafe-5113-class-3-microfiber-birdseye-short-sleeve-t-shrit-w-black-bottom-lime-3xl>: HTTP status code is not handled or not allowed 2026-01-24 22:36:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gssafe-2702-class-3-heavy-duty-safety-vest-orange-3xl>: HTTP status code is not handled or not allowed 2026-01-24 22:36:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/summit-ct661bisshhada-ada-compliant-built-in-undercounter-refrigerator-freezer-51-cu-ft-24w>: HTTP status code is not handled or not allowed 2026-01-24 22:36:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/summit-ct663bbiifada-ada-compliant-built-in-undercounter-refrigerator-freezer-51-cu-ft-24w>: HTTP status code is not handled or not allowed 2026-01-24 22:36:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-c-emergency-collet-steel>: HTTP status code is not handled or not allowed 2026-01-24 22:36:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-reclosable-sandwich-bags-in-dispenser-box-1-mil-6-12-6>: HTTP status code is not handled or not allowed 2026-01-24 22:36:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/navigator-5153-expandable-insulated-overboots-threaded-outsole-xs-15h-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:36:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/voyager-nylon-all-season-waterproof-overboots-threaded-outsole-m-11h-black>: HTTP status code is not handled or not allowed 2026-01-24 22:36:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/workbench-36-x-24-louvered-panel--3-shelves-steel-square-edge>: HTTP status code is not handled or not allowed 2026-01-24 22:36:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/base-cover-lpgred-26dia-x-41-1-4h-4-rings-yellow>: HTTP status code is not handled or not allowed 2026-01-24 22:36:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-stp-stl-all-direct-safet-roll-lad-wt-actu-24-perf-stp-wa-ad-093214p>: HTTP status code is not handled or not allowed 2026-01-24 22:36:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/piaggio-vespa-free-25cc-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:36:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelate-48w-x-18d-x-69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 22:36:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/razor-pocketrocket-electric-scooter-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 22:36:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clearance-bar-htgrd796yw-7dia-x-96l-yellow-w-white-tape-graphics>: HTTP status code is not handled or not allowed 2026-01-24 22:36:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelate-60w-x-14d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:36:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motofino-mf150qt-21-150cc-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:36:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jenoptik-125-zip-4t-125cc-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:36:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelate-60w-x-30d-x-80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 22:36:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelate-36w-x-21d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:36:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelate-42w-x-21d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:36:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vestil-fully-powered-counter-balanced-stacker-s-cb-62-1000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:36:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/big-reach-drawer-for-beverage-case-container-ul-app-bullet-resistant-faceplate>: HTTP status code is not handled or not allowed 2026-01-24 22:36:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drinking-cup-w-lanyard-5-oz-clear-1-case-dx1525m>: HTTP status code is not handled or not allowed 2026-01-24 22:36:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/piaggio-vespa-scatto-50cc-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:36:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelate-60w-x-21d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:36:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galvanized-steel-round-vent-144-sq-in-nfa-weathered-gray-1-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:36:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/koala-kare-174-diaper-kit-diaper-dispenser-kb113-99>: HTTP status code is not handled or not allowed 2026-01-24 22:36:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/econo-locking-tie-48>: HTTP status code is not handled or not allowed 2026-01-24 22:36:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/can-am-commander-1000-x-1000cc-utv-replacement-battery-2011-2013-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:36:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-ski-doo-gtx-800-ho-800cc-snowmobile-replacement-battery-2006-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:36:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-mx-z-1000-sdi-renegade-x-998cc-snowmobile-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:36:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polaris-iq-cruiser-euro-750cc-snowmobile-replacement-battery-2008-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:36:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/du2082-cantilever-rack-double-sided-upright-82d-x-20h>: HTTP status code is not handled or not allowed 2026-01-24 22:36:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/can-am-commander-800-800cc-utv-replacement-battery-2011-2016-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:36:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/econo-tag-extra-wide-upclabel-holder-1-14-x-3-38>: HTTP status code is not handled or not allowed 2026-01-24 22:36:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelate-48w-x-24d-x-60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 22:36:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drum-containment-dolly-1617y-70-gallon-cap-10-wheels-polyethylene>: HTTP status code is not handled or not allowed 2026-01-24 22:36:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/signo-gel-207-roller-ball-retractable-gel-pen-blue-ink-medium-36-box>: HTTP status code is not handled or not allowed 2026-01-24 22:36:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beaded-chain-6-ball-12l>: HTTP status code is not handled or not allowed 2026-01-24 22:36:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelate-60w-x-30d-x-69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 22:36:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dover-mailbox-post-5810-b-56h-black>: HTTP status code is not handled or not allowed 2026-01-24 22:36:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grip-tite-wire-basket-sign-holder-1>: HTTP status code is not handled or not allowed 2026-01-24 22:36:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelate-48w-x-24d-x-80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 22:36:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polaris-iq-turbo-750cc-snowmobile-replacement-battery-2011-2012-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:36:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-kit-for-manual-pallet-jack-gwk-cfp-lw-fits-lift-rite-model-cf>: HTTP status code is not handled or not allowed 2026-01-24 22:36:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/complete-wheel-kit-for-manual-pallet-jack-gwk-btu-ck-fits-bt-model-l-2000-u-l-2300-u>: HTTP status code is not handled or not allowed 2026-01-24 22:36:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/countertop-sign-holder-3l>: HTTP status code is not handled or not allowed 2026-01-24 22:36:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-piercing-flush-to-channel-sign-holder>: HTTP status code is not handled or not allowed 2026-01-24 22:36:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clip-strip-12-station-wtape-header-34-14l-white>: HTTP status code is not handled or not allowed 2026-01-24 22:36:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/import-hss-hex-re-threading-die-m20x2-5-carbon-steel>: HTTP status code is not handled or not allowed 2026-01-24 22:36:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-16028-nipple-34-body-size-34-female-sae-111612-unf-oring-boss-orb>: HTTP status code is not handled or not allowed 2026-01-24 22:36:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-storage-cabinet-7218dlx-03-welded-36w-x-18d-x-72h-black>: HTTP status code is not handled or not allowed 2026-01-24 22:36:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bookcase-storage-cabinet-bcd18-72-216-welded-36w-x-18d-x-72h-champagne-putty>: HTTP status code is not handled or not allowed 2026-01-24 22:36:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-nexelate-60w-x-14d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:36:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bombardier-tundra-300f-300cc-snowmobile-replacement-battery-2006-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 22:36:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-black-epoxy-36w-x-18d-x-60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 22:37:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wesco-174-long-deck-mobile-scissor-li-273230-with-oversized-80-x-30-table-top-2200-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:37:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ba-fans-aireye-directional-fan-with-swivel-mount-24-inch>: HTTP status code is not handled or not allowed 2026-01-24 22:37:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-frame-sheet-panel-truck-48-x-24-2000-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-24 22:37:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-3l200-v-belt-3-8-x-20>: HTTP status code is not handled or not allowed 2026-01-24 22:37:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-a75-4l770-v-belt-1-2-x-77>: HTTP status code is not handled or not allowed 2026-01-24 22:37:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interior-mattes-with-beveled-edges-1-inch-x-3-inch-1-16-inch-100-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:37:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-black-epoxy-36w-x-21d-x-60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 22:37:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-a55-4l570-v-belt-1-2-x-57>: HTTP status code is not handled or not allowed 2026-01-24 22:37:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-a146-v-belt-1-2-x-148>: HTTP status code is not handled or not allowed 2026-01-24 22:37:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/push-top-receptacle-36-gallon-9895>: HTTP status code is not handled or not allowed 2026-01-24 22:37:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 31530 pages (at 96 pages/min), scraped 14684 items (at 26 items/min) 2026-01-24 22:37:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24ft-fiberglass-aero-box-rail-extension-ladder-iaa-b7124-2>: HTTP status code is not handled or not allowed 2026-01-24 22:37:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-poly-green-42w-x-18d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:37:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-b42-5l450-v-belt-5-8-x-45>: HTTP status code is not handled or not allowed 2026-01-24 22:37:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-b138-v-belt-5-8-x-141>: HTTP status code is not handled or not allowed 2026-01-24 22:37:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-poly-green-54w-x-14d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:37:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8249z-s-hi-vis-surveyors-vest-class-2-economy-single-size-xl-orange>: HTTP status code is not handled or not allowed 2026-01-24 22:37:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-black-epoxy-60w-x-24d-x-69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 22:37:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8246z-s-two-tone-mesh-hi-vis-safety-vest-class-2-single-size-m-orange>: HTTP status code is not handled or not allowed 2026-01-24 22:37:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-lockout-padlock-keyed-different-1-1-2-plastic-steel-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:37:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/locks_lockout_devices/lockout_tagout_devices/lockout_padlocks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:37:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pump-motor-jl1301a-1-phase-033-hp-115-230-volts-1725-rpm-60-hz-open-56j>: HTTP status code is not handled or not allowed 2026-01-24 22:37:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-lockout-padlock-keyed-different-1-1-2-aluminum-steel-orange-6-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:37:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pump-motor-ejmm2334t-g-3-phase-20-hp-230-460-volts-1800-rpm-60-hz-tefc-256jm>: HTTP status code is not handled or not allowed 2026-01-24 22:37:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esd-3-8-thick-anti-fatigue-floor-mat-4-x-60-full-roll-brown-fm248720bn>: HTTP status code is not handled or not allowed 2026-01-24 22:37:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-8482-28-1-8-qt-plastic-recycling-wastebasket-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:37:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-motor-208-230-460-v-10-hp-1770-rpm-3-ph-215tc-opsb>: HTTP status code is not handled or not allowed 2026-01-24 22:37:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-motor-208-230-460-v-75-hp-1770-rpm-3-ph-213tc-tefc-B909050>: HTTP status code is not handled or not allowed 2026-01-24 22:37:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelite-54w-x-18d-x-80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 22:37:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/as-40x-1000x-monocular-student-microscope-with-coarse-focus-10-pc-slide-set-experiment-cards>: HTTP status code is not handled or not allowed 2026-01-24 22:37:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3817-glass-stacking-belgian-beer-10-oz-12-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:37:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/locks_lockout_devices/lockout_tagout_devices/lockout_padlocks>: HTTP status code is not handled or not allowed 2026-01-24 22:37:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/as-iqcrew-40x-1000x-dual-illumination-microscope-3mp-digital-eyepiece-slide-prep-book-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:37:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelite-60w-x-24d-x-80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 22:37:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-phase-motor-em4103t-5-1770-hp-25-rpm-284t-frame-c-face-mount-tefc-575-volts>: HTTP status code is not handled or not allowed 2026-01-24 22:37:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-best-value-wire-shelving-unit-36-w-x-24-d-x-74-h-400-lb-shelf-cap-zinc-chromate>: HTTP status code is not handled or not allowed 2026-01-24 22:37:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usb-rechargeable-tactical-flashlight-w-holster-900-lumens-green>: HTTP status code is not handled or not allowed 2026-01-24 22:37:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelite-42w-x-24d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:37:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evolution-deluxe-6151-cap-style-hard-hat-hdpe-shell-6-pt-suspension-ratchet-adj-neon-orange>: HTTP status code is not handled or not allowed 2026-01-24 22:37:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ken-forging-tb101-e-e-turnbuckle-assembly-eye-eye-1-4-20-x-10-carbon>: HTTP status code is not handled or not allowed 2026-01-24 22:37:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dynamic-dom-cap-style-dome-hard-hat-hdpe-shell-4-pt-suspension-rachet-adjustment-sky-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:37:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-motor-230-460-v-60-hp-1775-rpm-3-ph-364tc-opsb>: HTTP status code is not handled or not allowed 2026-01-24 22:37:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stromboli-cap-style-dome-hard-hat-abs-polycarbonate-shell-4-pt-suspension-rachet-adjustment-ylw>: HTTP status code is not handled or not allowed 2026-01-24 22:37:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-phase-motor-cl3612tm-1-ph-230-v-5-hp-1725-rpm-tefc-184tc-frame>: HTTP status code is not handled or not allowed 2026-01-24 22:37:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washdown-motor-idwnm22937t-3-phase-75-hp-1765-rpm-230-460-volts-tenv-254tc-fr>: HTTP status code is not handled or not allowed 2026-01-24 22:37:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/en1030-3bb-window-entry-awning-3-3-8-w-x-1-5-16-h-x-2-1-2-d-bright-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:37:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-motor-208-230-460-v-2-hp-3490-rpm-3-ph-145tc-tefc>: HTTP status code is not handled or not allowed 2026-01-24 22:37:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evolution-deluxe-6151-cap-style-hard-hat-hdpe-shell-6-pt-polyester-suspension-ratchet-adj-white>: HTTP status code is not handled or not allowed 2026-01-24 22:37:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/severe-duty-motor-cecp83580t-4-3-ph-1-hp-460-v-3450-rpm-tefc-143tc-frame>: HTTP status code is not handled or not allowed 2026-01-24 22:37:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-poly-green-30w-x-24d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:37:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dlr1060-dlr-series-1-watt-6-channel-digital-two-way-radio>: HTTP status code is not handled or not allowed 2026-01-24 22:37:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-poly-green-72wx21dx60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 22:37:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-poly-green-36w-x-21d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:37:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-poly-green-24w-x-21d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:37:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hardcap-a1-baseball-style-bump-cap-hdpe-protective-liner-w-adjustable-back-micro-brim-navy>: HTTP status code is not handled or not allowed 2026-01-24 22:37:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/light-duty-steel-wheel-875-8x175-2-1-2-centered-hub-1-2bb-diamond-tread>: HTTP status code is not handled or not allowed 2026-01-24 22:37:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-poly-green-54w-x-21d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:37:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8000240b-4-peg-hook-ball-tip-bright-silver-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 22:37:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pump-motor-vhecp3660t-3-phase-3-hp-230-460-volts-3500-rpm-60-hz-tefc-182hp>: HTTP status code is not handled or not allowed 2026-01-24 22:37:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-poly-green-60w-x-14d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:37:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/severe-duty-motor-ecp84115t-4-3-ph-50-hp-460-v-1770-rpm-tefc-326t-frame>: HTTP status code is not handled or not allowed 2026-01-24 22:37:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-weather-steel-diskus-padlock-24ib60-kd-keyed-different-price-per-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:37:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dc-explosion-proof-motor-cdx1850-05-hp-1750-rpm-xpfc-182cz>: HTTP status code is not handled or not allowed 2026-01-24 22:37:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magic-clamp-magnetic-clamp-50ml-erlenmeyer>: HTTP status code is not handled or not allowed 2026-01-24 22:37:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-nexelon-48w-x-18d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:37:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-poly-green-24wx14dx60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 22:37:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelon-54w-x-24d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:37:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/p550-electrical-switch-power-plug-lockout-device-220-550v-00451>: HTTP status code is not handled or not allowed 2026-01-24 22:37:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-poly-green-42w-x-24d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:37:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machined-chrome-steel-needle-roller-bearing-w-inner-ring-30-x-47-x-30mm>: HTTP status code is not handled or not allowed 2026-01-24 22:37:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-poly-green-48wx14dx60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 22:38:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/repair-coupling-drive-end-for-sc-tc-s02-gtc-702-shafts>: HTTP status code is not handled or not allowed 2026-01-24 22:38:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hvac-motor-em2531t-ci-3-ph-25-hp-230-460-v-1770-rpm-open-284t-frame>: HTTP status code is not handled or not allowed 2026-01-24 22:38:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2aa-ssmach-rod-end-machined-304-stainless-5-16-18-x-1-1-2>: HTTP status code is not handled or not allowed 2026-01-24 22:38:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-poly-green-60w-x-24d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:38:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/repair-coupling-brush-end-for-sc-tc-s03-gtc-703-shafts>: HTTP status code is not handled or not allowed 2026-01-24 22:38:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rolling-mortar-tub-w-8-inch-mold-on-rubber-casters-6-6-cu-ft-capacity-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:38:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machined-chrome-steel-needle-roller-bearing-w-inner-ring-55-x-72-x-35mm>: HTTP status code is not handled or not allowed 2026-01-24 22:38:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triangular-brush-attatchment-for-gvc-models-390-1100-1250-1502>: HTTP status code is not handled or not allowed 2026-01-24 22:38:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-round-chrome-cafeteria-table-with-8-stools-gray-mdf-core-top-gray-stools>: HTTP status code is not handled or not allowed 2026-01-24 22:38:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/automotive-duty-motor-aem4104-4-3-ph-460-v-30-hp-1775-rpm-tefc-326u-frame>: HTTP status code is not handled or not allowed 2026-01-24 22:38:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-poly-green-36wx14dx69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 22:38:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-phase-motor-l3516tm-2-hp-208-230-volts-1740-rpm-tefc-56hz-frame>: HTTP status code is not handled or not allowed 2026-01-24 22:38:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-phase-motor-l1410tm-5-hp-230-volts-1725-rpm-open-184t-frame>: HTTP status code is not handled or not allowed 2026-01-24 22:38:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 31625 pages (at 95 pages/min), scraped 14707 items (at 23 items/min) 2026-01-24 22:38:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-w-x-30-d-plastic-laminate-square-edge-workbench-adjustable-height-black>: HTTP status code is not handled or not allowed 2026-01-24 22:38:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-poly-green-60w-x-14d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:38:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machined-chrome-steel-needle-roller-bearing-w-inner-ring-12-x-24-x-22mm>: HTTP status code is not handled or not allowed 2026-01-24 22:38:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machined-chrome-steel-needle-roller-bearing-w-inner-ring-15-x-27-x-20mm>: HTTP status code is not handled or not allowed 2026-01-24 22:38:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wd-industrial-steel-mezzanine-21-7w-x-31-5d-x-8h-clearance>: HTTP status code is not handled or not allowed 2026-01-24 22:38:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-phase-motor-el1409t-5-hp-230-volts-3450-rpm-open-184t-frame>: HTTP status code is not handled or not allowed 2026-01-24 22:38:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machined-chrome-steel-needle-roller-bearing-w-inner-ring-32-x-47-x-20mm>: HTTP status code is not handled or not allowed 2026-01-24 22:38:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-poly-green-72w-x-18d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:38:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machined-chrome-steel-needle-roller-bearing-w-inner-ring-38-x-53-x-20mm>: HTTP status code is not handled or not allowed 2026-01-24 22:38:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machined-chrome-steel-needle-roller-bearing-w-inner-ring-45-x-68-x-40mm>: HTTP status code is not handled or not allowed 2026-01-24 22:38:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-poly-green-30wx21dx80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 22:38:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-poly-green-54wx21dx69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 22:38:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-poly-green-30w-x-21d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:38:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/outdoor-deluxe-slatted-steel-trash-can-with-access-door-36-gal-42-inch-h-black>: HTTP status code is not handled or not allowed 2026-01-24 22:38:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scotch-cushion-lock-protective-wrap-12-x-1000-pcw-121000>: HTTP status code is not handled or not allowed 2026-01-24 22:38:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evap-cooling-bandana-w-cooling-towel-h-l-lime-12576>: HTTP status code is not handled or not allowed 2026-01-24 22:38:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-poly-green-48w-x-18d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:38:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultratech-gloves-latex-cut-a4-impact-1-abrasion-4-puncture-5-gray-blue-m>: HTTP status code is not handled or not allowed 2026-01-24 22:38:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultratech-gloves-13-gauge-nylon-shell-nitrile-palm-fingers-tpr-back-red-black-2xl>: HTTP status code is not handled or not allowed 2026-01-24 22:38:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-poly-green-36w-x-18d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:38:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ninja-bnf-gloves-18-gauge-nylon-shell-nitrile-coated-palm-fingertips-red-black-l>: HTTP status code is not handled or not allowed 2026-01-24 22:38:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelon-54w-x-14d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:38:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelon-30w-x-14d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:38:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/encoder-tachometer-mounting-kits-w-o-tach-417708-41-dpg-dpfv-tenv-frames-c210-c250>: HTTP status code is not handled or not allowed 2026-01-24 22:38:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-acting-hollow-hole-aluminum-hydraulic-cylinder-60-ton-6-stroke>: HTTP status code is not handled or not allowed 2026-01-24 22:38:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-acting-cylinder-base-10-ton>: HTTP status code is not handled or not allowed 2026-01-24 22:38:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-blower-kit-417077-141-for-tebc-frame-fl-rl210-320>: HTTP status code is not handled or not allowed 2026-01-24 22:38:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/awn-en1030-8n-window-entry-awning-8-4-1-2-w-x-2-6-d-x-1-4-h-navy>: HTTP status code is not handled or not allowed 2026-01-24 22:38:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-nexelon-54w-x-24d-x-60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 22:38:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pmln7240ar-whistle-belt-clip-twin-pack-for-t400-series>: HTTP status code is not handled or not allowed 2026-01-24 22:38:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speedclean-indoor-window-cleaning-kit-ck053>: HTTP status code is not handled or not allowed 2026-01-24 22:38:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-microfiber-washing-pad-phw20>: HTTP status code is not handled or not allowed 2026-01-24 22:38:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-nexelon-36w-x-14d-x-60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 22:38:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/main-support-poles-for-cabinet-41800-and-42000>: HTTP status code is not handled or not allowed 2026-01-24 22:38:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ja-x-1-2-1-2-x-2-j-series-quick-detach-steel-bushing-1-2-bore>: HTTP status code is not handled or not allowed 2026-01-24 22:38:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelon-24w-x-18d-x-80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 22:38:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-locator-kit-w-inductive-toner-probe-and-tone-generator>: HTTP status code is not handled or not allowed 2026-01-24 22:38:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8744-ultra-containment-berm-modular-model-12-x-40-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:38:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vortex-tube-3204-small-4-scrm-275-btuhr>: HTTP status code is not handled or not allowed 2026-01-24 22:38:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waring-wdb600-hot-plate-double-burner-commercial-120v>: HTTP status code is not handled or not allowed 2026-01-24 22:38:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/abc3bw-3-lb-non-sparking-brass-hammer-125-wood-handle>: HTTP status code is not handled or not allowed 2026-01-24 22:38:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edge-protector-black-plastic-1-34-x-2-12>: HTTP status code is not handled or not allowed 2026-01-24 22:38:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vertical-kraft-paper-dispenser-36-material-width>: HTTP status code is not handled or not allowed 2026-01-24 22:38:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/38-push-on-seal-for-steel-2500case>: HTTP status code is not handled or not allowed 2026-01-24 22:38:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelon-54w-x-21d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:38:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waring-fp2200-food-processor-3-4-hp-6-quarts>: HTTP status code is not handled or not allowed 2026-01-24 22:38:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waring-wfp14sc-food-processor-commercial-with-liquid-lock-3-1-2-quart>: HTTP status code is not handled or not allowed 2026-01-24 22:38:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pistol-grip-with-t-handle-stretch-film-dispenser-12-to-20-rolls-8-per-case>: HTTP status code is not handled or not allowed 2026-01-24 22:38:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sump-insert-96-w-x-42-d-pallet-rack>: HTTP status code is not handled or not allowed 2026-01-24 22:38:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-gallon-stonetec-174-72041199-square-receptacle-with-dome-lid-gray-w-ashtone-panels>: HTTP status code is not handled or not allowed 2026-01-24 22:38:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-nexelon-72w-x-24d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 22:38:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spill-containment-shed-4062-bd-2-drum>: HTTP status code is not handled or not allowed 2026-01-24 22:38:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ibc-steel-spill-pallet-9469-bd>: HTTP status code is not handled or not allowed 2026-01-24 22:38:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-nexelon-72w-x-14d-x-69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 22:38:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bolted-teardrop-pallet-rack-starter-108inw-x-42ind-x-192inh-6420-lbs-shelf-cap>: HTTP status code is not handled or not allowed 2026-01-24 22:38:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/celltreat-scientific-products-2l-pipette-ovation-f2-fixed-volume-two-stroke>: HTTP status code is not handled or not allowed 2026-01-24 22:38:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/celltreat-scientific-products-022l-pipette-ovation-quickset-qs-red>: HTTP status code is not handled or not allowed 2026-01-24 22:38:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/knives-6l-recycled-polystyrene-1000-carton-black>: HTTP status code is not handled or not allowed 2026-01-24 22:38:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/we-mc400-16875-vhs-alternate-coupling-bore-size-16875-frame-404tp-or-405tp>: HTTP status code is not handled or not allowed 2026-01-24 22:38:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/urban-6-ft-plastic-picnic-table-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:38:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spoons-heavy-medium-weight-plastic-shatter-resistant-1000-carton-black>: HTTP status code is not handled or not allowed 2026-01-24 22:38:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/we-mc320-14375-vhs-alternate-coupling-bore-size-14375-frame-324tp-or-326tp>: HTTP status code is not handled or not allowed 2026-01-24 22:38:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/we-mc360-125-25-vhs-alternate-coupling-bore-size-125-frame-364tp-or-365tp>: HTTP status code is not handled or not allowed 2026-01-24 22:38:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/celltreat-scientific-products-4ml-pipette-mla-macro-white>: HTTP status code is not handled or not allowed 2026-01-24 22:38:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/celltreat-scientific-products-1010l-pipette-ovation-m-mechanical>: HTTP status code is not handled or not allowed 2026-01-24 22:38:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/celltreat-scientific-products-1000l-pipette-mla-precision-silver>: HTTP status code is not handled or not allowed 2026-01-24 22:39:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/omnicart-cab-30-cap-w-locking-panel-beige-49-3-4w-x-17d-x-43-3-4h>: HTTP status code is not handled or not allowed 2026-01-24 22:39:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jurni-flip-top-training-table-with-casters-60l-x-24w-x-29h-columbian-walnut>: HTTP status code is not handled or not allowed 2026-01-24 22:39:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/recycled-beverage-napkins-quad-fold-9-1-2-x-9-1-4-4000-carton-white>: HTTP status code is not handled or not allowed 2026-01-24 22:39:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n-ferno-6475-cold-storage-thermal-insulated-coverall-2xl-navy>: HTTP status code is not handled or not allowed 2026-01-24 22:39:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rocacrylic-3800-acrylic-enamel-paint-1-gallon-can-gloss-white>: HTTP status code is not handled or not allowed 2026-01-24 22:39:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/squids-3386-arm-band-idbadge-holder-one-size-orange>: HTTP status code is not handled or not allowed 2026-01-24 22:39:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/topoge-series-180-handhole-gasket-234-x-434-x-12-black-rubber-obround-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:39:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mototrbo-cm300d-two-way-mobile-radio-analogdigital-vhf-99-channels-25-watts-black>: HTTP status code is not handled or not allowed 2026-01-24 22:39:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/topoge-series-180-handhole-gasket-3-x-4-x-34-black-rubber-elliptical-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:39:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jurni-multi-purpose-table-with-post-legs--casters-60l-x-24w-x-29h-columbian-walnut>: HTTP status code is not handled or not allowed 2026-01-24 22:39:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 31726 pages (at 101 pages/min), scraped 14731 items (at 24 items/min) 2026-01-24 22:39:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/130g32-0022-f1-gas-engine-900-series-tiller-blower-log-splitter-horizontal-shaft>: HTTP status code is not handled or not allowed 2026-01-24 22:39:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/topoge-series-180-manway-gasket-11-x-15-x-134-black-rubber-elliptical-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:39:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/therm153-replacement-batteries-for-heated-glove-liner-black-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-24 22:39:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/therm153-heated-glove-liner-one-size-black>: HTTP status code is not handled or not allowed 2026-01-24 22:39:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/western-global-fuelcube-type-s-diesel-fuel-storage-tank-w-12v15gpm-pump-kit-119-gal-cap>: HTTP status code is not handled or not allowed 2026-01-24 22:39:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-nut-90-union-elbow-h5506x4-1-4-inch-tube-od>: HTTP status code is not handled or not allowed 2026-01-24 22:39:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thin-wall-collar-kit-9307-for-wall-thickness-1-to-5>: HTTP status code is not handled or not allowed 2026-01-24 22:39:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mechanix-wear-mpact174-gloves-synthetic-leatherd30174-palm-padding-blackgray-medium>: HTTP status code is not handled or not allowed 2026-01-24 22:39:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tennsco-deluxe-cabinet-36x18x78-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 22:39:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wearwell-weldsafe-anti-fatigue-mat-36x60>: HTTP status code is not handled or not allowed 2026-01-24 22:39:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-series-clamp-for-pipe-hdsscp025-1-4-inch-id>: HTTP status code is not handled or not allowed 2026-01-24 22:39:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simpson-oem-technologies-axial-cam-pump-kit-3300-psi-24-gpm>: HTTP status code is not handled or not allowed 2026-01-24 22:39:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smr7-ta-torque-arm-for-a-size-7-shaft-mount-reducer>: HTTP status code is not handled or not allowed 2026-01-24 22:39:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/achim-nexus-self-adhesive-vinyl-floor-tile-12-x-12-black-20-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:39:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/filter-assembly-hfa21015-10-micron-15-psi-bypass-1-1-4-inch-npt-ports-41-8-inch-npt-gage-ports>: HTTP status code is not handled or not allowed 2026-01-24 22:39:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/filter-assembly-hfa12515-25-micron-15-psi-bypass-3-4-inch-npt-ports-2-1-8-inch-npt-gage-ports>: HTTP status code is not handled or not allowed 2026-01-24 22:39:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/achim-nexus-self-adhesive-vinyl-floor-planks-6-x-36-walnut-10-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:39:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/achim-tivoli-self-adhesive-vinyl-floor-tile-12-x-12-classic-light-oak-diamond-parquet-45-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:39:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hydraulic-motor-cm064p-4-bolt-141-cipr-243-max-rpm-141-displacement>: HTTP status code is not handled or not allowed 2026-01-24 22:39:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hose-clamp-hk2517-1-1-2-inch-clamping-dia-17-32-inch-bolt-hole-dia-25-pcs-bag>: HTTP status code is not handled or not allowed 2026-01-24 22:39:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wsmr9-151-shaft-mount-reducer-size-9-151-ratio-4-1516-tapered-bore>: HTTP status code is not handled or not allowed 2026-01-24 22:39:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smr9-151-shaft-mount-reducer-size-9-151-ratio-4-1516-tapered-bore>: HTTP status code is not handled or not allowed 2026-01-24 22:39:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ediusa-full-face-gas-mask-w-double-filters-adjustable-suspension-system>: HTTP status code is not handled or not allowed 2026-01-24 22:39:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smr3-151-shaft-mount-reducer-size-3-151-ratio-2-316-tapered-bore>: HTTP status code is not handled or not allowed 2026-01-24 22:39:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9oz-30-sateen-cotton-jacket-black-4xl>: HTTP status code is not handled or not allowed 2026-01-24 22:39:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-applicator-with-granules-fg08832071>: HTTP status code is not handled or not allowed 2026-01-24 22:39:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rousseau-drawer-partition-kits-fits-front-drawer-height-4-30-compartment-kit-36wx24d>: HTTP status code is not handled or not allowed 2026-01-24 22:39:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quantum-giant-stacking-bins-19-7-8-x15-1-4-x15-3-4-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:39:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sa-mirella-conference-table-sitting-height-8-rectangle-sanddune>: HTTP status code is not handled or not allowed 2026-01-24 22:39:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-corn-brooms>: HTTP status code is not handled or not allowed 2026-01-24 22:39:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reducer-bushing-h3109x12x8-3-4-inch-x-1-2-inch-npt-male-to-female>: HTTP status code is not handled or not allowed 2026-01-24 22:39:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sa-learn-mobile-whiteboard-30-x-64-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:39:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metro-extra-shelf-for-open-wire-shelving-30x24>: HTTP status code is not handled or not allowed 2026-01-24 22:39:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pops-assorted-flavors-individually-wrapped-300-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:39:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/worldwide-electric-winl77-60-1-145tc-helical-inline-speed-reducer-145tc-input-flange-60-1-ratio>: HTTP status code is not handled or not allowed 2026-01-24 22:39:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/in-line-relief-valve-hrv07516-12-sae-30-gpm>: HTTP status code is not handled or not allowed 2026-01-24 22:39:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/worldwide-electric-winl137-40-1-284-6tchelical-inline-speed-reducer284-6tc-input-flange40-1-ratio>: HTTP status code is not handled or not allowed 2026-01-24 22:39:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:39:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-port-high-pressure-ball-valve-hbv3w038-5800-max-pressure-3-8-inch-nptf>: HTTP status code is not handled or not allowed 2026-01-24 22:39:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-port-high-pressure-ball-valve-hbvs075-5800-max-pressure-3-4-inch-nptf>: HTTP status code is not handled or not allowed 2026-01-24 22:39:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-32w-x-28l-navy>: HTTP status code is not handled or not allowed 2026-01-24 22:39:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zcc-12hd-40t3-zipcombo-centrifuge-with-12-place-microhematocrit-rotor>: HTTP status code is not handled or not allowed 2026-01-24 22:39:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ds-247050w-shrink-wrap-24w-x-50l-7mil-white>: HTTP status code is not handled or not allowed 2026-01-24 22:39:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ds-706w-heat-shrink-tape-6w-x-180l-9mil-white>: HTTP status code is not handled or not allowed 2026-01-24 22:39:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/foam/foam_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:39:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-vis-long-sleeve-shirt-tear-and-rip-resistant-s-navy>: HTTP status code is not handled or not allowed 2026-01-24 22:39:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clark-foam-products-1001215-foam-sheet-200100-poly-charcoal-4x24x72>: HTTP status code is not handled or not allowed 2026-01-24 22:39:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/foam/foam_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:39:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wind-pro-full-zip-hooded-sweater-reflective-flame-resistant-5xl-tall-orange>: HTTP status code is not handled or not allowed 2026-01-24 22:39:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-vis-full-zip-hooded-sweater-reflective-s-orange>: HTTP status code is not handled or not allowed 2026-01-24 22:39:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats>: HTTP status code is not handled or not allowed 2026-01-24 22:39:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hickory-half-zip-pull-over-long-sleeve-shirt-l-tall-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:39:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/le1-wood-cutting-reciprocating-saw-blade-6-tpi-6x34x050-25-pack-B925751>: HTTP status code is not handled or not allowed 2026-01-24 22:39:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-long-sleeve-button-down-work-shirt-s-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:39:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/view-adjust-floor-stand-reference-organizer-10-panel>: HTTP status code is not handled or not allowed 2026-01-24 22:39:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-locker-accessory-individual-slope-top-12w-x-12d-x-4h-707-marine-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:39:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-tablet-viewstation-black>: HTTP status code is not handled or not allowed 2026-01-24 22:39:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/foam/foam_sheets>: HTTP status code is not handled or not allowed 2026-01-24 22:39:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-32w-x-32l-brown>: HTTP status code is not handled or not allowed 2026-01-24 22:39:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grd-super-brute-3-speed-circulator-water-pump-ups-15-58-fc-59896341-115v-cast-iron>: HTTP status code is not handled or not allowed 2026-01-24 22:39:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_drawers_keyboard_trays already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:39:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/foam/foam_sheets>: HTTP status code is not handled or not allowed 2026-01-24 22:39:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-30w-x-31l-navy>: HTTP status code is not handled or not allowed 2026-01-24 22:39:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/truck_boxes_utility_boxes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:39:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/footrest-tube-c2ft-60-adjustable-c2-series-60>: HTTP status code is not handled or not allowed 2026-01-24 22:39:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-44w-x-36l-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:39:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-36w-x-34l-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:39:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-30w-x-28l-black>: HTTP status code is not handled or not allowed 2026-01-24 22:40:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-media-bollard-black-with-rubber-base-sspb>: HTTP status code is not handled or not allowed 2026-01-24 22:40:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-32w-x-28l-black>: HTTP status code is not handled or not allowed 2026-01-24 22:40:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/fans_blowers/cooling_fans/home_office_fans already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:40:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ctl-bandsaw-blade-10-1-long-x-1-2-wide-18-tpi-x-0025-thick>: HTTP status code is not handled or not allowed 2026-01-24 22:40:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vented-plastic-shelving-72x21x54-nexelon-finish-331481N>: HTTP status code is not handled or not allowed 2026-01-24 22:40:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_drawers_keyboard_trays>: HTTP status code is not handled or not allowed 2026-01-24 22:40:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-42w-x-34l-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:40:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:40:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/truck_boxes_utility_boxes>: HTTP status code is not handled or not allowed 2026-01-24 22:40:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-42w-x-32l-navy>: HTTP status code is not handled or not allowed 2026-01-24 22:40:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-34w-x-31l-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:40:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-28w-x-36l-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:40:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 31813 pages (at 87 pages/min), scraped 14746 items (at 15 items/min) 2026-01-24 22:40:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/301v-bike-shell-bike-locker-1-bike-cap-54l-x-30wf-x-4wb-x-74h-tan-t-handle>: HTTP status code is not handled or not allowed 2026-01-24 22:40:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5650100-sase-shlvg-48x18x63-add-on>: HTTP status code is not handled or not allowed 2026-01-24 22:40:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5464200-60x18x86-add-on-shelving>: HTTP status code is not handled or not allowed 2026-01-24 22:40:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-34w-x-33l-navy>: HTTP status code is not handled or not allowed 2026-01-24 22:40:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/fans_blowers/cooling_fans/home_office_fans>: HTTP status code is not handled or not allowed 2026-01-24 22:40:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/l41ledh-led-dock-light-head-only-38w-3500-lumens-5700k>: HTTP status code is not handled or not allowed 2026-01-24 22:40:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-36-x-14-3-shelf-media-stand-chrome>: HTTP status code is not handled or not allowed 2026-01-24 22:40:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-double-wall-cardboard-corrugated-box-12-x-12-x-18-275-lb-ect-48-15-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:40:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:40:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 22:40:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manchester-address-plate-w-3-gold-brass-numbers-in-bronze>: HTTP status code is not handled or not allowed 2026-01-24 22:40:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-44w-x-32l-green>: HTTP status code is not handled or not allowed 2026-01-24 22:40:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-inch-drive-ratcheting-head-micrometer-torque-wrench-30-150-ft-lbs-w-nist-certificate>: HTTP status code is not handled or not allowed 2026-01-24 22:40:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-31w-x-33l-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:40:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-literature-mailers-9-x-6-1-2-x-4-200lb-test-ect-32-b-50-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:40:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-38w-x-36l-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:40:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-handle-hex-key-1-8-inch-6-inch-long>: HTTP status code is not handled or not allowed 2026-01-24 22:40:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/la-72-24-70mm-stack-audio-unit-24v-acdc-pulse-or-continuous-75-db>: HTTP status code is not handled or not allowed 2026-01-24 22:40:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/la-tcl03sf532-70mm-complete-light-stack-120v-led-gray-term-g-a-r>: HTTP status code is not handled or not allowed 2026-01-24 22:40:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-54w-x-30l-navy>: HTTP status code is not handled or not allowed 2026-01-24 22:40:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-poly-vinyl-long-bag-black-42-e>: HTTP status code is not handled or not allowed 2026-01-24 22:40:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 22:40:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-34w-x-32l-black>: HTTP status code is not handled or not allowed 2026-01-24 22:40:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:40:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manchester-stucco-locking-column-mailbox-in-slate-gray-w-decorative-fleur-de-lis-door-in-black>: HTTP status code is not handled or not allowed 2026-01-24 22:40:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-46w-x-30l-brown>: HTTP status code is not handled or not allowed 2026-01-24 22:40:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-import-36-pc-letter-figure-steel-stamp-set>: HTTP status code is not handled or not allowed 2026-01-24 22:40:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/j68-oh-needle-bearing-drawn-cup-caged-oil-hole-bore-9525mm>: HTTP status code is not handled or not allowed 2026-01-24 22:40:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-replacement-rollers-10000728-12-pack-rollers-discs>: HTTP status code is not handled or not allowed 2026-01-24 22:40:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b812-oh-needle-bearing-drawn-cup-full-complement-oil-hole-bore-127mm>: HTTP status code is not handled or not allowed 2026-01-24 22:40:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washable-markers-conical-tip-nontoxic-assorted-12-set>: HTTP status code is not handled or not allowed 2026-01-24 22:40:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wood-tape-holder-holds-8-rolls-natural>: HTTP status code is not handled or not allowed 2026-01-24 22:40:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-46w-x-32l-black>: HTTP status code is not handled or not allowed 2026-01-24 22:40:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/el11206-33hp-48-frame-1800rpm-115-230v-odp-rigid-premium-efficiency>: HTTP status code is not handled or not allowed 2026-01-24 22:40:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vem31109-5hp-1200rpm-56c-frame-3ph-230-460v-odp-c-face-footless-premium-efficiency>: HTTP status code is not handled or not allowed 2026-01-24 22:40:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-metric-ruler-12-inch-long-clear>: HTTP status code is not handled or not allowed 2026-01-24 22:40:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:40:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 22:40:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/em31120-15hp-3600rpm-56-frame-3ph-230-460v-odp-rigid-premium-efficiency>: HTTP status code is not handled or not allowed 2026-01-24 22:40:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cel11307-75hp-56c-frame-1800rpm-115-230v-odp-c-face-rigid-premium-efficiency>: HTTP status code is not handled or not allowed 2026-01-24 22:40:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vel11301-33hp-56c-frame-1800rpm-115-230v-odp-c-face-footless-premium-efficiency>: HTTP status code is not handled or not allowed 2026-01-24 22:40:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-52w-x-33l-green>: HTTP status code is not handled or not allowed 2026-01-24 22:40:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:40:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-cal-eco-sport-1-interlocking-flooring-tiles-1-x-20-x-20-12-pack-coal-1>: HTTP status code is not handled or not allowed 2026-01-24 22:40:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-cal-dura-chef-interlock-kitchen-comfort-mats-5-8-x-3-x-3-red-rubber-mat-4-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:40:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-bulk-parts-lifter-4-to-5-lbs-pull-4375l-x-3125w-x-925h-nylon-base>: HTTP status code is not handled or not allowed 2026-01-24 22:40:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/727e-cup-glass-washer-high-temperature-with-built-in-booster-16-x-16-rack>: HTTP status code is not handled or not allowed 2026-01-24 22:40:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tb-1003-motor-terminal-conduit-box>: HTTP status code is not handled or not allowed 2026-01-24 22:40:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-56w-x-29l-navy>: HTTP status code is not handled or not allowed 2026-01-24 22:40:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-56w-x-31l-navy>: HTTP status code is not handled or not allowed 2026-01-24 22:40:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 22:40:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-32w-x-30l-navy>: HTTP status code is not handled or not allowed 2026-01-24 22:40:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-stainless-steel-compact-ash-bin-908>: HTTP status code is not handled or not allowed 2026-01-24 22:40:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-56w-x-29l-black>: HTTP status code is not handled or not allowed 2026-01-24 22:40:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wypall-l40-wipers-10-45-x-10-pop-up-box-white-90box-9-boxescarton-03046>: HTTP status code is not handled or not allowed 2026-01-24 22:40:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-60w-x-34l-black>: HTTP status code is not handled or not allowed 2026-01-24 22:40:50 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 22:40:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stock-tank-w-55bk-w-series-55-gal>: HTTP status code is not handled or not allowed 2026-01-24 22:40:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-28w-x-30l-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:40:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-28w-x-32l-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:40:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-12-gauge-cabinet-with-3-drawers-4-shelves-36w-x-24d-x-78h>: HTTP status code is not handled or not allowed 2026-01-24 22:40:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:40:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-ghs2032alv2-corrosive-ghs-label-vinyl-2-x-2-250-roll>: HTTP status code is not handled or not allowed 2026-01-24 22:40:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ctl-bandsaw-blade-7-foot-9-inch-long-x-3-4-inch-wide-8-12-tpi-x-0035-thick>: HTTP status code is not handled or not allowed 2026-01-24 22:40:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-work-coveralls-52-tall-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:40:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/69000-series-rectangular-return-shell-llr69388-42-w-x-24-d-x-29-1-2-h-mahogany>: HTTP status code is not handled or not allowed 2026-01-24 22:40:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-westex-31w-x-34l-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:40:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15153-tourn-knife-black-handle-high-carbon-steel-black-handle-2-1-2-l>: HTTP status code is not handled or not allowed 2026-01-24 22:41:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/weld-curtain-w-hardware-12w-x-8h>: HTTP status code is not handled or not allowed 2026-01-24 22:41:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-29w-x-30l-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:41:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/09463-vegetable-produce-knife-high-carbon-steel-stamped-white-handle-6-l>: HTTP status code is not handled or not allowed 2026-01-24 22:41:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/competitor-ventilated-three-tier-locker-starter-1-wide-12w-x-15d-x-24h-unassembled-black>: HTTP status code is not handled or not allowed 2026-01-24 22:41:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-double-tier-locker-2-wide-12w-x-12d-x-30h-assembled-black>: HTTP status code is not handled or not allowed 2026-01-24 22:41:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/r11i-tas-115-2303-rotary-screw-air-compressor-3-phase-230-volts-15hp-120-gal>: HTTP status code is not handled or not allowed 2026-01-24 22:41:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 22:41:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-mesh-baskets-14x14x8-price-each-for-qty-3>: HTTP status code is not handled or not allowed 2026-01-24 22:41:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:41:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wesco-24x16-open-deck-hardwood-dolly-4-casters-1200-lb-cap-987847>: HTTP status code is not handled or not allowed 2026-01-24 22:41:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wesco-36x24-open-deck-hardwood-dolly-4-casters-1200-lb-cap-987849>: HTTP status code is not handled or not allowed 2026-01-24 22:41:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:41:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-westex-52w-x-unhemmed-navy>: HTTP status code is not handled or not allowed 2026-01-24 22:41:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-steelepoxy-s-5-52345-2-1-oz-tubes>: HTTP status code is not handled or not allowed 2026-01-24 22:41:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 31903 pages (at 90 pages/min), scraped 14761 items (at 15 items/min) 2026-01-24 22:41:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-westex-30w-x-34l-navy>: HTTP status code is not handled or not allowed 2026-01-24 22:41:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-drawer-divider-set-for-whisper-medical-carts>: HTTP status code is not handled or not allowed 2026-01-24 22:41:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/performance-corner-work-station-african-mahogany>: HTTP status code is not handled or not allowed 2026-01-24 22:41:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jd-es-shutter-fan-20-inch-115v-1-10hp-1ph-variable-speed-aluminum-shutters-10ft-cord>: HTTP status code is not handled or not allowed 2026-01-24 22:41:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rmd-a-1-rubber-in-shear-mounts-3-1-8-l-x-1-3-4-w-x-1-1-4-h>: HTTP status code is not handled or not allowed 2026-01-24 22:41:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/remp-040401-rubber-eva-pads-4-l-x-4-w-x-7-8-h>: HTTP status code is not handled or not allowed 2026-01-24 22:41:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-work-coveralls-44-tall-black>: HTTP status code is not handled or not allowed 2026-01-24 22:41:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 22:41:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a136k-v-belt-kevlar-1-2-x-138>: HTTP status code is not handled or not allowed 2026-01-24 22:41:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a20k-v-belt-kevlar-1-2-x-22>: HTTP status code is not handled or not allowed 2026-01-24 22:41:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:41:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 22:41:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-westex-33w-x-34l-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:41:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:41:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-compartment-hanging-garment-dispenser-locker-silver-vein-w-turn-knob-locks>: HTTP status code is not handled or not allowed 2026-01-24 22:41:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b158k-v-belt-kevlar-5-8-x-161>: HTTP status code is not handled or not allowed 2026-01-24 22:41:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-westex-40w-x-unhemmed-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:41:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-westex-34w-x-34l-navy>: HTTP status code is not handled or not allowed 2026-01-24 22:41:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/en-14-hd-316ss-forged-eye-nut-1-1-2-6-style-b-316-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 22:41:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18900222-heating-block-used-for-50ml-tubes-8-holes>: HTTP status code is not handled or not allowed 2026-01-24 22:41:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/utility-jeans-basket-weave-flame-resistant-8-oz-42w-x-unhemmed-navy>: HTTP status code is not handled or not allowed 2026-01-24 22:41:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flanged-intake-door-1000>: HTTP status code is not handled or not allowed 2026-01-24 22:41:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cargo-pants-with-double-reinforced-knees-flame-resistant-44w-x-34l-black>: HTTP status code is not handled or not allowed 2026-01-24 22:41:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/utility-jeans-basket-weave-flame-resistant-8-oz-48w-x-34l-navy>: HTTP status code is not handled or not allowed 2026-01-24 22:41:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 22:41:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-scrubber-negative-air-machine-2000-1000-cfm-w-hepa-audible-visual-alarm>: HTTP status code is not handled or not allowed 2026-01-24 22:41:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-accessories-4-34-h-ylw-wall-mount-plate-for-wm412>: HTTP status code is not handled or not allowed 2026-01-24 22:41:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 22:41:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galvanized-steel-sheet-57836-36-inchl-x-24-inchw-x-28-ga>: HTTP status code is not handled or not allowed 2026-01-24 22:41:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stair-edging-32012-36-inchl-cherry-screw-nails>: HTTP status code is not handled or not allowed 2026-01-24 22:41:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-westex-54w-x-36l-black>: HTTP status code is not handled or not allowed 2026-01-24 22:41:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wallbase-dry-back-93245-4-inchw-x-20-footh-silver-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:41:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:41:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zinsser-b-i-n-advanced-synthetic-shellac-primer-white-gal-can-270976>: HTTP status code is not handled or not allowed 2026-01-24 22:41:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-wire-shelving-54-inch-w-x-24-inch-d-x-54-inch-h>: HTTP status code is not handled or not allowed 2026-01-24 22:41:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-wire-shelving-60-inch-w-x-21-inch-d-x-86-inch-h>: HTTP status code is not handled or not allowed 2026-01-24 22:41:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f1810pm0-18-gpm-particulate-spin-on-filter-18-gpm-in-line>: HTTP status code is not handled or not allowed 2026-01-24 22:41:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/utility-jeans-basket-weave-flame-resistant-8-oz-35w-x-30l-navy>: HTTP status code is not handled or not allowed 2026-01-24 22:41:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mh7004-7-magnetic-wall-mount-barrier-cleaning-in-progress-banner>: HTTP status code is not handled or not allowed 2026-01-24 22:41:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-wire-shelving-24-inch-w-x-24-inch-d-x-54-inch-h>: HTTP status code is not handled or not allowed 2026-01-24 22:41:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-westex-48w-x-32l-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:41:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:41:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-wire-shelving-30-inch-w-x-14-inch-d-x-86-inch-h>: HTTP status code is not handled or not allowed 2026-01-24 22:41:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-work-pants-reflective-material-flame-resistant-30w-x-34l-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:41:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cesm-21cb-3660-safety-switch-mat-36-x-60-black>: HTTP status code is not handled or not allowed 2026-01-24 22:41:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-wire-shelving-30-inch-w-x-18-inch-d-x-74-inch-h>: HTTP status code is not handled or not allowed 2026-01-24 22:41:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 22:41:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-westex-54w-x-32l-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:41:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/painting_supplies/paint_primer already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:41:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-wire-shelving-add-on-36-inch-w-x-14-inch-d-x-74-inch-h>: HTTP status code is not handled or not allowed 2026-01-24 22:41:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-work-pants-reflective-material-flame-resistant-50w-x-34l-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:41:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:41:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-48w-x-21d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 22:41:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cargo-pants-with-double-reinforced-knees-flame-resistant-40w-x-30l-black>: HTTP status code is not handled or not allowed 2026-01-24 22:41:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 22:41:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-42w-x-24d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 22:41:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-30w-x-18d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 22:41:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-42w-x-18d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 22:41:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-decor-decorative-flat-enamel-12-oz-aerosol-can-black-792245>: HTTP status code is not handled or not allowed 2026-01-24 22:41:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-decor-decorative-gloss-enamel-12-oz-aerosol-can-clear-792264>: HTTP status code is not handled or not allowed 2026-01-24 22:41:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:41:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/70inch-square-resin-outdoor-dining-table-and-chair-set-4-chairs>: HTTP status code is not handled or not allowed 2026-01-24 22:41:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/x-o-rust-12-oz-aerosol-rust-preventative-paint-primer-in-one-blue-gloss-125846>: HTTP status code is not handled or not allowed 2026-01-24 22:41:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/painting_supplies/paint_primer>: HTTP status code is not handled or not allowed 2026-01-24 22:41:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gpm-1-gallon-bottle-house-siding-wash-250-300-sq-ft-coverage-880765>: HTTP status code is not handled or not allowed 2026-01-24 22:41:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 22:41:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-9a-3ph-reset-button-24-60v-4-2-5-7a>: HTTP status code is not handled or not allowed 2026-01-24 22:41:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/x-o-rust-brush-on-enamel-gloss-royal-blue-matches-ford-blue-gallon-776536>: HTTP status code is not handled or not allowed 2026-01-24 22:41:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-epoxy-wire-shelving-36w-x-24d-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 22:41:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-9a-1ph-remote-start-start-stop-24-60v-4-2-5-7a>: HTTP status code is not handled or not allowed 2026-01-24 22:41:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hinged-portable-speed-bump-8-sections-black-yellow-psb-118-10-1>: HTTP status code is not handled or not allowed 2026-01-24 22:41:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-9a-1ph-remote-start-terminals-reset-button-100-250v-1-7-2-3a>: HTTP status code is not handled or not allowed 2026-01-24 22:41:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nestable-plastic-shelf-bin-6-5-8w-x-23-5-8d-x-8h-clear>: HTTP status code is not handled or not allowed 2026-01-24 22:41:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 22:41:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-9a-1ph-start-stop-24-60v-2-3-3-1a>: HTTP status code is not handled or not allowed 2026-01-24 22:41:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-12a-3ph-remote-start-terminals-start-stop-250-500v-3-1-4-2a>: HTTP status code is not handled or not allowed 2026-01-24 22:41:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manitowoc-ice-ud-0310a-neo-undercounter-ice-maker-cube-style-air-cooled-self-contained>: HTTP status code is not handled or not allowed 2026-01-24 22:41:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-12a-3ph-remote-start-start-stop-24-60v-3-1-4-2a>: HTTP status code is not handled or not allowed 2026-01-24 22:41:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ez-gld-2-fixed-drp-dn-lad-rack-for-comp-long-2265-3-01>: HTTP status code is not handled or not allowed 2026-01-24 22:42:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-12a-3ph-remote-start-reset-button-24-60v-4-2-5-7a>: HTTP status code is not handled or not allowed 2026-01-24 22:42:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/security-shelf-door-10-1-2-in-36-in-9503-3-01>: HTTP status code is not handled or not allowed 2026-01-24 22:42:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-12a-3ph-separate-coil-voltage-reset-button-100-250v-4-2-5-7a>: HTTP status code is not handled or not allowed 2026-01-24 22:42:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-12a-1ph-separate-coil-voltage-reset-button-100-250v-5-7-7-6a>: HTTP status code is not handled or not allowed 2026-01-24 22:42:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1500102-single-tool-holster-belt-with-retractor>: HTTP status code is not handled or not allowed 2026-01-24 22:42:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lab-tables-24-w-x-48-l-x-36-h-black-granite>: HTTP status code is not handled or not allowed 2026-01-24 22:42:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-square-table-ladder-back-bar-ht-set-maple-fusion-laminate-table-black-vinyl-chair>: HTTP status code is not handled or not allowed 2026-01-24 22:42:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swing-bar-door-guard-u-9897-brass>: HTTP status code is not handled or not allowed 2026-01-24 22:42:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/olf72-a31-light-accy-supt-reqr-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:42:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/501l-floor-mount-sink-with-double-foot-pedal-valve>: HTTP status code is not handled or not allowed 2026-01-24 22:42:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/56fal-3-person-floor-mount-wash-station-with-ac-powered-sensors>: HTTP status code is not handled or not allowed 2026-01-24 22:42:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 31997 pages (at 94 pages/min), scraped 14772 items (at 11 items/min) 2026-01-24 22:42:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chair-mat-for-hard-floors-heavy-use-60-x-72-rectangle-clear-straight-edge>: HTTP status code is not handled or not allowed 2026-01-24 22:42:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deep-groove-ball-bearings-inch-r20-2rs-sealed-light-duty-125-inch-bore-225-inch-od>: HTTP status code is not handled or not allowed 2026-01-24 22:42:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-8482-outdoor-metal-slatted-receptacle-w-access-door-rain-bonnet-lid-36-gallon-green>: HTTP status code is not handled or not allowed 2026-01-24 22:42:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-16a-1ph-remote-start-terminals-start-stop-100-250v-13-16a>: HTTP status code is not handled or not allowed 2026-01-24 22:42:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/viper-plus-gt-full-reclining-wheelchair-detachable-desk-arms-16-seat-pla416rbdda>: HTTP status code is not handled or not allowed 2026-01-24 22:42:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-16a-1ph-direct-online-reset-button-100-250v-10-13a>: HTTP status code is not handled or not allowed 2026-01-24 22:42:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-16a-1ph-separate-coil-voltage-reset-button-100-250v-7-6-10a>: HTTP status code is not handled or not allowed 2026-01-24 22:42:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deep-groove-ball-bearings-metric-6212-zz-2-metal-shields-medium-duty-60mm-bore-110mm-od>: HTTP status code is not handled or not allowed 2026-01-24 22:42:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-26a-3ph-direct-online-start-stop-100-250v-13-16a>: HTTP status code is not handled or not allowed 2026-01-24 22:42:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-26a-3ph-direct-online-start-stop-250-500v-10-13a>: HTTP status code is not handled or not allowed 2026-01-24 22:42:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/662047-machine-screw-m3-x-0-5-x-10mm-pan-head-phillips-drive-steel-zinc-pkg-of-1000>: HTTP status code is not handled or not allowed 2026-01-24 22:42:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-16a-1ph-remote-start-reset-button-100-250v-13-16a>: HTTP status code is not handled or not allowed 2026-01-24 22:42:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deep-groove-ball-bearings-inch-r4-2rs-sealed-light-duty-025-inch-bore-0625-inch-od>: HTTP status code is not handled or not allowed 2026-01-24 22:42:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-hinged-lid-bayhead-products-2-2-cu-yd-self-dumping-plastic-hopper>: HTTP status code is not handled or not allowed 2026-01-24 22:42:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-38a-3ph-direct-online-start-stop-250-500v-20-24a>: HTTP status code is not handled or not allowed 2026-01-24 22:42:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/305184-nylon-insert-lock-nut-7-16-20-low-carbon-steel-zinc-unf-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:42:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-26a-1ph-separate-coil-voltage-reset-button-100-250v-13-16a>: HTTP status code is not handled or not allowed 2026-01-24 22:42:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/800-wood-classic-2-drawers-gun-storage-cabinet-8-long-guns>: HTTP status code is not handled or not allowed 2026-01-24 22:42:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/812008-nylon-insert-lock-nut-m3-x-0-5-steel-zinc-unc-class-8-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:42:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/857380-serrated-hex-flange-nut-5-8-11-zinc-case-hardened-steel-unc-pkg-of-50>: HTTP status code is not handled or not allowed 2026-01-24 22:42:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/302090-tee-nut-5-16-18-x-5-8-3-prong-steel-zinc-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:42:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/825190-all-metal-hex-lock-nut-1-2-13-medium-carbon-steel-zinc-and-wax-grade-c-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:42:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/489028-carriage-bolt-1-4-20-x-3-round-head-steel-zinc-grade-a-ft-pkg-of-50>: HTTP status code is not handled or not allowed 2026-01-24 22:42:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tm3g-traffic-sign-no-parking-fire-lane-18-x-12-whitered>: HTTP status code is not handled or not allowed 2026-01-24 22:42:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/write-on-scoreboard-this-department-has-worked-days-without-a-lost-time-accident-28-x-20-whtgrn>: HTTP status code is not handled or not allowed 2026-01-24 22:42:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-52a-3ph-remote-start-start-stop-250-500v-25-33a>: HTTP status code is not handled or not allowed 2026-01-24 22:42:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e8c-extension-spring-025-od-x-00286-wire-dia-x-125-oal-328-lbs-in-mbhd>: HTTP status code is not handled or not allowed 2026-01-24 22:42:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dl12ap-dot-shipping-labels-corrosive-8-4-x-4-whiteblack-25-per-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:42:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-mini-split-dual-fan-stand-qsms1202-38l-x-38w-x-12h>: HTTP status code is not handled or not allowed 2026-01-24 22:42:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/d18p-osha-sign-danger-diesel-no-smoking-7-x-10-whiteredblack>: HTTP status code is not handled or not allowed 2026-01-24 22:42:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-52a-3ph-remote-start-start-stop-24-60v-30-40a>: HTTP status code is not handled or not allowed 2026-01-24 22:42:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-52a-1ph-separate-coil-voltage-start-stop-100-250v-36-47a>: HTTP status code is not handled or not allowed 2026-01-24 22:42:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precision-compression-spring-06-od-x-0055-x-15-music-wire-pkg-of-5>: HTTP status code is not handled or not allowed 2026-01-24 22:42:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precision-compression-spring-03-od-x-0032-x-1-music-wire-pkg-of-5>: HTTP status code is not handled or not allowed 2026-01-24 22:42:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-52a-1ph-remote-start-start-stop-100-250v-22-28a>: HTTP status code is not handled or not allowed 2026-01-24 22:42:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/patio-heater-cover-hvd-sgtcv-econ-fits-heavy-duty-for-91-square-glass-tube-models-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:42:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-65a-3ph-start-stop-24-60v-50-60a>: HTTP status code is not handled or not allowed 2026-01-24 22:42:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-65a-3ph-direct-online-start-stop-100-250v-50-60a>: HTTP status code is not handled or not allowed 2026-01-24 22:42:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tm18g-traffic-sign-10-mph-speed-limit-sign-18-x-12-whiteblack>: HTTP status code is not handled or not allowed 2026-01-24 22:42:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rpt30-tags-danger-6-x-3-whiteredblack-25pk>: HTTP status code is not handled or not allowed 2026-01-24 22:42:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-65a-1ph-remote-start-terminals-reset-button-100-250v-44-53a>: HTTP status code is not handled or not allowed 2026-01-24 22:42:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-80a-3ph-remote-start-start-stop-100-250v-48-60a>: HTTP status code is not handled or not allowed 2026-01-24 22:42:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-80a-1ph-direct-online-start-stop-100-250v-57-68a>: HTTP status code is not handled or not allowed 2026-01-24 22:42:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cymbidium-with-decorative-vase-silk-arrangement-burgundy>: HTTP status code is not handled or not allowed 2026-01-24 22:42:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vsm-abrasive-belt-217481-zirconia-alumina-2-1-2-x-72-50-grit>: HTTP status code is not handled or not allowed 2026-01-24 22:42:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelate-wire-shelving-72-w-x-18-d-x-86-h>: HTTP status code is not handled or not allowed 2026-01-24 22:42:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelate-wire-shelving-add-on-72-w-x-24-d-x-86-h>: HTTP status code is not handled or not allowed 2026-01-24 22:42:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vsm-abrasive-belt-283001-ceramic-37-x-75-120-grit>: HTTP status code is not handled or not allowed 2026-01-24 22:42:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cabinet-exhaust-fan-w-shutter-48-inch-prop-15hp-23735-cfm-3-phase-belt-drive>: HTTP status code is not handled or not allowed 2026-01-24 22:42:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/loaded-gear-vx-100-tactical-vest-and-leg-platform-22l-x-38-50w-x-h>: HTTP status code is not handled or not allowed 2026-01-24 22:42:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commander-tactical-headset-kenwood-hyt-relm-radios>: HTTP status code is not handled or not allowed 2026-01-24 22:42:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-shot-6-unit-battery-charger-for-bantam-radio>: HTTP status code is not handled or not allowed 2026-01-24 22:42:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ac-inpass-2-1-2-inch-npt-coalescing-filter-716-cfm-8-000-hr-0-1-micron-232-psi>: HTTP status code is not handled or not allowed 2026-01-24 22:42:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-polypropylene-tagging-fasteners-803-standard-red-3l-5000-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:42:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-step-800-lb-capacity-work-platform-with-36-w-x-48-l-platform>: HTTP status code is not handled or not allowed 2026-01-24 22:42:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/olympia-i2-k-5025-bn-single-lever-pull-down-kitchen-faucet-pvd-brushed-nickel>: HTTP status code is not handled or not allowed 2026-01-24 22:42:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/folding-mezzanine-ladder-120h>: HTTP status code is not handled or not allowed 2026-01-24 22:42:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/folding-mezzanine-ladder-72h>: HTTP status code is not handled or not allowed 2026-01-24 22:42:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-in-bronze-glass-door-knob-set>: HTTP status code is not handled or not allowed 2026-01-24 22:42:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pioneer-legacy-3lg260h-single-lever-bathroom-faucet-with-pop-up-polished-chrome>: HTTP status code is not handled or not allowed 2026-01-24 22:42:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/k-5342-two-handle-kitchen-faucet-with-spray-polished-chrome>: HTTP status code is not handled or not allowed 2026-01-24 22:42:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pioneer-del-mar-2dm201-two-handle-kitchen-faucet-with-spray-polished-chrome>: HTTP status code is not handled or not allowed 2026-01-24 22:42:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nord-lock-1080-wedge-locking-washer-316-stainless-steel-m6-pkg-of-200>: HTTP status code is not handled or not allowed 2026-01-24 22:42:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/olympia-i1-l-6030-single-lever-bathroom-faucet-with-pop-up-polished-chrome>: HTTP status code is not handled or not allowed 2026-01-24 22:42:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thermostat-ta155-018-manual-changeover-heat-cool-no-switch-24-277vac>: HTTP status code is not handled or not allowed 2026-01-24 22:42:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-step-configurable-forward-descent-roll-ladder-perforated-ukdec106246>: HTTP status code is not handled or not allowed 2026-01-24 22:43:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-step-configurabl-forward-descent-roll-ladder-grip-strut-ukdec112242>: HTTP status code is not handled or not allowed 2026-01-24 22:43:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/41-ibc-stairway-stair36i-6-41>: HTTP status code is not handled or not allowed 2026-01-24 22:43:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-variable-height-sheet-rack-knock-down-1-500-lb-capacity-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:43:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/r120-rivet-pack-assortment-120-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:43:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plus-arborpantsseries-1-isea-ansi-107-class-3-high-visibility-prolar-lite-large>: HTTP status code is not handled or not allowed 2026-01-24 22:43:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/semi-automatic-stretch-wrap-machine-with-ntep-scale>: HTTP status code is not handled or not allowed 2026-01-24 22:43:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/92-824-69-piece-fractional-metric-laser-etched-socket-set>: HTTP status code is not handled or not allowed 2026-01-24 22:43:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/92-839-99-piece-fractional-metric-laser-etched-socket-set>: HTTP status code is not handled or not allowed 2026-01-24 22:43:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/82-115-4-piece-pick-and-hook-set>: HTTP status code is not handled or not allowed 2026-01-24 22:43:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fao-first-aid-kit-50-person-ansi-compliant-class-b-metal-case>: HTTP status code is not handled or not allowed 2026-01-24 22:43:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fao-refill-kit-50-person-ansi-compliant-class-a>: HTTP status code is not handled or not allowed 2026-01-24 22:43:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/28-140-nylon-handle-stiff-putty-knife-1-1-4-inch-wide-blade>: HTTP status code is not handled or not allowed 2026-01-24 22:43:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/hand_tools_1/hand_saws_cutting_tools_scrapers/knives_blades/utility_knives_1 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:43:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 32094 pages (at 97 pages/min), scraped 14790 items (at 18 items/min) 2026-01-24 22:43:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/129-ibc-stairway-stair36i-19-129>: HTTP status code is not handled or not allowed 2026-01-24 22:43:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-set-screw-coupling-mrc25-black-oxide-25mm-2pk>: HTTP status code is not handled or not allowed 2026-01-24 22:43:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/139-osha-stairway-stair36o-16-139>: HTTP status code is not handled or not allowed 2026-01-24 22:43:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-60-inch-w-x-21-inch-d-x-62-inch-h-2000-lbs-cap-4-open-1-solid-shelf-dunnage-stand>: HTTP status code is not handled or not allowed 2026-01-24 22:43:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-60-inch-w-x-21-inch-d-x-86-inch-h-2000-lbs-cap-3-open-1-solid-shelf-dunnage-stand>: HTTP status code is not handled or not allowed 2026-01-24 22:43:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compressor-l800053-model-lpw6548a-single-stage-compressor-pump-3-cylinder-25-5-hp>: HTTP status code is not handled or not allowed 2026-01-24 22:43:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-42-inch-w-x-24-inch-d-x-86-inch-h-2000-lbs-cap-5-open-grid-shelves>: HTTP status code is not handled or not allowed 2026-01-24 22:43:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24mm-locking-assembly-c193-series-c193m24-metric-m5-x-18>: HTTP status code is not handled or not allowed 2026-01-24 22:43:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-60-inch-w-x-18-inch-d-x-62-inch-h-2000-lbs-cap-5-open-grid-shelves-dunnage-stand>: HTTP status code is not handled or not allowed 2026-01-24 22:43:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-frame-folding-tank-stf-5000-207l-x-207w-x-29h-5000-gallon-cap-orange>: HTTP status code is not handled or not allowed 2026-01-24 22:43:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/hand_tools_1/hand_saws_cutting_tools_scrapers/knives_blades/utility_knives_1>: HTTP status code is not handled or not allowed 2026-01-24 22:43:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-72-inch-w-x-21-inch-d-x-50-inch-h-2000-lbs-cap-5-open-grid-shelves-dunnage-stand>: HTTP status code is not handled or not allowed 2026-01-24 22:43:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-72-inch-w-x-21-inch-d-x-62-inch-h-2000-lbs-cap-4-solid-shelves-dunnage-stand>: HTTP status code is not handled or not allowed 2026-01-24 22:43:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-frame-folding-tank-alf-750-88l-x-88w-x-29h-750-gallon-cap-black>: HTTP status code is not handled or not allowed 2026-01-24 22:43:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-keyless-rigid-coupling-c600m8-c600-series-steel-8mm-shaft>: HTTP status code is not handled or not allowed 2026-01-24 22:43:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-shaft-arbor-al5l-lefthand-type-d-214l-thread-fits-58-shaft>: HTTP status code is not handled or not allowed 2026-01-24 22:43:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-keyless-rigid-coupling-c600m24-c600-series-steel-24mm-shaft>: HTTP status code is not handled or not allowed 2026-01-24 22:43:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-keyless-rigid-coupling-c600m25-c600-series-steel-25mm-shaft>: HTTP status code is not handled or not allowed 2026-01-24 22:43:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-shaft-arbor-ga1-screw-lock-type-e-fits-5811-shaft>: HTTP status code is not handled or not allowed 2026-01-24 22:43:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-48-inch-w-x-24-inch-d-x-74-inch-h-2000-lbs-cap-5-solid-shelves>: HTTP status code is not handled or not allowed 2026-01-24 22:43:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shower-commode-wheelchair-nrs185006-aluminum-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:43:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drink-water-bladder-tank-bt-300pw-96l-x-72w-x-15h-300-gallon-cap-powder-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:43:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-36-inch-w-x-24-inch-d-x-86-inch-h-2000-lbs-cap-4-open-grid-shelves>: HTTP status code is not handled or not allowed 2026-01-24 22:43:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-no-28h-cable-knife-8-28-mm-common-round-cables>: HTTP status code is not handled or not allowed 2026-01-24 22:43:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dsv521-2x-ansi-class-2-heavy-duty-surveyor-vest-2xl>: HTTP status code is not handled or not allowed 2026-01-24 22:43:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dpg54-2d-ansi-z87-1-protector-smoke>: HTTP status code is not handled or not allowed 2026-01-24 22:43:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dsv521-5x-ansi-class-2-heavy-duty-surveyor-vest-5xl>: HTTP status code is not handled or not allowed 2026-01-24 22:43:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-36-inch-w-x-21-inch-d-x-74-inch-h-2000-lbs-cap-4-solid-shelves>: HTTP status code is not handled or not allowed 2026-01-24 22:43:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quick-adjust-wire-shelving-36-x-18-x-86-poly-z-brite>: HTTP status code is not handled or not allowed 2026-01-24 22:43:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-30-inch-w-x-21-inch-d-x-50-inch-h-2000-lbs-cap-4-open-grid-shelves>: HTTP status code is not handled or not allowed 2026-01-24 22:43:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-30-inch-w-x-21-inch-d-x-62-inch-h-2000-lbs-cap-4-open-grid-shelves>: HTTP status code is not handled or not allowed 2026-01-24 22:43:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drainage-channel-frame-jd2al-double-leaf-316-stainless-steel-hardware-48l-x-48w>: HTTP status code is not handled or not allowed 2026-01-24 22:43:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-stage-oil-less-direct-drive-air-compressor-15-gallon-1-hp-horizontal>: HTTP status code is not handled or not allowed 2026-01-24 22:43:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/puma-tuk-5080vm-two-stage-belt-drive-air-compressor-80-gallon-5-hp-vertical>: HTTP status code is not handled or not allowed 2026-01-24 22:43:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10010028-lbl-dt-z-perform-2000d-2-w-x-1-l-perforated-1-core-5-od>: HTTP status code is not handled or not allowed 2026-01-24 22:43:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24x48-metromax-shelf>: HTTP status code is not handled or not allowed 2026-01-24 22:43:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b-rate-depository-safe-front-loading-digital-lock-20w-x-20d-x-30h>: HTTP status code is not handled or not allowed 2026-01-24 22:43:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/contempo-exhibit-post-24h-black-post-7-retractable-black-yellow-belt>: HTTP status code is not handled or not allowed 2026-01-24 22:43:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cotton-undercast-padding-regular-finish-6-x-4-yards-6-roll-bag-6-bag-case>: HTTP status code is not handled or not allowed 2026-01-24 22:43:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/american-white-cross-fabric-fingertip-bandage-3-non-metal-blue-bulk-1400-case>: HTTP status code is not handled or not allowed 2026-01-24 22:43:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medina-corner-support-for-hutch-15w-x-15d-x-18h-gray-steel>: HTTP status code is not handled or not allowed 2026-01-24 22:43:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-modular-frl-wend-ports-c33352-gauge-poly-bowl-manual-fog-lubricator>: HTTP status code is not handled or not allowed 2026-01-24 22:43:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medina-63w-left-curved-desk-return-with-pencil-box-file-pedestal-sea-salt>: HTTP status code is not handled or not allowed 2026-01-24 22:43:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/applicator-jar-7-x-3-6-case>: HTTP status code is not handled or not allowed 2026-01-24 22:43:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/versatile-4c-mailbox-4c16s-09-56-12h-9-mailboxes-1-parcel-front-loading-brown-usps>: HTTP status code is not handled or not allowed 2026-01-24 22:43:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sensi-wrap-self-adherent-bandage-wo-rubber-latex-6w-x-5-yards-assorted-12-pcs>: HTTP status code is not handled or not allowed 2026-01-24 22:43:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-phillips-power-bit-12062x-6l-x-1-4-number2-tip-1-4-hexagon-drive>: HTTP status code is not handled or not allowed 2026-01-24 22:43:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vitamins-ad-ointment-1-oz-tube-pack-of-72>: HTTP status code is not handled or not allowed 2026-01-24 22:43:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-torx-power-bit-16032x-1-4-hexagon-t30-drive-1-15-16-long>: HTTP status code is not handled or not allowed 2026-01-24 22:43:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-torx-power-bit-16640x-1-4-hexagon-t40-drive-6-long>: HTTP status code is not handled or not allowed 2026-01-24 22:43:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/abdominal-binder-small-fits-24-x-30-waist-12h-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 22:43:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sae-hex-tip-impact-socket-7441-3-8-1-2-drive-3-long-3-8-hex-tip>: HTTP status code is not handled or not allowed 2026-01-24 22:43:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/home-safe-hs30-with-electronic-lock-1-hour-fire-rating-20-x-20-x-30-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:43:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tm620h-traffic-sign-no-parking-fire-lane-double-arrow-18-x-12-white>: HTTP status code is not handled or not allowed 2026-01-24 22:43:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paper-surgical-tape-2w-x-10-yards-pack-of-72>: HTTP status code is not handled or not allowed 2026-01-24 22:43:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tm062k-traffic-sign-no-parking-fire-lane-tow-away-zone-18-x-12-white>: HTTP status code is not handled or not allowed 2026-01-24 22:43:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/siligentle-silicone-non-adhesive-foam-dressing-6l-x-6w-120-pcs>: HTTP status code is not handled or not allowed 2026-01-24 22:43:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tms323g-traffic-sign-reserved-parking-nevada-18-x-12-white>: HTTP status code is not handled or not allowed 2026-01-24 22:43:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cloth-surgical-tape-2w-x-10-yards-pack-of-72>: HTTP status code is not handled or not allowed 2026-01-24 22:43:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medi-cut-safety-sterile-scalpels-size-11-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 22:43:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dynaride-s3-lite-wheelchair-flip-desk-arm-foot-rest-18w-seat>: HTTP status code is not handled or not allowed 2026-01-24 22:43:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dynaride-s3-lite-wheelchair-flip-desk-elevating-legrest-20w-seat>: HTTP status code is not handled or not allowed 2026-01-24 22:44:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-bariatric-quad-cane-w-large-base-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-24 22:44:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-sheet-panel-truck-30-x-72>: HTTP status code is not handled or not allowed 2026-01-24 22:44:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-tall-hand-truck-w-foot-kick-10-solid-rubber>: HTTP status code is not handled or not allowed 2026-01-24 22:44:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/welded-storage-locker-w-adjustable-center-shelf-mesh-sides-61w-x-33d-x-52h>: HTTP status code is not handled or not allowed 2026-01-24 22:44:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premier-built-in-glass-door-under-counter-freezer-ada-abt-ucbi-0420-ada-4-cuft>: HTTP status code is not handled or not allowed 2026-01-24 22:44:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-hand-held-body-massage-tool-3-point>: HTTP status code is not handled or not allowed 2026-01-24 22:44:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-berm-builder-wall-section-6-feet-feet-2-qty-yellow>: HTTP status code is not handled or not allowed 2026-01-24 22:44:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sl-led-mission-style-outdoor-rectangle-light-fixture-12w-800-lumens-black-finish>: HTTP status code is not handled or not allowed 2026-01-24 22:44:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yellow-quick-mount-barricade-18l-caution-do-not-enter-belt-modified-mount>: HTTP status code is not handled or not allowed 2026-01-24 22:44:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 32188 pages (at 94 pages/min), scraped 14814 items (at 24 items/min) 2026-01-24 22:44:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anatomical-model-loose-bones-leg-skeleton-right>: HTTP status code is not handled or not allowed 2026-01-24 22:44:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/amscope-dust-cover-full-size-standard-microscopes-m>: HTTP status code is not handled or not allowed 2026-01-24 22:44:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/amscope-biology-pathology-prepared-microscope-slides-set-50-pc>: HTTP status code is not handled or not allowed 2026-01-24 22:44:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/amscope-halogen-microscope-bulb-6v-30w>: HTTP status code is not handled or not allowed 2026-01-24 22:44:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anatomical-chart-nervous-system-chart-paper>: HTTP status code is not handled or not allowed 2026-01-24 22:44:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anatomical-model-shorty-the-mini-skeleton-on-hanging-stand>: HTTP status code is not handled or not allowed 2026-01-24 22:44:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/monitor-riser-ktkms500-65-x-20-x-133-black-up-to-50lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:44:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-laser-pointer-qrtmp1200q-class-3a-plastic-black>: HTTP status code is not handled or not allowed 2026-01-24 22:44:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cleaning-wipe-rearr1391-screen-kleen-cleaning-wipe-40-bx>: HTTP status code is not handled or not allowed 2026-01-24 22:44:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stackable-large-storage-cube-black>: HTTP status code is not handled or not allowed 2026-01-24 22:44:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eco-stackable-storage-cube-natural>: HTTP status code is not handled or not allowed 2026-01-24 22:44:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-8482-thermoplastic-32-gallon-mesh-recycling-receptacle-w-rain-bonnet-lid-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:44:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ct-usa-endurance-open-chest-30-inch-x-17-1-2-inch-midnight-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:44:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/39x18x47-trolley-1-perfo-panels-louvered-panel>: HTTP status code is not handled or not allowed 2026-01-24 22:44:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:44:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbk-athletic-work-shoe-ultraknit-flexweave-black-white-7-5w>: HTTP status code is not handled or not allowed 2026-01-24 22:44:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbk-athletic-work-shoe-ultraknit-flexweave-black-white-10w>: HTTP status code is not handled or not allowed 2026-01-24 22:44:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:44:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/77300-emery-cloth-shop-rolls-grit-80-1-x-10-yards>: HTTP status code is not handled or not allowed 2026-01-24 22:44:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fl-casual-work-three-quarter-nubuck-leather-black-9-5d>: HTTP status code is not handled or not allowed 2026-01-24 22:44:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/08806-abrasive-saw-blade-metal-cutting-6-1-2-x-3-32-arbor-5-8>: HTTP status code is not handled or not allowed 2026-01-24 22:44:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-twin-hammer-impact-wrench-up7976-1-drive-4000-rpm-1500-ftlbs-torque>: HTTP status code is not handled or not allowed 2026-01-24 22:44:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss304-60010-1-class-150-90-degree-elbow-stainless-steel-304>: HTTP status code is not handled or not allowed 2026-01-24 22:44:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rp-works-ruched-bootie-full-grain-leather-black-10-5w>: HTTP status code is not handled or not allowed 2026-01-24 22:44:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glassless-mirror-free-standing-3-panel-24w-x-72h-panels>: HTTP status code is not handled or not allowed 2026-01-24 22:44:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss304-63740-4-class-150-cap-stainless-steel-304>: HTTP status code is not handled or not allowed 2026-01-24 22:44:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-24 22:44:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbk-athletic-work-shoe-ultraknit-flexweave-black-8-5w>: HTTP status code is not handled or not allowed 2026-01-24 22:44:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:44:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots>: HTTP status code is not handled or not allowed 2026-01-24 22:44:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heat-lamp-infrared-two-bulb-portable-clamp-on-120v-6004-2>: HTTP status code is not handled or not allowed 2026-01-24 22:44:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ia-6-inch-waterproof-work-boot-waterproof-leather-brown-14m>: HTTP status code is not handled or not allowed 2026-01-24 22:44:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/loading-dock-safety-net-with-posts-4-feet-x-18-feet>: HTTP status code is not handled or not allowed 2026-01-24 22:44:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rp-works-ruched-bootie-full-grain-leather-black-10-5m>: HTTP status code is not handled or not allowed 2026-01-24 22:44:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/horizontal-fixed-sign-frame-50-1141sh-s-mb-8-5-x-11-slotted-matte-black>: HTTP status code is not handled or not allowed 2026-01-24 22:44:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f18933-0011-lab-aire-ii-single-sided-non-electric-benchtop-or-wallmount-dryer-1-tier>: HTTP status code is not handled or not allowed 2026-01-24 22:44:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ia-6-inch-waterproof-work-boot-lime-resistant-full-grain-leather-brown-8-5m>: HTTP status code is not handled or not allowed 2026-01-24 22:44:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ia-11-inch-pull-on-workboot-leather-brown-9w>: HTTP status code is not handled or not allowed 2026-01-24 22:44:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-freestanding-office-partition-panel-24-1-4-w-x-61-1-2-h-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:44:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/freestanding-office-partition-panel-24-1-4-w-x-72-h-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:44:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/restraining-cable-24-inch-for-36-inch-hose-for-dormont-rdc-36>: HTTP status code is not handled or not allowed 2026-01-24 22:44:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slatted-steel-outdoor-ashtray-13-gallon-black>: HTTP status code is not handled or not allowed 2026-01-24 22:44:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 22:44:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oven-valve-1-4-mpt-x-7-16-cc-for-garland-1150399>: HTTP status code is not handled or not allowed 2026-01-24 22:44:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gas-control-for-southbend-1164804>: HTTP status code is not handled or not allowed 2026-01-24 22:44:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uy1818-1a-maintenance-free-quiet-locker-single-tier-18x21x72-1-door-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:44:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ml-lockout-toolbox-premier-electrical-device-assortment-4-zen-padlocks-red>: HTTP status code is not handled or not allowed 2026-01-24 22:44:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-gallon-instant-powder-mix-lemonade>: HTTP status code is not handled or not allowed 2026-01-24 22:44:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-gallon-instant-powder-mix-lemon-lime>: HTTP status code is not handled or not allowed 2026-01-24 22:44:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ironcat-heavyweight-top-grain-pigskin-mig-welding-gloves-natural-xl-all-leather>: HTTP status code is not handled or not allowed 2026-01-24 22:44:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kcr24-steel-locker-accessory-coat-rod-assembly-24>: HTTP status code is not handled or not allowed 2026-01-24 22:44:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/400-series-solid-door-storage-cabinet-36wx18dx72h-parchment-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 22:44:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u3282-3a-me-black-tie-locker-triple-tier-12xx18x24-9-doors-assembled-black>: HTTP status code is not handled or not allowed 2026-01-24 22:44:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/awa282-222-gym-pe-locker-double-tier-36x18x36-assembled>: HTTP status code is not handled or not allowed 2026-01-24 22:44:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/400-series-solid-door-storage-cabinet-48wx24dx72h-midnight-ebony-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 22:44:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u1282-2mp-black-tie-locker-double-tier-12x18x36-2-doors-unassembled-black-light-grey>: HTTP status code is not handled or not allowed 2026-01-24 22:44:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-chrome-2-tier-wire-shelving-add-on-unit-48w-x-30d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 22:44:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inner-ring-for-machined-type-needle-roller-bearing-metric-14mm-bore-17mm-od-17mm-width>: HTTP status code is not handled or not allowed 2026-01-24 22:44:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-chrome-2-tier-wire-shelving-add-on-unit-42w-x-14d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 22:44:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tsv1260-red-12-x-60-shadow-board-red-vinyl-self-adhesive-tape-roll>: HTTP status code is not handled or not allowed 2026-01-24 22:44:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sa-heat-pump-dryer-14-dry-programs-24-inch-wide-208-240v-white>: HTTP status code is not handled or not allowed 2026-01-24 22:44:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/119-01-field-station-1207-cu-ft-steel-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:44:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enclosed-power-control-center-pspw2rb4-4a-breaker-switch-120vac-2-outlets-wires>: HTTP status code is not handled or not allowed 2026-01-24 22:44:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pathcade-barrier-system-w-hip-sheeting-2-side-72-inch-x-3-inch-x-38-inch-orange>: HTTP status code is not handled or not allowed 2026-01-24 22:44:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compact-springless-sign-stand-for-roll-up-signs-36-inch-and-48-inch-powder-coated-orange-steel>: HTTP status code is not handled or not allowed 2026-01-24 22:45:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-gauge-staple-2-length-7-16-crown-304-stainless-steel-pkg-of-5000-made-in-usa>: HTTP status code is not handled or not allowed 2026-01-24 22:45:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stmi-fiberglass-umbrella-72-inch-octagon-valance-8-feet-pole-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:45:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-variable-speed-scroll-saw-dw788-13-amps->: HTTP status code is not handled or not allowed 2026-01-24 22:45:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbwp-6-bushel-antimicrobial-vinyl-basket-truck-w-antimicrobial-bumper-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:45:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aircraft-black-oxide-extended-length-drill-bit-dw1607-1-4-diameter-6-long>: HTTP status code is not handled or not allowed 2026-01-24 22:45:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbwp-8-bushel-ups-fedex-able-antimicrobial-basket-all-swivel-casters-red>: HTTP status code is not handled or not allowed 2026-01-24 22:45:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pistol-case-w-springfield-xd-insert-locks-qf300bklxd-watertight-10-11-16x9-3-4x4-13-16>: HTTP status code is not handled or not allowed 2026-01-24 22:45:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pistol-case-qf345grl-watertight-10-11-16-x9-3-4-x4-13-16-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:45:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbwp-12-bushel-ups-fedex-able-antimicrobial-basket-all-swivel-casters-red>: HTTP status code is not handled or not allowed 2026-01-24 22:45:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-standing-60-l-x-24-w-x-34-to-42-height-workstation-black-granite>: HTTP status code is not handled or not allowed 2026-01-24 22:45:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/floor-marking-aisle-tape-white-3w-x-108l-roll>: HTTP status code is not handled or not allowed 2026-01-24 22:45:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbwp-10-bushel-antimicrobial-vinyl-basket-truck-all-swivel-casters-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:45:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/floor-marking-tape-blue-t-shape-25pkg>: HTTP status code is not handled or not allowed 2026-01-24 22:45:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tt-heel-grounder-35mm-red-velcro-1meg>: HTTP status code is not handled or not allowed 2026-01-24 22:45:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/horizon-mfg-70-pair-safety-glass-dispenser-clear-acrylic-with-clear-lid-5148-w-9-1-4l>: HTTP status code is not handled or not allowed 2026-01-24 22:45:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/floor-marking-tape-orange-t-shape-25pkg>: HTTP status code is not handled or not allowed 2026-01-24 22:45:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 32283 pages (at 95 pages/min), scraped 14831 items (at 17 items/min) 2026-01-24 22:45:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbwp-14-bushel-antimicrobial-vinyl-basket-truck-w-antimicrobial-bumper-red>: HTTP status code is not handled or not allowed 2026-01-24 22:45:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/horizon-mfg-double-bottle-eyewash-station-5182-5-1-2l-10w-10h>: HTTP status code is not handled or not allowed 2026-01-24 22:45:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbwp-14-bushel-vinyl-basket-truck-all-swivel-casters-yellow>: HTTP status code is not handled or not allowed 2026-01-24 22:45:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bath-fan-retrofit-kit-bfrk100-115v-1-ph-100-cfm-4-duct>: HTTP status code is not handled or not allowed 2026-01-24 22:45:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/little-giant-174-aerial-safety-cage-5-9-w-click-casters-18509-240>: HTTP status code is not handled or not allowed 2026-01-24 22:45:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-safety-netting-for-120-in-bay-standard-attachment-390-lbs-black>: HTTP status code is not handled or not allowed 2026-01-24 22:45:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-office-partition-panel-with-window-36-1-4w-x-61-1-2h-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:45:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-office-partition-panel-with-window-48-1-4w-x-61-1-2h-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:45:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-roof-mount-curb-centrifugal-direct-drive-ventilator-rec-6-115v-1-ph-227-cfm>: HTTP status code is not handled or not allowed 2026-01-24 22:45:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/td-1-28-gpf-round-bowl-toilet-19-1-8-inch-w-x-26-3-8-inch-d-x-30-11-16-inch-h-bone>: HTTP status code is not handled or not allowed 2026-01-24 22:45:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ajc-skytech-skytech-sc46-6v-4-5ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-24 22:45:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/td-1-28-gpf-elongated-bowl-toilet-19-1-16-inch-w-x-28-3-8-inch-d-x-30-11-16-inch-h-bone>: HTTP status code is not handled or not allowed 2026-01-24 22:45:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ajc-kung-long-wp3-3-12-12v-3-2ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-24 22:45:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/freestanding-office-partition-panel-with-partial-window-48-1-4-w-x-72-h-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:45:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ajc-gs-portalac-pxl12072-pxl-12072-12v-8ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-24 22:45:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vertical-file-accessory-lock-kit-50-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:45:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thhn-wire-w-non-metallic-connectors-3-4-x-4>: HTTP status code is not handled or not allowed 2026-01-24 22:45:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-x-2-signature-wheel-with-sealed-ball-bearing-for-1-2-axle-400-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:45:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-x-1-1-4-polyolefin-wheel-with-ball-bearing-for-3-8-axle-300-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:45:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-x-1-1-2-urethane-on-iron-wheel-with-roller-bearing-for-1-2-axle-600-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:45:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-rinsing-indoor-concentrate-coil-cleaner-1-gal-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-24 22:45:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-series-6-1-2height-5-gt-wheel-swivel-caster>: HTTP status code is not handled or not allowed 2026-01-24 22:45:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-x-2-urethane-on-iron-wheel-with-roller-bearing-for-1-2-axle-1050-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:45:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/46-series-9-1-2height-8-urethane-polypropylene-wheel-with-total-lock-brake-swivel-caster>: HTTP status code is not handled or not allowed 2026-01-24 22:45:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/comfort-multipoise-gas-furnace-45000-btu-17-1-2-w>: HTTP status code is not handled or not allowed 2026-01-24 22:45:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/comfort-single-stage-ecm-multipoise-gas-furnace-80000-btu-21-w>: HTTP status code is not handled or not allowed 2026-01-24 22:45:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/contactor-30-amp-24v-2-pole>: HTTP status code is not handled or not allowed 2026-01-24 22:45:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-inches-height-adjustable-masonite-stool-with-backrest-gray-frame>: HTTP status code is not handled or not allowed 2026-01-24 22:45:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heat-pump-high-wall-18-seer-36000-btu-3-ton>: HTTP status code is not handled or not allowed 2026-01-24 22:45:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/104431-acrylic-vertical-horizontal-block-frame-3-x-8-sold-each>: HTTP status code is not handled or not allowed 2026-01-24 22:45:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/142726-vertical-top-load-acrylic-sign-holder-4-x-6-acrylic>: HTTP status code is not handled or not allowed 2026-01-24 22:45:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/112760-horizontal-slanted-l-shaped-acrylic-sign-holder-8-5-x-2-5-10-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:45:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/608f-wall-mount-sink-with-faucet>: HTTP status code is not handled or not allowed 2026-01-24 22:45:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/152722-vertical-double-sided-stand-up-sign-holder-5-x-7-acrylic>: HTTP status code is not handled or not allowed 2026-01-24 22:45:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wern-35-feet-vertical-lifeline-w-5-8-inch-poly-blend-rope-snap-hook>: HTTP status code is not handled or not allowed 2026-01-24 22:45:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/162724-vertical-wall-mount-acrylic-sign-holder-3-5-x-5-acrylic>: HTTP status code is not handled or not allowed 2026-01-24 22:45:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/532b-wall-mount-scrub-sink-with-battery-powered-sensor>: HTTP status code is not handled or not allowed 2026-01-24 22:45:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/225533-48-slot-mascara-cosmetic-tray-for-pegboard-slatwall-12-x-2-acrylic>: HTTP status code is not handled or not allowed 2026-01-24 22:45:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/225551-36-compartment-lipstick-tray-for-pegboard-slatwall-6-x-1-25-acrylic>: HTTP status code is not handled or not allowed 2026-01-24 22:45:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/252066-6-pocket-wall-mount-brochure-holder-14-5-x-16-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:45:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-utility-tool-trolley-w-one-drawer-550lb-cap>: HTTP status code is not handled or not allowed 2026-01-24 22:45:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/515430-acrylic-easel-w-1-75-lip-7-x-6-5-acrylic>: HTTP status code is not handled or not allowed 2026-01-24 22:45:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wern-positioning-belt-for-harness-gray-xxl>: HTTP status code is not handled or not allowed 2026-01-24 22:45:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/701180-8-safety-metal-loop-hook-galvanized-50-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:45:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/771322-gre-pegboard-wall-panel-13-5-x-22-green-opaque>: HTTP status code is not handled or not allowed 2026-01-24 22:45:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-306l-royal-series-single-deck-mount-pantry-faucet-6-spout-low-lead>: HTTP status code is not handled or not allowed 2026-01-24 22:45:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700220-blu-12-pegboard-revolving-countertop-display-4-sided-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:45:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-814l-royal-series-3-4-full-flow-wall-mount-faucet-14-spout-low-lead-compliant>: HTTP status code is not handled or not allowed 2026-01-24 22:45:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-121l-royal-series-knee-pedal-valve-low-lead>: HTTP status code is not handled or not allowed 2026-01-24 22:45:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/771660-wht-white-pegboard-wall-panel-16-x-60-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:45:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/belleville-disc-spring-2-od-x-1-id-x-0084-thick-x-0136-oah-1074-carbon-steel>: HTTP status code is not handled or not allowed 2026-01-24 22:45:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/701414-yel-interlocking-pegboard-countertop-display-14-x-20-yellow-opaque>: HTTP status code is not handled or not allowed 2026-01-24 22:45:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/275-piece-sheet-metal-screw-assortment-6-to-1-4-phillips-pan-head-zinc>: HTTP status code is not handled or not allowed 2026-01-24 22:45:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/igg-high-precision-double-square-w-6-inch-steel-blade-black>: HTTP status code is not handled or not allowed 2026-01-24 22:45:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/igg-absolute-origin-indicator-w-remote-readout-0-0003-inch-accuracy>: HTTP status code is not handled or not allowed 2026-01-24 22:45:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/igg-digital-electronic-protractor-w-12-inch-stainless-steel-blade>: HTTP status code is not handled or not allowed 2026-01-24 22:45:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inplant-office-vinyl-cov-drywall-12x12-2-wall-3-lgt-4-dplx-1-dr-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:45:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/igg-digital-caliper-ip54-w-24-inch-range>: HTTP status code is not handled or not allowed 2026-01-24 22:45:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inplant-office-pntd-steel-drywall-10x20-3-wall-3-lgt-4-dplx-1-dr-gry>: HTTP status code is not handled or not allowed 2026-01-24 22:45:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-vsr-pistol-grip-drill-kit-dwd110k-8-amps-650w-0-2500-rpm-single-gear-reduction>: HTTP status code is not handled or not allowed 2026-01-24 22:45:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lithium-ion-compact-battery-pack-15-ah-2-pack-dcb201-2-20vmax>: HTTP status code is not handled or not allowed 2026-01-24 22:45:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/highperformance-bench-grinder-brush-dw4907-8-diameter-5-8-arbor-4000-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:45:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c-clamp-task-lamp-95030-18-flex-arm>: HTTP status code is not handled or not allowed 2026-01-24 22:45:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inplant-office-pntd-steel-drywall-10x10-3-wall-2-lgt-2-dplx-1-dr-gry>: HTTP status code is not handled or not allowed 2026-01-24 22:46:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inplant-office-vinyl-cov-drywall-8x10-2-wall-2-lgt-2-dplx-1-dr-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:46:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-m-xtract-5-inch-pneumatic-random-orbital-sander-self-generated-vacuum-3-8-inch-orbit-12000-rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:46:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-green-wire-shelf-24-x-24>: HTTP status code is not handled or not allowed 2026-01-24 22:46:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-green-wire-ledge-54-x-4>: HTTP status code is not handled or not allowed 2026-01-24 22:46:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cartp536-heavy-duty-mobile-stand-with-locking-casters-36w-x-34h-x-28d>: HTTP status code is not handled or not allowed 2026-01-24 22:46:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-plastic-barrier-chain-1-1-2-inch-x-25-feet-l-slate-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:46:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/work-benches/bench_tops_accessories/work_surface_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:46:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/circuit-board-shipping-and-storage-box-no-foam-23l-x-15-3-8w-x-3-1-4h>: HTTP status code is not handled or not allowed 2026-01-24 22:46:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mounting-kit-with-18-w-louver-36-w-whiteboard-72-w-workbench-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:46:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11-step-aluminum-rolling-ladder-24inw-ribbed-tread-21ind-top-step-32in-handrails>: HTTP status code is not handled or not allowed 2026-01-24 22:46:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-step-aluminum-rolling-ladder-16inw-grip-tread-21ind-top-step-32in-handrails>: HTTP status code is not handled or not allowed 2026-01-24 22:46:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-step-aluminum-rolling-ladder-24inw-grip-tread-21ind-top-step-w-o-handrails>: HTTP status code is not handled or not allowed 2026-01-24 22:46:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esd-open-bin-box-for-reels-and-rail-dip-tubes-4w-x-24d-x-9h>: HTTP status code is not handled or not allowed 2026-01-24 22:46:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 32382 pages (at 99 pages/min), scraped 14854 items (at 23 items/min) 2026-01-24 22:46:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pink-cushion-grade-static-dissipative-foam48l-x-40w-x-1-4h>: HTTP status code is not handled or not allowed 2026-01-24 22:46:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-utility-stakes-in-a-pail-12-inch-h-yellow-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-24 22:46:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-3-inch-heavy-duty-stowable-stanchion-kit-w-2-inch-x-30-feet-l-chain-white-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-24 22:46:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-3-inch-heavy-duty-ground-pole-kit-w-2-inch-x-50-feet-l-chain-black-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-24 22:46:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/work-benches/bench_tops_accessories/work_surface_mats>: HTTP status code is not handled or not allowed 2026-01-24 22:46:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/excelsior-executive-high-back-chair-bm9000-blk-black-fabric-non-adjustable-arms>: HTTP status code is not handled or not allowed 2026-01-24 22:46:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-series-swivel-plate-caster-404109929-brk7-polyurethane-with-brake-4dia-600-lb-cap>: HTTP status code is not handled or not allowed 2026-01-24 22:46:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hydraulic-stacker-step-type-440-lbs-capacity-354-lift>: HTTP status code is not handled or not allowed 2026-01-24 22:46:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dispensing-jug-118500000-hdpe-20-liters-1-pk-clear>: HTTP status code is not handled or not allowed 2026-01-24 22:46:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wash-bottle-116430238-ldpe-250ml-capacity-write-on-3-pk-natural-cap-wide-mouth>: HTTP status code is not handled or not allowed 2026-01-24 22:46:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-series-swivel-plate-caster-404109139-cast-iron-4dia-1000-lb-cap>: HTTP status code is not handled or not allowed 2026-01-24 22:46:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/biohazard-disposal-can-w-lift-up-cover-131970000-pp-fits-bags-14w-x-19h-red>: HTTP status code is not handled or not allowed 2026-01-24 22:46:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wash-bottle-116210016-ldpe-500ml-capacity-12-pk-natural-cap-narrow>: HTTP status code is not handled or not allowed 2026-01-24 22:46:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-8-ne-nylon-insert-lock-nut-grade-2-carbon-steel-zinc-clear-trivalent-coarse-pkg-of-5>: HTTP status code is not handled or not allowed 2026-01-24 22:46:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-2-6-x-5-hex-head-cap-screw-grade-8-partial-thread-unc-alloy-steel-plain>: HTTP status code is not handled or not allowed 2026-01-24 22:46:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-wall-container-24-x-15-x-11-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:46:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scented-exercise-putty-vanilla-tan-xx-light-5-pound>: HTTP status code is not handled or not allowed 2026-01-24 22:46:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bedford-inside-corner-for-moulding-profiles-mic03x03be-3d-x-3h>: HTTP status code is not handled or not allowed 2026-01-24 22:46:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-putty-4-ounce-set-of-6-6-colors>: HTTP status code is not handled or not allowed 2026-01-24 22:46:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-gable-vent-louver-gvro18f-18w-x-18h>: HTTP status code is not handled or not allowed 2026-01-24 22:46:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scented-exercise-putty-banana-yellow-x-light-5-pound>: HTTP status code is not handled or not allowed 2026-01-24 22:46:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microwaveable-exercise-putty-soft-orange-1-pound>: HTTP status code is not handled or not allowed 2026-01-24 22:46:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-5-tools-4-x-5-lb-putties-easy-tan-yellow-red-green>: HTTP status code is not handled or not allowed 2026-01-24 22:46:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rocker-board-wooden-with-carpet-side-to-side-60l-x-30w-x-12h>: HTTP status code is not handled or not allowed 2026-01-24 22:46:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/balance-system-blue-ball-only-level-4-1-each>: HTTP status code is not handled or not allowed 2026-01-24 22:46:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-circular-wobble-rocker-board-2h-green>: HTTP status code is not handled or not allowed 2026-01-24 22:46:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/designer-wood-locker-33365-triple-tier-3-wide-12w-x-15d-x-24h-gray-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 22:46:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-wide-designer-wood-locker-23368-triple-tier-3-wide-15x18x24-maple-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 22:46:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/designer-wood-locker-37361-z-style-3-wide-12w-x-21d-x-72h-cherry-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 22:46:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-unit-storage-shelving-chrome-72w-x-21d-x-86h-14-ft>: HTTP status code is not handled or not allowed 2026-01-24 22:46:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-unit-storage-shelving-chrome-48w-x-21d-x-74h-14-ft>: HTTP status code is not handled or not allowed 2026-01-24 22:46:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/styleview-cart-with-lcd-pivot-sla-powered-2-drawers>: HTTP status code is not handled or not allowed 2026-01-24 22:46:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-pe-foam-roller-with-blue-round-4-dia-x-12l>: HTTP status code is not handled or not allowed 2026-01-24 22:46:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-eva-foam-roller-half-round-6-dia-x-36l>: HTTP status code is not handled or not allowed 2026-01-24 22:46:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-eva-foam-roller-round-6-dia-x-36l>: HTTP status code is not handled or not allowed 2026-01-24 22:46:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/progressive-hand-trainer-6-sheet-refills-blue-heavy>: HTTP status code is not handled or not allowed 2026-01-24 22:46:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/y-foam-roller-6-dia-x-15l-black>: HTTP status code is not handled or not allowed 2026-01-24 22:46:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hand-exerciser-base-1-each>: HTTP status code is not handled or not allowed 2026-01-24 22:46:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exerciser-moderate-green-medium>: HTTP status code is not handled or not allowed 2026-01-24 22:46:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-composite-foam-roller-half-round-6-dia-x-36-case-of-24>: HTTP status code is not handled or not allowed 2026-01-24 22:46:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hand-exerciser-basic-starter-pack-yellow-x-light>: HTTP status code is not handled or not allowed 2026-01-24 22:46:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-composite-foam-roller-round-6-dia-x-36l>: HTTP status code is not handled or not allowed 2026-01-24 22:46:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exerciser-light-red-small>: HTTP status code is not handled or not allowed 2026-01-24 22:46:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/com-pax-ial-explosion-proof-blower-with-15-duct-canister-9538-15-8-dia-1-3hp-900-cfm>: HTTP status code is not handled or not allowed 2026-01-24 22:46:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/venture-tape-ul181b-fx-polypropylene-duct-tape-1599b-silver-2-13-16-x-360-3-mil>: HTTP status code is not handled or not allowed 2026-01-24 22:46:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/molded-vinyl-inflatable-exercise-roll-70-cm-28-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:46:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ball-stabilizer-base-medium-for-65-75-cm-balls>: HTTP status code is not handled or not allowed 2026-01-24 22:46:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inflatable-exercise-jump-ball-blue-22-55-cm>: HTTP status code is not handled or not allowed 2026-01-24 22:46:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48w-x-30d-adjustable-height-assembly-table-on-casters-with-maple-top-black>: HTTP status code is not handled or not allowed 2026-01-24 22:46:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-ton-cap-aluminum-gantry-crane-12-ft-span-adj-height-10-10-min-13-4-max>: HTTP status code is not handled or not allowed 2026-01-24 22:46:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-ton-cap-aluminum-gantry-crane-8-ft-span-adj-height-8-6-min-11-0-max>: HTTP status code is not handled or not allowed 2026-01-24 22:46:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-stp-steel-safe-roll-lad-wgt-act-lock-24-w-perf-stp-org-wa154014p-o>: HTTP status code is not handled or not allowed 2026-01-24 22:46:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-hotel-safe-electronic-lock-w-card-slot-18-w-x-15-d-x-9-h-keyed-alike-off-white>: HTTP status code is not handled or not allowed 2026-01-24 22:46:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/55004-1-z-4-no-hub-cleanout-round-nikaloy-top>: HTTP status code is not handled or not allowed 2026-01-24 22:46:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/geogrid-light-duty-cellular-paving-system-195-square-geol01>: HTTP status code is not handled or not allowed 2026-01-24 22:46:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chemical-classifier-strips-10case>: HTTP status code is not handled or not allowed 2026-01-24 22:46:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4in-x-4mm-shed-resistant-motech-mini-roller-covers-2-pack-24-case-4emtc025d>: HTTP status code is not handled or not allowed 2026-01-24 22:46:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-part-reeved-cross-mount-assembly-model-20655k>: HTTP status code is not handled or not allowed 2026-01-24 22:46:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-part-reeved-cross-parallel-mount-assembly-model-3668ut>: HTTP status code is not handled or not allowed 2026-01-24 22:46:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-part-reeved-cross-parallel-mount-assembly-model-33337903>: HTTP status code is not handled or not allowed 2026-01-24 22:46:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90617-first-aid-refill-kit-50-person-ansi-compliant-class-a>: HTTP status code is not handled or not allowed 2026-01-24 22:46:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/truck-lid-rmtt50lid-1-2-cu-yd>: HTTP status code is not handled or not allowed 2026-01-24 22:46:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/902002-1-1-2b-3-ton-broaching-ratchet-leverage-press>: HTTP status code is not handled or not allowed 2026-01-24 22:46:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/902006-4m-5-ton-ratchet-leverage-press-floor-type>: HTTP status code is not handled or not allowed 2026-01-24 22:46:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/903001-2b-6-ton-compound-leverage-press>: HTTP status code is not handled or not allowed 2026-01-24 22:46:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eagle-electric-portable-compressor-p3124v1-3hp-24-gal-6-cfm-100-psi>: HTTP status code is not handled or not allowed 2026-01-24 22:46:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eagle-electric-portable-compressor-p4125h1-5hp-220v-25-gal-125-cfm-100-psi>: HTTP status code is not handled or not allowed 2026-01-24 22:46:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hand-tool-adapter-aluminum-black-metallic-sw66630>: HTTP status code is not handled or not allowed 2026-01-24 22:46:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stage-pie-unit-with-carpet-for-48w-x-24h-stage-units-black>: HTTP status code is not handled or not allowed 2026-01-24 22:46:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8ab-x-3-4-security-self-drilling-screw-pan-6-lobe-head-410-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:46:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sewn-liquid-bag-filter-nylon-mesh-731-dia-x-325-l-800-micron-snap-band-pkg-qty-50>: HTTP status code is not handled or not allowed 2026-01-24 22:47:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/welded-liquid-bag-filter-polyester-felt-7-1-8-d-x-32-l-3-micron-plastic-flange-pkg-50>: HTTP status code is not handled or not allowed 2026-01-24 22:47:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/welded-liquid-bag-filter-polyester-felt-7-1-8-d-x-32-l-50-micron-plastic-flange-pkg-50>: HTTP status code is not handled or not allowed 2026-01-24 22:47:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-x-1-2-security-sheet-metal-screw-flat-phillips-head-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:47:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-x-3-4-spanner-sheet-metal-screw-pan-head-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:47:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-7-balldriver-screwdrivers-5-64-3-16-10645>: HTTP status code is not handled or not allowed 2026-01-24 22:47:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-x-3-security-sheet-metal-screw-flat-phillips-head-18-8-stainless-steel-pkg-of-50>: HTTP status code is not handled or not allowed 2026-01-24 22:47:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-balldriver-power-bit-10812>: HTTP status code is not handled or not allowed 2026-01-24 22:47:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-16-hex-end-l-wrench-6-long-22710>: HTTP status code is not handled or not allowed 2026-01-24 22:47:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sewn-liquid-bag-filter-polyester-felt-5-1-2-d-x-15-l-1-micron-steel-ring-pkg-50>: HTTP status code is not handled or not allowed 2026-01-24 22:47:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 32479 pages (at 97 pages/min), scraped 14871 items (at 17 items/min) 2026-01-24 22:47:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10ab-x-1-security-sheet-metal-screw-button-torx-head-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:47:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/werner-24-1a-fiberglass-3-section-compact-extension-ladder-d6224-3>: HTTP status code is not handled or not allowed 2026-01-24 22:47:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-16-balldriver-t-handle-13110>: HTTP status code is not handled or not allowed 2026-01-24 22:47:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-32-x-3-4-security-machine-screw-button-torx-head-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:47:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m4-x-0-7-x-12mm-security-cap-screw-button-socket-head-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:47:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-office-partition-panel-with-partial-window-and-pass-thru-cable-48-1-4w-x-47-1-2h-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:47:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-25-torx-tamperproof-key>: HTTP status code is not handled or not allowed 2026-01-24 22:47:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-male-branch-tee-metal-release-collet-5-16-tube-x-3-8-swift-fit>: HTTP status code is not handled or not allowed 2026-01-24 22:47:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sf-swivel-4-position-y-5-32-tube-x-1-4-swift-fit>: HTTP status code is not handled or not allowed 2026-01-24 22:47:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelving-42w-x-24d-x-14h-1-shelf-starter-chrome>: HTTP status code is not handled or not allowed 2026-01-24 22:47:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/needle-valve-12mm-tube-x-3-8-metal-release-collet-knob-adjustmet>: HTTP status code is not handled or not allowed 2026-01-24 22:47:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/male-w-barb-10mm-x-1-4>: HTTP status code is not handled or not allowed 2026-01-24 22:47:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-straight-galvanized-steel-skatewheel-conveyor-15x10x10g-15oaw-10-wpf>: HTTP status code is not handled or not allowed 2026-01-24 22:47:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelving-54w-x-18d-x-14h-1-shelf-starter-chrome>: HTTP status code is not handled or not allowed 2026-01-24 22:47:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelving-48w-x-14d-x-63h-4-shelf-starter-chrome>: HTTP status code is not handled or not allowed 2026-01-24 22:47:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelving-54w-x-24d-x-54h-3-shelf-add-on-chrome>: HTTP status code is not handled or not allowed 2026-01-24 22:47:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelving-72w-x-14d-x-54h-3-shelf-add-on-chrome>: HTTP status code is not handled or not allowed 2026-01-24 22:47:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelving-24w-x-18d-x-54h-3-shelf-starter-chrome>: HTTP status code is not handled or not allowed 2026-01-24 22:47:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelving-54w-x-14d-x-34h-2-shelf-add-on-chrome>: HTTP status code is not handled or not allowed 2026-01-24 22:47:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-male-branch-tee-metal-release-collet-6mm-tube-x-1-8>: HTTP status code is not handled or not allowed 2026-01-24 22:47:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-control-3-8-tube-x-1-4-swift-fit-flow-in-screw-adjustment>: HTTP status code is not handled or not allowed 2026-01-24 22:47:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultrapipe-sock-oil-sediment-for-16inches-diameter-pipe>: HTTP status code is not handled or not allowed 2026-01-24 22:47:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/battery-connector-din-a320-male-connector-with-70mm-pin-contacts>: HTTP status code is not handled or not allowed 2026-01-24 22:47:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/battery-connector-din-a160-male-connector-with-35mm-pin-contacts>: HTTP status code is not handled or not allowed 2026-01-24 22:47:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chemo-trans-bag-12w-15l>: HTTP status code is not handled or not allowed 2026-01-24 22:47:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-48-x-36-adj-height-workbench-w-drawer-gray-maple-safety-edge-top>: HTTP status code is not handled or not allowed 2026-01-24 22:47:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-48-x-36-adj-height-workbench-w-drawer-riser-gray-maple-square-edge-top>: HTTP status code is not handled or not allowed 2026-01-24 22:47:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/i-compact-stirrer-with-telemodul-20-c-controller-100-240v>: HTTP status code is not handled or not allowed 2026-01-24 22:47:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-density-red-infectious-waste-liners-44-gal-36w-48l>: HTTP status code is not handled or not allowed 2026-01-24 22:47:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-step-plastic-step-stand-w-handrails-blue-27-w-x-38-d-x-44-h-st-4-bl>: HTTP status code is not handled or not allowed 2026-01-24 22:47:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-step-plastic-step-stand-yellow-19-1-2-w-x-14-d-x-12-h-st-1-yel>: HTTP status code is not handled or not allowed 2026-01-24 22:47:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multipack-tags-dirty-clean-and-repair-2-5-16w-4-3-4l>: HTTP status code is not handled or not allowed 2026-01-24 22:47:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/karcher-9-807-722-0-4000psi-4gpm-gas-pressure-washer>: HTTP status code is not handled or not allowed 2026-01-24 22:47:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crash-cart-drug-tray-security-covers-20w-29l>: HTTP status code is not handled or not allowed 2026-01-24 22:47:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-epoxy-wire-shelving-189249gz-30w-x-18d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 22:47:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-96-w-x-24-d-x-72-h-starter-3-shelf-levels-steel-deck-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:47:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-digit-desktop-calculator-12004-dual-power-7-1-4-x-6-3-8-x-1-2-black>: HTTP status code is not handled or not allowed 2026-01-24 22:47:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/open-platform-kit-8-x-18-x-54>: HTTP status code is not handled or not allowed 2026-01-24 22:47:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-96-w-x-48-d-x-72-h-starter-3-shelf-levels-wood-deck-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:47:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-readybuilt-locker-36-w-x-12-d-x-78-h-725-hallowell-gray-single-tier-3-wide-assembled>: HTTP status code is not handled or not allowed 2026-01-24 22:47:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-readybuilt-locker-36-w-x-15-d-x-78-h-707-marine-blue-double-tier-3-wide-assembled>: HTTP status code is not handled or not allowed 2026-01-24 22:47:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-readybuilt-locker-36-w-x-15-d-x-78-h-729-parchment-double-tier-3-wide-assembled>: HTTP status code is not handled or not allowed 2026-01-24 22:47:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-800-series-storage-cabinet-36w-x-18d-x-78h-midnight-ebony-single-tier-knockdown>: HTTP status code is not handled or not allowed 2026-01-24 22:47:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/block-heater-4-x-50-ml-conical-tubes>: HTTP status code is not handled or not allowed 2026-01-24 22:47:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/500ml-erlenmeyer-flask-clamp-30156bi-for-use-with-maxq-shaker-platforms>: HTTP status code is not handled or not allowed 2026-01-24 22:47:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-72-w-x-48-d-x-84-h-starter-3-shelf-levels-steel-deck-medium-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:47:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-48-w-x-36-d-x-84-h-add-on-3-shelf-levels-wood-deck-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:47:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-coil-thread-repair-kit-for-metal-10-32-x-15d>: HTTP status code is not handled or not allowed 2026-01-24 22:47:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-coil-thread-repair-kit-for-metal-1-2-20-x-15d>: HTTP status code is not handled or not allowed 2026-01-24 22:47:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-value-max-locker-36-w-x-12-d-x-78-h-725-hallowell-gray-triple-tier-3-wide-assembled>: HTTP status code is not handled or not allowed 2026-01-24 22:47:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/legacy-42-single-pedestal-desk-ash-grey>: HTTP status code is not handled or not allowed 2026-01-24 22:47:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-72-w-x-24-d-x-72-h-starter-3-shelf-levels-wood-deck-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:47:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/legacy-264-modular-racetrack-conference-table-with-3-power-data-grommets-ash-grey>: HTTP status code is not handled or not allowed 2026-01-24 22:47:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-storage-clipboard-21117>: HTTP status code is not handled or not allowed 2026-01-24 22:47:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-48-w-x-48-d-x-84-h-add-on-3-shelf-levels-steel-deck-medium-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:47:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/arts_crafts_supplies/craft_basics already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:47:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-series-wheel-20000291-2-1-2-x-1-1-4-polyurethane-on-polyolefin-3-8-delrin-bushing>: HTTP status code is not handled or not allowed 2026-01-24 22:47:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/avery-8962-inkjet-dvd-labels-matte-white-20pack>: HTTP status code is not handled or not allowed 2026-01-24 22:47:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-hi-tech-metal-bin-shelving-36-w-x-18-d-x-87-h-725-hallowell-gray-starter-unit-36-bins>: HTTP status code is not handled or not allowed 2026-01-24 22:47:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/calcium-lime-rust-remover-42oz>: HTTP status code is not handled or not allowed 2026-01-24 22:47:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/calcium-lime-rust-remover-1-gal>: HTTP status code is not handled or not allowed 2026-01-24 22:47:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-series-wheel-500008559-ws-8-x-2-performa-rubber-1-2-straight-roller-bearing-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:47:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-72-w-x-24-d-x-84-h-add-on-3-shelf-levels-steel-deck-sand>: HTTP status code is not handled or not allowed 2026-01-24 22:47:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-hi-tech-metal-bin-shelving-36-w-x-12-d-x-87-h-725-hallowell-gray-starter-unit-24-bins>: HTTP status code is not handled or not allowed 2026-01-24 22:47:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-hi-tech-metal-bin-shelving-36-w-x-12-d-x-87-h-725-hallowell-gray-starter-unit-38-bins>: HTTP status code is not handled or not allowed 2026-01-24 22:47:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-series-wheel-200005944-5-x-1-1-4-polyurethane-on-polyolefin-3-8-delrin-bushing-maroon>: HTTP status code is not handled or not allowed 2026-01-24 22:47:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/air_compressors/rotary_screw_air_compressors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:48:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/281673-cap-outside-round>: HTTP status code is not handled or not allowed 2026-01-24 22:48:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-platter-for-nsf-listed-primo-scales>: HTTP status code is not handled or not allowed 2026-01-24 22:48:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:48:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/arts_crafts_supplies/craft_basics>: HTTP status code is not handled or not allowed 2026-01-24 22:48:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/advantus-75317-panel-wall-acrylic-frame-11-x-8-12-clear>: HTTP status code is not handled or not allowed 2026-01-24 22:48:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1111144-washer-seat-push-button-gray-for-ts-brass-bronze-works>: HTTP status code is not handled or not allowed 2026-01-24 22:48:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-48-w-x-36-d-x-72-h-add-on-3-shelf-levels-wood-deck-sand>: HTTP status code is not handled or not allowed 2026-01-24 22:48:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2241142-bushing-guide-rod-7-16-inch-id-for-nemco>: HTTP status code is not handled or not allowed 2026-01-24 22:48:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1211007-glide-1-4-20-x-1-inch-1-inch-od>: HTTP status code is not handled or not allowed 2026-01-24 22:48:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ribbed-3-step-18-alum-rolling-ladder-10-top-step-spring-load-a3sh>: HTTP status code is not handled or not allowed 2026-01-24 22:48:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/serrated-5-step-24-alum-rolling-ladder-10-top-step-spring-load-a5sh30g>: HTTP status code is not handled or not allowed 2026-01-24 22:48:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1061183-stem-assembly-dipperwell-leadfree>: HTTP status code is not handled or not allowed 2026-01-24 22:48:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/air_compressors/rotary_screw_air_compressors>: HTTP status code is not handled or not allowed 2026-01-24 22:48:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 32573 pages (at 94 pages/min), scraped 14888 items (at 17 items/min) 2026-01-24 22:48:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1341120-closer-door-4-silver>: HTTP status code is not handled or not allowed 2026-01-24 22:48:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pl-l-lamp-ft18w2g11841-18w-4100k>: HTTP status code is not handled or not allowed 2026-01-24 22:48:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks>: HTTP status code is not handled or not allowed 2026-01-24 22:48:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ironwood-50-Open-folio-storage-maple>: HTTP status code is not handled or not allowed 2026-01-24 22:48:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-96-w-x-48-d-x-72-h-add-on-3-shelf-levels-wood-deck-medium-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:48:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1901340-nut-motor-receptacle-mounting-for-bunn-o-matic>: HTTP status code is not handled or not allowed 2026-01-24 22:48:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precision-high-performance-incubator-mechanical-convection-11-2-cu-ft>: HTTP status code is not handled or not allowed 2026-01-24 22:48:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bookcases_hutches already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:48:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alera-aleva635632my-valencia-series-bookcase-4-shelves-31-34w-x-14d-x-55h-mahogany>: HTTP status code is not handled or not allowed 2026-01-24 22:48:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alera-aleva256624my-valencia-series-credenza-shells-65w-x-23-35d-mahogany>: HTTP status code is not handled or not allowed 2026-01-24 22:48:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-shelf-level-for-bulk-storage-rack-72-w-x-36-d-wood-deck-medium-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:48:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/champion-sports-qs1-ring-toss-set-plasticwood-assorted-colors-4-rings5-pegsset>: HTTP status code is not handled or not allowed 2026-01-24 22:48:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2181300-knob-drive-shaft-for-star-manufacturing>: HTTP status code is not handled or not allowed 2026-01-24 22:48:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-ton-in-line-bottle-jack-shorty-10208>: HTTP status code is not handled or not allowed 2026-01-24 22:48:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/85-1090-white-ptfe-teflon-tape-1-2-inch-x-260-inch>: HTTP status code is not handled or not allowed 2026-01-24 22:48:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/681299-cond-fan-motor115v-6w>: HTTP status code is not handled or not allowed 2026-01-24 22:48:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/credenzas already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:48:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-96-w-x-36-d-x-96-h-starter-3-shelf-levels-wood-deck-medium-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:48:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-brake-drum-handler-46000>: HTTP status code is not handled or not allowed 2026-01-24 22:48:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bookcases_hutches>: HTTP status code is not handled or not allowed 2026-01-24 22:48:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-40-14w-cabinet-11-drawer-162-compart-classic-blue-master-keyed>: HTTP status code is not handled or not allowed 2026-01-24 22:48:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1621154-grate-top-18-inch-for-montague-oven>: HTTP status code is not handled or not allowed 2026-01-24 22:48:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/375x4-fcrb03-child-resistant-flat-bags-3-3-4-x-4-5-mil-black-250-case>: HTTP status code is not handled or not allowed 2026-01-24 22:48:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-40-14w-mobile-cabinet-6-drawers-84-compart-bright-blue-master-keyed>: HTTP status code is not handled or not allowed 2026-01-24 22:48:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-1120-orifice-60>: HTTP status code is not handled or not allowed 2026-01-24 22:48:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bungee-exercise-cord-with-attachments-7-cord-red>: HTTP status code is not handled or not allowed 2026-01-24 22:48:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2631042-o-ring-1-inch-od-3-4-inch-id-blk-for-hobart>: HTTP status code is not handled or not allowed 2026-01-24 22:48:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/credenzas>: HTTP status code is not handled or not allowed 2026-01-24 22:48:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extremity-cuff-strap-with-loop-for-exercise-band-tubing-16-1-each>: HTTP status code is not handled or not allowed 2026-01-24 22:48:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-ae14ep-nexelate-safety-push-handle-14w>: HTTP status code is not handled or not allowed 2026-01-24 22:48:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-28-14w-drawer-cabinet-4-drawer-26-compart-bright-blue-keyed-alike>: HTTP status code is not handled or not allowed 2026-01-24 22:48:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-battery-cable-200102-050-6-gauge-50-ft-roll>: HTTP status code is not handled or not allowed 2026-01-24 22:48:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-al442s-stainless-steel-wire-ledge-42l-x-4h>: HTTP status code is not handled or not allowed 2026-01-24 22:48:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-free-exercise-tubing-silver-25-roll-bag>: HTTP status code is not handled or not allowed 2026-01-24 22:48:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-40-14w-cabinet-16-drawer-270-compart-classic-blue-no-lock>: HTTP status code is not handled or not allowed 2026-01-24 22:48:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8014260-basket-fry-teal-handle-12-1-8-inch-x-6-5-16-inch>: HTTP status code is not handled or not allowed 2026-01-24 22:48:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-free-exercise-tubing-red-100-roll-box-10-5722>: HTTP status code is not handled or not allowed 2026-01-24 22:48:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/w-450dts-18-ptfe-coated-direct-heat-foot-sealer-with-15mm-serrated-seal-width>: HTTP status code is not handled or not allowed 2026-01-24 22:48:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gwsz-400-16-portable-direct-heat-sealer-with-pfte-coated-bars-16mm-seal-width>: HTTP status code is not handled or not allowed 2026-01-24 22:48:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tubing-with-cuff-exerciser-35-circumference-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:48:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1111059-washer-bonnet-push-button-ts-for-ts-brass-bronze-works>: HTTP status code is not handled or not allowed 2026-01-24 22:48:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/441672-relay-blower-3a-for-cres-cor>: HTTP status code is not handled or not allowed 2026-01-24 22:48:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-tin-plated-copper-lugs-5958-050hl-4-0-gauge-50-pcs>: HTTP status code is not handled or not allowed 2026-01-24 22:48:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unisex-jade-cotton-polyester-fashion-blend-reversible-scrub-pants-size-2xl>: HTTP status code is not handled or not allowed 2026-01-24 22:48:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/starter-ground-magnalugs-6401-005f-2-1-gauge-5-pcs>: HTTP status code is not handled or not allowed 2026-01-24 22:48:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-free-exercise-band-pear-level-0-50-yard-roll-box>: HTTP status code is not handled or not allowed 2026-01-24 22:48:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-band-yellow-48-l-band-box-of-40>: HTTP status code is not handled or not allowed 2026-01-24 22:48:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-inplant-office-w-2-windows-class-a-8-w-x-12-d-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:48:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ironwood-2-compartment-wardrobe-storage-cabinet-solid-door-oiled-cherry-color-10>: HTTP status code is not handled or not allowed 2026-01-24 22:48:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ironwood-3-compartment-open-storage-locker-cactus-star-color-finish>: HTTP status code is not handled or not allowed 2026-01-24 22:48:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-free-exercise-band-gold-6-yard-roll-1-roll-box>: HTTP status code is not handled or not allowed 2026-01-24 22:48:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/wiring_devices/wire_clips_clamps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:48:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-dual-post-insulator-terminal-protectors-5713-005b-5-pcs>: HTTP status code is not handled or not allowed 2026-01-24 22:48:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/77164-16-condensation-squeegee-red>: HTTP status code is not handled or not allowed 2026-01-24 22:48:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2491081-magnetron-for-amana-litton>: HTTP status code is not handled or not allowed 2026-01-24 22:48:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/econoco-non-adjustable-ballet-rack-psbb-41w-x-64h-x-22d-anthracite-grey>: HTTP status code is not handled or not allowed 2026-01-24 22:48:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/71258-10-single-blade-ultra-hygiene-bench-squeegee-purple>: HTTP status code is not handled or not allowed 2026-01-24 22:48:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-free-exercise-band-50-yard-roll-box-5-color-set>: HTTP status code is not handled or not allowed 2026-01-24 22:48:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tx4000-esd-cleanroom-apparel-hard-sole-boot-cover-s-navy-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:48:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-free-exercise-band-5-strips-blue-30-strips-box>: HTTP status code is not handled or not allowed 2026-01-24 22:48:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/38-1089-13-32-inch-x-1-1-2-inch-15-amp-fast-acting-fuse-600v>: HTTP status code is not handled or not allowed 2026-01-24 22:48:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/53709-216-2-4-tube-brush-medium-black>: HTTP status code is not handled or not allowed 2026-01-24 22:48:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-free-exercise-band-5-strips-red-30-strips-box>: HTTP status code is not handled or not allowed 2026-01-24 22:49:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009599-motor-blower-for-amana-litton>: HTTP status code is not handled or not allowed 2026-01-24 22:49:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/wiring_devices/wire_clips_clamps>: HTTP status code is not handled or not allowed 2026-01-24 22:49:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vs-232007-anystream-shower-head-w-hand-shower-combination-shower-system>: HTTP status code is not handled or not allowed 2026-01-24 22:49:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/55006-cleaning-pad-holder-yellow>: HTTP status code is not handled or not allowed 2026-01-24 22:49:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s-2007-bn-e2-anystream-multi-function-shower-head>: HTTP status code is not handled or not allowed 2026-01-24 22:49:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-tier-5-door-clear-view-locker-w-digital-lock-12-w-x-18-d-x-66-h-tan-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 22:49:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-free-exercise-band-red-6-yard-roll-box>: HTTP status code is not handled or not allowed 2026-01-24 22:49:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2061300-capacitor-start-for-robot-coupe>: HTTP status code is not handled or not allowed 2026-01-24 22:49:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bungee-exercise-cord-with-attachments-7-cord-gold>: HTTP status code is not handled or not allowed 2026-01-24 22:49:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fieldpro-purofort-boots-plain-toe-15h-size-12-green>: HTTP status code is not handled or not allowed 2026-01-24 22:49:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-free-exercise-band-peach-6-yard-roll-box>: HTTP status code is not handled or not allowed 2026-01-24 22:49:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-band-loop-10-long-gold-10-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:49:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cylinder-pallet-stand-32w-x-48d-x-37-1-4h-12-cylinder-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:49:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 32666 pages (at 93 pages/min), scraped 14908 items (at 20 items/min) 2026-01-24 22:49:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-band-loop-10-long-red-10-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:49:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/barnstead-1-2-size-mixed-bed-with-oxygen-removal-cartridge-d8822-1-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:49:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/seapro-purofort-full-safety-boots-steel-toe-15h-size-4-blueyellow>: HTTP status code is not handled or not allowed 2026-01-24 22:49:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-tier-18-door-clear-view-locker-w-digital-lock-36-w-x-12-d-x-78-h-tan-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 22:49:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/miracool-bandana-assorted-colors-24-pack-no-header-card>: HTTP status code is not handled or not allowed 2026-01-24 22:49:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2271122-decal-for-henny-penny>: HTTP status code is not handled or not allowed 2026-01-24 22:49:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-speed-fan-assembly-kit>: HTTP status code is not handled or not allowed 2026-01-24 22:49:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-double-robe-hook-bright-polished>: HTTP status code is not handled or not allowed 2026-01-24 22:49:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/synthetic-winch-line-with-stainless-steel-thimble-38-dia-x-125l>: HTTP status code is not handled or not allowed 2026-01-24 22:49:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-floor-standing-open-top-waste-receptacle-13-gal>: HTTP status code is not handled or not allowed 2026-01-24 22:49:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/record-storage-open-72x30x84-1>: HTTP status code is not handled or not allowed 2026-01-24 22:49:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-24-x-3-4-machine-screw-round-head-phillips-steel-zinc-ft-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:49:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22-1594-2-inch-black-warmer-steamer-thermostat-knob-with-silver-insert-off-1-10>: HTTP status code is not handled or not allowed 2026-01-24 22:49:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/52-bollard-flex-concrete-installation-yellow-cover-red-tapes>: HTTP status code is not handled or not allowed 2026-01-24 22:49:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-stainless-steel-shelf-24-w-x-8-d>: HTTP status code is not handled or not allowed 2026-01-24 22:49:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/52-bollard-flex-asphalt-installation-white-cover-yellow-tapes>: HTTP status code is not handled or not allowed 2026-01-24 22:49:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gn851-latch-type-toggle-clamp-320evu7-t2-size-320-steel>: HTTP status code is not handled or not allowed 2026-01-24 22:49:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/screws/machine_screws/pan_head_machine_screws already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:49:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/52-bollard-flex-natural-ground-installation-black-cover-red-tapes-2>: HTTP status code is not handled or not allowed 2026-01-24 22:49:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/52-bollard-flex-with-sign-post-concrete-installation-yellow-cover-red-tapes>: HTTP status code is not handled or not allowed 2026-01-24 22:49:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-flexible-expandable-conveyor-steel-skate-wheels-175-lbs-per-foot-1>: HTTP status code is not handled or not allowed 2026-01-24 22:49:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/321814-brush-lettuce-cutter>: HTTP status code is not handled or not allowed 2026-01-24 22:49:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-truck-4-shelves-48-x-24-3000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:49:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-1355-3-8-inch-24-locknut>: HTTP status code is not handled or not allowed 2026-01-24 22:49:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/52-bollard-flex-with-sign-post-asphalt-installation-red-cover-white-tapes>: HTTP status code is not handled or not allowed 2026-01-24 22:49:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-soapstone-pencil-refill-5l-x-12w-x-316h-pack-of-144>: HTTP status code is not handled or not allowed 2026-01-24 22:49:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deck-mop-cotton-5720-size-20>: HTTP status code is not handled or not allowed 2026-01-24 22:49:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/56863-3-gallon-bucket-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:49:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/red-hopper-front-security-bulk-truck-48-cu>: HTTP status code is not handled or not allowed 2026-01-24 22:49:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keyboard-tray-for-series-carts>: HTTP status code is not handled or not allowed 2026-01-24 22:49:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/29352-51-aluminum-handle-green>: HTTP status code is not handled or not allowed 2026-01-24 22:49:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/screws/machine_screws/pan_head_machine_screws landed on page that is not a product page. 2026-01-24 22:49:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-shelf-liner-54w-x-18d-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:49:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/120-clearance-bar-yellow-bar-white-tapes>: HTTP status code is not handled or not allowed 2026-01-24 22:49:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelate-wire-shelf-utility-cart-36-w-x-18-d-2-shelves-800-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:49:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-w-x-36-d-production-workbench-maple-butcher-block-safety-edge-complete-bench-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:49:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-w-x-30-d-production-workbench-esd-laminate-square-edge-complete-bench-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:49:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/29113-nylon-floor-scraper-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:49:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/108-w-so-nylon-gate-arm-cover-black-cover-white-tapes>: HTTP status code is not handled or not allowed 2026-01-24 22:49:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-fiberglass-filter-16-w-x-20-h-x-2-d>: HTTP status code is not handled or not allowed 2026-01-24 22:49:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cat-6-plenum-1000-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:49:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1971157-orifice-lower-burner-49-lp-for-duke-manufacturing>: HTTP status code is not handled or not allowed 2026-01-24 22:49:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-w-x-30-d-production-workbench-maple-butcher-block-square-edge-complete-bench-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:49:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/29625-59-ultra-hygiene-handle-white>: HTTP status code is not handled or not allowed 2026-01-24 22:49:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fir-plywood-premium-grade-crate-premium76-46-59-collapsible-inside-76l-x-46w-x-59>: HTTP status code is not handled or not allowed 2026-01-24 22:49:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010609-fuse-block-for-cleveland-range>: HTTP status code is not handled or not allowed 2026-01-24 22:49:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/small-bussing-cart-w-panels-18-x-36-black-cc2036p03>: HTTP status code is not handled or not allowed 2026-01-24 22:49:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/presto-stationary-tilt-table-hand-operated-1>: HTTP status code is not handled or not allowed 2026-01-24 22:49:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/35874-medium-hand-brush-medium-red>: HTTP status code is not handled or not allowed 2026-01-24 22:49:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-wardrobe-cabinet-with-4-drawers-48-x-24-x-78>: HTTP status code is not handled or not allowed 2026-01-24 22:49:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-rack-120x42x96-starter-unit-blue-orange-6700-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-24 22:49:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/419288-short-handle-scrubbing-brush-stiff-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:49:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cabinet-shop-desk-with-pegboard-riser>: HTTP status code is not handled or not allowed 2026-01-24 22:49:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rolling-plastic-hand-basket-43-liter-red>: HTTP status code is not handled or not allowed 2026-01-24 22:49:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-sign-please-wait-here-the-next-available-cashier-11-x14-w-adapter-1>: HTTP status code is not handled or not allowed 2026-01-24 22:49:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/equipment-stand-with-stainless-undershelf-36-w-x-36-d-2>: HTTP status code is not handled or not allowed 2026-01-24 22:49:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bolted-teardrop-pallet-rack-starter-48inw-x-48ind-x-144inh-6590-lbs-shelf-cap>: HTTP status code is not handled or not allowed 2026-01-24 22:49:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tensacone-topper-yellow-with-black-yellow-diagonal-striped-belt>: HTTP status code is not handled or not allowed 2026-01-24 22:49:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-mesh-security-cage-48x24x36>: HTTP status code is not handled or not allowed 2026-01-24 22:49:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dinex-lidfits-aladdin-8-oz-mug-5-oz-bowl-2000-cs-translucent-dx21259000>: HTTP status code is not handled or not allowed 2026-01-24 22:49:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vinyl-coated-cast-iron-dumbbell-set-with-floor-rack-20-piece-set>: HTTP status code is not handled or not allowed 2026-01-24 22:49:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/orange-bomb-gel-8-ounce-cup>: HTTP status code is not handled or not allowed 2026-01-24 22:50:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hot-power-1-2-gallon>: HTTP status code is not handled or not allowed 2026-01-24 22:50:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-weight-bar-5-piece-set>: HTTP status code is not handled or not allowed 2026-01-24 22:50:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30882-small-utility-brush-medium-green>: HTTP status code is not handled or not allowed 2026-01-24 22:50:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-weight-bar-7-5-lb-black-stripe>: HTTP status code is not handled or not allowed 2026-01-24 22:50:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/robinair-34788-r-134a-recycle-recharging-station>: HTTP status code is not handled or not allowed 2026-01-24 22:50:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-glad-hand-lock-keyed-different>: HTTP status code is not handled or not allowed 2026-01-24 22:50:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-ss-nsf-wall-shelf-single-bar-pot-rack-29>: HTTP status code is not handled or not allowed 2026-01-24 22:50:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aerospec-ss-nsf-wall-shelf-34>: HTTP status code is not handled or not allowed 2026-01-24 22:50:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-level-carton-stand-with-3-dividers>: HTTP status code is not handled or not allowed 2026-01-24 22:50:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-ss-nsf-wall-shelf-17>: HTTP status code is not handled or not allowed 2026-01-24 22:50:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6500-tl-semi-automatic-l-clip-label-applicator>: HTTP status code is not handled or not allowed 2026-01-24 22:50:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ccc10040-socket-set-screw>: HTTP status code is not handled or not allowed 2026-01-24 22:50:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-sectional-stackbin-unit-37-w-x-15-1-2-d-x-6-h-black-5-compartments>: HTTP status code is not handled or not allowed 2026-01-24 22:50:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 32764 pages (at 98 pages/min), scraped 14932 items (at 24 items/min) 2026-01-24 22:50:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stretch-unlined-bib-overall-m-black>: HTTP status code is not handled or not allowed 2026-01-24 22:50:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/550-10-1-digital-chair-scale-with-17w-seat-660-lb-x-0-2-lb>: HTTP status code is not handled or not allowed 2026-01-24 22:50:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stretch-unlined-bib-overall-m-brown>: HTTP status code is not handled or not allowed 2026-01-24 22:50:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ms1-angle-mount-for-6-bore-nfpa-cylinder>: HTTP status code is not handled or not allowed 2026-01-24 22:50:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-poly-oxford-5-in-1-safety-jacket-mt-fluorescent-green>: HTTP status code is not handled or not allowed 2026-01-24 22:50:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ms1-angle-mount-for-2-1-2-bore-nfpa-cylinder>: HTTP status code is not handled or not allowed 2026-01-24 22:50:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50-splash-rescue-tube-royal-bluewhite-10-213-roywhi>: HTTP status code is not handled or not allowed 2026-01-24 22:50:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-80mm-bore-x-360mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 22:50:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/riser-tapered-hardboard-60l-x-18w-x-8h>: HTTP status code is not handled or not allowed 2026-01-24 22:50:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/viseaccessories-wilton14842>: HTTP status code is not handled or not allowed 2026-01-24 22:50:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-1477-stainless-steel-phillips-truss-head-8-32-x-1-2-inch-machine-screw-100-box>: HTTP status code is not handled or not allowed 2026-01-24 22:50:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16h-steps-stages>: HTTP status code is not handled or not allowed 2026-01-24 22:50:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24h-steps-stages>: HTTP status code is not handled or not allowed 2026-01-24 22:50:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/first-responder-bag-navy-10-108-nvy>: HTTP status code is not handled or not allowed 2026-01-24 22:50:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clamps-wilton-22006>: HTTP status code is not handled or not allowed 2026-01-24 22:50:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8ft-long-box-pleat-skirting-24h-stage-black>: HTTP status code is not handled or not allowed 2026-01-24 22:50:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-stage-carpet-96l-x-48w-x-32h-grey>: HTTP status code is not handled or not allowed 2026-01-24 22:50:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clamps-wilton14756>: HTTP status code is not handled or not allowed 2026-01-24 22:50:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fireproof-2-drawer-vertical-file-cabinet-letter-31-9-16-lx17-3-4-wx27-3-4-h-parchment>: HTTP status code is not handled or not allowed 2026-01-24 22:50:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-stage-carpet-96l-x-48w-x-8h-grey>: HTTP status code is not handled or not allowed 2026-01-24 22:50:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kit-clevis-bracket-mount-al-160-for-iso-15552-cylinders>: HTTP status code is not handled or not allowed 2026-01-24 22:50:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-100mm-bore-x-300mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 22:50:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uncoated-fiberglass-welding-blanket-6w-x-6h-24-oz-fabric-white>: HTTP status code is not handled or not allowed 2026-01-24 22:50:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-wire-shelving-36-w-x-18-d-x-63-h>: HTTP status code is not handled or not allowed 2026-01-24 22:50:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-deluxe-throw-line-with-ball-10-222-60>: HTTP status code is not handled or not allowed 2026-01-24 22:50:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fireproof-4-drawer-vertical-safe-in-file-legal-31-9-16-lx20-13-16-wx52-3-4-h-champagne>: HTTP status code is not handled or not allowed 2026-01-24 22:50:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-80mm-bore-x-550mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 22:50:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8ft-long-shirred-pleat-skirting-16h-stage-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:50:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1831149-damper-frame-assembly-for-roundup-food-equipment>: HTTP status code is not handled or not allowed 2026-01-24 22:50:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-63mm-bore-x-660mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 22:50:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smead-supertab-pocket-redrope-5-14-accordion-expansion-letter-10box>: HTTP status code is not handled or not allowed 2026-01-24 22:50:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009490-valve-burner-for-imperial-cooking-equipment>: HTTP status code is not handled or not allowed 2026-01-24 22:50:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rwg557m-axis-cut-resistant-polyurethane-palm-gloves-orange-black-m-1-pair>: HTTP status code is not handled or not allowed 2026-01-24 22:50:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/74-h-nexel-chrome-wire-shelving-36-w-x-18-d>: HTTP status code is not handled or not allowed 2026-01-24 22:50:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stage-pie-unit-carpet-36w-x-16h-stage-units-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:50:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8ft-long-shirred-pleat-skirting-24h-stage-black>: HTTP status code is not handled or not allowed 2026-01-24 22:50:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-100mm-bore-x-150mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 22:50:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/pneumatic_cylinders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:50:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smead-hanging-file-folders-15-tab-11-point-stock-legal-green-25box>: HTTP status code is not handled or not allowed 2026-01-24 22:50:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/416-14-3-composite-adjustable-spreader-lifting-beam-14-ton-capacity-3-hook-spread>: HTTP status code is not handled or not allowed 2026-01-24 22:50:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/414450-j-7015-8-x-13-horizontal-bandsaw-115-230v1-phase>: HTTP status code is not handled or not allowed 2026-01-24 22:50:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-80mm-bore-x-50mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 22:50:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-poly-oxford-5-in-1-safety-jacket-xs-fluorescent-green>: HTTP status code is not handled or not allowed 2026-01-24 22:50:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/261792-burner-jetbrass-71-for-groen>: HTTP status code is not handled or not allowed 2026-01-24 22:50:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stage-pie-unit-carpet-36w-x-8h-stage-units-grey>: HTTP status code is not handled or not allowed 2026-01-24 22:50:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-poly-oxford-5-in-1-safety-jacket-4xlt-fluorescent-green>: HTTP status code is not handled or not allowed 2026-01-24 22:50:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-ripstop-4-in-1-safety-jacket-2xl-fluorescent-orange>: HTTP status code is not handled or not allowed 2026-01-24 22:50:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14awg-2c-stranded-control-cable-cmr-500-ft-box-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:50:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/pneumatic_cylinders>: HTTP status code is not handled or not allowed 2026-01-24 22:50:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/type-r-class-2-two-tone-surveyor-safety-vest-4xl-green-sv6g4x>: HTTP status code is not handled or not allowed 2026-01-24 22:50:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/moist-heat-pack-cover-foam-filled-terry-oversize-24-5-x-36-12-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:50:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stage-set-hardboard-96l-x-48w-8h-16h-24h-two-guard-rails>: HTTP status code is not handled or not allowed 2026-01-24 22:50:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compress-large-10-x-13>: HTTP status code is not handled or not allowed 2026-01-24 22:50:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/needle-valve-89973-08-08-screw-adj-12-tube-x-12-swift-fit-metallic-collet>: HTTP status code is not handled or not allowed 2026-01-24 22:50:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foot-plate-package-of-4>: HTTP status code is not handled or not allowed 2026-01-24 22:50:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1502528-pump-mega-kit-w-5-caps-for-san-jamar>: HTTP status code is not handled or not allowed 2026-01-24 22:50:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-male-connector-60000-05-04-516-tube-x-14-swift-fit-univ-thread-ss>: HTTP status code is not handled or not allowed 2026-01-24 22:50:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/type-r-class-2-heavy-duty-engineer-vest-green-l-sv55-2zgd-l>: HTTP status code is not handled or not allowed 2026-01-24 22:50:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8016225-sweep-38-inch-for-kolpak-refrigeration>: HTTP status code is not handled or not allowed 2026-01-24 22:50:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-wire-shelf-instrument-cart-60x24-3-shelves-1200-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:50:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-stage-carpet-96l-x-48w-x-24h-red>: HTTP status code is not handled or not allowed 2026-01-24 22:50:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-quilted-safety-freezer-jacket-5xl-fluorescent-green>: HTTP status code is not handled or not allowed 2026-01-24 22:50:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-ripstop-fleece-lined-safety-jacket-m-black>: HTTP status code is not handled or not allowed 2026-01-24 22:50:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-ripstop-fleece-lined-safety-jacket-s-black>: HTTP status code is not handled or not allowed 2026-01-24 22:50:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reducer-82280n-02-32-18-male-nptf-x-1032-unf-female-thread-nickel-plated-brass>: HTTP status code is not handled or not allowed 2026-01-24 22:51:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wallplate-1-g-gfci-ss>: HTTP status code is not handled or not allowed 2026-01-24 22:51:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insulated-cold-gear-coverall-m-black>: HTTP status code is not handled or not allowed 2026-01-24 22:51:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chefs-trousers-4x-hounds-tooth>: HTTP status code is not handled or not allowed 2026-01-24 22:51:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speedy-board-dock-board-6>: HTTP status code is not handled or not allowed 2026-01-24 22:51:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/321876-filter-hose-2-ftw-1-2-inch-fittings>: HTTP status code is not handled or not allowed 2026-01-24 22:51:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-additional-level-wide-span-rack-60x48-wood-deck-1200-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:51:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-electric-heating-pad-dry-heat-3-settings-small-12-x-15>: HTTP status code is not handled or not allowed 2026-01-24 22:51:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-extra-shelving-48x12x96-with-5-shelves-wire-deck-1>: HTTP status code is not handled or not allowed 2026-01-24 22:51:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1591182-bx62-goodyear-belt>: HTTP status code is not handled or not allowed 2026-01-24 22:51:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-electric-heating-pad-moist-or-dry-heat-3-settings-small-12-x-15>: HTTP status code is not handled or not allowed 2026-01-24 22:51:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 32861 pages (at 97 pages/min), scraped 14955 items (at 23 items/min) 2026-01-24 22:51:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-z-brite-wire-shelving-24-w-x-18-d-x-74-h>: HTTP status code is not handled or not allowed 2026-01-24 22:51:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-coupler-80621-04-14-x-14-male-nptf>: HTTP status code is not handled or not allowed 2026-01-24 22:51:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-quilted-safety-freezer-jacket-xs-black>: HTTP status code is not handled or not allowed 2026-01-24 22:51:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/needle-valve-55920-5-14-screw-adj-5mm-tube-x-14-swift-fit-composite-body>: HTTP status code is not handled or not allowed 2026-01-24 22:51:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-additional-level-wide-span-rack-60w-x-36d-no-deck-1200-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:51:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-run-tee-55222-10-12-10mm-tube-x-12-swift-fit-univ-thread-composite-body>: HTTP status code is not handled or not allowed 2026-01-24 22:51:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-control-85963-53-02-screw-adj-532-tube-x-18-swift-fit-composite-body>: HTTP status code is not handled or not allowed 2026-01-24 22:51:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-1-4-flexi-spring-end>: HTTP status code is not handled or not allowed 2026-01-24 22:51:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-black-urethane-reusable-cold-pack-standard-11-x-14-12-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:51:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-50mm-bore-x-480mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 22:51:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1171348-control-faucet-thft-prf-leadfr>: HTTP status code is not handled or not allowed 2026-01-24 22:51:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-flexi-closet-auger>: HTTP status code is not handled or not allowed 2026-01-24 22:51:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-control-85953-08-08-screw-adj-12-tube-x-12-swift-fit-composite-body>: HTTP status code is not handled or not allowed 2026-01-24 22:51:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compression-sleeve-small>: HTTP status code is not handled or not allowed 2026-01-24 22:51:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-4-drain-flush-bag>: HTTP status code is not handled or not allowed 2026-01-24 22:51:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-32mm-bore-x-420mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 22:51:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-32mm-bore-x-400mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 22:51:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-32mm-bore-x-125mm-stroke>: HTTP status code is not handled or not allowed 2026-01-24 22:51:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-16>: HTTP status code is not handled or not allowed 2026-01-24 22:51:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beige-outdoor-ashtray-1-gallon>: HTTP status code is not handled or not allowed 2026-01-24 22:51:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-black-urethane-reusable-cold-pack-oversize-12-1-2-x-18-1-2>: HTTP status code is not handled or not allowed 2026-01-24 22:51:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reusable-hot-and-cold-compress-8-x-14-case-of-12>: HTTP status code is not handled or not allowed 2026-01-24 22:51:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-control-85953-06-06-screw-adj-38-tube-x-38-swift-fit-composite-body>: HTTP status code is not handled or not allowed 2026-01-24 22:51:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wraps-6-x-18-blue-package-of-3>: HTTP status code is not handled or not allowed 2026-01-24 22:51:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vollrath-bell-creamer-5-oz>: HTTP status code is not handled or not allowed 2026-01-24 22:51:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-expandable-add-on-rack-48x12x84-gray-with-3-level-wood-deck-1500lb-cap-per-level>: HTTP status code is not handled or not allowed 2026-01-24 22:51:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-48-w-x-18-d-x-63-h>: HTTP status code is not handled or not allowed 2026-01-24 22:51:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/needle-valve-85973-04-02-screw-adj-14-tube-x-18-swift-fit-composite-body>: HTTP status code is not handled or not allowed 2026-01-24 22:51:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/moto-vest-4-pockets-cottonpolyester-4xl-dark-brown>: HTTP status code is not handled or not allowed 2026-01-24 22:51:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/male-y-55320-12-38-12mm-tube-x-38-swift-fit-univ-thread-composite-body>: HTTP status code is not handled or not allowed 2026-01-24 22:51:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1031234-coupling-disconnect-fem-1-2-inch-for-ultrafryer>: HTTP status code is not handled or not allowed 2026-01-24 22:51:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-galvanized-steel-shelving-48-w-x-24-d-x-63-h>: HTTP status code is not handled or not allowed 2026-01-24 22:51:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-male-connector-60020-4-18-4mm-tube-x-18-bspp-thread-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 22:51:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eagle-bench-can-metal-red-2-qt>: HTTP status code is not handled or not allowed 2026-01-24 22:51:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-galvanized-steel-shelving-72-w-x-18-d-x-74-h>: HTTP status code is not handled or not allowed 2026-01-24 22:51:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eagle-bench-can-metal-red-8-qt>: HTTP status code is not handled or not allowed 2026-01-24 22:51:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eagle-poly-acid-corrosive-cabinet-manual-close-44-gallon>: HTTP status code is not handled or not allowed 2026-01-24 22:51:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eagle-dot-approved-transport-can-5-8flexable-hose-type-ii-red-5-gal>: HTTP status code is not handled or not allowed 2026-01-24 22:51:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/38-inch-id-x-58-inch-od-x-0001-inch-steel-arbor-shim-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 22:51:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009425-heating-element-120v-1700w-for-carter-hoffmann>: HTTP status code is not handled or not allowed 2026-01-24 22:51:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/34-inch-id-x-1-18-inch-od-steel-arbor-spacer-assortment>: HTTP status code is not handled or not allowed 2026-01-24 22:51:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flap-disc-967a-4-1-2-inch-x-7-8-inch-40-grit-ceramic-grain-type-27-wheel>: HTTP status code is not handled or not allowed 2026-01-24 22:51:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eagle-flammable-liquid-tower-safety-cabinet-self-close-12-gallon>: HTTP status code is not handled or not allowed 2026-01-24 22:51:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-heavy-duty-shelving-36x18x72-with-4-shelves-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 22:51:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hydraulic-raise-n-roll-machinery-dolly-10000-lb-capacity-pair>: HTTP status code is not handled or not allowed 2026-01-24 22:51:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cargo-load-binder-hydraulic-raise-n-roll-machinery-dolly>: HTTP status code is not handled or not allowed 2026-01-24 22:51:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-step-roll-fold-steel-rolling-ladder-perforated-tread>: HTTP status code is not handled or not allowed 2026-01-24 22:51:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/264040-tube-overflow>: HTTP status code is not handled or not allowed 2026-01-24 22:51:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-step-roll-fold-steel-rolling-ladder-perforated-tread>: HTTP status code is not handled or not allowed 2026-01-24 22:51:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plug-ind-grd-15a-125v-5-15p>: HTTP status code is not handled or not allowed 2026-01-24 22:51:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0005-inch-brass-shim-stock-12-inch-x-120-inch-roll>: HTTP status code is not handled or not allowed 2026-01-24 22:51:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-branch-tee-60210-04-04-14-tube-x-14-swift-fit-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 22:51:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-extra-heavy-duty-boltless-shelving-add-on-48-in-w-x-24-in-d-x-72-in-h>: HTTP status code is not handled or not allowed 2026-01-24 22:51:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/finish-thompson-40-drum-pump-kit-ss-pump-tube-tefc-115v-motor>: HTTP status code is not handled or not allowed 2026-01-24 22:51:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-extra-hd-boltless-shelving-add-on-36-in-w-x-18-in-d-x-72-in-h-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 22:51:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22mm-x-40mm-metric-keystock-plain-finish-1-meter-length>: HTTP status code is not handled or not allowed 2026-01-24 22:51:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/union-55040-10-6-10mm-tube-x-6mm-tube-composite-body>: HTTP status code is not handled or not allowed 2026-01-24 22:51:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-pair-of-brake-casters-for-412559>: HTTP status code is not handled or not allowed 2026-01-24 22:51:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-branch-tee-60215-10-14-10mm-tube-x-14-male-bspp-thread-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 22:51:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1131014-hook-wall-pre-rinse>: HTTP status code is not handled or not allowed 2026-01-24 22:51:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-additional-shelf-level-boltless-wood-deck-32>: HTTP status code is not handled or not allowed 2026-01-24 22:51:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/locking-plug-30a-3ph-480v-l16-30pb-w>: HTTP status code is not handled or not allowed 2026-01-24 22:51:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulkhead-union-60050-06-38-tube-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 22:52:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultrasonic-thickness-gauge--nist-0004-resolution>: HTTP status code is not handled or not allowed 2026-01-24 22:52:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/side-mount-above-ground-post-for-weekend-away-vault-dvjr0060-dvjr0060smpa-sand>: HTTP status code is not handled or not allowed 2026-01-24 22:52:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/truflate-plug-80291-04-14-x-14-male-nptf>: HTTP status code is not handled or not allowed 2026-01-24 22:52:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0889373-rk-957957rpda-rt-2-12-4-total-rubber-kit>: HTTP status code is not handled or not allowed 2026-01-24 22:52:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50054-500-plastic-barrier-chain-hdpe-2-x500-8-51mm-green>: HTTP status code is not handled or not allowed 2026-01-24 22:52:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0794107-lfrk-909-s-4-1st-or-2nd-check-seat>: HTTP status code is not handled or not allowed 2026-01-24 22:52:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1271154-door-gasket-13-1-2-x-20-3-4-od>: HTTP status code is not handled or not allowed 2026-01-24 22:52:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-piece-metric-steel-shim-stock-assortment-150mm-x-300mm-sheets>: HTTP status code is not handled or not allowed 2026-01-24 22:52:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deposit-vault-mailbox-and-parcel-drop-dvcs0020-free-standing-rear-access-white>: HTTP status code is not handled or not allowed 2026-01-24 22:52:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-184-dp-1ph-1740rpm-m184k17dk13aa>: HTTP status code is not handled or not allowed 2026-01-24 22:52:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 32957 pages (at 96 pages/min), scraped 14978 items (at 23 items/min) 2026-01-24 22:52:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/top-mount-above-ground-post-for-weekend-away-mail-protector-dvjr0060-dvcs0070-dvjr0060pa-sd>: HTTP status code is not handled or not allowed 2026-01-24 22:52:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/360-series-high-back-seat-with-arm-rests--integrated-suspension-vinyl-black>: HTTP status code is not handled or not allowed 2026-01-24 22:52:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1631021-beater-flat-7-qt-stainless-steel-nsf-for-kitchen-aid>: HTTP status code is not handled or not allowed 2026-01-24 22:52:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/51054-100-heavy-duty-plastic-barrier-chain-hdpe-2-x100-54mm-green>: HTTP status code is not handled or not allowed 2026-01-24 22:52:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m8-x-1-25-x-16mm-machine-screw-phillips-pan-head-304-stainless-steel-din-7985-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:52:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-male-elbow-60110-06-08-38-tube-x-12-swift-fit-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 22:52:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-rack-safety-strap-for-144in-bay-j-hook-attachment-yellow>: HTTP status code is not handled or not allowed 2026-01-24 22:52:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36x24x74-stainless-steel-solid-shelving>: HTTP status code is not handled or not allowed 2026-01-24 22:52:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/union-elbow-60130-04-14-tube-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 22:52:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/union-elbow-60130-06-38-tube-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 22:52:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-poster-stand-sign-holder-11-in-w-x-7-in-h-insert-black-chrome>: HTTP status code is not handled or not allowed 2026-01-24 22:52:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-56h-dp-1ph-1725rpm-p6k17dk12gg>: HTTP status code is not handled or not allowed 2026-01-24 22:52:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-56-tefc-1ph-1725rpm-m6c17fc13jj>: HTTP status code is not handled or not allowed 2026-01-24 22:52:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/superior-graphite-30732r-seed-slik-talc-8-pound-pail>: HTTP status code is not handled or not allowed 2026-01-24 22:52:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m5-x-0-8-x-16mm-phillips-oval-head-machine-screw-304-stainless-steel-din-966-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:52:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lineshaft-90-deg-slave-curve-lssc19x16-16-3-90-t-2-12-1-1116-dia-16bf>: HTTP status code is not handled or not allowed 2026-01-24 22:52:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-branch-tee-89210-08-06-12-tube-x-38-swift-fit-metallic-collet>: HTTP status code is not handled or not allowed 2026-01-24 22:52:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/refuel-300-industrial-led-headlamp-300-lumens-89m-beam-distance-green>: HTTP status code is not handled or not allowed 2026-01-24 22:52:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-7074-cut-resistant-gloves-polyurethane-coated-ansi-a2-xs-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:52:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m2-5-x-0-45-x-16mm-socket-head-cap-screw-304-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:52:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b60h-part-ss-worm-gear-hose-clamp-3-516-inch-4-14-inch-clamping-diameter-box-of-10>: HTTP status code is not handled or not allowed 2026-01-24 22:52:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-20-ga-36-wx18-dx85-h-closed-clip-style-5-shelf-starter>: HTTP status code is not handled or not allowed 2026-01-24 22:52:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/220-piece-stainless-steel-set-screw-assortment>: HTTP status code is not handled or not allowed 2026-01-24 22:52:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-20-ga-48-wx30-dx85-h-closed-clip-style-5-shelf-add-on>: HTTP status code is not handled or not allowed 2026-01-24 22:52:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011040-door-gasket-26-1-8-inch-x-26-3-4-inch-for-turbo-air-refrigeration>: HTTP status code is not handled or not allowed 2026-01-24 22:52:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-tecgen-flame-resistant-work-shirt-3xl-short-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:52:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/45-x-30-letterboard-top-hinged-single-door-message-center-cedar-frame>: HTTP status code is not handled or not allowed 2026-01-24 22:52:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1191102-leg-4-inch-h-stainless-steel-5-8-11>: HTTP status code is not handled or not allowed 2026-01-24 22:52:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd8sn-916-inch-band-heavy-duty-4-pc-hose-clamp-716-inch-1-inch-clamping-diam-10-pcs>: HTTP status code is not handled or not allowed 2026-01-24 22:52:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eco-products-compostable-sugarcane-dinnerware-12-oz-bowl-white-1000carton>: HTTP status code is not handled or not allowed 2026-01-24 22:52:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-gallon-dome-lid-square-waste-container-beige-with-stainless-steel-horz-lines-panels>: HTTP status code is not handled or not allowed 2026-01-24 22:52:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/solo-cup-company-bistro-design-hot-drink-cups-paper-16-oz-maroon-300carton>: HTTP status code is not handled or not allowed 2026-01-24 22:52:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lid-3-4-cu-yd-gray-self-dumping-hopper>: HTTP status code is not handled or not allowed 2026-01-24 22:52:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/l-5550-123-7q-cord-reel-quad-outlet-box>: HTTP status code is not handled or not allowed 2026-01-24 22:52:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-145-tefc-1ph-1740rpm-p145k17fc2mm>: HTTP status code is not handled or not allowed 2026-01-24 22:52:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qr28h-quick-release-part-ss-hose-clamp-1-inch-2-14-inch-clamping-diam-10-pcs>: HTTP status code is not handled or not allowed 2026-01-24 22:52:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lite-insulated-drum-heater-pbl15-15-gallon-capacity-145f-fixed>: HTTP status code is not handled or not allowed 2026-01-24 22:52:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-tecgen-flame-resistant-work-shirt-m-short-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:52:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-kit-for-electric-pallet-truck-gwk-102tf-lw-fits-raymond-model-102tf>: HTTP status code is not handled or not allowed 2026-01-24 22:52:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-kit-for-electric-pallet-truck-gwk-w40xt-xl-lw-fits-hyster-model-w40-xt>: HTTP status code is not handled or not allowed 2026-01-24 22:52:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-partition-flexible-seven-panel-2>: HTTP status code is not handled or not allowed 2026-01-24 22:52:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/warning-line-system-galvanized-steel-4-stanchions-and-100-pennants>: HTTP status code is not handled or not allowed 2026-01-24 22:52:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/100-perimeter-warning-line-pennants-replacement-yellow-plastic-nylon>: HTTP status code is not handled or not allowed 2026-01-24 22:52:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-56-tefc-1ph-3450rpm-c6c34fk36ff>: HTTP status code is not handled or not allowed 2026-01-24 22:52:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-56-tefc-3ph-1750-1450rpm-c6t17fb150bb>: HTTP status code is not handled or not allowed 2026-01-24 22:52:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-waterproof-hard-case-w-p-tear-foam-11-l-x-9-11-16-w-x-4-3-16-h-black>: HTTP status code is not handled or not allowed 2026-01-24 22:52:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-partition-flexible-seven-panel-12>: HTTP status code is not handled or not allowed 2026-01-24 22:52:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nswduf251jwj-1-white-diamond-series-ultra-low-temperature-freezer-25-cu-ft-230v>: HTTP status code is not handled or not allowed 2026-01-24 22:52:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conductivity-benchtop-meter-standard-kit-vstar22>: HTTP status code is not handled or not allowed 2026-01-24 22:52:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-flame-resistant-helix-work-shirt-4xl-regular-dark-khaki>: HTTP status code is not handled or not allowed 2026-01-24 22:52:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/antimicrobial-vinyl-medical-stool-with-backrest-beige>: HTTP status code is not handled or not allowed 2026-01-24 22:52:52 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-24 22:52:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-flame-resistant-dh-air153-work-shirt-5xl-regular-navy>: HTTP status code is not handled or not allowed 2026-01-24 22:52:53 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:15 2026-01-24 22:52:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sets-irwin-vise-grip-2078705>: HTTP status code is not handled or not allowed 2026-01-24 22:52:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-plastic-box-truck-14-bushel-medium-duty>: HTTP status code is not handled or not allowed 2026-01-24 22:52:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thread-repair-kit-for-metal-standard-wall-8-32-x-5-16-18-ez-329-008>: HTTP status code is not handled or not allowed 2026-01-24 22:52:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/solo-cup-company-hot-cup-lids-fits-8-oz-paper-hot-cups-white-1000carton>: HTTP status code is not handled or not allowed 2026-01-24 22:52:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-strippers-irwin-vise-grip-2078309>: HTTP status code is not handled or not allowed 2026-01-24 22:52:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thread-repair-kit-for-metal-thin-wall-5-16-24-x-7-16-14-ez-310-524>: HTTP status code is not handled or not allowed 2026-01-24 22:52:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thread-repair-kit-for-metal-thin-wall-10-24-x-5-16-18-ez-310-3>: HTTP status code is not handled or not allowed 2026-01-24 22:53:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-tecgen-flame-resistant-work-shirt-xl-long-tall-light-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:53:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thread-repair-kit-for-metal-standard-wall-1-2-20-x-3-4-10-ez-329-820>: HTTP status code is not handled or not allowed 2026-01-24 22:53:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s9172-14w-led-s14-ceramic-blue-medium-base>: HTTP status code is not handled or not allowed 2026-01-24 22:53:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/182060-rfx-lp600-wireless-solar-wheel-load-6-scale-system-120000-lb-x-20-lb>: HTTP status code is not handled or not allowed 2026-01-24 22:53:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/american-forge-and-foundry-bench-grinder-stand>: HTTP status code is not handled or not allowed 2026-01-24 22:53:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1291241-kason-67002001020-filter-baffle-for-kason>: HTTP status code is not handled or not allowed 2026-01-24 22:53:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/line-voltage-programmable-thermostat-k302pe-double-pole-heat-only-120-208-240v-15a>: HTTP status code is not handled or not allowed 2026-01-24 22:53:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bin-nestable-clear-5>: HTTP status code is not handled or not allowed 2026-01-24 22:53:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-piece-brass-shim-stock-assortment-6-inch-x-12-inch-sheets>: HTTP status code is not handled or not allowed 2026-01-24 22:53:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vci-co-extruded-gusseted-bags-40-x-27-x-60-3-mil-50-bags-per-roll>: HTTP status code is not handled or not allowed 2026-01-24 22:53:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rotom-45-5dvr-45-5-mfd-370-440-volts-run-round>: HTTP status code is not handled or not allowed 2026-01-24 22:53:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/010mm-steel-shim-stock-150mm-x-25m-roll>: HTTP status code is not handled or not allowed 2026-01-24 22:53:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/red-baked-epoxy-steel-step-on-can-16-quart>: HTTP status code is not handled or not allowed 2026-01-24 22:53:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 33055 pages (at 98 pages/min), scraped 15004 items (at 26 items/min) 2026-01-24 22:53:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-flame-resistant-zip-front-hoodie-4xl-regular-navy>: HTTP status code is not handled or not allowed 2026-01-24 22:53:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-gantry-crane-adjustable-height-pneumatic-casters-1-500-lb-capacity-1>: HTTP status code is not handled or not allowed 2026-01-24 22:53:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exo2-migr-05-xlmotor-impact-glove>: HTTP status code is not handled or not allowed 2026-01-24 22:53:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1201115-caster-plate-5-inch-rgd-gry>: HTTP status code is not handled or not allowed 2026-01-24 22:53:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1021172-drain-lock-floor-35-inch-rd-josam>: HTTP status code is not handled or not allowed 2026-01-24 22:53:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/550va-compact-low-profile-standby-8-outlet-db9>: HTTP status code is not handled or not allowed 2026-01-24 22:53:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exo2-pig-03-mproject-impact-glove>: HTTP status code is not handled or not allowed 2026-01-24 22:53:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wstk-02-sstick-welder-glove>: HTTP status code is not handled or not allowed 2026-01-24 22:53:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-12-inch-id-x-2-18-inch-od-steel-arbor-shim-assortment>: HTTP status code is not handled or not allowed 2026-01-24 22:53:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-3371-4-inch-swivel-stem-caster-for-1-5-8-inch-od-tubing-250-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:53:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/isobar-6-outlet-10ft-1790-joules>: HTTP status code is not handled or not allowed 2026-01-24 22:53:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/litefit-standard-harness-pass-through-legs-s>: HTTP status code is not handled or not allowed 2026-01-24 22:53:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forklift-jack-4-ton-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:53:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carriage-bolt-12-13-1-34-round-head-18-8-stainless-steel-unc-pkg-of-50-781402>: HTTP status code is not handled or not allowed 2026-01-24 22:53:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-hand-lever-actuated-valve-spring-centered-closed-center-1-8-nptf-ports>: HTTP status code is not handled or not allowed 2026-01-24 22:53:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-inch-plastic-classroom-chair-set-of-five-red>: HTTP status code is not handled or not allowed 2026-01-24 22:53:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2331007-gasket-ref-20-1-4-x-62-1-4-for-star-manufacturing>: HTTP status code is not handled or not allowed 2026-01-24 22:53:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-hand-lever-actuated-valve-1-4-nptf-ports>: HTTP status code is not handled or not allowed 2026-01-24 22:53:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-carabiner-2>: HTTP status code is not handled or not allowed 2026-01-24 22:53:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vacuum-hose-18-gallon-wet-dry-squeegee-vacuum>: HTTP status code is not handled or not allowed 2026-01-24 22:53:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/13-panel-divider-6-ft-high-x-24-ft-1-inch-long-fabric-summer-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:53:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/321894-gh-26120zh-large-spindle-bore-lathe-w-newall-dp700-dro-taper-attachment>: HTTP status code is not handled or not allowed 2026-01-24 22:53:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11-panel-divider-7-ft-4-inch-high-x-20-ft-5-in-long-fabric-summer-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:53:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5674021-bandsaw-blade-1-x-035-x-135-x-6-10vt-bi-metal>: HTTP status code is not handled or not allowed 2026-01-24 22:53:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ccgrc-led-combo-exitemergency-unit-w-remote-capacity-green-letters-white-ni-cad-battery>: HTTP status code is not handled or not allowed 2026-01-24 22:53:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lyon-bench-height-modular-storage-drawer-cabinet-7>: HTTP status code is not handled or not allowed 2026-01-24 22:53:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cwc1rw-ct-led-combo-exitemerg-light-wet-listed-red-letters-white-one-face-cold-temp>: HTTP status code is not handled or not allowed 2026-01-24 22:53:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lab-pack-open-head-poly-drum-plastic-lever-lock-20-gallon-capacity-black>: HTTP status code is not handled or not allowed 2026-01-24 22:53:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gray-outdoor-ashtray>: HTTP status code is not handled or not allowed 2026-01-24 22:53:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sign-receptacle-2pc-unglazed-porcelain-medium-base-w-screw-terminals>: HTTP status code is not handled or not allowed 2026-01-24 22:53:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mixer-flat-beater-40772-for-30-quart-mixer>: HTTP status code is not handled or not allowed 2026-01-24 22:53:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-way-keyless-mechanism-w-screw-terminals>: HTTP status code is not handled or not allowed 2026-01-24 22:53:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1181063-inlet-spun-18-ga>: HTTP status code is not handled or not allowed 2026-01-24 22:53:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/open-shelf-file-cabinet-credenza-walnut-29-1-2-22-23-1-9>: HTTP status code is not handled or not allowed 2026-01-24 22:53:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-panel-display-tower-6-ft-5-inch-high-fabric-sea-green>: HTTP status code is not handled or not allowed 2026-01-24 22:53:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-panel-display-tower-5-ft-9-inch-high-fabric-greysmoke>: HTTP status code is not handled or not allowed 2026-01-24 22:53:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mixer-dough-hook-40767-for-20-quart-mixer>: HTTP status code is not handled or not allowed 2026-01-24 22:53:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/phenolic-candelabra-socket-with-paper-liner-3-in-screw-terminals-double-leg>: HTTP status code is not handled or not allowed 2026-01-24 22:53:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-2398-5-inch-swivel-stem-caster-for-1-inch-od-tubing-300-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:53:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wireway-husky-tear-drop-pallet-rack-with-wire-deck-96x36x96>: HTTP status code is not handled or not allowed 2026-01-24 22:53:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8013889-cooking-basket-ptfe-xl-mesh-for-turbochef>: HTTP status code is not handled or not allowed 2026-01-24 22:53:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tabletop-personal-safety-partition-30w-x-30h-clear>: HTTP status code is not handled or not allowed 2026-01-24 22:53:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1131129-backflow-preventer-leadfree-ss-3-4-for-fisher-manufacturing>: HTTP status code is not handled or not allowed 2026-01-24 22:53:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washdown-motor-56-tefc-3ph-3450-2850rpm-c6t34wk12cc>: HTTP status code is not handled or not allowed 2026-01-24 22:53:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slide-plug-polarized-18-2-spt-2-black>: HTTP status code is not handled or not allowed 2026-01-24 22:53:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/push-to-connect-tube-fitting-nylon-plastic-straight-adapter-8mm-tube-od-x-14-npt-male>: HTTP status code is not handled or not allowed 2026-01-24 22:53:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-flat-washer-sae-9-32-i-d-steel-plain-grade-2-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:53:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/push-to-connect-tube-fitting-nylon-plastic-straight-reducer-8mm-tube-od-x-4mm-tube-od>: HTTP status code is not handled or not allowed 2026-01-24 22:53:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wear-ever-shallow-sauce-pan-with-traditional-handle-4020-8-gauge-5-quart-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:53:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ribbed-surface-mat-2-foot-wide-black-with-yellow-borders>: HTTP status code is not handled or not allowed 2026-01-24 22:53:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8004166-elec-ce-dv-smt-cpu-assembly-for-frymaster>: HTTP status code is not handled or not allowed 2026-01-24 22:53:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-internal-tooth-lock-washer-grade-2-steel-zinc-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:53:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/optio-fry-pan-3812-2-1-4-depth-21-gauge-natural-finish>: HTTP status code is not handled or not allowed 2026-01-24 22:53:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arkadia-4-1-2-quart-sauce-pan-7344-9-gauge-4-5-8-depth>: HTTP status code is not handled or not allowed 2026-01-24 22:53:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-countersunk-finishing-washer-22-236-i-d-012-02-thick-brass-plain-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:53:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylon-syringe-filter-0-45um-30mm-bulk-packed-non-sterile-100-case>: HTTP status code is not handled or not allowed 2026-01-24 22:53:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1292059-filter-baffle-grease-gal12x16>: HTTP status code is not handled or not allowed 2026-01-24 22:53:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/richmond-8-leaning-rail-in-ground-mount-brown>: HTTP status code is not handled or not allowed 2026-01-24 22:53:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-countersunk-finishing-washer-185-161-i-d-012-02-thick-brass-plain-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:53:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/infinity-locker-single-tier-12x15x60-1-door-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:53:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/681534-motor-conveyor-for-middleby>: HTTP status code is not handled or not allowed 2026-01-24 22:53:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medium-base-pull-chain-current-tap-ivory>: HTTP status code is not handled or not allowed 2026-01-24 22:53:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1331755-pan-food-full-4-inch-d-clear-for-carter-hoffmann>: HTTP status code is not handled or not allowed 2026-01-24 22:53:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-length-squeeze-disher-47393-green-2-8-oz-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:53:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-countersunk-finishing-washer-322-299-i-d-steel-plain-grade-2-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:53:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2hp-northstar-rep-motor>: HTTP status code is not handled or not allowed 2026-01-24 22:53:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/end-panel-12-x-73>: HTTP status code is not handled or not allowed 2026-01-24 22:53:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-square-table-with-2-imme-armless-barstools-black-seatdesigner-white-top>: HTTP status code is not handled or not allowed 2026-01-24 22:54:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/redco-onion-king-511-11-blades-1-4-cut>: HTTP status code is not handled or not allowed 2026-01-24 22:54:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metal-storage-cabinet-36x18x72-black>: HTTP status code is not handled or not allowed 2026-01-24 22:54:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2ml-serological-pipet-bulk-packed-in-bags-sterile-case-of-800>: HTTP status code is not handled or not allowed 2026-01-24 22:54:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mixing-bowl-xmix0702-7-quart-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:54:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012444-relay-flange-basew-cover-for-electro-freeze>: HTTP status code is not handled or not allowed 2026-01-24 22:54:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25x48-black-top-shelf-for-a301>: HTTP status code is not handled or not allowed 2026-01-24 22:54:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wear-ever-professional-standard-strength-loaf-pan-5435-anodized-5-x-10-x-4>: HTTP status code is not handled or not allowed 2026-01-24 22:54:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-7-drawer-40-shallow-depth-mobile-cabinet-w-butcher-top-blue-no-lock>: HTTP status code is not handled or not allowed 2026-01-24 22:54:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-round-table-with-4-imme-armless-chairs-navy-seatmaple-top>: HTTP status code is not handled or not allowed 2026-01-24 22:54:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traex-bar-keep-storage-jar-3628-4-quarts>: HTTP status code is not handled or not allowed 2026-01-24 22:54:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 33155 pages (at 100 pages/min), scraped 15026 items (at 22 items/min) 2026-01-24 22:54:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-red-pin-heavy-duty-dock-boards-4>: HTTP status code is not handled or not allowed 2026-01-24 22:54:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/266332-magnetic-latch-strike>: HTTP status code is not handled or not allowed 2026-01-24 22:54:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cdn-stick-ups-thermometers>: HTTP status code is not handled or not allowed 2026-01-24 22:54:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mixer-wire-whisk-40778-for-60-quart-mixer>: HTTP status code is not handled or not allowed 2026-01-24 22:54:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slotted-hook-hangers-24-pack-charcoal-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:54:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acid-corrosive-cabinet-manual-single-door-vertical-storage-1>: HTTP status code is not handled or not allowed 2026-01-24 22:54:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-round-table-with-4-imme-armless-barstools-coral-seatdesigner-white-top>: HTTP status code is not handled or not allowed 2026-01-24 22:54:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-plastic-tote-without-lid-18-1-2x13-1-4x12-yellow>: HTTP status code is not handled or not allowed 2026-01-24 22:54:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/lathes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:54:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traex-dripcut-dredges-caps-4908-01-extra-large-brown-lid>: HTTP status code is not handled or not allowed 2026-01-24 22:54:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-starter-rack-48x48x84-with-3-level-steel-deck-1500lb-cap-per-shelf-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:54:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1681325-tray-sediment-for-frymaster>: HTTP status code is not handled or not allowed 2026-01-24 22:54:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/321507-gh-1440zx-large-spindle-bore-lathe-w-acu-rite-200s-taper-attachment-collet-closer>: HTTP status code is not handled or not allowed 2026-01-24 22:54:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/equipto-vg-bulk-rack-60-w-x-15-d-x-60-h-starter-with-corrugated-steel-decking-textured-putty>: HTTP status code is not handled or not allowed 2026-01-24 22:54:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-container-lid-2>: HTTP status code is not handled or not allowed 2026-01-24 22:54:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/diamond-plate-ergonomic-mat-9-16-thick-36-wide-black-yellow-border-up-to-75>: HTTP status code is not handled or not allowed 2026-01-24 22:54:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-round-table-with-4-imme-armless-chairs-coral-seatdesigner-white-top>: HTTP status code is not handled or not allowed 2026-01-24 22:54:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-round-table-with-4-imme-armless-barstools-moonbeam-seatespresso-top>: HTTP status code is not handled or not allowed 2026-01-24 22:54:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acry-fab-flush-mount-stir-stick-holder-fms-5>: HTTP status code is not handled or not allowed 2026-01-24 22:54:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-task-chair>: HTTP status code is not handled or not allowed 2026-01-24 22:54:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/machining/machinery/lathes>: HTTP status code is not handled or not allowed 2026-01-24 22:54:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-add-on-rack-60x48x84-with-3-levels-wire-deck-1300-lb-cap-per-level-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:54:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vises-wilton11127>: HTTP status code is not handled or not allowed 2026-01-24 22:54:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-add-on-rack-48x36x84-with-3-levels-wire-deck-1000-lb-cap-per-level-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:54:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pre-built-roller-bearing-unit-28-1-2-w-x-20-1-2-d-x-32-h-4-steel-bins-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:54:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cm-valuestar-electric-chain-hoist-with-chain-container-500-lb-capacity-1>: HTTP status code is not handled or not allowed 2026-01-24 22:54:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wraparound-floor-savers-gray-16-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:54:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/577003-j-4002-1w-x-42l-bench-belt-disc-sander>: HTTP status code is not handled or not allowed 2026-01-24 22:54:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-round-table-with-4-imme-armless-barstools-black-seatmaple-top>: HTTP status code is not handled or not allowed 2026-01-24 22:54:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobilepro-anti-fatigue-mat-3-4-thick-2-x-1-5-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:54:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-square-table-with-4-imme-armless-caster-chairs-sky-blue-seatmaple-top>: HTTP status code is not handled or not allowed 2026-01-24 22:54:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hl-series-lateral-file-36-wide-5-drawer-lava-red>: HTTP status code is not handled or not allowed 2026-01-24 22:54:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wear-ever-tapered-sauce-pan-434212-6-1-2-bottom-diameter-2-3-4-quart-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:54:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pedestal-stand-for-bench-grinders-11-inch-x-10-inch-mounting-surface>: HTTP status code is not handled or not allowed 2026-01-24 22:54:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-deck-narrow-aisle-high-end-platform-truck-60-l-x-24-w>: HTTP status code is not handled or not allowed 2026-01-24 22:54:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-round-table-with-4-imme-armless-chairs-black-seatstudio-teak-top>: HTTP status code is not handled or not allowed 2026-01-24 22:54:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011679-evaporator-fan-motor-230v-for-master-bilt-products>: HTTP status code is not handled or not allowed 2026-01-24 22:54:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/data-commander-2-hour-digital-fire-water-resistant-media-safe-7-87-cu-ft-off-white>: HTTP status code is not handled or not allowed 2026-01-24 22:54:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/norton-66252843219-bluefire-cutting-wheel-4-1-2-inch-x-1-16-inch-x-7-8-inch-zirconia-alumina-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-24 22:54:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/appliances/cooking_equipment/toasters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:54:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acry-fab-1-column-lid-holder-fml-1-1-column-7-1-4-x-6-1-2-x-5-3-8>: HTTP status code is not handled or not allowed 2026-01-24 22:54:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-high-profile-machinery-guard-4>: HTTP status code is not handled or not allowed 2026-01-24 22:54:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/foodservice_disposables/disposable_food_beverage_trays already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:54:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010586-front-burner-kit-for-garland-range>: HTTP status code is not handled or not allowed 2026-01-24 22:54:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cable-tie-uvb-14-75-lb-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:54:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/personnel-locker-double-tier-98x24x78-8-doors-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:54:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/light-duty-induction-range-wih200>: HTTP status code is not handled or not allowed 2026-01-24 22:54:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/norton-07660701617-gemini-right-angle-cut-off-wheel-4-1-2-inch-x-040-inch-x-7-8-inch-60-grit-alum-oxide-type-1>: HTTP status code is not handled or not allowed 2026-01-24 22:54:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/norton-66261131626-paper-sheet-9-inch-x-11-inch-p320-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-24 22:54:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-square-table-with-4-imme-armless-barstools-black-seatstudio-teak-top>: HTTP status code is not handled or not allowed 2026-01-24 22:54:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esd-tractor-stool-polyurethane-black>: HTTP status code is not handled or not allowed 2026-01-24 22:54:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/appliances/cooking_equipment/toasters>: HTTP status code is not handled or not allowed 2026-01-24 22:54:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/norton-66623340032-bear-tex-non-woven-quick-change-disc-3-4-inch-dia-med-grit-aluminum-oxide-type-iii>: HTTP status code is not handled or not allowed 2026-01-24 22:54:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/foodservice_disposables/disposable_food_beverage_trays>: HTTP status code is not handled or not allowed 2026-01-24 22:54:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8231tvk-hi-vis-tool-tethering-safety-vest-kit-type-r-class-2-2xl3xl>: HTTP status code is not handled or not allowed 2026-01-24 22:54:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grid-wall-bin-accessory-bin-divider-sold-pack-of-6-1>: HTTP status code is not handled or not allowed 2026-01-24 22:54:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1503530-tray-chilled-condiment-3-qt-for-san-jamar>: HTTP status code is not handled or not allowed 2026-01-24 22:54:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gearmotor-ac-parallel-157rpm-c4c17fz32c>: HTTP status code is not handled or not allowed 2026-01-24 22:54:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hcb-103am-ntep-approved-portable-precision-balance-100-g-x-0-001-g>: HTTP status code is not handled or not allowed 2026-01-24 22:54:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/norton-66623399156-bluefire-plastic-flat-flap-disc-t27-7-inch-x-5-8-11-inch-p40-grit-zirconia-alumina>: HTTP status code is not handled or not allowed 2026-01-24 22:54:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/263779-pump-sp-for-server-products>: HTTP status code is not handled or not allowed 2026-01-24 22:54:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gearmotor-dc-parallel-42rpm-cm34d25nz75a>: HTTP status code is not handled or not allowed 2026-01-24 22:55:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/first-aid-only-3500-first-responder-kit-151-piece-fabric-case>: HTTP status code is not handled or not allowed 2026-01-24 22:55:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cross-stack-nest-tote-box-25-1-8x16x8-1-2-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:55:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cross-stack-nest-tote-box-25-1-8x16x8-1-2-white>: HTTP status code is not handled or not allowed 2026-01-24 22:55:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-display-cabinet-40842-refrigerated-36-x-21-x-32-1-2>: HTTP status code is not handled or not allowed 2026-01-24 22:55:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wiregard-174-orange-gb-3-small-jar-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:55:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-two-tone-mesh-vest-reflective-binding-4xl-5xl-type-r-class-2-lime>: HTTP status code is not handled or not allowed 2026-01-24 22:55:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tote-box-accessory-long-divider-sold-pack-of-6-7>: HTTP status code is not handled or not allowed 2026-01-24 22:55:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n-ferno-6492-2-piece-wind-resistant-multi-band-american-flag>: HTTP status code is not handled or not allowed 2026-01-24 22:55:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2231296-plate-julienne-1-8-inch-as3x-for-dito-dean>: HTTP status code is not handled or not allowed 2026-01-24 22:55:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1681561-hanger-basket-14-5-16-inch-wide-for-frymaster>: HTTP status code is not handled or not allowed 2026-01-24 22:55:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rw30-3z1y-general-purpose-rain-jacket-hi-viz-lime-2xl>: HTTP status code is not handled or not allowed 2026-01-24 22:55:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hot-food-display-case-40732>: HTTP status code is not handled or not allowed 2026-01-24 22:55:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 33249 pages (at 94 pages/min), scraped 15049 items (at 23 items/min) 2026-01-24 22:55:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-disposable-respirator-storage-bags-50-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:55:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/38-reversible-drill-65>: HTTP status code is not handled or not allowed 2026-01-24 22:55:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kyorene-pro-cut-resistant-gloves-hct-micro-foam-nitrile-coated-ansi-a4-xs-yellow-12-pair>: HTTP status code is not handled or not allowed 2026-01-24 22:55:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/equipto-30-w-modular-cabinet-18-drawers-w-dividers-59-h-no-lock-textured-regal-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:55:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rs1-21-rad-sequel-safety-eyewear-smoke-anti-fog>: HTTP status code is not handled or not allowed 2026-01-24 22:55:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/french-fry-rack-3681-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 22:55:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gpx-9-in-portable-dvd-player>: HTTP status code is not handled or not allowed 2026-01-24 22:55:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1411060-fan-ceiling-exhaust-115v>: HTTP status code is not handled or not allowed 2026-01-24 22:55:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s-225-rekoil-hose-safety-blow-gun-kit-1-4-npt>: HTTP status code is not handled or not allowed 2026-01-24 22:55:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ibc-tilt-stand-with-fork-pockets-and-adjustable-legs-4400-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:55:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s-184-turbo-blow-gun-engine-cleaner-3-8-npt>: HTTP status code is not handled or not allowed 2026-01-24 22:55:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-3889-10-inch-h-x-20-inch-w-x-2-inch-t-stainless-steel-hood-filter-ridged-baffles>: HTTP status code is not handled or not allowed 2026-01-24 22:55:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compact-magnetic-lid-lifter-500-lbs>: HTTP status code is not handled or not allowed 2026-01-24 22:55:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-rated-ul2043-filtered-return-grille-20-square-ductwhite-str-erfg-w-fr>: HTTP status code is not handled or not allowed 2026-01-24 22:55:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/appliances/refrigerators_freezers/refrigerators/refrigerated_merchandisers_display_cases already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:55:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8014007-element-assembly-208v-7kw-lve-for-henny-penny>: HTTP status code is not handled or not allowed 2026-01-24 22:55:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-use-indoor-outdoor-control-box-2-lights-2-sensors-brackets-15-cord>: HTTP status code is not handled or not allowed 2026-01-24 22:55:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mq54pe-metromax-q-stationary-post-54>: HTTP status code is not handled or not allowed 2026-01-24 22:55:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-8482-46-child-size-square-outdoor-steel-picnic-table-expanded-metal-green>: HTTP status code is not handled or not allowed 2026-01-24 22:55:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/al-sight-guard-return-grille-w-45-degree-sight-proof-core-10-x-10-white-4-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:55:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-3-point-drum-lifter-for-30-and-55-gallon-drums-1000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:55:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_closers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:55:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1291140-kason-67003002520-filter-baffle>: HTTP status code is not handled or not allowed 2026-01-24 22:55:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/al-stratus-cone-diffuser-with-14-neck-white-str-c-14w>: HTTP status code is not handled or not allowed 2026-01-24 22:55:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/visual-feedback-meter-only-for-biofeedback-device>: HTTP status code is not handled or not allowed 2026-01-24 22:55:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/263129-alum-tube-50ft-roll-1-2-inch>: HTTP status code is not handled or not allowed 2026-01-24 22:55:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/321234-kason-401sa6064284-strip-curtain-for-kason>: HTTP status code is not handled or not allowed 2026-01-24 22:55:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/appliances/refrigerators_freezers/refrigerators/refrigerated_merchandisers_display_cases>: HTTP status code is not handled or not allowed 2026-01-24 22:55:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/framing-nails-fr113-238b-2-3-8-x-113-plastic-collated-coated-smooth-shank-2000-bx>: HTTP status code is not handled or not allowed 2026-01-24 22:55:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gml105rd-gauge-marking-decal>: HTTP status code is not handled or not allowed 2026-01-24 22:55:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2241170-block-pusher-3-16-inch-tom-slcr-ii-for-nemco>: HTTP status code is not handled or not allowed 2026-01-24 22:55:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ignition-20-4-way-stretch-task-chair-gray-lumbar-support-17-21h-seat-basaltwhite>: HTTP status code is not handled or not allowed 2026-01-24 22:55:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ruck-task-stool-laminate-seat-mid-back-pinnacle>: HTTP status code is not handled or not allowed 2026-01-24 22:55:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-syle-1-23-3-4-x-23-3-4-pvc-lay-in-tile-in-polished-copper-pl5025>: HTTP status code is not handled or not allowed 2026-01-24 22:55:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_closers>: HTTP status code is not handled or not allowed 2026-01-24 22:55:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vpl907led-sign-projector-with-lens>: HTTP status code is not handled or not allowed 2026-01-24 22:55:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-syle-2-48-3-8-x-24-3-8-pvc-glue-up-tile-in-brushed-aluminum-pg5108>: HTTP status code is not handled or not allowed 2026-01-24 22:55:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/221596-kason-10056l05020-latch-for-kason>: HTTP status code is not handled or not allowed 2026-01-24 22:55:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kee-klamp-straight-coupling-2>: HTTP status code is not handled or not allowed 2026-01-24 22:55:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16x16-3-milslide-seal-reclosable-poly-bags>: HTTP status code is not handled or not allowed 2026-01-24 22:55:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/receiving-shop-desk-beige>: HTTP status code is not handled or not allowed 2026-01-24 22:55:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-2922-5-inch-swivel-stem-caster-for-1-inch-od-tubing-260-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:55:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-3254-5-inch-swivel-stem-caster-with-brake-for-7-8-inch-square-post-260-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:55:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/noheat-hand-dryer-with-noise-reduction-nozzle-stainless-steel-hepa-110-120v>: HTTP status code is not handled or not allowed 2026-01-24 22:55:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1571111-connector-gas-3-4-inch-x36-inch-for-dormont-mfg>: HTTP status code is not handled or not allowed 2026-01-24 22:55:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spring-retractable-fuel-delivery-hose-reel-with-3-4-in-x-25-ft-l-hose-250-psi>: HTTP status code is not handled or not allowed 2026-01-24 22:55:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8014154-hood-filter-type6-stainless-steel-20x16>: HTTP status code is not handled or not allowed 2026-01-24 22:55:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-1-2-x1075-4-mil-poly-tubing>: HTTP status code is not handled or not allowed 2026-01-24 22:55:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1331481-cart-oil-short-shuttle-8-1-2-inch-h-for-worchester-industrial-products>: HTTP status code is not handled or not allowed 2026-01-24 22:55:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chain-driven-live-roller-conveyor-1-9-dia-rollers-1-2-hp-10l-x-48w>: HTTP status code is not handled or not allowed 2026-01-24 22:55:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/no-touch-menstrual-care-dual-vendor-stainless-steel-battery-operated>: HTTP status code is not handled or not allowed 2026-01-24 22:55:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4x4-2-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-24 22:55:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10197215-harness-back-hip-d-rings-tongue-buckle-leg-straps-standard>: HTTP status code is not handled or not allowed 2026-01-24 22:55:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/piston-pump-non-corrosive-fluids-2>: HTTP status code is not handled or not allowed 2026-01-24 22:55:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blickman-newark-height-adjustable-39-1-2-62-stainless-steel-stand-with-tru-loc>: HTTP status code is not handled or not allowed 2026-01-24 22:55:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-heavy-duty-boltless-shelving-starter-72in-w-x-48in-d-x-72in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 22:55:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blickman-howard-instrument-shelf-table-36l-x-24w-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 22:55:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swing-glass-door-merchandiser-63-high>: HTTP status code is not handled or not allowed 2026-01-24 22:55:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-36-x-48-dark-yellow-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:55:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10194976-harness-back-d-ring-tongue-buckle-leg-straps-extra-small>: HTTP status code is not handled or not allowed 2026-01-24 22:55:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/professional-manual-pot-pan-dish-detergent-original-scent-5-gallon-cube>: HTTP status code is not handled or not allowed 2026-01-24 22:55:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1971190-faceplate-fwm3-34-for-duke-manufacturing>: HTTP status code is not handled or not allowed 2026-01-24 22:55:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cylinder-storage-mobile-forkli-able-caddy-1>: HTTP status code is not handled or not allowed 2026-01-24 22:56:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8x8x1-1-4-literature-mailers>: HTTP status code is not handled or not allowed 2026-01-24 22:56:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-starter-72in-w-x-24in-d-x-72in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 22:56:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10193130-standard-single-leg-energy-absorbing-6-tie-back-lanyard-csa-z259-11-17>: HTTP status code is not handled or not allowed 2026-01-24 22:56:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009358-thermostat-ego-for-moffat>: HTTP status code is not handled or not allowed 2026-01-24 22:56:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blickman-glass-door-30w-x-26-5-8d-x-24-1-2h-blanket-warming-medical-cabinet>: HTTP status code is not handled or not allowed 2026-01-24 22:56:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tek-tough-jr-mat-3-ft-x-5-ft-red>: HTTP status code is not handled or not allowed 2026-01-24 22:56:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-18-x-18-black-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:56:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10193124-standard-single-leg-fixed-lanyard-3-36c-small-snaphooks>: HTTP status code is not handled or not allowed 2026-01-24 22:56:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-extra-heavy-duty-boltless-shelving-starter-36in-w-x-18in-d-x-72in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 22:56:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-arm-with-2-lip-cantilever-rack>: HTTP status code is not handled or not allowed 2026-01-24 22:56:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-21-x-30-white-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:56:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-mat-ii-reversible-drainage-mat-3-ft-x-4-ft-black>: HTTP status code is not handled or not allowed 2026-01-24 22:56:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/621160-infrared-thermometer>: HTTP status code is not handled or not allowed 2026-01-24 22:56:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/closed-workbench-w-sliding-door-8-dove-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:56:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 33347 pages (at 98 pages/min), scraped 15071 items (at 22 items/min) 2026-01-24 22:56:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10191758-standard-twin-leg-adjustable-lanyard-6-36cl-large-snaphooks>: HTTP status code is not handled or not allowed 2026-01-24 22:56:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16x12x6-double-wall-boxes>: HTTP status code is not handled or not allowed 2026-01-24 22:56:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-21-x-42-dark-yellow-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:56:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tutco-crankcase-100-230-30-1316>: HTTP status code is not handled or not allowed 2026-01-24 22:56:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/weblok-assembly-877010-2-synthetic-to-synthetic-3-8-5000-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:56:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16x9x9-corrugated-boxes>: HTTP status code is not handled or not allowed 2026-01-24 22:56:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tutco-crankcase-25-120-21>: HTTP status code is not handled or not allowed 2026-01-24 22:56:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-extra-heavy-duty-boltless-shelving-starter-48in-w-x-18in-d-x-96in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 22:56:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-2-plastic-chain-magenta>: HTTP status code is not handled or not allowed 2026-01-24 22:56:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-14-x-54-light-grey-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:56:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10x5x4-corrugated-boxes>: HTTP status code is not handled or not allowed 2026-01-24 22:56:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galvanized-shelf-truck-36x24x80-1200-pound-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:56:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/224467-knob-indexing-for-hobart>: HTTP status code is not handled or not allowed 2026-01-24 22:56:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-plastic-chain-traffic-orange>: HTTP status code is not handled or not allowed 2026-01-24 22:56:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-21-x-54-red-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:56:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24x12x8-double-wall-boxes>: HTTP status code is not handled or not allowed 2026-01-24 22:56:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/er70s-6-mild-steel-solid-mig-welding-wire-030-10-lbs-spool>: HTTP status code is not handled or not allowed 2026-01-24 22:56:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-wire-document-holder>: HTTP status code is not handled or not allowed 2026-01-24 22:56:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1u-6-vented-light-duty-rack-shelf-black-ldrs1u6v>: HTTP status code is not handled or not allowed 2026-01-24 22:56:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e7018-aluminum-welding-rod-5-32-40-lbs-package>: HTTP status code is not handled or not allowed 2026-01-24 22:56:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22x20x20-corrugated-boxes>: HTTP status code is not handled or not allowed 2026-01-24 22:56:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-shelf-heavy-duty-boltless-shelving-add-on-72in-w-x-30in-d-x-36in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 22:56:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galaxy-12-gal-box-carpet-extractor-100-psi-pump-150-inch-waterlift-without-heat-2000sx-hr>: HTTP status code is not handled or not allowed 2026-01-24 22:56:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-plastic-platform-truck-36-x-24-1000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:56:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/floodbuster-flood-pumper-with-50-gpm-pump-fb50>: HTTP status code is not handled or not allowed 2026-01-24 22:56:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/266206-stop-door-concave-bumper>: HTTP status code is not handled or not allowed 2026-01-24 22:56:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/266245-kit-spring-edgemount-hinge>: HTTP status code is not handled or not allowed 2026-01-24 22:56:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-inch-teflon-wand-glide-1012ac-gl>: HTTP status code is not handled or not allowed 2026-01-24 22:56:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012425-ignition-module-ssb-3-5-10g-for-groen>: HTTP status code is not handled or not allowed 2026-01-24 22:56:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-add-on-36in-w-x-18in-d-x-84in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 22:56:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stackable-plastic-shipping-tote-29-5x19-5x15-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:56:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/book-truck-6-shelves-42-wide-18-deep-42-high-oiled-cherry>: HTTP status code is not handled or not allowed 2026-01-24 22:56:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lxt-174-cordless-7-1-4-circular-saw-tool-only-lithium-ion-brushless-18v-6000rpm>: HTTP status code is not handled or not allowed 2026-01-24 22:56:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/book-return-unit-36-wide-30-deep-32-high-oiled-cherry>: HTTP status code is not handled or not allowed 2026-01-24 22:56:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-x2900-1-5-mil-poly-tubing>: HTTP status code is not handled or not allowed 2026-01-24 22:56:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-mobile-rack-with-bins-5>: HTTP status code is not handled or not allowed 2026-01-24 22:56:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1381066-thermometer-100-400f-2-inch-dia>: HTTP status code is not handled or not allowed 2026-01-24 22:56:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-18-x-54-red-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:56:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-21-x-48-grey-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:56:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-add-on-48in-w-x-18in-d-x-96in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 22:56:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-add-on-48in-w-x-36in-d-x-72in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 22:56:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/locking-storage-cabinet-36-w-x-18-d-x-42-h-with-removable-bins-3>: HTTP status code is not handled or not allowed 2026-01-24 22:56:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gp8000e-8000-watt-portable-generator-gasoline-electric-recoil-120-240v>: HTTP status code is not handled or not allowed 2026-01-24 22:56:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-heavy-duty-boltless-shelving-add-on-48in-w-x-18in-d-x-72in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 22:56:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:56:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/outdoor-enclosed-bulletin-board-cabinet-1-door-3-feet-h-x-3-feet-w-coffee-trim-pacific-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:56:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8013230-switch-rocker-250v-green-dpst-for-hatco-corp>: HTTP status code is not handled or not allowed 2026-01-24 22:56:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergomat-complete-smooth-2-x-5>: HTTP status code is not handled or not allowed 2026-01-24 22:56:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-grabber-skid-puller-chain-20-long>: HTTP status code is not handled or not allowed 2026-01-24 22:56:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergomat-classic-4-x-12>: HTTP status code is not handled or not allowed 2026-01-24 22:56:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7x6-red-paper-face-packing-list-enclosed-important-papers-enclosed-envelopes>: HTTP status code is not handled or not allowed 2026-01-24 22:56:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7x5-1-2-environmental-packing-list-enclosed-envelopes>: HTTP status code is not handled or not allowed 2026-01-24 22:56:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-1-2x10-blue-packing-list-enclosed-envelopes>: HTTP status code is not handled or not allowed 2026-01-24 22:56:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durastripe-32-octagone-sign-stop>: HTTP status code is not handled or not allowed 2026-01-24 22:56:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/881015-liq-filter-drierdcl163>: HTTP status code is not handled or not allowed 2026-01-24 22:56:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xtreme-48-oz-blender-programmable-electronic-keypad-bpa-free-copolyester-jar-2-spd>: HTTP status code is not handled or not allowed 2026-01-24 22:56:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-add-on-72in-w-x-15in-d-x-60in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 22:56:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks>: HTTP status code is not handled or not allowed 2026-01-24 22:56:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-21-x-60-red-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:56:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-add-on-72in-w-x-30in-d-x-72in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 22:56:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torq-bar-blender-64-oz-copolyester-jar-toggle-switch-2-speed-2-hp>: HTTP status code is not handled or not allowed 2026-01-24 22:56:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torq-bar-blender-48-oz-copolyester-jar-toggle-switch-2-speed-2-hp>: HTTP status code is not handled or not allowed 2026-01-24 22:56:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-starter-36inw-x-12ind-x-84inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 22:56:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-heavy-duty-boltless-shelving-starter-36inw-x-12ind-x-72inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 22:56:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-starter-48inw-x-48ind-x-96inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 22:56:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-2-single-solenoid-valve-pilot-spr-return-1-8-nptf-220v-ac-5va-coil-blk-connection>: HTTP status code is not handled or not allowed 2026-01-24 22:56:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-starter-48inw-x-48ind-x-72inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 22:56:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durastripe-20-round-sign-notice-safety-equipment-must-be-worn-no-exceptions>: HTTP status code is not handled or not allowed 2026-01-24 22:57:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-open-single-solenoid-valve-pilot-spr-return-1-4-nptf-110v-ac-5va-coil-blk-conn>: HTTP status code is not handled or not allowed 2026-01-24 22:57:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durastripe-30-octagone-sign-stop-watch-for-forklifts>: HTTP status code is not handled or not allowed 2026-01-24 22:57:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-terrain-dolly-sl1500d6r-6-casters-1500-lb-cap-red>: HTTP status code is not handled or not allowed 2026-01-24 22:57:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-pres-cntr-dbl-solenoid-vlv-pilot-spr-cntred-1-2-nptf-24v-dc-3w-coil-black-conn>: HTTP status code is not handled or not allowed 2026-01-24 22:57:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durastripe-12-round-sign-notice-safety-equipment-must-be-worn-no-exceptions>: HTTP status code is not handled or not allowed 2026-01-24 22:57:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enclosure-kit-slide-door-18-x-36-x-16-light-grey>: HTTP status code is not handled or not allowed 2026-01-24 22:57:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/snap-hooks-slcaeashi2-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:57:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durastripe-24x16-arrow-sign-pedestrians>: HTTP status code is not handled or not allowed 2026-01-24 22:57:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durastripe-supreme-v-2-x-100-orange>: HTTP status code is not handled or not allowed 2026-01-24 22:57:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-fold-paper-towels-9-1-4-x-9-1-2-white-150-pack-8-ctn>: HTTP status code is not handled or not allowed 2026-01-24 22:57:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enclosure-kit-slide-door-14-x-36-x-18-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:57:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 33441 pages (at 94 pages/min), scraped 15087 items (at 16 items/min) 2026-01-24 22:57:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-high-capacity-bulk-rack-with-wood-decking-72x24x72-add-on>: HTTP status code is not handled or not allowed 2026-01-24 22:57:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-half-inch-fnpt-brass-check-valve-buna-s-rubber-poppet>: HTTP status code is not handled or not allowed 2026-01-24 22:57:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/translucent-liner-14-x-36>: HTTP status code is not handled or not allowed 2026-01-24 22:57:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-1134-on-off-mini-micro-pin-switch-15a-125-250v>: HTTP status code is not handled or not allowed 2026-01-24 22:57:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/water-resistant-mid-arm-cryogenic-glove-medium-blue-g99crbermdma>: HTTP status code is not handled or not allowed 2026-01-24 22:57:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kit2cv11b2x11-12-cal-ultrasoft-arc-flash-kit-coverall-balaclava-2xl-glove-size-11>: HTTP status code is not handled or not allowed 2026-01-24 22:57:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-dexterity-performance-work-glove-mechanics-plus-large>: HTTP status code is not handled or not allowed 2026-01-24 22:57:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enclosure-kit-slide-door-18-x-36-x-18-white>: HTTP status code is not handled or not allowed 2026-01-24 22:57:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8300-vortex-cooling-vest-standard>: HTTP status code is not handled or not allowed 2026-01-24 22:57:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-high-capacity-bulk-rack-with-steel-decking-96x24x72-add-on>: HTTP status code is not handled or not allowed 2026-01-24 22:57:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1491066-guard-toggle-switch-for-ultrafryer>: HTTP status code is not handled or not allowed 2026-01-24 22:57:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pl4081-plus-stanchionstake-replacement-spike>: HTTP status code is not handled or not allowed 2026-01-24 22:57:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enclosure-kit-slide-door-24-x-60-x-18-white>: HTTP status code is not handled or not allowed 2026-01-24 22:57:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-pres-cntr-dbl-solenoid-vlv-pilot-spr-cntred-1-2-nptf-220v-ac-5va-coil-black-conn>: HTTP status code is not handled or not allowed 2026-01-24 22:57:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/translucent-liner-12-x-30>: HTTP status code is not handled or not allowed 2026-01-24 22:57:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-high-capacity-bulk-rack-without-decking-96x48x96-starter>: HTTP status code is not handled or not allowed 2026-01-24 22:57:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sc-layout-slotted-groove-trays-51-compartments-2-h>: HTTP status code is not handled or not allowed 2026-01-24 22:57:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-unlined-bib-overall-30-x-32-brown-bib6dcm30x32>: HTTP status code is not handled or not allowed 2026-01-24 22:57:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enclosure-kit-slide-door-24-x-36-x-18-dark-yellow>: HTTP status code is not handled or not allowed 2026-01-24 22:57:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-high-capacity-bulk-rack-without-decking-96x24x72-add-on>: HTTP status code is not handled or not allowed 2026-01-24 22:57:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hs-layout-slotted-groove-trays>: HTTP status code is not handled or not allowed 2026-01-24 22:57:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011782-solenoid-valve-left-for-vulcan-hart>: HTTP status code is not handled or not allowed 2026-01-24 22:57:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-high-capacity-bulk-rack-without-decking-96x48x96-add-on>: HTTP status code is not handled or not allowed 2026-01-24 22:57:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-2-single-solenoid-pilot-valve-spring-return-1-4-110v-ac-5va-coil-black-connection>: HTTP status code is not handled or not allowed 2026-01-24 22:57:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-long-sleeve-t-shirt-m-navy-c541nnblsmd>: HTTP status code is not handled or not allowed 2026-01-24 22:57:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-heavy-duty-boltless-shelving-add-on-42inw-x-30ind-x-60inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 22:57:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/381284-locking-connector-for-cres-cor>: HTTP status code is not handled or not allowed 2026-01-24 22:57:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14l-kevlar-terrycloth-gloves-yellow-large-g51ktlw00214>: HTTP status code is not handled or not allowed 2026-01-24 22:57:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-shelf-bin-6-5-8-x-17-7-8-x-7>: HTTP status code is not handled or not allowed 2026-01-24 22:57:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kit2cv11xl10-12-cal-cm2-ultrasoft-arc-flash-kit-with-fr-coverall-xl-glove-size-10>: HTTP status code is not handled or not allowed 2026-01-24 22:57:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-1-2-npt-double-solenoid-valve-pilot-24v-dc-3w-coil-black-connection>: HTTP status code is not handled or not allowed 2026-01-24 22:57:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-2-single-solenoid-valve-pilot-spr-return-1-4-nptf-220v-ac-5va-coil-blk-connection>: HTTP status code is not handled or not allowed 2026-01-24 22:57:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ws72-waveseal-360-constant-tension-hose-clamp-319-481-dia-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 22:57:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drink-mat-14041-mlb-philadelphia-phillies-3-1-4-x-24-x-1>: HTTP status code is not handled or not allowed 2026-01-24 22:57:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kit2cv11-sm12-12-cal-cm2-ultrasoft-arc-flash-kit-with-fr-coverall-sm-glove-size-12>: HTTP status code is not handled or not allowed 2026-01-24 22:57:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-heavy-duty-boltless-shelving-add-on-36inw-x-24ind-x-60inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 22:57:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/isolated-ground-receptacle-package-of-4>: HTTP status code is not handled or not allowed 2026-01-24 22:57:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-in-w-x-64-in-h-panel-with-electric-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:57:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-in-w-x-46-in-h-panel-with-electric-blue>: HTTP status code is not handled or not allowed 2026-01-24 22:57:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-x-48-kraft-heavy-duty-mailing-tubes-with-caps>: HTTP status code is not handled or not allowed 2026-01-24 22:57:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-compartment-wood-literature-sorter-23-1-2-h-oak>: HTTP status code is not handled or not allowed 2026-01-24 22:57:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-1017-infinite-control-switch-kit-15a-120v>: HTTP status code is not handled or not allowed 2026-01-24 22:57:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-32-x-1-4-ph-flat-undercut-head-mach-screw-pkg-of-100-dwa88008>: HTTP status code is not handled or not allowed 2026-01-24 22:57:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-add-on-60inw-x-24ind-x-60inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 22:57:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-closed-single-solenoid-valve-ext-pilot-g-1-8-24v-dc-2w-coil-led-connection>: HTTP status code is not handled or not allowed 2026-01-24 22:57:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-starter-72inw-x-24ind-x-96inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 22:57:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010720-high-limit-switch-manual-for-imperial-cooking-equipment>: HTTP status code is not handled or not allowed 2026-01-24 22:57:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-work-shirt-2xl-ln-royal-blue-tcg01130226>: HTTP status code is not handled or not allowed 2026-01-24 22:57:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-hose-female-male-reducer-adapter-369-2061524-2-npt-x-1-1-2-nh-aluminum>: HTTP status code is not handled or not allowed 2026-01-24 22:57:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pebble-surface-mat-black-24-es-wide>: HTTP status code is not handled or not allowed 2026-01-24 22:57:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/structural-plastic-shelving-96x36x51>: HTTP status code is not handled or not allowed 2026-01-24 22:57:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-add-on-60inw-x-48ind-x-60inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 22:57:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m12andno8482-cordless-screwdriver-bare-tool-2401-20>: HTTP status code is not handled or not allowed 2026-01-24 22:57:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-pressure-cntr-g1-8-dbl-solenoid-valve-pilot-spring-cntred-24v-dc-2w-coil-led-conn>: HTTP status code is not handled or not allowed 2026-01-24 22:57:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-starter-48inw-x-36ind-x-96inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 22:57:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/orbital-jig-saw-6268-21>: HTTP status code is not handled or not allowed 2026-01-24 22:57:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-8-thick-anti-fatigue-mat-black-24x66>: HTTP status code is not handled or not allowed 2026-01-24 22:57:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2441092-strain-relief-cord-for-star-manufacturing>: HTTP status code is not handled or not allowed 2026-01-24 22:57:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/av-base-layer-bottoms-l-black-bsbavlg>: HTTP status code is not handled or not allowed 2026-01-24 22:57:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/combination-toggle-bolt-1-8-32-x-3-phillips-slotted-truss-head-steel-zinc-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:57:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultrawedge-wedge-anchor-1-2-13-x-2-3-4-steel-zinc-pkg-of-50-157180>: HTTP status code is not handled or not allowed 2026-01-24 22:58:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lightweight-long-sleeve-fr-t-shirt-2xl-t-desert-sand-df2-cm-446ls-ds-2xlt>: HTTP status code is not handled or not allowed 2026-01-24 22:58:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-add-on-96inw-x-48ind-x-96inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 22:58:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bench-rack-with-bins-2>: HTTP status code is not handled or not allowed 2026-01-24 22:58:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-starter-96inw-x-36ind-x-84inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 22:58:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-dbl-solenoid-vlv-ext-pilot-open-cntr-spr-cntred-1-8-nptf-110v-ac-5va-coil-led>: HTTP status code is not handled or not allowed 2026-01-24 22:58:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/264568-flue-deflector-for-frymaster>: HTTP status code is not handled or not allowed 2026-01-24 22:58:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beam-seating-guest-chairs-3-seater-navy-white>: HTTP status code is not handled or not allowed 2026-01-24 22:58:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-dbl-solenoid-vlv-ext-pilot-pres-cntr-spr-cntred-1-8-npt-110v-ac-5va-coil-led>: HTTP status code is not handled or not allowed 2026-01-24 22:58:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-drawcord-nylon-blue-straight>: HTTP status code is not handled or not allowed 2026-01-24 22:58:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-ropeless-nylon-white-straight>: HTTP status code is not handled or not allowed 2026-01-24 22:58:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 33542 pages (at 101 pages/min), scraped 15118 items (at 31 items/min) 2026-01-24 22:58:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-ropeless-nylon-yellow-straight>: HTTP status code is not handled or not allowed 2026-01-24 22:58:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-dbl-solenoid-vlv-ext-pilot-closed-cntr-spr-cntred-1-4-nptf-12v-dc-3w-coil-led>: HTTP status code is not handled or not allowed 2026-01-24 22:58:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quantum-red-exit-w-battery>: HTTP status code is not handled or not allowed 2026-01-24 22:58:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grip-24w-12-step-steel-rolling-ladder-14d-top-step-wcal-osha-handrail-ca>: HTTP status code is not handled or not allowed 2026-01-24 22:58:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/open-sided-wire-exchange-truck-five-wire-shelves-1000-lb-cap-1>: HTTP status code is not handled or not allowed 2026-01-24 22:58:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-24w-11-step-steel-rolling-ladder-21d-top-step-wcal-osha-handrail-ca>: HTTP status code is not handled or not allowed 2026-01-24 22:58:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-58-200-230460v-1075rpm-reversible-46-amp>: HTTP status code is not handled or not allowed 2026-01-24 22:58:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/charles-gallo-drain-gun-for-ac-condensate-lines-gg-1>: HTTP status code is not handled or not allowed 2026-01-24 22:58:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-58-15-amp208-230v-1075rpm-cwse>: HTTP status code is not handled or not allowed 2026-01-24 22:58:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drainable-blade-fixed-louver-with-flange-36w-x-24h-dfl-f-36x24>: HTTP status code is not handled or not allowed 2026-01-24 22:58:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/264221-90-degree-elbow-street-fitting-3-8-inch>: HTTP status code is not handled or not allowed 2026-01-24 22:58:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h-d-shelving-poly-z-c-o-14>: HTTP status code is not handled or not allowed 2026-01-24 22:58:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-work-shirt-m-ln-gray-tcg01150217>: HTTP status code is not handled or not allowed 2026-01-24 22:58:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-starter-36inw-x-18ind-x-84inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 22:58:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-open-g-1-8-single-solenoid-valve-pilot-spr-return-110v-ac-5va-coil-led-connection>: HTTP status code is not handled or not allowed 2026-01-24 22:58:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/282287-handle-meat-grip-for-hobart>: HTTP status code is not handled or not allowed 2026-01-24 22:58:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stationary-dunnage-rack-24-w-x-18-d-2>: HTTP status code is not handled or not allowed 2026-01-24 22:58:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-work-shirt-5x-light-blue-tcg01190234>: HTTP status code is not handled or not allowed 2026-01-24 22:58:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jrs-10l-18w-straight-gal-steel-roller-conveyor-1-3-8-roller-dia-16-bf-15-axle-center>: HTTP status code is not handled or not allowed 2026-01-24 22:58:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-cal-flame-resistant-ultrasoft-bib-overall-3xl-navy-c45up3xl32>: HTTP status code is not handled or not allowed 2026-01-24 22:58:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poly-liner-20-bu-blue-tub-only>: HTTP status code is not handled or not allowed 2026-01-24 22:58:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/dunnage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:58:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-2-double-solenoid-valve-ext-pilot-g-1-8-12v-dc-3w-coil-led-connection>: HTTP status code is not handled or not allowed 2026-01-24 22:58:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-58-56-amp-115v-1075rpm-reversible>: HTTP status code is not handled or not allowed 2026-01-24 22:58:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-work-shirt-4x-gray-tcg01150231>: HTTP status code is not handled or not allowed 2026-01-24 22:58:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-work-shirt-3xl-ln-gray-tcg01150229>: HTTP status code is not handled or not allowed 2026-01-24 22:58:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-open-g-1-2-single-solenoid-valve-pilot-spr-return-220v-ac-5va-coil-blk-connection>: HTTP status code is not handled or not allowed 2026-01-24 22:58:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-18-ga-48-wx12-dx85-h-open-clip-style-7-shelf>: HTTP status code is not handled or not allowed 2026-01-24 22:58:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4c-horizontal-mailbox-n1029411-51-1-4h-single-column-7-doors-1-parcel-usps>: HTTP status code is not handled or not allowed 2026-01-24 22:58:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stanchion-2-white-w-c-hooks-incl-chain-extra-2-pole-14-base-41-height>: HTTP status code is not handled or not allowed 2026-01-24 22:58:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-work-shirt-l-ln-royal-blue-tcg01130220>: HTTP status code is not handled or not allowed 2026-01-24 22:58:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-single-shaft-open-blower-115v-1625rpm-110hp>: HTTP status code is not handled or not allowed 2026-01-24 22:58:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nekoosa-fast-pack-digital-carbonless-paper-nek17392-8-1-2-11-pink-canary-white-2500-sheets>: HTTP status code is not handled or not allowed 2026-01-24 22:58:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/capacitor-start-rigid-base-1725rpm-115208-230v-1-12hp>: HTTP status code is not handled or not allowed 2026-01-24 22:58:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-winter-knit-cap-12-x-7-1-4-black-hnc2bk>: HTTP status code is not handled or not allowed 2026-01-24 22:58:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-starter-36inw-x-12ind-x-72inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 22:58:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-month-wall-calendar-blue-and-red-22-x-29-2015-aagpm928>: HTTP status code is not handled or not allowed 2026-01-24 22:58:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/utility_racks/dunnage_racks>: HTTP status code is not handled or not allowed 2026-01-24 22:58:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-18-ga-48-wx30-dx97-h-open-clip-style-7-shelf>: HTTP status code is not handled or not allowed 2026-01-24 22:58:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-locker-accessory-closed-side-base-12d-x-6h-725-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:58:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1481161-shelf-ref-16-inch-d-x-15-9-16-w-left-side-w-clips-for-true-manufacturing>: HTTP status code is not handled or not allowed 2026-01-24 22:58:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lyon-locker-double-tier-12x12x36-2-door-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-24 22:58:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oil-only-drumtop-pads-btop-8-gallon-cap-22-dia-polypropylene-25-per-pack>: HTTP status code is not handled or not allowed 2026-01-24 22:58:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-58-stock-115208-230v-1550rpm-18hp>: HTTP status code is not handled or not allowed 2026-01-24 22:58:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-add-on-60inw-x-48ind-x-72inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 22:58:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/282296-washer-slide-rod-pk-5-for-hobart>: HTTP status code is not handled or not allowed 2026-01-24 22:58:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grade-e-1-5um-binderless-glass-microfiber-filter-4-7cm-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:58:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c-thru-deluxe-unassembled-steel-wardrobe-cabinet-36w-x-18d-x-78h-black>: HTTP status code is not handled or not allowed 2026-01-24 22:58:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/264258-reducing-coupling-1-2-inch-ftg-x-3-8-inch>: HTTP status code is not handled or not allowed 2026-01-24 22:58:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/direct-drive-blower-1075rpm-115v-14hp-4-12>: HTTP status code is not handled or not allowed 2026-01-24 22:58:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m5-x-0-8-x-10mm-security-machine-screw-flat-torx-head-18-8-stainless-steel-ft-unc-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:58:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-dbl-solenoid-valve-pilot-spring-cntred-closed-cntr-g1-2-24v-dc-2w-coil-black-conn>: HTTP status code is not handled or not allowed 2026-01-24 22:58:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lightweight-long-sleeve-fr-t-shirt-3xl-desert-sand-df2-cm-446ls-ds-3xl>: HTTP status code is not handled or not allowed 2026-01-24 22:58:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/control-hygienic-bath-tissue-2-ply-250-pack-36-carton>: HTTP status code is not handled or not allowed 2026-01-24 22:58:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-16-18-x-3-security-machine-screw-round-one-way-head-steel-zinc-ft-unc-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:58:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-2-single-solenoid-valve-ext-pilot-spring-return-g-1-8-220v-ac-5va-coil-led-conn>: HTTP status code is not handled or not allowed 2026-01-24 22:58:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c-thru-deluxe-unassembled-steel-wardrobe-cabinet-36w-x-24d-x-78h-black>: HTTP status code is not handled or not allowed 2026-01-24 22:58:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/x-back-swivel-metal-barstool-vinyl-black-hercules-series>: HTTP status code is not handled or not allowed 2026-01-24 22:58:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6ab-x-2-security-sheet-metal-screw-button-hex-socket-head-302hq-18-8-stainless-steel-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:58:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-2-single-solenoid-valve-ext-pilot-spring-return-g-1-4-110v-ac-5va-coil-black-conn>: HTTP status code is not handled or not allowed 2026-01-24 22:58:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silicone-foam-ring-3-1-2-id-x-4-od-x-1-8>: HTTP status code is not handled or not allowed 2026-01-24 22:58:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pegboard-standard-tool-storage-kit-blue-white-48-x-32-x-9>: HTTP status code is not handled or not allowed 2026-01-24 22:58:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-person-box-locker-with-legs-12w-x-18d-x-12h-assembled-black>: HTTP status code is not handled or not allowed 2026-01-24 22:58:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/lockers/standard_lockers/steel_standard_lockers/standard_duty_steel_lockers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:58:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-32-x-1-2-security-machine-screw-button-torx-head-alloy-steel-black-oxide-ft-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:58:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyurethane-foam-sheet-with-acrylic-adhesive-1-8-x-39-x-78>: HTTP status code is not handled or not allowed 2026-01-24 22:58:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-shelf-truck-36x18x80-1200-pound-capacity-with-brakes>: HTTP status code is not handled or not allowed 2026-01-24 22:58:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pegboard-hobby-craft-organizer-storage-kit-white-black-32-x-32-x-9>: HTTP status code is not handled or not allowed 2026-01-24 22:59:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pegboard-master-workbench-kit-blue-white-96-x-32-x-9>: HTTP status code is not handled or not allowed 2026-01-24 22:59:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-24-x-1-1-2-security-machine-screw-flat-hex-socket-head-302hq-18-8-ss-ft-unc-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:59:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pegboard-garden-tool-board-organizer-red-blue-48-x-32-x-9>: HTTP status code is not handled or not allowed 2026-01-24 22:59:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-end-wood-shelf-truck-60-x-30-2400-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:59:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zip-seal-vinyl-pouches-5-8-in-plain>: HTTP status code is not handled or not allowed 2026-01-24 22:59:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-4978-boxwood-1-light-outdoor-hanging-clear-beveled-rustic-bronze-8w-x-13-75h>: HTTP status code is not handled or not allowed 2026-01-24 22:59:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/265088-magnet-reed-switch-for-jackson-dalton-dishwasher>: HTTP status code is not handled or not allowed 2026-01-24 22:59:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/label-holders-3-8-in-6-in-clear-self-adheres>: HTTP status code is not handled or not allowed 2026-01-24 22:59:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/lockers/standard_lockers/steel_standard_lockers/standard_duty_steel_lockers>: HTTP status code is not handled or not allowed 2026-01-24 22:59:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-20-x-1-security-machine-screw-button-hex-socket-head-alloy-steel-black-oxide-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:59:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/label-holders-2-in-6-in-clear-removable>: HTTP status code is not handled or not allowed 2026-01-24 22:59:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/262904-collar-for-hobart>: HTTP status code is not handled or not allowed 2026-01-24 22:59:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 33637 pages (at 95 pages/min), scraped 15135 items (at 17 items/min) 2026-01-24 22:59:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magliner-aluminum-platform-truck-with-smooth-deck-48-x-24-1400-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:59:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-3246-3-light-ceiling-semi-flush-frosted-white-brushed-nickel-15-25w-x-15-5h>: HTTP status code is not handled or not allowed 2026-01-24 22:59:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-1822-dupont-5-light-chandelier-satin-white-brushed-nickel-21-5w-x-16-75h>: HTTP status code is not handled or not allowed 2026-01-24 22:59:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2901006-lid-plastic-black-welded>: HTTP status code is not handled or not allowed 2026-01-24 22:59:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-king-sk3000-pallet-rack-108w-x-36d-x-144h-4040-lbs-cap-per-level-add-on-unit>: HTTP status code is not handled or not allowed 2026-01-24 22:59:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magliner-aluminum-platform-truck-with-smooth-deck-60-x-30-1200-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:59:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-bright-tea-co-iced-green-tea-with-honey-011-oz-pouch-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 22:59:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-184-ballerina-3-light-chandelier-alabaster-bell-old-bronze-20w-x-17h>: HTTP status code is not handled or not allowed 2026-01-24 22:59:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-round-hole-pegboard-24w-x-14d-x-48h-natural-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-24 22:59:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/266056-track-set-for-randell>: HTTP status code is not handled or not allowed 2026-01-24 22:59:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dough-retarding-box-18x26x3-5>: HTTP status code is not handled or not allowed 2026-01-24 22:59:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shaker-large-25-hole-base-and-s-s-top>: HTTP status code is not handled or not allowed 2026-01-24 22:59:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8ab-x-5-8-security-sheet-metal-screw-flat-spanner-head-18-8-stainless-steel-usa-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:59:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/utility-scoop-5-oz-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:59:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/machine_mounts_pads_vibration_dampers/vibration_isolating_machine_levelers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:59:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010600-handhole-plate-cover-assembly-for-cleveland-range>: HTTP status code is not handled or not allowed 2026-01-24 22:59:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inertia-womens-safety-work-boots-steel-toe-7h-size-95w-matte-black>: HTTP status code is not handled or not allowed 2026-01-24 22:59:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-plate-caster-6-mold-on-rubber-wheel-500-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 22:59:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-exit-sign-self-powered-single-face>: HTTP status code is not handled or not allowed 2026-01-24 22:59:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-16-inch-1-inch-stainless-steel-worm-gear-clamp-with-1-2-inch-band>: HTTP status code is not handled or not allowed 2026-01-24 22:59:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/digital-power-distribution-unit-vertical-pdu-16-nema5-15r-15a>: HTTP status code is not handled or not allowed 2026-01-24 22:59:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/square-edge-work-bench-top-plastic-72-w-x-36-d-x-1-5-8-thick>: HTTP status code is not handled or not allowed 2026-01-24 22:59:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-4756-gemini-4-light-pendant-slate-gray-fabric-shade-brushed-nickel-23-5w-x-22-875h>: HTTP status code is not handled or not allowed 2026-01-24 22:59:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/square-edge-work-bench-top-plastic-96-w-x-36-d-x-1-5-8-thick>: HTTP status code is not handled or not allowed 2026-01-24 22:59:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010619-rotating-lever-lock-for-cleveland-range>: HTTP status code is not handled or not allowed 2026-01-24 22:59:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/77-192-3-light-vanity-wall-strip-polished-chrome-18w-x-4-75h>: HTTP status code is not handled or not allowed 2026-01-24 22:59:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-32-x-1-security-machine-screw-button-torx-head-302hq-stainless-steel-ft-unc-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:59:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/transformers_uninterrupted_power_supplies/uninterrupted_power_supply_backup_systems already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 22:59:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/fasteners/machine_mounts_pads_vibration_dampers/vibration_isolating_machine_levelers>: HTTP status code is not handled or not allowed 2026-01-24 22:59:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22-red-buffing-pad-5-per-case>: HTTP status code is not handled or not allowed 2026-01-24 22:59:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-chrome-wire-shelving-add-on-36w-x-12-d-x-63-h>: HTTP status code is not handled or not allowed 2026-01-24 22:59:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wide-span-rack-72x30x96-with-3-shelves-no-deck-900-lb-capaity-per-level>: HTTP status code is not handled or not allowed 2026-01-24 22:59:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/omega-womens-safety-work-boots-steel-toe-6h-size-75w-black>: HTTP status code is not handled or not allowed 2026-01-24 22:59:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/77-859-1-light-round-cage-wall-fixt-polysynthetic-body-lens-black-10h>: HTTP status code is not handled or not allowed 2026-01-24 22:59:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iron-eagle-174-jacket-gold-inner-cuffs-storm-fly-front-hood-snaps-small>: HTTP status code is not handled or not allowed 2026-01-24 22:59:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-32-x-1-4-security-machine-screw-flat-torx-head-302hq-stainless-steel-ft-unc-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:59:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-in-x-30-in-work-bench-base-tan>: HTTP status code is not handled or not allowed 2026-01-24 22:59:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-532-1-light-die-cast-bulk-head-frosted-diffuser-semi-gloss-white-6-5w-x-10h>: HTTP status code is not handled or not allowed 2026-01-24 22:59:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iron-eagle-174-jacket-green-inner-cuffs-storm-fly-front-hood-snaps-xl>: HTTP status code is not handled or not allowed 2026-01-24 22:59:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wide-span-rack-96x24x84-with-3-shelves-wood-deck-1100-lb-capacity-per-level>: HTTP status code is not handled or not allowed 2026-01-24 22:59:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/transformers_uninterrupted_power_supplies/uninterrupted_power_supply_backup_systems>: HTTP status code is not handled or not allowed 2026-01-24 22:59:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-40-x-3-8-security-machine-screw-button-hex-socket-head-302hq-18-8-ss-ft-unc-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:59:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/265850-saw-guide-lower-w-plug-for-hobart>: HTTP status code is not handled or not allowed 2026-01-24 22:59:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-24-x-1-2-security-machine-screw-flat-torx-plus-head-18-8-stainless-steel-ft-unc-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 22:59:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-pallet-truck-ulm-pm-2745-30>: HTTP status code is not handled or not allowed 2026-01-24 22:59:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-sink-3-station-with-manual-faucets-wall-brackets-60l-x20wx8d>: HTTP status code is not handled or not allowed 2026-01-24 22:59:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/77-324-1-light-4-tier-garden-light-green-6-125w-x-8-625h>: HTTP status code is not handled or not allowed 2026-01-24 22:59:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-sink-3-station-with-knee-valve-operatefaucets-rounlegs-60l-x20wx8d>: HTTP status code is not handled or not allowed 2026-01-24 22:59:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gravity-womens-safety-work-boots-steel-toe-4h-size-12-cognac-brown>: HTTP status code is not handled or not allowed 2026-01-24 22:59:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/badger-neoprene-boots-plain-toe-upper-rubber-sole-steel-shank-15-h-blk-size-12>: HTTP status code is not handled or not allowed 2026-01-24 22:59:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/badger-neoprene-boots-plain-toe-upper-rubber-sole-steel-shank-12-h-blk-size-5>: HTTP status code is not handled or not allowed 2026-01-24 22:59:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-sink-3-station-with-manual-faucets-straight-legs-72l-x20wx8d>: HTTP status code is not handled or not allowed 2026-01-24 22:59:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/66-158-100-plus-8-piece-screwdriver-set>: HTTP status code is not handled or not allowed 2026-01-24 22:59:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/badger-neoprene-boots-steel-toe-upper-rubber-sole-steel-shank-15-h-blk-size-11>: HTTP status code is not handled or not allowed 2026-01-24 22:59:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-coat-rack-with-bars-wall-mount-60-w-x-24-d-x-6-h>: HTTP status code is not handled or not allowed 2026-01-24 22:59:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/77-700-1-light-flood-light-w-adjust-swivel-black-4-5w-x-8h>: HTTP status code is not handled or not allowed 2026-01-24 22:59:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/441744-sensor-for-hobart>: HTTP status code is not handled or not allowed 2026-01-24 22:59:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aware-wear-ansi-class-2-economy-mesh-vest-61451-lime>: HTTP status code is not handled or not allowed 2026-01-24 22:59:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pulsar-knee-boot-plain-toe-chevron-plus-174-outsole-15-h-blk-gray-size-7>: HTTP status code is not handled or not allowed 2026-01-24 22:59:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oil-resistant-squeegee-blade-for-b-250-r-bp-2sb-r-100-scrubber-brown-6-273-252-0>: HTTP status code is not handled or not allowed 2026-01-24 22:59:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reflective-t-shirt-long-sleeve-1-pocket-silver-tape-type-r-class-3-fl-lime-md>: HTTP status code is not handled or not allowed 2026-01-24 22:59:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-cylindrical-brush-for-b-250-r-bp-2sb-r-120-scrubber-soft-6-906-855-0>: HTTP status code is not handled or not allowed 2026-01-24 22:59:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-4966-boxwood-1-light-outdoor-wall-clear-beveled-textured-black-9-875w-x-19h>: HTTP status code is not handled or not allowed 2026-01-24 22:59:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009902-roundup-set-screw-forsprocket-7000771-10p-for-roundup-food-equipment>: HTTP status code is not handled or not allowed 2026-01-24 22:59:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/76-622-1-light-wall-mount-industrial-style-frosted-metallic-silver-5-5w-x-11h>: HTTP status code is not handled or not allowed 2026-01-24 22:59:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/icon-153-overall-fluorescent-lime-black-snap-fly-front-4xl>: HTTP status code is not handled or not allowed 2026-01-24 22:59:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-526-1-light-die-cast-bulk-head-clear-diffuser-semi-gloss-white-9w-x-11h>: HTTP status code is not handled or not allowed 2026-01-24 22:59:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aware-wear-ansi-class-2-economy-mesh-vest-61660-orange>: HTTP status code is not handled or not allowed 2026-01-24 22:59:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mover-tricycle-500-cap-3speed-coaster-brake-w-20-in-wheels-black>: HTTP status code is not handled or not allowed 2026-01-24 22:59:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2661028-kit-tune-up-shake-freezer-for-taylor-freezer>: HTTP status code is not handled or not allowed 2026-01-24 23:00:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reflector-3-in-w-x-6-in-h-white>: HTTP status code is not handled or not allowed 2026-01-24 23:00:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/valence-womens-safety-work-boots-steel-toe-7h-size-9w-chestnut-brown>: HTTP status code is not handled or not allowed 2026-01-24 23:00:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp178-emergency-shower-eyewash-sign-8-x-10-3-4>: HTTP status code is not handled or not allowed 2026-01-24 23:00:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hss-tmx-jewelers-saw-6-inch-dia-x-064-inch-face-x-1-inch-hole-x-232-teeth>: HTTP status code is not handled or not allowed 2026-01-24 23:00:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/science-lab-table-36h-with-black-phenolic-resin-top-72-30>: HTTP status code is not handled or not allowed 2026-01-24 23:00:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-2859-odeon-2-light-ceiling-flush-dome-satin-white-brushed-nickel-13-125w-x-4-875h>: HTTP status code is not handled or not allowed 2026-01-24 23:00:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cobalt-tmx-silver-and-deming-drill-1-2-inch-shank-135-splt-pt-5-8-inch-dia-x-3-inch-flute-l-x-6-inch-oal>: HTTP status code is not handled or not allowed 2026-01-24 23:00:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-x-30-18-gauge-stainless-steel-workbench>: HTTP status code is not handled or not allowed 2026-01-24 23:00:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/881301-relay-overload-andstart-capacitor-combo>: HTTP status code is not handled or not allowed 2026-01-24 23:00:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-4906-adams-1-light-outdoor-wall-frosted-textured-black-6-125w-x-13-75h>: HTTP status code is not handled or not allowed 2026-01-24 23:00:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1511046-timer-tracker-6-channel-120v>: HTTP status code is not handled or not allowed 2026-01-24 23:00:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1831191-switch-rocker-dpst-on-off-for-roundup-food-equipment>: HTTP status code is not handled or not allowed 2026-01-24 23:00:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 33733 pages (at 96 pages/min), scraped 15151 items (at 16 items/min) 2026-01-24 23:00:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/77-110-1-light-ceiling-fixt-white-ball-polished-chrome-6w-x-7-25h>: HTTP status code is not handled or not allowed 2026-01-24 23:00:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-3241-ballerina-3-light-chandelier-frosted-white-brushed-nickel-20w-x-17h>: HTTP status code is not handled or not allowed 2026-01-24 23:00:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-8482-7-x-20-6-oz-tarp-white>: HTTP status code is not handled or not allowed 2026-01-24 23:00:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hss-tmx-t-shank-gp-drills-118-11-32-inch-diameter-x-6-3-8-inch-oal-number-1-mt>: HTTP status code is not handled or not allowed 2026-01-24 23:00:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-boltless-shelving-add-on-36inw-x-18ind-x-84inh-no-deck>: HTTP status code is not handled or not allowed 2026-01-24 23:00:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hss-tmx-plain-slitting-saw-5-inch-dia-x-1-16-inch-face-x-1-1-4-inch-hole-x-40-teeth>: HTTP status code is not handled or not allowed 2026-01-24 23:00:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-8482-12-x-20-super-8-oz-tarp-brown>: HTTP status code is not handled or not allowed 2026-01-24 23:00:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a2108-18-stabilizer-metal>: HTTP status code is not handled or not allowed 2026-01-24 23:00:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-high-legal-starter-empty-bone-white>: HTTP status code is not handled or not allowed 2026-01-24 23:00:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-boltless-shelving-starter-48inw-x-18ind-x-96inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 23:00:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-high-letter-starter-empty-bone-white>: HTTP status code is not handled or not allowed 2026-01-24 23:00:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-air-push-to-connect-7-piece-outlet-block-kit>: HTTP status code is not handled or not allowed 2026-01-24 23:00:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-24-w-7-step-steel-rolling-ladder-14-d-top-step-lock-style-a>: HTTP status code is not handled or not allowed 2026-01-24 23:00:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grip-16-w-1-step-steel-rolling-ladder-10-d-top-step-1>: HTTP status code is not handled or not allowed 2026-01-24 23:00:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ego-ht2411-600cfm-blower-kit-5-0ah-battery-210w-charger>: HTTP status code is not handled or not allowed 2026-01-24 23:00:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grip-16-w-2-step-steel-rolling-ladder-10-d-top-step-1>: HTTP status code is not handled or not allowed 2026-01-24 23:00:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grip-16-w-5-step-steel-rolling-ladder-14-d-top-step>: HTTP status code is not handled or not allowed 2026-01-24 23:00:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keyway-broach-set-number-c-10-coll-36-comb>: HTTP status code is not handled or not allowed 2026-01-24 23:00:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-4-copper-fitting-x-press-w-90-176-long-turn-street-elbow>: HTTP status code is not handled or not allowed 2026-01-24 23:00:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-boltless-shelving-add-on-48inw-x-18ind-x-96inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 23:00:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-boltless-shelving-add-on-48inw-x-18ind-x-96inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 23:00:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-m-style-swivel-steel-plug-1-4-x-1-4-male-npt-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 23:00:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-boltless-shelving-starter-36inw-x-12ind-x-48inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 23:00:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unhoused-simple-spring-mount-2-long-2-wide-purple>: HTTP status code is not handled or not allowed 2026-01-24 23:00:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-305-4-light-vanity-wall-old-bronze-24w-x-4-75h>: HTTP status code is not handled or not allowed 2026-01-24 23:00:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-boltless-shelving-starter-48inw-x-12ind-x-84inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-24 23:00:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-3-8-50-ft-300-psi-air-water-with-hose>: HTTP status code is not handled or not allowed 2026-01-24 23:00:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-step-24-w-grip-all-directional-steel-ladder>: HTTP status code is not handled or not allowed 2026-01-24 23:00:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/outdoor-sling-led-dusk-to-dawn-7900l-56w-type-5-dist-120-277v-dark-bronze>: HTTP status code is not handled or not allowed 2026-01-24 23:00:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-boltless-shelving-starter-48inw-x-12ind-x-60inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 23:00:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camlok-hgc-plate-clamp-w-grip-500-lbs-0-3-8jaw>: HTTP status code is not handled or not allowed 2026-01-24 23:00:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sb-app-connector-2-0-wire-gauge-350-amp-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:00:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cyclone-hand-chain-hoist-on-plain-trolley-6-ton-10-ft-lift>: HTTP status code is not handled or not allowed 2026-01-24 23:00:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009784-clamp-rear-for-high-limit-hp-for-henny-penny>: HTTP status code is not handled or not allowed 2026-01-24 23:00:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-cantilever-heavy-duty-shelf-18-wx60-l>: HTTP status code is not handled or not allowed 2026-01-24 23:00:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/configurable-multi-platform-access-system-63-h-tower-support-mpasst7>: HTTP status code is not handled or not allowed 2026-01-24 23:00:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-pedestal-3-4-50-ft-250-psi-air-water-with-hose>: HTTP status code is not handled or not allowed 2026-01-24 23:00:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-boltless-shelving-starter-48inw-x-12ind-x-84inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 23:00:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cyclone-hand-chain-hoist-on-geared-trolley-1-4-ton-10-ft-lift>: HTTP status code is not handled or not allowed 2026-01-24 23:00:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sb-smh-connector-1-0-wire-gauge-350-amp-blue>: HTTP status code is not handled or not allowed 2026-01-24 23:00:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-single-rod-30-degree-bend-1-4-dia-steel-pegboard-hook-3-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:00:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-handtruck-fbal48-10fpn-double-handle-pneumatic-wheels-600-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 23:00:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/three-conductor-grounded-orange-cord-300v-1>: HTTP status code is not handled or not allowed 2026-01-24 23:00:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sawtrax-yel-low-safety-dolly-30-25-5-1000-lb-capysd>: HTTP status code is not handled or not allowed 2026-01-24 23:00:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-gallon-steel-drum-closed-head>: HTTP status code is not handled or not allowed 2026-01-24 23:00:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:00:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-boltless-shelving-add-on-36inw-x-12ind-x-84inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 23:00:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hurricane-360-hand-chain-hoist-1-ton-20-ft-lift>: HTTP status code is not handled or not allowed 2026-01-24 23:00:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wheeled-dual-bale-lift-pallet-lifter-2000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 23:00:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-series-36-wide-door-unheated-115160-obsidian-black>: HTTP status code is not handled or not allowed 2026-01-24 23:00:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-bin-cup-for-quantum-tip-out-storage-bin-qtb303-clear>: HTTP status code is not handled or not allowed 2026-01-24 23:00:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-boltless-shelving-add-on-36inw-x-24ind-x-96inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-24 23:00:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coil-lifter-2000-lb-capacity-14466>: HTTP status code is not handled or not allowed 2026-01-24 23:00:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/full-motion-and-pull-down-tv-wall-mount-for-43-in-73-in-tv-screen>: HTTP status code is not handled or not allowed 2026-01-24 23:00:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-cabinet-with-removable-bins-35>: HTTP status code is not handled or not allowed 2026-01-24 23:00:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safe-rotary-hopper-drop-safe-14-w-x-17-d-x-21-h-digtal-lock-black>: HTTP status code is not handled or not allowed 2026-01-24 23:00:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs>: HTTP status code is not handled or not allowed 2026-01-24 23:00:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/royal-910-16>: HTTP status code is not handled or not allowed 2026-01-24 23:01:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premier-pharma-vaccine-refrigerator-and-freezer-combo-stainless-steel-9-cu-ft>: HTTP status code is not handled or not allowed 2026-01-24 23:01:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/342284-element-cal-rod-190-inch-500w>: HTTP status code is not handled or not allowed 2026-01-24 23:01:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/locking-vertical-plate-clamp-lifting-attachment-4000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 23:01:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/light-duty-service-door-double-panel-4ft-x-7ft>: HTTP status code is not handled or not allowed 2026-01-24 23:01:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bi-metal-bandsaw-blade-1-x-0035-x-128-12-46t-for-ehb-916v>: HTTP status code is not handled or not allowed 2026-01-24 23:01:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/75t4-cl-75w-halogen-w-bi-pin-base-12v>: HTTP status code is not handled or not allowed 2026-01-24 23:01:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-inch-analog-wall-clock-with-black-narrow-bezel-frame>: HTTP status code is not handled or not allowed 2026-01-24 23:01:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/light-to-medium-duty-service-door-single-panel-beige-3ft-6in-x-8ft>: HTTP status code is not handled or not allowed 2026-01-24 23:01:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/150t3q-cl-150w-halogen-w-double-ended-base-mol-3-1-8>: HTTP status code is not handled or not allowed 2026-01-24 23:01:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/light-to-medium-duty-service-door-double-panel-black-6ft-x-7ft>: HTTP status code is not handled or not allowed 2026-01-24 23:01:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/200t3q-cl-200w-halogen-w-double-ended-base-mol-3-1-8>: HTTP status code is not handled or not allowed 2026-01-24 23:01:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/342302-element-cal-rod-45-inch-150w120v>: HTTP status code is not handled or not allowed 2026-01-24 23:01:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72073-unitzed-economy-fiberglass-disc-4-1-2-inch-x-5-8-11-inch-fine-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-24 23:01:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 33826 pages (at 93 pages/min), scraped 15174 items (at 23 items/min) 2026-01-24 23:01:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-68-inch-high-post-poly-z-brite-4-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:01:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caged-walk-through-fixed-access-ladder-gray-3>: HTTP status code is not handled or not allowed 2026-01-24 23:01:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-4-shelf-boltless-shelving-add-on-36inw-x-18ind-x-96inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 23:01:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-4-shelf-boltless-shelving-add-on-48inw-x-18ind-x-96inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 23:01:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-4-shelf-boltless-shelving-add-on-48inw-x-24ind-x-96inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 23:01:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2661103-switch-assembly-358-for-taylor-freezer>: HTTP status code is not handled or not allowed 2026-01-24 23:01:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-threaded-handle-knob-powered-roller-conveyors-box-of-10>: HTTP status code is not handled or not allowed 2026-01-24 23:01:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-63-inch-high-post-poly-z-brite-4-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:01:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-in-room-cabinet-with-5-quart-sharps-container-beige>: HTTP status code is not handled or not allowed 2026-01-24 23:01:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rigid-liner-rubbermaid-plaza-receptacle-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:01:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/water-swirl-hose-1-4-x-1-4-x-48-for-dormont-71-1418-48>: HTTP status code is not handled or not allowed 2026-01-24 23:01:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3dDiopter-fluorescent-magnifier-lamp-w-ac-receptacle-white>: HTTP status code is not handled or not allowed 2026-01-24 23:01:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-edge-work-bench-top-esd-48-w-x-30-d-x-1-1-4-thick>: HTTP status code is not handled or not allowed 2026-01-24 23:01:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2661138-drip-tray-ss-long-for-taylor-freezer>: HTTP status code is not handled or not allowed 2026-01-24 23:01:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-54-inch-high-post-silver-epoxy-4-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:01:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-ldpe-pallet-top-sheet-75in-x-75in-pkg-qty-200>: HTTP status code is not handled or not allowed 2026-01-24 23:01:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/100a21-f-34v-100w-incandescent-w-medium-base>: HTTP status code is not handled or not allowed 2026-01-24 23:01:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-density-poly-tubing-12in-6-mil-x-725ft>: HTTP status code is not handled or not allowed 2026-01-24 23:01:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-security-computer-cabinet-gray-2>: HTTP status code is not handled or not allowed 2026-01-24 23:01:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fb32t8-865-env-32w-fluorescent-w-medium-bi-pin-base-warm>: HTTP status code is not handled or not allowed 2026-01-24 23:01:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torque-limiting-socket-1-2-drive-21mm-100-ft-lbs>: HTTP status code is not handled or not allowed 2026-01-24 23:01:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-gauge-heavy-duty-36-w-x-18-d-x-84-h-cabinet>: HTTP status code is not handled or not allowed 2026-01-24 23:01:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8013255-switch-magnetic-harvest-lh-for-manitowoc-machines>: HTTP status code is not handled or not allowed 2026-01-24 23:01:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/461634-main-pcb-for-master-bilt-products>: HTTP status code is not handled or not allowed 2026-01-24 23:01:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-3-wall-specimen-transfer-bag-biohazard-pkg-qty-1000>: HTTP status code is not handled or not allowed 2026-01-24 23:01:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-density-mattress-bag-king-pkg-qty-100>: HTTP status code is not handled or not allowed 2026-01-24 23:01:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/100t4-cl-100w-halogen-w-bi-pin-base>: HTTP status code is not handled or not allowed 2026-01-24 23:01:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/266238-handle-lever-w-fl-strike-ir>: HTTP status code is not handled or not allowed 2026-01-24 23:01:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/300ps35-cl-300w-incandescent-w-mogul-base>: HTTP status code is not handled or not allowed 2026-01-24 23:01:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/federal-signal-mounting-kit-27xst-lights>: HTTP status code is not handled or not allowed 2026-01-24 23:01:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torque-stick-1-5-16-3-4-drive-475-ft-lbs-brown>: HTTP status code is not handled or not allowed 2026-01-24 23:01:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/light-duty-floor-crane-500-lb-capacity-7>: HTTP status code is not handled or not allowed 2026-01-24 23:01:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009824-drain-line-heater-19208v>: HTTP status code is not handled or not allowed 2026-01-24 23:01:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-mailers-6-12l-x-3-14w-x-1-14h-kraft-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-24 23:01:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kx20fr-3m-mc-20w-halogen-w-mini-can-base>: HTTP status code is not handled or not allowed 2026-01-24 23:01:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/long-chassis-jack-20-ton-heavy-duty>: HTTP status code is not handled or not allowed 2026-01-24 23:01:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/homak-rd04072160-rs-pro-series-72-w-x-24-d-x-40-3-8-h-16-drawer-red-roller-tool-cabinet>: HTTP status code is not handled or not allowed 2026-01-24 23:01:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/homak-bl02027052-pro-ii-series-26-1-2-w-x-24-1-4-d-x-21-3-8-h-5-drawer-blue-tool-chest>: HTTP status code is not handled or not allowed 2026-01-24 23:01:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010281-water-curtain-for-manitowoc-machines>: HTTP status code is not handled or not allowed 2026-01-24 23:01:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10500-series-l-workstation-right-pedestal-desk-rectangle-harvest>: HTTP status code is not handled or not allowed 2026-01-24 23:01:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/office-desk-with-2-drawers-60-x-24-cherry>: HTTP status code is not handled or not allowed 2026-01-24 23:01:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-drawer-parts-storage-cabinet-18-drawers-4>: HTTP status code is not handled or not allowed 2026-01-24 23:01:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bl-pc-replacement-post-carbon-filter-for-bottless-coolers>: HTTP status code is not handled or not allowed 2026-01-24 23:01:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bl-uvl-replacement-uv-light-for-blu-ct-curve-for-bottless-coolers>: HTTP status code is not handled or not allowed 2026-01-24 23:01:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24w-x-24-l-2-step-adjustable-height-step-stands-carbon-steel>: HTTP status code is not handled or not allowed 2026-01-24 23:01:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bin-nestable-6-5-8-w-x-17-7-8-d-x-4-h-blue-2>: HTTP status code is not handled or not allowed 2026-01-24 23:01:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/surface-bumper-guard-neon-fluorescent-type-c-39-3-8-inch-l-60-6722-4>: HTTP status code is not handled or not allowed 2026-01-24 23:01:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5t2c-41-5w-compact-fluorescent-w-candelabra-base-cool-white>: HTTP status code is not handled or not allowed 2026-01-24 23:01:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-wall-corrugated-boxes-w-hand-holes-20l-x-20w-x-12h-kraft-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 23:01:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/z-swivel-360-degree-1-4-npt-m-x-1-4-npt-m-heavy-duty-steel>: HTTP status code is not handled or not allowed 2026-01-24 23:01:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-gallon-hazmat-wheeled-dash-spill-kit>: HTTP status code is not handled or not allowed 2026-01-24 23:01:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8015536-evaporator-coil-for-true-manufacturing>: HTTP status code is not handled or not allowed 2026-01-24 23:01:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roller-follower-full-comp-inch-cry12vuu-double-sealed-34-od>: HTTP status code is not handled or not allowed 2026-01-24 23:01:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/separable-roller-follower-metric-nast40r-crowned-od-open-40-mm-bore-80-mm-od>: HTTP status code is not handled or not allowed 2026-01-24 23:01:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24w-x-36-l-adjustable-height-step-stands-carbon-steel>: HTTP status code is not handled or not allowed 2026-01-24 23:01:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zerk-kit-80pc-metric>: HTTP status code is not handled or not allowed 2026-01-24 23:01:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/separable-roller-follower-metric-nast15zzr-crowned-od-double-shielded-15-mm-bore-35-mm-od>: HTTP status code is not handled or not allowed 2026-01-24 23:01:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/-uca-u-channel-adaptor-bases-post-mount>: HTTP status code is not handled or not allowed 2026-01-24 23:01:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lev-r-matic-push-lever-seat-cvr-dsp-whi>: HTTP status code is not handled or not allowed 2026-01-24 23:01:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-wall-flat-panel-corrugated-tv-boxes-56l-x-8w-x-36h-kraft-pack-of-5>: HTTP status code is not handled or not allowed 2026-01-24 23:01:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2532000-module-control-solid-state>: HTTP status code is not handled or not allowed 2026-01-24 23:01:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-depth-cardboard-corrugated-boxes-12l-x-12w-x-4h-kraft-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-24 23:01:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a20p20-panel-nema-12-1700x1700-fits-20x20-steelwhite>: HTTP status code is not handled or not allowed 2026-01-24 23:01:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bin-nestable-2>: HTTP status code is not handled or not allowed 2026-01-24 23:01:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cushion-stat-dyna-shield-anti-static-mat-3-by-10-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:01:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/locking-tire-sling-for-use-with-large-tires-up-to-39-6-500-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-24 23:01:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tire-inflation-safety-cage-5-bar-heavy-duty-steel>: HTTP status code is not handled or not allowed 2026-01-24 23:01:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-ton-ironworker-3-phase-575v-iw60-3p575>: HTTP status code is not handled or not allowed 2026-01-24 23:01:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/120-ton-ironworker-3-phase-380v-50-hz-w-hydraulic-accessory-pack-iw120-3p380-ac900>: HTTP status code is not handled or not allowed 2026-01-24 23:01:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/checkmate-loose-wheel-nut-indicator-bag-of-100-high-temp-38mm-hi-vis-yellow>: HTTP status code is not handled or not allowed 2026-01-24 23:01:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tote-box-dividable-20>: HTTP status code is not handled or not allowed 2026-01-24 23:01:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2881082-probe-thermostat-for-accu-temp>: HTTP status code is not handled or not allowed 2026-01-24 23:01:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tote-box-dividable-27>: HTTP status code is not handled or not allowed 2026-01-24 23:01:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-decorative-light-fixture-9w-3000k-120v-4-12l-juneau-lensbrushed-nickel>: HTTP status code is not handled or not allowed 2026-01-24 23:01:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scotch-brite-griddle-polish-pad-4x525-320>: HTTP status code is not handled or not allowed 2026-01-24 23:01:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/481129-hi-limit-fenwal-for-henny-penny>: HTTP status code is not handled or not allowed 2026-01-24 23:01:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tote-box-accessory-short-divider-17>: HTTP status code is not handled or not allowed 2026-01-24 23:01:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/481159-hi-limit-for-blodgett-oven>: HTTP status code is not handled or not allowed 2026-01-24 23:01:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ind-qlty-steel-wool-hand-pad-00-fine-126>: HTTP status code is not handled or not allowed 2026-01-24 23:01:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/travira-36-bar-table-tekwood-vintage>: HTTP status code is not handled or not allowed 2026-01-24 23:01:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tote-box-accessory-short-divider-22>: HTTP status code is not handled or not allowed 2026-01-24 23:01:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/511514-pilot-burner-w-electrode-for-bk-industries>: HTTP status code is not handled or not allowed 2026-01-24 23:01:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-1-thick-x-4-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 23:02:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straddle-type-floor-lock-for-lifts>: HTTP status code is not handled or not allowed 2026-01-24 23:02:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uhmw-polyethylene-plastic-rod-4-diameter-x-1-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 23:02:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/strain-relief-cord-connectors-15>: HTTP status code is not handled or not allowed 2026-01-24 23:02:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3ca9-flicker-3w-incandescent-w-medium-base>: HTTP status code is not handled or not allowed 2026-01-24 23:02:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kdw-flush-access-doors-for-drywall-surf-lock-18wx18h-kdwp1818l>: HTTP status code is not handled or not allowed 2026-01-24 23:02:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-inch-traffic-cone-reflective-solid-orange-base>: HTTP status code is not handled or not allowed 2026-01-24 23:02:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-pin-compact-fluorescent-light-bulb-gx23-2-base-6500k-13w-daylight-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 23:02:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/loom-clamps-plated-steel-8>: HTTP status code is not handled or not allowed 2026-01-24 23:02:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:02:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1171379-cartridge-waterfilter-hf25-s-for-cuno-inc>: HTTP status code is not handled or not allowed 2026-01-24 23:02:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h20-reusable-earplugs-corded-nrr-26-db-400-pairscase>: HTTP status code is not handled or not allowed 2026-01-24 23:02:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 33926 pages (at 100 pages/min), scraped 15180 items (at 6 items/min) 2026-01-24 23:02:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scratch-resistant-strip-door-curtain-9-w-x-7-h>: HTTP status code is not handled or not allowed 2026-01-24 23:02:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lab-tray-25-1-2l-x-22w-25-gallon-capacity>: HTTP status code is not handled or not allowed 2026-01-24 23:02:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:02:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-1-2-thick-x-2-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 23:02:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rlph1ns-long-sleeve-pullover-hoodie-with-uv-protection-xl-hi-vis-orange>: HTTP status code is not handled or not allowed 2026-01-24 23:02:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 23:02:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hybrid-air-service-hose-gates-27571>: HTTP status code is not handled or not allowed 2026-01-24 23:02:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-acetal-plastic-bar-3-4-thick-x-1-1-2-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 23:02:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 23:02:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-stripe-2ply-straight-coolant-hose-gates-24222>: HTTP status code is not handled or not allowed 2026-01-24 23:02:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cantilever-rack-single-sided-upright-26>: HTTP status code is not handled or not allowed 2026-01-24 23:02:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/933-series-airtight-watertight-case-with-foam-933-1001-19-7-8-l-x-16-1-8-w-x-10-1-8-h-black>: HTTP status code is not handled or not allowed 2026-01-24 23:02:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/923-series-airtight-watertight-case-w-divider-923-2007-18-5-8-l-x-14-3-8-w-x-6-5-16-h-graphite>: HTTP status code is not handled or not allowed 2026-01-24 23:02:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/material-handling/cranes_hoists/hoists/electric_powered_hoists already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:02:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plt-fluorescent-bulb-gx24q3-base-32w-2400-lumens-5000k-super-white-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-24 23:02:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:02:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/boxguard-lockable-outdoor-storage-box-63-cu-ft-20-1-2-x-22-x-33-1-4-tan>: HTTP status code is not handled or not allowed 2026-01-24 23:02:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ribbed-bollard-post-sleeve-6-green>: HTTP status code is not handled or not allowed 2026-01-24 23:02:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/omni-pir-ceiling-low-voltage-sensor-with-1500-sq-ft-range-relay-photocell-off-white>: HTTP status code is not handled or not allowed 2026-01-24 23:02:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-steel-reel-holder-workbench-72-w>: HTTP status code is not handled or not allowed 2026-01-24 23:02:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-replaceable-blade-sn-ip>: HTTP status code is not handled or not allowed 2026-01-24 23:02:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/material-handling/cranes_hoists/hoists/electric_powered_hoists>: HTTP status code is not handled or not allowed 2026-01-24 23:02:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-welded-steel-container-unpainted-9>: HTTP status code is not handled or not allowed 2026-01-24 23:02:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milton-s-727w-m-style-industrial-plug-1-4-mnpt-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:02:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-24 23:02:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paper-bag-35-pound-base-weight-white-6-6-x-3-58-x-11-116-500-bundle>: HTTP status code is not handled or not allowed 2026-01-24 23:02:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-acetal-plastic-bar-1-thick-x-2-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 23:02:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-3-8-thick-x-2-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 23:02:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milton-s-776w-a-style-aro-coupler-1-4-mnpt>: HTTP status code is not handled or not allowed 2026-01-24 23:02:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perfectouch-hot-cups-16-oz-coffee-dreams-design-50bag>: HTTP status code is not handled or not allowed 2026-01-24 23:02:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-rack-accessory-bin-2>: HTTP status code is not handled or not allowed 2026-01-24 23:02:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/840-style-woven-wire-hinge-door-3w-x-7h-12-5-1-4-overall-height>: HTTP status code is not handled or not allowed 2026-01-24 23:02:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-g1sc-143-b>: HTTP status code is not handled or not allowed 2026-01-24 23:02:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/photoluminescent-ul1994-listed-wall-handrail-marking-tape-1-x-164-feet>: HTTP status code is not handled or not allowed 2026-01-24 23:02:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/composite-reel-spring-rewind-with-1-4-hose-3>: HTTP status code is not handled or not allowed 2026-01-24 23:02:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/self_luminous_emergency_exit_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:02:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-steel-reel-spring-rewind-with-1-2-hose-1>: HTTP status code is not handled or not allowed 2026-01-24 23:02:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reel-1>: HTTP status code is not handled or not allowed 2026-01-24 23:02:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/tool_holders/lathe_tool_posts_post_holders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:02:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/self_luminous_emergency_exit_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:02:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grip-16-w-5-step-steel-rolling-ladder-10-d-top-step>: HTTP status code is not handled or not allowed 2026-01-24 23:02:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-electronic-48-key-safe-cabinet-sand>: HTTP status code is not handled or not allowed 2026-01-24 23:03:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nsf-fabricated-2-compartment-24l-x-24w-bowl-812h-splash-18-left-drainboard-16-ga>: HTTP status code is not handled or not allowed 2026-01-24 23:03:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reebokrb1975-mens-retro-jogger-oxford-navy-blue-size-7-w>: HTTP status code is not handled or not allowed 2026-01-24 23:03:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-round-regressed-downlight-fixture-18w-1360-lumens-4-12dia-white-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-24 23:03:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/self_luminous_emergency_exit_signs>: HTTP status code is not handled or not allowed 2026-01-24 23:03:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-metric-hss-tm-x-hand-taps-set-of-3-m12-x-1-75-taper-plug-and-bottoming-chamfer>: HTTP status code is not handled or not allowed 2026-01-24 23:03:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m42-x-4-5-d10-taper-chamfer-iso-metric-hss-tm-x-hand-tap-ground-rh-6-flutes>: HTTP status code is not handled or not allowed 2026-01-24 23:03:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60x30-phenolic-safety-edge-power-apron-production-bench-blue-32313795>: HTTP status code is not handled or not allowed 2026-01-24 23:03:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/machining/tool_holders/lathe_tool_posts_post_holders>: HTTP status code is not handled or not allowed 2026-01-24 23:03:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/self_luminous_emergency_exit_signs landed on page that is not a product page. 2026-01-24 23:03:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60x30-shop-top-square-edge-power-apron-production-bench-blue-32313799>: HTTP status code is not handled or not allowed 2026-01-24 23:03:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m14-x-1-5-d5-bottoming-chamfer-iso-metric-hss-tm-x-hand-tap-ground-rh-4-flutes>: HTTP status code is not handled or not allowed 2026-01-24 23:03:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 34020 pages (at 94 pages/min), scraped 15227 items (at 47 items/min) 2026-01-24 23:03:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/180-torsion-spring-1-082-coil-dia-0-125-wire-dia-wound-right-302-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 23:03:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009473-hi-limit-assembly-for-vollrath-idea-company>: HTTP status code is not handled or not allowed 2026-01-24 23:03:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:03:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-a19-light-bulb-e26-base-11w-1100-lumens-2700k-warm-white-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-24 23:03:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/garage_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:03:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-round-recessed-downlight-fixture-91318w-2000-lumens-5-1516dia-white>: HTTP status code is not handled or not allowed 2026-01-24 23:03:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sand-pan-with-wing-nut-sable-fg3976l2sble>: HTTP status code is not handled or not allowed 2026-01-24 23:03:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-a19-colored-light-bulb-e26-base-3w-100-lumens-red-pack-of-3>: HTTP status code is not handled or not allowed 2026-01-24 23:03:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/notebooks_pads already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:03:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ulti-mate-garage-2-0-series-3-piece-cabinet-set-83w-x-21d-x-80h-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:03:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-w-x-30-d-maple-butcher-block-square-edge-with-drawer>: HTTP status code is not handled or not allowed 2026-01-24 23:03:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-24 23:03:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ulti-mate-garage-2-0-series-7-piece-cabinet-set-71w-x-21-14d-wall-x-80h-silver>: HTTP status code is not handled or not allowed 2026-01-24 23:03:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/cabinets/garage_cabinets>: HTTP status code is not handled or not allowed 2026-01-24 23:03:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-par38-spotlight-bulb-e26-base-15w-1200-lumens-6500k-daylight-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-24 23:03:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-hand-ctrl-remote-logic-unit-only>: HTTP status code is not handled or not allowed 2026-01-24 23:03:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/std050322-10-hp-oil-less-scroll-compressor-80-gallon-horizontal-116-psi-3-phase-208v>: HTTP status code is not handled or not allowed 2026-01-24 23:03:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-gsc-243-b>: HTTP status code is not handled or not allowed 2026-01-24 23:03:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/law-og112afotg-over-the-glass-frame-gray-anti-fog-lens-silver-frame>: HTTP status code is not handled or not allowed 2026-01-24 23:03:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/paper_stationery/notebooks_pads>: HTTP status code is not handled or not allowed 2026-01-24 23:03:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-step-steel-safety-rolling-ladder-w-weight-actuated-lock-step-24-w-serrated-step-w-cal-osha-handrail-cal-wa144014g>: HTTP status code is not handled or not allowed 2026-01-24 23:03:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/venture-patio-umbrella-with-bronze-pole-108w-olefin-antique-beige-fabric>: HTTP status code is not handled or not allowed 2026-01-24 23:03:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/venture-patio-umbrella-with-bronze-pole-108w-olefin-woven-granite-fabric>: HTTP status code is not handled or not allowed 2026-01-24 23:03:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-back-support-with-suspenders-size-25-to-30-1>: HTTP status code is not handled or not allowed 2026-01-24 23:03:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/linear_light_fixtures already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:03:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-linear-bar-vanity-light-fixture-20w-1400-lumens-18l-brushed-nickel>: HTTP status code is not handled or not allowed 2026-01-24 23:03:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-coupling-gscc-075-075-kw>: HTTP status code is not handled or not allowed 2026-01-24 23:03:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/made-in-usa-hss-zero-flute-countersink-deburring-tool-82-1>: HTTP status code is not handled or not allowed 2026-01-24 23:03:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/werner-ladder-fiberglass-extension-with-250-lb-capacity-16-h>: HTTP status code is not handled or not allowed 2026-01-24 23:03:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ia5018-6-internal-met-guard-slip-on-work-boot-steel-toe-men-s-sz-13-w-wide-brown>: HTTP status code is not handled or not allowed 2026-01-24 23:03:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-deck-truck-36-l-x-24-w-1200-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 23:03:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-deck-truck-48-l-x-30-w-1400-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 23:03:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/linear_light_fixtures landed on page that is not a product page. 2026-01-24 23:03:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-deck-truck-48-l-x-30-w-1200-lb-capacity-with-ergo-handle>: HTTP status code is not handled or not allowed 2026-01-24 23:03:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-valve-5-port-2-pos-dbl-pressure-piloted-1-4-nptf>: HTTP status code is not handled or not allowed 2026-01-24 23:03:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-under-cabinet-light-fixture-4w-320-lumens-80-cri-3000k-white>: HTTP status code is not handled or not allowed 2026-01-24 23:03:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-step-steel-safety-rolling-ladder-w-weight-actuated-lock-24-w-perforated-step-orange-w-cal-osha-handrail-cal-wa073214p-o>: HTTP status code is not handled or not allowed 2026-01-24 23:03:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/educational_materials_books/safety_training_materials already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:03:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-w-x-16-h-x-4-d-mini-pleated-merv-14-air-filter-box-style-global-industrial>: HTTP status code is not handled or not allowed 2026-01-24 23:03:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/euro-truck-with-3-wire-sides-wood-deck-60-x-30-1200-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 23:03:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-w-x-14-h-x-1-d-pleated-merv-8-standard-capacity-air-filter-wire-back-global-industrial>: HTTP status code is not handled or not allowed 2026-01-24 23:03:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-step-steel-safety-rolling-ladder-w-weight-actuated-lock-24-w-expanded-step-orange-w-cal-osha-handrail-cal-wa093214x-o>: HTTP status code is not handled or not allowed 2026-01-24 23:03:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fr-tacoma-heavyweight-zip-front-hoodie-mdt>: HTTP status code is not handled or not allowed 2026-01-24 23:03:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergotron-ds100-triple-monitor-desk-stand>: HTTP status code is not handled or not allowed 2026-01-24 23:03:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/osha-recordkeeping-compliance-manual>: HTTP status code is not handled or not allowed 2026-01-24 23:04:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergotron-200-series-combo-arm-mounting-kit-white>: HTTP status code is not handled or not allowed 2026-01-24 23:04:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bca-365218-w-bookcase-all-welded-36-wx18-dx52-h-white>: HTTP status code is not handled or not allowed 2026-01-24 23:04:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-w-x-20-h-x-1-d-pleated-merv-11-high-capacity-air-filter-global-industrial>: HTTP status code is not handled or not allowed 2026-01-24 23:04:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/educational_materials_books/safety_training_materials>: HTTP status code is not handled or not allowed 2026-01-24 23:04:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bca-367213-l-grn-bookcase-all-welded-36-wx13-dx72-h-leaf-green>: HTTP status code is not handled or not allowed 2026-01-24 23:04:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-step-stock-picking-ladder-perf-step>: HTTP status code is not handled or not allowed 2026-01-24 23:04:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-seat-rizer-with-removable-arms>: HTTP status code is not handled or not allowed 2026-01-24 23:04:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss-hose-braid-w-grooved-pipe-ends-4x14>: HTTP status code is not handled or not allowed 2026-01-24 23:04:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 34111 pages (at 91 pages/min), scraped 15271 items (at 44 items/min) 2026-01-24 23:04:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-w-x-20-h-x-1-d-pleated-merv-8-standard-capacity-air-filter-wire-back-global-industrial>: HTTP status code is not handled or not allowed 2026-01-24 23:04:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oxivir174-tb-disinfecting-wipes-characteristic-scent-160bucket-4-bucketscase>: HTTP status code is not handled or not allowed 2026-01-24 23:04:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polymer-pilaster-w-shoe-24w-x-82h-cream>: HTTP status code is not handled or not allowed 2026-01-24 23:04:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/import-copper-fitting-with-90-short-radius-elbow-c-x-c-12-od-x-38-id-10-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:04:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/earpiece-receive-only-with-translucent-tube-includes-a-3-5mm-connector>: HTTP status code is not handled or not allowed 2026-01-24 23:04:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-terrain-heavy-duty-wheel-chock-w-aluminum-cleat-polyethylene-15l-x-8w-x-8h>: HTTP status code is not handled or not allowed 2026-01-24 23:04:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oxivir-five-16-concentrate-one-step-disinfectant-cleaner-845-oz-bottle-2case>: HTTP status code is not handled or not allowed 2026-01-24 23:04:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1340161-pro-stretch-shock-absorbing-lanyard-6l-130-310-cap-lbs>: HTTP status code is not handled or not allowed 2026-01-24 23:04:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-locker-single-tier-12x18x72-without-legs-3-doors-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:04:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1340040-pro-tie-back-lanyard-6l-130-310-cap-lbs>: HTTP status code is not handled or not allowed 2026-01-24 23:04:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-pilaster-w-shoe-22w-x-82h-sage>: HTTP status code is not handled or not allowed 2026-01-24 23:04:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-step-steel-stock-picking-ladder-expanded-tread-w-cal-osha-handrail-cal-spl-6-nx>: HTTP status code is not handled or not allowed 2026-01-24 23:04:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-fold-portable-wheelchair-scooter-ramp-w-carry-handle-6-foot-30-inch>: HTTP status code is not handled or not allowed 2026-01-24 23:04:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-panel-privacy-screen>: HTTP status code is not handled or not allowed 2026-01-24 23:04:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/two_way_radios_accessories/two_way_radios already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:04:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-closed-linear-bearing-wseals-sw16uu-1id-225l>: HTTP status code is not handled or not allowed 2026-01-24 23:04:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/peninsulas already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:04:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s201013-12-poly-web-sling-endless-4240-5300-10600-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-24 23:04:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bradley-wash-fountain-circular-raising-vent>: HTTP status code is not handled or not allowed 2026-01-24 23:04:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-closed-linear-bearing-wseals-sws20uu-14id-2625l>: HTTP status code is not handled or not allowed 2026-01-24 23:04:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-squeegee-front-for-minuteman-260242>: HTTP status code is not handled or not allowed 2026-01-24 23:04:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bradley-eye-wash-station-s19-210>: HTTP status code is not handled or not allowed 2026-01-24 23:04:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1803-9-32-hss-heavy-duty-steam-oxide-135-aircraft-extension-drill>: HTTP status code is not handled or not allowed 2026-01-24 23:04:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelate-wire-shelving-36-w-x-24-d-x-63-h>: HTTP status code is not handled or not allowed 2026-01-24 23:04:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacment-hub-disk-for-nobles-tennant-377374-nobles-tennant-1042966-nobles-tennant-377364>: HTTP status code is not handled or not allowed 2026-01-24 23:04:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/two_way_radios_accessories/two_way_radios>: HTTP status code is not handled or not allowed 2026-01-24 23:04:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b151026-4-single-leg-chain-sling-w-sling-hook>: HTTP status code is not handled or not allowed 2026-01-24 23:04:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-squeegee-front-3-16-neo-a-for-nilfisk-advance-30937a>: HTTP status code is not handled or not allowed 2026-01-24 23:04:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roll-form-t-post-84h-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:04:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/warehouse-safety-meeting-kit>: HTTP status code is not handled or not allowed 2026-01-24 23:04:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/educational_materials_books/safety_training_materials already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:04:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/950-series-heavy-duty-closer-duranodic>: HTTP status code is not handled or not allowed 2026-01-24 23:04:44 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/peninsulas landed on page that is not a product page. 2026-01-24 23:04:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-squeegee-rear-a-for-nobles-tennant-1025336>: HTTP status code is not handled or not allowed 2026-01-24 23:04:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b151052-5-double-leg-chain-sling-w-sling-hook>: HTTP status code is not handled or not allowed 2026-01-24 23:04:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b152021-12-quad-leg-chain-sling-w-sling-hook>: HTTP status code is not handled or not allowed 2026-01-24 23:04:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-tier-6-door-digital-locker-12inwx12indx78inh-gray-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 23:04:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-tier-6-door-digital-locker-12inwx18indx78inh-tan-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 23:04:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s152065-2-poly-web-sling-endless-4900-6200-12400-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-24 23:04:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-cylindrical-brush-3-sr-for-nilfisk-advance-56507350>: HTTP status code is not handled or not allowed 2026-01-24 23:04:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ground-pad-for-containment-berm-12l-x-60w-black>: HTTP status code is not handled or not allowed 2026-01-24 23:04:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1818-1-4-12in-oal-hss-heavy-duty-sand-blasted-118-point-carbide-tipped-masonry-drill>: HTTP status code is not handled or not allowed 2026-01-24 23:04:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bag-spill-kit-4-capacity-gallon-bag-universal>: HTTP status code is not handled or not allowed 2026-01-24 23:04:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd-cyl-locksets-entry-lock-stainless-steel-key-different>: HTTP status code is not handled or not allowed 2026-01-24 23:04:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/educational_materials_books/safety_training_materials>: HTTP status code is not handled or not allowed 2026-01-24 23:04:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-pad-holder-center-lok-3-for-factory-cat-40-433>: HTTP status code is not handled or not allowed 2026-01-24 23:04:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-tier-6-door-digital-locker-36inwx15indx78inh-gray-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 23:04:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/en53372-five-lobed-knob-6316-19750>: HTTP status code is not handled or not allowed 2026-01-24 23:04:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polymer-pilaster-w-shoe-6w-x-82h-moss>: HTTP status code is not handled or not allowed 2026-01-24 23:04:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-partition-panel-wo-brackets-5412w-sage>: HTTP status code is not handled or not allowed 2026-01-24 23:04:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edge-sealing-acrylic-lacquer-250-millaliter-non-flammable-83-0783>: HTTP status code is not handled or not allowed 2026-01-24 23:05:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-couplings-recessed-screw-cc-series-climax-2cc-062-050>: HTTP status code is not handled or not allowed 2026-01-24 23:05:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-couplings-recessed-screw-w-keyway-cc-series-climax-2cc-062-050-kw>: HTTP status code is not handled or not allowed 2026-01-24 23:05:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/narrow-mouth-polycarbonate-bottle-with-closure-4-liter-1-each>: HTTP status code is not handled or not allowed 2026-01-24 23:05:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1898-14-00mm-hss-general-purpose-bright-118-point-jobber-length-drill>: HTTP status code is not handled or not allowed 2026-01-24 23:05:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-industry-standard-clamping-couplings-w-keyway-climax-2iscc-037-037kw>: HTTP status code is not handled or not allowed 2026-01-24 23:05:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-couplings-recessed-screw-w-keyway-climax-cc-150-150-kw>: HTTP status code is not handled or not allowed 2026-01-24 23:05:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-squeegee-set-urethane-for-viper-vf91116>: HTTP status code is not handled or not allowed 2026-01-24 23:05:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 34206 pages (at 95 pages/min), scraped 15311 items (at 40 items/min) 2026-01-24 23:05:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/southwire-67962901-nm-b-pcs-12-3-g-16-2-pegy-200c-romex-simpull-nm-b-pcs-duo-cable-200>: HTTP status code is not handled or not allowed 2026-01-24 23:05:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-lightweight-short-sleeve-flame-resistant-safety-shirt-l-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:05:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1875r-17-64-hss-h-d-black-gold-135-split-point-3-flatted-shank-mechanics-length-drill>: HTTP status code is not handled or not allowed 2026-01-24 23:05:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-two-piece-industry-standard-clamping-couplings-w-keyway-climax-2miscc-35-35skw>: HTTP status code is not handled or not allowed 2026-01-24 23:05:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sk-1-heavy-duty-steak-knife-5l-black-polypropylene-handle-high-carbon-steel-pointed-tip>: HTTP status code is not handled or not allowed 2026-01-24 23:05:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slo-11-colander-w-base-11-qt-14d-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 23:05:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-safety-pant-40-waist-x-34l-inseam-navy>: HTTP status code is not handled or not allowed 2026-01-24 23:05:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-climax-2c-250-z>: HTTP status code is not handled or not allowed 2026-01-24 23:05:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/150-9-32-hss-general-purpose-steam-oxide-118-point-jobber-length-drill>: HTTP status code is not handled or not allowed 2026-01-24 23:05:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/tables/laboratory_tables already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:05:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pn-10-piano-wire-whip-10l-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 23:05:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/150asp-tn-11-hss-heavy-duty-tin-135-split-point-jobber-length-drill>: HTTP status code is not handled or not allowed 2026-01-24 23:05:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/150asp-tn-19-hss-heavy-duty-tin-135-split-point-jobber-length-drill>: HTTP status code is not handled or not allowed 2026-01-24 23:05:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/composites/composite_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:05:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-g-10-fr-4-garolite-rod-3-4-diameter-x-3-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 23:05:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-cotton-flame-resistant-henley-shirt-2xl-navy>: HTTP status code is not handled or not allowed 2026-01-24 23:05:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/tables/laboratory_tables>: HTTP status code is not handled or not allowed 2026-01-24 23:05:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/composites/composite_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:05:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-hi-vis-safety-vest-class-2-m-hi-vis-yellow>: HTTP status code is not handled or not allowed 2026-01-24 23:05:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/water_safety/life_jackets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:05:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergonomic-shovel-large-blade-yellow>: HTTP status code is not handled or not allowed 2026-01-24 23:05:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/composites/composite_rod_stock landed on page that is not a product page. 2026-01-24 23:05:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-laminate-panel-wo-brackets-5712w-khaki-brown>: HTTP status code is not handled or not allowed 2026-01-24 23:05:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-climax-c-043-s>: HTTP status code is not handled or not allowed 2026-01-24 23:05:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-climax-2c-287>: HTTP status code is not handled or not allowed 2026-01-24 23:05:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/composites/composite_sheet_stock landed on page that is not a product page. 2026-01-24 23:05:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/professional-series-1-2-hp-submersible-cast-iron-effluent-pump-vertical-switch>: HTTP status code is not handled or not allowed 2026-01-24 23:05:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/water_safety/life_jackets>: HTTP status code is not handled or not allowed 2026-01-24 23:05:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-climax-1c-250-s>: HTTP status code is not handled or not allowed 2026-01-24 23:05:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-laminate-pilaster-w-shoe-6w-x-82h-sand>: HTTP status code is not handled or not allowed 2026-01-24 23:05:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/desk_accessories/desk_organizers_letter_trays already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:05:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powered-air-purifying-respirator-kit-sr-500-520-hood-medium-large-h06-0621>: HTTP status code is not handled or not allowed 2026-01-24 23:05:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-oil-filled-cast-nylon-rod-4-500-dia-ft>: HTTP status code is not handled or not allowed 2026-01-24 23:05:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-couplings-recessed-screw-climax-cc-062-062>: HTTP status code is not handled or not allowed 2026-01-24 23:05:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-duct-tape-1-88-x-30-yards>: HTTP status code is not handled or not allowed 2026-01-24 23:05:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/marine-dock-bench-box-130-gallon-60-l-x-24-w-x-24-1-4-h-white>: HTTP status code is not handled or not allowed 2026-01-24 23:05:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-canvas-ce-phenolic-sheet-0-625-thick-36-x-48>: HTTP status code is not handled or not allowed 2026-01-24 23:05:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tough-and-clear-mounting-tape-1-x-60>: HTTP status code is not handled or not allowed 2026-01-24 23:05:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-laminate-pilaster-w-shoe-3w-x-82h-storm-solidz>: HTTP status code is not handled or not allowed 2026-01-24 23:05:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/optional-adjustable-height-shelf-for-rc2448-nashville-wire-cart>: HTTP status code is not handled or not allowed 2026-01-24 23:05:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/basic_supplies/desk_accessories/desk_organizers_letter_trays>: HTTP status code is not handled or not allowed 2026-01-24 23:05:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-laminate-pilaster-w-shoe-3w-x-82h-mission-white>: HTTP status code is not handled or not allowed 2026-01-24 23:05:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-canvas-ce-phenolic-sheet-0-375-thick-36-x-48>: HTTP status code is not handled or not allowed 2026-01-24 23:05:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-fluid-level-sight-w-esg-glass-w-o-reflector-g-1-pipe-thread-r29-b>: HTTP status code is not handled or not allowed 2026-01-24 23:05:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-guard-guardrail-base-with-toeboard-slots-yellow-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-24 23:05:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-fluid-level-sight-glass-g-1-pipe-thread-6311025>: HTTP status code is not handled or not allowed 2026-01-24 23:05:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o-201-general-purpose-polyolefin-cf-shrink-film-60-ga-10w-x-4375l-clear-1-roll>: HTTP status code is not handled or not allowed 2026-01-24 23:06:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-x-6-x-6-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 23:06:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rx723019rcmbbk-250-professional-72-19-drawer-matte-black-triple-bank-roller-cabinet>: HTTP status code is not handled or not allowed 2026-01-24 23:06:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-x-12-x-12-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 23:06:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrosion-resistant-two-piece-clamping-collar-cr-climax-cr2c-050>: HTTP status code is not handled or not allowed 2026-01-24 23:06:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tim-85d-digital-timer>: HTTP status code is not handled or not allowed 2026-01-24 23:06:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-makrofol-de1-1-g-g-pc-film-0-015-x-24-x-48-75-sheet-case>: HTTP status code is not handled or not allowed 2026-01-24 23:06:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ppok-050-measuring-pourer-1-2-oz>: HTTP status code is not handled or not allowed 2026-01-24 23:06:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-x-6-x-4-2-multi-depth-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 23:06:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-canvas-ce-phenolic-rod-1-000-dia-x-48-000-inch>: HTTP status code is not handled or not allowed 2026-01-24 23:06:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 34303 pages (at 97 pages/min), scraped 15356 items (at 45 items/min) 2026-01-24 23:06:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/phenolic-black-core-outward-swing-partition-door-w-hardware-36w-graphite-grafix>: HTTP status code is not handled or not allowed 2026-01-24 23:06:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-x-12-x-12-10-8-6-multi-depth-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 23:06:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-two-piece-industry-standard-clamping-couplings-climax-2miscc-12-12>: HTTP status code is not handled or not allowed 2026-01-24 23:06:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rigid-coupling-4-set-screws-2-90-rc4h-series-climax-rc-037-4h90>: HTTP status code is not handled or not allowed 2026-01-24 23:06:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-trash-can-with-lift-handle-32-gal-cap-black>: HTTP status code is not handled or not allowed 2026-01-24 23:06:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-set-screw-collar-climax-mc-15-s>: HTTP status code is not handled or not allowed 2026-01-24 23:06:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelf-18wx18dx76h-gray-all-welded>: HTTP status code is not handled or not allowed 2026-01-24 23:06:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-x-12-kra-mailing-tubes-with-caps-case>: HTTP status code is not handled or not allowed 2026-01-24 23:06:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-gutter-guard-rt-with-5-weir-108l-x-8-dia>: HTTP status code is not handled or not allowed 2026-01-24 23:06:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stretch-wrap-film-for-hand-dispenser-blown-90-gauge-12w-x-1500l-clear-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-24 23:06:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ptp-32a-pebbled-tumblers-32-oz-1475h>: HTTP status code is not handled or not allowed 2026-01-24 23:06:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/patriot-fully-framed-gear-locker-24wx24dx72h-blue-unassembled>: HTTP status code is not handled or not allowed 2026-01-24 23:06:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qf-reducer-5-4-5-dia-galvanized-steel>: HTTP status code is not handled or not allowed 2026-01-24 23:06:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commercial-kitchen-100-silicone-sealant-9-8-oz-stainless-steel-7079808660>: HTTP status code is not handled or not allowed 2026-01-24 23:06:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qf-pipe-5-dia-galvanized-steel>: HTTP status code is not handled or not allowed 2026-01-24 23:06:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c-1080-economy-chafer-8-qt-folding-top>: HTTP status code is not handled or not allowed 2026-01-24 23:06:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-linen-le-phenolic-sheet-0-125-thick-48-x-96>: HTTP status code is not handled or not allowed 2026-01-24 23:06:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qf-clamp-with-pin-9-dia-galvanized-steel>: HTTP status code is not handled or not allowed 2026-01-24 23:06:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0034-10-stanford-table-spoon>: HTTP status code is not handled or not allowed 2026-01-24 23:06:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/132c-ice-logger-boot-regular-tan-9-5>: HTTP status code is not handled or not allowed 2026-01-24 23:06:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/gas_engines already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:06:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0386r-iron-tuff-enhanced-visibility-high-bib-overall-regular-navy-large>: HTTP status code is not handled or not allowed 2026-01-24 23:06:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-climax-2c-162-s>: HTTP status code is not handled or not allowed 2026-01-24 23:06:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-two-piece-clamping-collar-climax-m2c-30>: HTTP status code is not handled or not allowed 2026-01-24 23:06:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0345r-iron-tuff-low-bib-overall-tall-navy-medium>: HTTP status code is not handled or not allowed 2026-01-24 23:06:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelfsecurity-boxvented24wx24dx76hblall-welded>: HTTP status code is not handled or not allowed 2026-01-24 23:06:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0385l2-hivis-iron-tuff-high-bib-overall-short-hivis-lime-yellow-4xl>: HTTP status code is not handled or not allowed 2026-01-24 23:06:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/motors_power_transmission/gas_engines>: HTTP status code is not handled or not allowed 2026-01-24 23:06:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-vision-link-light-body>: HTTP status code is not handled or not allowed 2026-01-24 23:06:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-climax-1c-018-s>: HTTP status code is not handled or not allowed 2026-01-24 23:06:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0057-silver-magic-mask-black>: HTTP status code is not handled or not allowed 2026-01-24 23:06:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-compartment-mobile-storage-locker-30-48-2000-lbs-cap-6-poly-wheels>: HTTP status code is not handled or not allowed 2026-01-24 23:06:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelate-silver-epoxy-wire-shelf-36x18>: HTTP status code is not handled or not allowed 2026-01-24 23:06:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelf-footlocker-vented-24wx24dx78hgrall-welded>: HTTP status code is not handled or not allowed 2026-01-24 23:06:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:06:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jtm-1050-mill-with-3-axis-acu-rite-g-2-millpwr-cnc>: HTTP status code is not handled or not allowed 2026-01-24 23:06:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-day-capture-cartridges-for-terminator-replacement-carts-4pk-73891>: HTTP status code is not handled or not allowed 2026-01-24 23:06:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0526-cooler-wear-trousers-regular-navy-5xl>: HTTP status code is not handled or not allowed 2026-01-24 23:06:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0526-cooler-wear-trousers-regular-navy-medium>: HTTP status code is not handled or not allowed 2026-01-24 23:06:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/azar-700778-clr-two-sided-pegboard-floor-display-w-studio-base-17-w-x-17-d-x-60-h>: HTTP status code is not handled or not allowed 2026-01-24 23:06:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ghs-exclamation-mark-pictogram-label-2-x-2-500-roll>: HTTP status code is not handled or not allowed 2026-01-24 23:06:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0344r-iron-tuff-minus-50-suit-short-navy-large>: HTTP status code is not handled or not allowed 2026-01-24 23:06:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s2454s-stainless-steel-wire-shelf-54w-x-24d>: HTTP status code is not handled or not allowed 2026-01-24 23:06:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/max-lift-standard-scissor-lift-table-hand-operated-4000-lb-capacity-72l-x-36w>: HTTP status code is not handled or not allowed 2026-01-24 23:06:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-24 23:06:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jbc-2-heavy-duty-beam-clamp-2-ton-capacity>: HTTP status code is not handled or not allowed 2026-01-24 23:07:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0381r-iron-tuff-minus-50-suit-tall-navy-4xl>: HTTP status code is not handled or not allowed 2026-01-24 23:07:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-nova-2000-extra-length-nylon-cape>: HTTP status code is not handled or not allowed 2026-01-24 23:07:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-14-x-3-self-drilling-screw-unslotted-indented-hex-washer-head-410-stainless-steel-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 23:07:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-8-thick-x-4-w-corrugated-vinyl-runner-custom-cut-length>: HTTP status code is not handled or not allowed 2026-01-24 23:07:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-vision-link-2-bay-battery-charger>: HTTP status code is not handled or not allowed 2026-01-24 23:07:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-partition-flexible-five-panel-27>: HTTP status code is not handled or not allowed 2026-01-24 23:07:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 34399 pages (at 96 pages/min), scraped 15402 items (at 46 items/min) 2026-01-24 23:07:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powerquad153-vacuum-kit-with-1-14-inlet-5-gallon-capacity>: HTTP status code is not handled or not allowed 2026-01-24 23:07:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/icd-8-ice-cream-disher-w-plastic-handle-size-8-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:07:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bash-brass-sledge-hammer-4-lb-head-12-steel-handle>: HTTP status code is not handled or not allowed 2026-01-24 23:07:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-bh-62-552-soft-face-sledge-hammer-95-lb-head-20l-steel-core-handle>: HTTP status code is not handled or not allowed 2026-01-24 23:07:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-inch-brass-interlocking-stencil-letters-and-numbers-ninty-two-piece-set>: HTTP status code is not handled or not allowed 2026-01-24 23:07:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ms3a-8s-single-fine-mesh-8-diameter-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 23:07:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-c40-silencer-assembly>: HTTP status code is not handled or not allowed 2026-01-24 23:07:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jlh-75wo-30-lever-hoist-with-overload-protection-34-ton-capacity-30-lift>: HTTP status code is not handled or not allowed 2026-01-24 23:07:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-zlink-weld-visor-without-adf-lens>: HTTP status code is not handled or not allowed 2026-01-24 23:07:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-inch-x-5-yards-self-adhesive-bandage-non-latex-kelly-green-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-24 23:07:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/01221-6-foot-shock-absorbing-lanyard-rebar-and-snap-hook>: HTTP status code is not handled or not allowed 2026-01-24 23:07:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jlh-300-15psh-lever-hoist-with-overload-protection-shipyard-hooks-3-ton-cap-15-lift>: HTTP status code is not handled or not allowed 2026-01-24 23:07:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-cover-dividable-grid-container-sold-per-carton-2>: HTTP status code is not handled or not allowed 2026-01-24 23:07:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/first_aid_wound_treatment/emergency_blankets_bivouac_sacks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:07:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0486-hoodie-sweatshirt-regular-navy-2xl>: HTTP status code is not handled or not allowed 2026-01-24 23:07:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jla-300-5-lever-hoist-3-ton-capacity-5-lift>: HTTP status code is not handled or not allowed 2026-01-24 23:07:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-double-wide-climax-d2c-018>: HTTP status code is not handled or not allowed 2026-01-24 23:07:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-30-maple-safety-edge-top-pedestal-workbench-w-6-drawers>: HTTP status code is not handled or not allowed 2026-01-24 23:07:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/maxflow-life-vest-mesh-uscgtc-type-iii-m-orange>: HTTP status code is not handled or not allowed 2026-01-24 23:07:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/k-438w-steak-knife-4-3-8l-wood-handle-pointed-tip>: HTTP status code is not handled or not allowed 2026-01-24 23:07:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-climax-c-237-s>: HTTP status code is not handled or not allowed 2026-01-24 23:07:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/first_aid_wound_treatment/emergency_blankets_bivouac_sacks>: HTTP status code is not handled or not allowed 2026-01-24 23:07:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0311-string-knit-liner-white-small>: HTTP status code is not handled or not allowed 2026-01-24 23:07:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-radex-complete-system>: HTTP status code is not handled or not allowed 2026-01-24 23:07:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dcn44-nfpa-label-number-4-5-x-5-whiteblack-5pk>: HTTP status code is not handled or not allowed 2026-01-24 23:07:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ribbed-bollard-post-sleeve-8-lime-green>: HTTP status code is not handled or not allowed 2026-01-24 23:07:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1026-midsize-regulator-34-npt-250-psi-gauge-not-included>: HTTP status code is not handled or not allowed 2026-01-24 23:07:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-carbon-brushes-for-pipe-threading-machines-604050-604051>: HTTP status code is not handled or not allowed 2026-01-24 23:07:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jtm-1254rvs-milling-machine-with-x-axis-powerfeed-5-hp-230460v>: HTTP status code is not handled or not allowed 2026-01-24 23:07:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=bench-yoke-vise-1-8-4-pipe-capacity already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:07:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-climax-c-450>: HTTP status code is not handled or not allowed 2026-01-24 23:07:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-one-piece-clamping-collar-climax-m1c-20-s>: HTTP status code is not handled or not allowed 2026-01-24 23:07:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/twpo-4-offset-spatulas-4-1-4l-blade-white-polypropylene-handle>: HTTP status code is not handled or not allowed 2026-01-24 23:07:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-locker-double-tier-12x12x36-without-legs-2-doors-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:07:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-rdb-500-rotary-draw-hydraulic-tube-pipe-bender-3-pipe-cap-3ph-220v>: HTTP status code is not handled or not allowed 2026-01-24 23:07:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-green-3-tier-wire-shelving-starter-unit-60-w-x-21-d-x-34-h>: HTTP status code is not handled or not allowed 2026-01-24 23:07:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=bench-yoke-vise-1-8-4-pipe-capacity>: HTTP status code is not handled or not allowed 2026-01-24 23:07:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mmpr-10w-round-plate-10-1-4d-round-melamine-white>: HTTP status code is not handled or not allowed 2026-01-24 23:07:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cb90pb-universal-circuit-breaker>: HTTP status code is not handled or not allowed 2026-01-24 23:07:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gauging-table-kit-for-oversize-punch-assembly-40t-100t-elite-ironworkers>: HTTP status code is not handled or not allowed 2026-01-24 23:07:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1308005-meyer-blade-guide-09916>: HTTP status code is not handled or not allowed 2026-01-24 23:07:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ut-12ht-utility-tong-coiled-spring-coiled-spring-12l-extra-heavyweight-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 23:08:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g-118-classic-shakers-w-flat-tops-2-oz>: HTTP status code is not handled or not allowed 2026-01-24 23:08:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/utility-cart-2-shelf-poly-green-36l-x-18w-x-42h-pneumatic-rigid-casters>: HTTP status code is not handled or not allowed 2026-01-24 23:08:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/viton-rubber-sheet-no-adhesive-75a-18-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 23:08:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-tier-shelving-add-on-unit-solid-galvanized-steel-24w-x-24d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 23:08:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spring-rewind-hose-reel-for-air-water-14-i-d-25-hose-300-psi>: HTTP status code is not handled or not allowed 2026-01-24 23:08:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 34497 pages (at 98 pages/min), scraped 15456 items (at 54 items/min) 2026-01-24 23:08:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/combination_metalworking_machines already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:08:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spring-rewind-hose-reel-for-air-water-14-i-d-50-hose-300-psi>: HTTP status code is not handled or not allowed 2026-01-24 23:08:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/hose_cord_reels/power_cord_reels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:08:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/no-tree-paper-bowls-16-oz-4-4-diameter-x-3-h-natural-500-carton>: HTTP status code is not handled or not allowed 2026-01-24 23:08:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vinyl-replacement-liner-24-bushel-blue-2>: HTTP status code is not handled or not allowed 2026-01-24 23:08:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paper-bowls-24-oz-4-4-diameter-x-4-4-h-white-500-carton>: HTTP status code is not handled or not allowed 2026-01-24 23:08:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-12-cubic-yard-utility-truck-base-for-rubbermaid-utility-truck>: HTTP status code is not handled or not allowed 2026-01-24 23:08:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-punch-die-set-38>: HTTP status code is not handled or not allowed 2026-01-24 23:08:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-5-rigid-plate-caster-with-hardware-includes-1-caster-and-1-hardware-kit>: HTTP status code is not handled or not allowed 2026-01-24 23:08:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-punch-die-set-532>: HTTP status code is not handled or not allowed 2026-01-24 23:08:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/viton-rubber-roll-no-adhesive-75a-18-thick-x-36-wide-x-4-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 23:08:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/combination_metalworking_machines landed on page that is not a product page. 2026-01-24 23:08:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dandux-vinyl-basket-truck-bulk-truck-10-bushel-red>: HTTP status code is not handled or not allowed 2026-01-24 23:08:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/hose_cord_reels/power_cord_reels landed on page that is not a product page. 2026-01-24 23:08:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fb-40-heavy-duty-fry-basket-rectangle-yellow-plastic>: HTTP status code is not handled or not allowed 2026-01-24 23:08:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fiberglass-tote-box-20-1-2x12-7-8x8-yellow>: HTTP status code is not handled or not allowed 2026-01-24 23:08:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/br-25-coffee-urn-brush>: HTTP status code is not handled or not allowed 2026-01-24 23:08:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-subject-notebook-wide-ruled-70-page-assorted-48-each-set>: HTTP status code is not handled or not allowed 2026-01-24 23:08:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-lock-assembly-with-keys-and-hardware-includes-1-lock-2-keys-and-mounting-hardware>: HTTP status code is not handled or not allowed 2026-01-24 23:08:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-series-spring-rewind-power-cord-reel-fluorescent-tube-light-50-cord-16-awg>: HTTP status code is not handled or not allowed 2026-01-24 23:08:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/d1-6-12-faceplate-for-elite-egh-1740-egh-1760-lathe>: HTTP status code is not handled or not allowed 2026-01-24 23:08:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-avant-caster-w--swivel-plate-and-directional-lock-2-1-2-x-3-5-8-plate-220-lb-cap>: HTTP status code is not handled or not allowed 2026-01-24 23:08:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brn-8p-plastic-brush-w-nylon-bristles-8-l>: HTTP status code is not handled or not allowed 2026-01-24 23:08:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spring-rewind-static-discharge-hand-crank-cable-reel-100-cable>: HTTP status code is not handled or not allowed 2026-01-24 23:08:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:08:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/112w1-100-100-series-welding-cable-14-twin-100-hose-cable-200-psi>: HTTP status code is not handled or not allowed 2026-01-24 23:08:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/84251-water-proof-sanding-sheets-9-inch-x-11-inch-120-grit-silicon-carbide>: HTTP status code is not handled or not allowed 2026-01-24 23:08:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/bins_totes_containers/baskets_organizers/closet_garment_organizers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:08:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tempco-immersion-heater-tank-1500-120v-316-ss>: HTTP status code is not handled or not allowed 2026-01-24 23:08:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heat-resistant-dust-collection-hose-2-12-dia-x-2l>: HTTP status code is not handled or not allowed 2026-01-24 23:08:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:08:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s-2130-s-c-carbon-shackle-bta-5-8-3-1-4t-wll>: HTTP status code is not handled or not allowed 2026-01-24 23:08:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-row-national-rep-aluminum-bleacher-with-guard-rail-21-wide>: HTTP status code is not handled or not allowed 2026-01-24 23:08:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls>: HTTP status code is not handled or not allowed 2026-01-24 23:08:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-hp-35a-hydraulic-shop-press-35-ton-max-pressure>: HTTP status code is not handled or not allowed 2026-01-24 23:08:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/bins_totes_containers/baskets_organizers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:08:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/outdoor-hazardous-building-4-drum>: HTTP status code is not handled or not allowed 2026-01-24 23:08:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets>: HTTP status code is not handled or not allowed 2026-01-24 23:08:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:08:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gd-12154-ferrules-and-stops-14-inch-aluminumpack-of-2>: HTTP status code is not handled or not allowed 2026-01-24 23:08:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s-4320-latch-kit-new-style-7-1-2t-carbon-11t-alloy-hook-3-56-l-x-1-25-w>: HTTP status code is not handled or not allowed 2026-01-24 23:08:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leeson-3-phase-washguard-duty-motor-3-2hp-3450-2850rpm-145tc-tefc-208-230-460v-60-50hz-rigid-c>: HTTP status code is not handled or not allowed 2026-01-24 23:08:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ts050-010-electric-chain-hoist-w-trolley-4-button-pendant-12-ton-cap-10-lift-460v>: HTTP status code is not handled or not allowed 2026-01-24 23:08:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:08:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/bins_totes_containers/baskets_organizers>: HTTP status code is not handled or not allowed 2026-01-24 23:08:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-strip-with-acrylic-adhesive-50a-38-thick-x-4-wide-x-5-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 23:08:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leeson-3-phase-multi-speed-motor-3-1-5hp-1725-850rpm-184-tefc-460v-60hz-40c-1-0sf-rigid>: HTTP status code is not handled or not allowed 2026-01-24 23:08:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keyboard-drawer-black>: HTTP status code is not handled or not allowed 2026-01-24 23:09:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-24 23:09:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/electric_motor_parts_accessories/motor_brakes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:09:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-profile-lithium-battery-powered-electric-pallet-jack-4400-lb-cap21wx45-516l-forks>: HTTP status code is not handled or not allowed 2026-01-24 23:09:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-hook-stake-black>: HTTP status code is not handled or not allowed 2026-01-24 23:09:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delfin-21-14-gallon-wet-dry-vacuum-2-7-hp-mistral-802-wd>: HTTP status code is not handled or not allowed 2026-01-24 23:09:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:09:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delfin-21-14-gallon-wet-dry-vacuum-air-80-wd-2v>: HTTP status code is not handled or not allowed 2026-01-24 23:09:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-24 23:09:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3754-2-12mm-extra-long-length-ball-end-screwdriver>: HTTP status code is not handled or not allowed 2026-01-24 23:09:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-sheet-with-acrylic-adhesive-50a-116-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 23:09:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/panel-truck-wood-ends>: HTTP status code is not handled or not allowed 2026-01-24 23:09:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 34585 pages (at 88 pages/min), scraped 15495 items (at 39 items/min) 2026-01-24 23:09:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/l100-200wo-30-hand-chain-hoist-with-overload-protection-2-ton-capacity-30-lift>: HTTP status code is not handled or not allowed 2026-01-24 23:09:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-0-hp-pistol-grip-non-reversible-drill-2600-rpm-3-8-inch-chuck>: HTTP status code is not handled or not allowed 2026-01-24 23:09:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/electric_motor_parts_accessories/motor_brakes>: HTTP status code is not handled or not allowed 2026-01-24 23:09:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hydraulic-box-and-pan-brake-3-hp-3-phase-220v-bb-9612h>: HTTP status code is not handled or not allowed 2026-01-24 23:09:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-solid-tarp-7-1-2-ft-x-20-ft-3011354>: HTTP status code is not handled or not allowed 2026-01-24 23:09:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls>: HTTP status code is not handled or not allowed 2026-01-24 23:09:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elkay-lzstl8wslk-water-refilling-station-bi-level-wfilter-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:09:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gripeez-finger-tips-size-11-1-2-medium-blue-1-dozen>: HTTP status code is not handled or not allowed 2026-01-24 23:09:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-tarp-system-with-mesh-tarp-6-1-2-ft-x-15-ft-dtr6515>: HTTP status code is not handled or not allowed 2026-01-24 23:09:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=sheet-metal-corner-notcher-3-hp-3-phase-220v-sn-f09-ms already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:09:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/d-island-71w-x-36d-x-29-1-2h-mahogany>: HTTP status code is not handled or not allowed 2026-01-24 23:09:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-metal-dust-collector-110v-single-phase-1-5-hp-mdc-1200-hd>: HTTP status code is not handled or not allowed 2026-01-24 23:09:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/side-mount-12-v-aluminum-mount-system-10-1-2-ft-x-8-1-3-ft-5544365>: HTTP status code is not handled or not allowed 2026-01-24 23:09:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=manual-slip-roll-24-width-20-gauge-mild-steel-capacity already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:09:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/seal-mailer-000-4-x-8-golden-yellow>: HTTP status code is not handled or not allowed 2026-01-24 23:09:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:09:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-x-30-shop-top-safety-edge-top-pedestal-workbench-w-4-drawers-cabinet>: HTTP status code is not handled or not allowed 2026-01-24 23:09:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-mesh-service-window-8-security-room>: HTTP status code is not handled or not allowed 2026-01-24 23:09:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/basic-medium-moving-boxes-18l-x-15w-x-14h-kraft-blue-8-ctn>: HTTP status code is not handled or not allowed 2026-01-24 23:09:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=sheet-metal-corner-notcher-3-hp-3-phase-220v-sn-f09-ms>: HTTP status code is not handled or not allowed 2026-01-24 23:09:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rapidroll-centre-shaft-and-crank-handle-assembly-70-7001>: HTTP status code is not handled or not allowed 2026-01-24 23:09:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-mesh-partition-security-room-10x10x8-without-roof-4-sides>: HTTP status code is not handled or not allowed 2026-01-24 23:09:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=manual-slip-roll-24-width-20-gauge-mild-steel-capacity>: HTTP status code is not handled or not allowed 2026-01-24 23:09:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls>: HTTP status code is not handled or not allowed 2026-01-24 23:09:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/hand_tools_1/pipe_tube_benders/conduit_benders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:09:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quickpick-double-hopper-bin-6-5-8x12-1-2x5-yellow>: HTTP status code is not handled or not allowed 2026-01-24 23:09:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/decorative-grid-kit-for-hs45gsp-high-intensity-heaters>: HTTP status code is not handled or not allowed 2026-01-24 23:09:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-roll-no-adhesive-60a-316-thick-x-36-wide-x-9-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 23:09:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/35111-workbrutes-hitop-work-overshoes-black-cleated-outsole-sm>: HTTP status code is not handled or not allowed 2026-01-24 23:09:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/35111-workbrutes-hitop-work-overshoes-black-cleated-outsole-xl>: HTTP status code is not handled or not allowed 2026-01-24 23:09:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-roll-no-adhesive-50a-14-thick-x-36-wide-x-10-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 23:09:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iteq2100-circuit-breaker-type-qp-2-pole-100a>: HTTP status code is not handled or not allowed 2026-01-24 23:09:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v41108-safetyflex-zipper-fly-front-hooded-coverall-sm>: HTTP status code is not handled or not allowed 2026-01-24 23:09:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/walls_ceilings/wall_tiles already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:09:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:09:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/hand_tools_1/pipe_tube_benders/conduit_benders>: HTTP status code is not handled or not allowed 2026-01-24 23:09:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/94-l-l-trim-in-louvre-granite-5-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:09:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/56-quart-lockable-storage-trunk-24-inch-l-x-15-inch-w-x-13-inch-h-od-green>: HTTP status code is not handled or not allowed 2026-01-24 23:09:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-shelving-open-shelving-36x12x39-16>: HTTP status code is not handled or not allowed 2026-01-24 23:09:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:09:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-high-security-brass-solid-body-padlocks-no-6840>: HTTP status code is not handled or not allowed 2026-01-24 23:09:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/counter_height_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:09:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-industrial-spray-nozzle-safety-orange-brass-swivel-hose-adapter>: HTTP status code is not handled or not allowed 2026-01-24 23:09:56 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/walls_ceilings/wall_tiles landed on page that is not a product page. 2026-01-24 23:09:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls>: HTTP status code is not handled or not allowed 2026-01-24 23:09:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-shelving-closed-shelving-36x12x39-9>: HTTP status code is not handled or not allowed 2026-01-24 23:10:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-x32-safety-barrier-net-net-only-orange>: HTTP status code is not handled or not allowed 2026-01-24 23:10:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips>: HTTP status code is not handled or not allowed 2026-01-24 23:10:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/cabinets/counter_height_cabinets>: HTTP status code is not handled or not allowed 2026-01-24 23:10:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/capital-locker-single-tier-12x15x72-3-door-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:10:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluidity-02f-direct-acting-solenoid-valve-3-2-nc-fkm-seal-1-2-nptf-3-mm-24v-dc>: HTTP status code is not handled or not allowed 2026-01-24 23:10:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-strip-with-acrylic-adhesive-60a-116-thick-x-4-wide-x-5-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 23:10:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-delrin-homopolymer-sheet-0-750-thick-x-12-w-x-12-l>: HTTP status code is not handled or not allowed 2026-01-24 23:10:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-framed-director-black-15w-54h>: HTTP status code is not handled or not allowed 2026-01-24 23:10:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thermoplastic-rubber-wall-base-6-in-x-48-in-fawn>: HTTP status code is not handled or not allowed 2026-01-24 23:10:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 34674 pages (at 89 pages/min), scraped 15537 items (at 42 items/min) 2026-01-24 23:10:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:10:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-framed-letter-board-message-center-24w-36h>: HTTP status code is not handled or not allowed 2026-01-24 23:10:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-ketron-peek-sheet-q-1-250-thick-x-24-w-x-24-l>: HTTP status code is not handled or not allowed 2026-01-24 23:10:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:10:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:10:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foam-insulated-shipping-kit-17l-x-17w-x-9h-white>: HTTP status code is not handled or not allowed 2026-01-24 23:10:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-430-nipple-5-8-hose-barb-industrial-interchange-3-8>: HTTP status code is not handled or not allowed 2026-01-24 23:10:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-tag-pre-strung-8-6-14l-x-3-18w-fluorescent-orange-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 23:10:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-sheet-no-adhesive-60a-116-thick-x-36-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-24 23:10:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 23:10:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-af-blend-sheet-2-000-thick-x-12-w-x-24-l>: HTTP status code is not handled or not allowed 2026-01-24 23:10:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merit-alum-40-pipe-nipple-12-6-npt-male>: HTTP status code is not handled or not allowed 2026-01-24 23:10:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls>: HTTP status code is not handled or not allowed 2026-01-24 23:10:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vinyl-wall-base-coil-4-in-x-125-in-x-120-ft-lunar-dust>: HTTP status code is not handled or not allowed 2026-01-24 23:10:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-24 23:10:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o53122-comfortbrite-snap-fly-front-overall-fluorescent-lime-md>: HTTP status code is not handled or not allowed 2026-01-24 23:10:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-large-nesting-tote-17-1-4-x-11x8-white>: HTTP status code is not handled or not allowed 2026-01-24 23:10:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silencer-plus-band-saw-blade-1-1-4-inch-w-042-thick-gauge-4-6-tpi-333-346197>: HTTP status code is not handled or not allowed 2026-01-24 23:10:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tyvek174-shipping-tag-pre-wired-5-34l-x-2-78w-white-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 23:10:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-delrin-homopolymer-rod-0-875-dia>: HTTP status code is not handled or not allowed 2026-01-24 23:10:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gray-activity-table-square-edge-top-20-w-x-40-l>: HTTP status code is not handled or not allowed 2026-01-24 23:10:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/j31107-webdri-storm-fly-front-hooded-jacket-yellow-lg>: HTTP status code is not handled or not allowed 2026-01-24 23:10:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-delrin-homopolymer-sheet-0-500-thick-x-24-000-w-x-48-000-l>: HTTP status code is not handled or not allowed 2026-01-24 23:10:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyurethane-drafting-stool-black>: HTTP status code is not handled or not allowed 2026-01-24 23:10:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-delrin-homopolymer-sheet-3-000-thick-x-12-w-x-12-l>: HTTP status code is not handled or not allowed 2026-01-24 23:10:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-delrin-homopolymer-sheet-0-625-thick-x-24-w-x-48-l>: HTTP status code is not handled or not allowed 2026-01-24 23:10:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medium-vortex-tube-15-scfm-1000-btu-hr>: HTTP status code is not handled or not allowed 2026-01-24 23:10:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-1-2x7-8-z3-36-t27-ultimate-flap-disc>: HTTP status code is not handled or not allowed 2026-01-24 23:10:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1724s11-osha-stairway-24width-11-stairs>: HTTP status code is not handled or not allowed 2026-01-24 23:10:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/din468-off-set-crank-handles-cast-iron-fixed-handle-square-bore-22mm-3-94-h-handle>: HTTP status code is not handled or not allowed 2026-01-24 23:10:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v73852-job-sight-class-2-surveyor-style-vest-fluorescent-lime-4xl5xl>: HTTP status code is not handled or not allowed 2026-01-24 23:10:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-delrin-homopolymer-rod-3-500-dia>: HTTP status code is not handled or not allowed 2026-01-24 23:10:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slatwall-cherry-shelves-3-4-x8-x14-finished-on-2-sides-and-3-edges>: HTTP status code is not handled or not allowed 2026-01-24 23:10:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/abrasives/sanding_discs/quick_change_sanding_discs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:10:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/instant-read-beverage-thermometer>: HTTP status code is not handled or not allowed 2026-01-24 23:10:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:10:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-shelf-steel-shelving-with-48-akro-mils-shelf-bins-4>: HTTP status code is not handled or not allowed 2026-01-24 23:10:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/abrasives/flap_specialty_wheels/flap_wheels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:10:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:10:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylatron-nsm-rod-6-000-dia>: HTTP status code is not handled or not allowed 2026-01-24 23:10:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-drawer-steel-x-tall-emergency-cart-lever-lock-red>: HTTP status code is not handled or not allowed 2026-01-24 23:10:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/j22208-iron-eagle-storm-fly-front-jacket-green-hood-snaps-4xl>: HTTP status code is not handled or not allowed 2026-01-24 23:10:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/l-shaped-electric-height-adjustable-desk-72-w-24-d-white-w-black-base>: HTTP status code is not handled or not allowed 2026-01-24 23:10:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/easy-cut-antivibration-neoprene-rubber-pad-6-x-6-cut-size-ecrmp-060634-fn>: HTTP status code is not handled or not allowed 2026-01-24 23:10:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chipboard-pads-8l-x-8w-kraft-675pack>: HTTP status code is not handled or not allowed 2026-01-24 23:10:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/abrasives/sanding_discs/quick_change_sanding_discs landed on page that is not a product page. 2026-01-24 23:10:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-1-2-x7-8-t27-a-cubedreg-40-grit-flap-disc>: HTTP status code is not handled or not allowed 2026-01-24 23:10:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/abrasives/flap_specialty_wheels/flap_wheels landed on page that is not a product page. 2026-01-24 23:10:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets>: HTTP status code is not handled or not allowed 2026-01-24 23:10:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-1-2x7-8-sc-80-t27-regsilicon-carbide-flap-dis>: HTTP status code is not handled or not allowed 2026-01-24 23:10:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 23:11:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/ink_toner/inkjet_ink_cartridges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:11:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/neoprene-rubber-roll-no-adhesive-70a-116-thick-x-36-wide-x-6-ft-long>: HTTP status code is not handled or not allowed 2026-01-24 23:11:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/haz-mat-boxes-for-1-gallon-plastic-jug-6l-x-6w-x-12-34h-kraft-20pack>: HTTP status code is not handled or not allowed 2026-01-24 23:11:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-sheet-no-adhesive-70a-14-thick-x-36-wide-x-36-long>: HTTP status code is not handled or not allowed 2026-01-24 23:11:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-sheet-no-adhesive-70a-18-thick-x-36-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 23:11:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tygon-3350-sanitary-silicone-tubing-abw00046-0-625-id-x-875-od-x-50-l>: HTTP status code is not handled or not allowed 2026-01-24 23:11:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 34765 pages (at 91 pages/min), scraped 15578 items (at 41 items/min) 2026-01-24 23:11:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rock-pull-plate-6-l-x-15-h-x-3-4-satin-stainless-steel-6-ctc>: HTTP status code is not handled or not allowed 2026-01-24 23:11:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/ink_toner/inkjet_ink_cartridges landed on page that is not a product page. 2026-01-24 23:11:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/expandable-add-on-rack-96x48x84-gray-with-3-levels-no-deck-800-lb-cap-per-level>: HTTP status code is not handled or not allowed 2026-01-24 23:11:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s62211-tuffenuff-plus-2-pc-suit-blue-3xl>: HTTP status code is not handled or not allowed 2026-01-24 23:11:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c53122-comfortbrite-coat-fluorescent-lime-md>: HTTP status code is not handled or not allowed 2026-01-24 23:11:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-step-heavy-duty-600-lb-cap-safety-angle-steel-rolling-ladder-perfor>: HTTP status code is not handled or not allowed 2026-01-24 23:11:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tyvek174-shipping-tag-4-34l-x-2-38w-blue-100pack>: HTTP status code is not handled or not allowed 2026-01-24 23:11:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/accuglide-2-lhb-taping-head-upper-head-2>: HTTP status code is not handled or not allowed 2026-01-24 23:11:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o53129-comfortbrite-snap-fly-front-overall-fluorescent-orange-5xl>: HTTP status code is not handled or not allowed 2026-01-24 23:11:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s62217-tuffenuff-plus-2-pc-suit-yellow-2xl>: HTTP status code is not handled or not allowed 2026-01-24 23:11:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10011r-extinguisher-cabinet-hammer-assembly-red>: HTTP status code is not handled or not allowed 2026-01-24 23:11:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machine-table-3-shelves-24-x-48>: HTTP status code is not handled or not allowed 2026-01-24 23:11:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-profile-spill-control-platform-2-drum-33-gallon>: HTTP status code is not handled or not allowed 2026-01-24 23:11:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/j22207-iron-eagle-storm-fly-front-jacket-gold-hood-snaps-4xl>: HTTP status code is not handled or not allowed 2026-01-24 23:11:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/weeb-workez-executive-aluminum-laptop-tablet-cooling-stand-black>: HTTP status code is not handled or not allowed 2026-01-24 23:11:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/accuglide2-stainless-steel-taping-head-lower-taping-head-3>: HTTP status code is not handled or not allowed 2026-01-24 23:11:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/coat_racks_umbrella_stands/luggage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:11:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mitutoyo-500-196-30cert-0-6-digimatic-digital-caliper-w-certification>: HTTP status code is not handled or not allowed 2026-01-24 23:11:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reverse-tuck-folding-cartons-2l-x-2w-x-4h-kraft-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 23:11:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/red-gpo-3-sheet-1-250-thick-x-24-w-x-48-l>: HTTP status code is not handled or not allowed 2026-01-24 23:11:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/retainer-lip-60-x-4-9706-3-02>: HTTP status code is not handled or not allowed 2026-01-24 23:11:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-54-w-x-18-d-x-63-h>: HTTP status code is not handled or not allowed 2026-01-24 23:11:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/coat_racks_umbrella_stands/luggage_racks>: HTTP status code is not handled or not allowed 2026-01-24 23:11:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-bound-wood-deck-truck-60x30-2000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 23:11:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/hoses_tubes/commercial_industrial_tubing already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:11:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/battery-operated-high-li-skid-truck-3300-lb-capacity-27-x-44>: HTTP status code is not handled or not allowed 2026-01-24 23:11:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-vinyl-tubing-1-2-i-d-x-1-8-wall-75-feet-reel>: HTTP status code is not handled or not allowed 2026-01-24 23:11:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/906-ni-m12x1-5-a-stainless-threaded-plug-with-m12-x-1-5-tapered-thread>: HTTP status code is not handled or not allowed 2026-01-24 23:11:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/great-lakes-tin-jamestown-2-x-2-nail-up-tin-ceiling-tile-in-copper-t51-08>: HTTP status code is not handled or not allowed 2026-01-24 23:11:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gift-boxes-4l-x-4w-x-4h-black-100pack>: HTTP status code is not handled or not allowed 2026-01-24 23:11:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gift-boxes-5l-x-5w-x-3h-kraft-100pack>: HTTP status code is not handled or not allowed 2026-01-24 23:11:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g2-116t-ap-3hp-oil-injected-rotary-screw-71-gallon-tank-mount-116psi-1ph-230v>: HTTP status code is not handled or not allowed 2026-01-24 23:11:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/plumbing/hoses_tubes/commercial_industrial_tubing landed on page that is not a product page. 2026-01-24 23:11:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/file-storage-boxes-with-lid-15l-x-12w-x-10h-wood-grain-12pack>: HTTP status code is not handled or not allowed 2026-01-24 23:11:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_tools/brushes_sponges_squeegees/sponges_scouring_pads already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:11:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bth-199-commercial-tank-type-water-heater-nat-gas-100-gal-cyclone-xi-199000-btu-input-high-efficiency>: HTTP status code is not handled or not allowed 2026-01-24 23:11:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dr3-b-touch-screen-digital-timer-black>: HTTP status code is not handled or not allowed 2026-01-24 23:12:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/great-lakes-tin-syracuse-2-x-2-nail-up-tin-ceiling-tile-in-clear-t50-04>: HTTP status code is not handled or not allowed 2026-01-24 23:12:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/great-lakes-tin-toronto-2-x-2-nail-up-tin-ceiling-tile-in-argento-t59-07>: HTTP status code is not handled or not allowed 2026-01-24 23:12:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/great-lakes-tin-toronto-2-x-2-nail-up-tin-ceiling-tile-in-penny-vein-t59-05>: HTTP status code is not handled or not allowed 2026-01-24 23:12:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/power_tools/power_saws_blades/power_saw_blades/hole_saws already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:12:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/great-lakes-tin-jamestown-2-x-2-lay-in-tin-ceiling-tile-in-unfinished-y51-03>: HTTP status code is not handled or not allowed 2026-01-24 23:12:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/74h-nexel-chrome-wire-shelving-60wx21d>: HTTP status code is not handled or not allowed 2026-01-24 23:12:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/sockets_bits/power_drill_driver_bits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:12:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/74h-nexel-chrome-wire-shelving-add-on-36wx21d>: HTTP status code is not handled or not allowed 2026-01-24 23:12:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 34859 pages (at 94 pages/min), scraped 15633 items (at 55 items/min) 2026-01-24 23:12:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_tools/brushes_sponges_squeegees/sponges_scouring_pads landed on page that is not a product page. 2026-01-24 23:12:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bright-air-scent-gems-odor-eliminator-cool-and-clean-blue-10-oz-6-case>: HTTP status code is not handled or not allowed 2026-01-24 23:12:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/power_tools/power_saws_blades/power_saw_blades/hole_saws landed on page that is not a product page. 2026-01-24 23:12:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gear-1230r-5234521>: HTTP status code is not handled or not allowed 2026-01-24 23:12:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/illinois-engineered-products-double-folding-gate-12w-to-14w-and-6h>: HTTP status code is not handled or not allowed 2026-01-24 23:12:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/sockets_bits/power_drill_driver_bits landed on page that is not a product page. 2026-01-24 23:12:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-purpose-safety-cleaning-towels-75-wipes-can>: HTTP status code is not handled or not allowed 2026-01-24 23:12:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/illinois-engineered-products-double-folding-gate-16w-to-18w-and-8h>: HTTP status code is not handled or not allowed 2026-01-24 23:12:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sauce-pan-w-cover-helper-handle-76-qt-1>: HTTP status code is not handled or not allowed 2026-01-24 23:12:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lf-74-heavy-duty-wire-feeder-bench-model-24-42-vac>: HTTP status code is not handled or not allowed 2026-01-24 23:12:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nipple-cga-500-510-580-590>: HTTP status code is not handled or not allowed 2026-01-24 23:12:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/versatile-stain-remover-regular-scent-7-22-lb-box>: HTTP status code is not handled or not allowed 2026-01-24 23:12:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keyguard-coverall-elastic-wrists-ankles-attached-hood-zipper-front-white-xl-25cs>: HTTP status code is not handled or not allowed 2026-01-24 23:12:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyethylene-container-20-lx13-wx6-1-4-h-3>: HTTP status code is not handled or not allowed 2026-01-24 23:12:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keyguard-lab-coat-3-pockets-elastic-wrists-snap-front-single-collar-white-2xl-30case>: HTTP status code is not handled or not allowed 2026-01-24 23:12:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fda-plastic-storage-container-with-casters-43-1-2-x-43-1-2-x-34-1-2>: HTTP status code is not handled or not allowed 2026-01-24 23:12:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/food-safe-motor-cfswdnm3554t-e-ip69-15-hp-1750-rpm-3-ph-60-hz-145t-c-3540m-tenv>: HTTP status code is not handled or not allowed 2026-01-24 23:12:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-mil-heavy-duty-polyethylene-boot-covers-clear-lg-50box-5-boxescase>: HTTP status code is not handled or not allowed 2026-01-24 23:12:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/seating/stools/task_office_stools/task_work_stools already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:12:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-storage-drawer-cabinet-36x24x60-3>: HTTP status code is not handled or not allowed 2026-01-24 23:12:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tyvek-bouffant-caphead-cover-100-latex-free-white-24-50bag>: HTTP status code is not handled or not allowed 2026-01-24 23:12:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cfswdl3507-ip69-75hp-1740rpm-1ph-60hz-56c-3532l-tefc-f1>: HTTP status code is not handled or not allowed 2026-01-24 23:12:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-storage-drawer-cabinet-36x24x40-4>: HTTP status code is not handled or not allowed 2026-01-24 23:12:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/seating/stools/task_office_stools/task_work_stools>: HTTP status code is not handled or not allowed 2026-01-24 23:12:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sharpie-peel-off-china-marker-red>: HTTP status code is not handled or not allowed 2026-01-24 23:12:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2221vs-screw-hex-hd-5510258>: HTTP status code is not handled or not allowed 2026-01-24 23:12:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-1100sf-standard-back-support-black-2-x-large>: HTTP status code is not handled or not allowed 2026-01-24 23:12:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/advanced-facial-tissue-2-ply-white-cube-box-94-sheets-box-36-boxes-case>: HTTP status code is not handled or not allowed 2026-01-24 23:12:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scrubbers-lightdutybrushdisc-261477>: HTTP status code is not handled or not allowed 2026-01-24 23:12:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/viking153-3250d-fgs-powered-air-purifying-respirator-w-standard-battery>: HTTP status code is not handled or not allowed 2026-01-24 23:13:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-lab-coat-3-pockets-elastic-wrists-snap-front-single-collar-white-3xl-30cs>: HTTP status code is not handled or not allowed 2026-01-24 23:13:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-drawer-unit-divider-kit-20-compartment-16>: HTTP status code is not handled or not allowed 2026-01-24 23:13:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jumbo-roll-bathroom-tissue-septic-safe-2-ply-white-3-4-x-1000-ft-12-rolls-case>: HTTP status code is not handled or not allowed 2026-01-24 23:13:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24k-btu-dual-zone-heat-pump-split-system-with-two-12k-indoor-high-wall-units>: HTTP status code is not handled or not allowed 2026-01-24 23:13:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 34956 pages (at 97 pages/min), scraped 15697 items (at 64 items/min) 2026-01-24 23:13:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sms-coverall-elastic-wrists-ankles-zipper-front-single-collar-white-4xl-25case>: HTTP status code is not handled or not allowed 2026-01-24 23:13:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-mil-pvc-apron-with-sewn-edge-blue-35-x-45-12bag->: HTTP status code is not handled or not allowed 2026-01-24 23:13:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/turbo-boosting-de-icer-system>: HTTP status code is not handled or not allowed 2026-01-24 23:13:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nrt4850s-gy-floor-grabber-high-traffic-mat-gray-50-by-48-1-roll>: HTTP status code is not handled or not allowed 2026-01-24 23:13:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/desk-bridge-42wx24dx29-5h-driftwood>: HTTP status code is not handled or not allowed 2026-01-24 23:13:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7060-hhcs-m8x25-5512155>: HTTP status code is not handled or not allowed 2026-01-24 23:13:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-adjustable-heavy-duty-shelf-18-wx48-l>: HTTP status code is not handled or not allowed 2026-01-24 23:13:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wooden-reception-desk-71wx30dx42h-driftwood>: HTTP status code is not handled or not allowed 2026-01-24 23:13:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/air_conditioners_chillers/ductless_air_conditioners/ductless_full_systems already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:13:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-adjustable-heavy-duty-shelf-18-wx60-l>: HTTP status code is not handled or not allowed 2026-01-24 23:13:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/81-h-stationary-unit-w-vesa-mount-beige>: HTTP status code is not handled or not allowed 2026-01-24 23:13:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2oz-natural-hdpe-wide-mouth-lab-style-bottle-with-28-415-natural-pp-linerless-cap-72pk>: HTTP status code is not handled or not allowed 2026-01-24 23:13:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/java-wicker-stacking-armchair-charcoal-pack-of-16>: HTTP status code is not handled or not allowed 2026-01-24 23:13:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hndl-grip-719a-6294170>: HTTP status code is not handled or not allowed 2026-01-24 23:13:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-dolly-handle-fdh-2436pl8-4sml-24-x-36-8-polyolefin-casters-3000-lbs>: HTTP status code is not handled or not allowed 2026-01-24 23:13:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/air_conditioners_chillers/ductless_air_conditioners/ductless_full_systems landed on page that is not a product page. 2026-01-24 23:13:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8935-class-hw-hi-vis-ranger-hat-orange-l-xl>: HTTP status code is not handled or not allowed 2026-01-24 23:13:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dl160-dual-input-true-rms-ac-voltagecurrent-datalogger10to600v01v>: HTTP status code is not handled or not allowed 2026-01-24 23:13:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_bottles_jars already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:13:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jet-jtm-949evs-mill-3-axis-newall-dp700-dro-knee-x-and-y-axis-powerfeeds-air-powered-drawbar>: HTTP status code is not handled or not allowed 2026-01-24 23:13:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergodyne-655-neoprene-elbow-sleeve-w-strap-black-x-large>: HTTP status code is not handled or not allowed 2026-01-24 23:13:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jet-jtm-949evs-mill-acu-rite-200s-dro-x-and-y-axis-powerfeeds-air-powered-drawbar-690513>: HTTP status code is not handled or not allowed 2026-01-24 23:13:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/food_beverages/snacks_candies/nutrition_bars_mixes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:13:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tool-rest-post-6295853>: HTTP status code is not handled or not allowed 2026-01-24 23:13:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rhino-34-white-permanent-poly-labels>: HTTP status code is not handled or not allowed 2026-01-24 23:13:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_bottles_jars>: HTTP status code is not handled or not allowed 2026-01-24 23:13:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jet-jtm-1-mill-3-axis-newall-dp500-dro-quill-x-axis-powerfeed-690614>: HTTP status code is not handled or not allowed 2026-01-24 23:13:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-adjustable-solid-brute-shelf-24-wx42-l>: HTTP status code is not handled or not allowed 2026-01-24 23:13:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-10w-x-40l-2-mil-clear-500pack>: HTTP status code is not handled or not allowed 2026-01-24 23:13:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-dram-vial-w-attached-24-400-green-pp-hole-cap-ptfe-ptfe-silicone-septa-144pk>: HTTP status code is not handled or not allowed 2026-01-24 23:13:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-container-site-box-od-45-x-22-1-2-x-31-with-casters-3>: HTTP status code is not handled or not allowed 2026-01-24 23:13:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-x-32mm-2ml-clear-glass-snap-top-vial-w-11mm-neck-finish-vial-only-1000pk>: HTTP status code is not handled or not allowed 2026-01-24 23:13:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=jet-jtm-1050evs2-230-mill-3-axis-acu-rite-200s-dro-quill-x-y-axis-pwrfeed-air-power-drawbar already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:13:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-3w-x-5l-1-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 23:13:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/food_beverages/snacks_candies/nutrition_bars_mixes landed on page that is not a product page. 2026-01-24 23:13:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavyweight-indoor-entrance-mat-3-8-thick-48-x-72-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:13:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-cantilever-t-bar-shelf-24-wx48-l>: HTTP status code is not handled or not allowed 2026-01-24 23:13:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/wall_mount_belts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:13:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/long-sleeve-wicking-birdseye-t-shirt-w-pocket-hi-vis-orange-xl>: HTTP status code is not handled or not allowed 2026-01-24 23:13:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/bin_shelving/steel_bin_shelving already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:13:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=jet-jtm-1050evs2-230-mill-3-axis-acu-rite-200s-dro-quill-x-y-axis-pwrfeed-air-power-drawbar>: HTTP status code is not handled or not allowed 2026-01-24 23:13:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crew-sweatshirt-hi-vis-orange-3xl>: HTTP status code is not handled or not allowed 2026-01-24 23:13:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/credenza-shell-dark-cherry-office-furniture-groupings-1>: HTTP status code is not handled or not allowed 2026-01-24 23:13:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergodyne-400-universal-wrist-wrap-black-one-size>: HTTP status code is not handled or not allowed 2026-01-24 23:13:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-20w-x-36l-1-mil-clear-500pack>: HTTP status code is not handled or not allowed 2026-01-24 23:13:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-cal-block-grip-rubber-runner-mat-2mm-x-4-x-40-rubber-rolls-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:14:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-visibility-value-bomber-jacket-hi-viz-yellow-large>: HTTP status code is not handled or not allowed 2026-01-24 23:14:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/stamps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:14:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paramount-clear-view-storage-cabinet-easy-assembly-48x24x78-tan-1>: HTTP status code is not handled or not allowed 2026-01-24 23:14:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/wall_mount_belts landed on page that is not a product page. 2026-01-24 23:14:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classic-flame-resistant-short-sleeve-t-shirt-ansi-hi-viz-yellow-medium>: HTTP status code is not handled or not allowed 2026-01-24 23:14:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washer-jdp-20mf-11406202>: HTTP status code is not handled or not allowed 2026-01-24 23:14:04 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/bin_shelving/steel_bin_shelving landed on page that is not a product page. 2026-01-24 23:14:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inventory-control-cone-with-dry-erase-decal-orange>: HTTP status code is not handled or not allowed 2026-01-24 23:14:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-3w-x-36l-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 23:14:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-5-in-1-parka-hi-viz-orange-medium>: HTTP status code is not handled or not allowed 2026-01-24 23:14:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 35046 pages (at 90 pages/min), scraped 15738 items (at 41 items/min) 2026-01-24 23:14:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/basic_supplies/stamps>: HTTP status code is not handled or not allowed 2026-01-24 23:14:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-clear-front-report-cover-tang-fasteners-letter-size-dark-blue-25-box>: HTTP status code is not handled or not allowed 2026-01-24 23:14:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-hanging-file-folder-plastic-index-tabs-1-3-tab-cut-3-1-2-tab-clear-25-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:14:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-60w-x-60l-6-mil-clear-50pack>: HTTP status code is not handled or not allowed 2026-01-24 23:14:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galvanized-shelf-truck-with-dolly-base-48x18x70-1600-pound-capacity>: HTTP status code is not handled or not allowed 2026-01-24 23:14:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-write-on-erasable-indexes-w-white-tabs-5-tab-letter-white-5-set>: HTTP status code is not handled or not allowed 2026-01-24 23:14:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/operating-elementb-424-2>: HTTP status code is not handled or not allowed 2026-01-24 23:14:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/plumbing_insulation already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:14:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speed-collection-premium-cold-weather-coverall-hi-viz-yellow-medium>: HTTP status code is not handled or not allowed 2026-01-24 23:14:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multipurpose-paper-wiper-9-3-4-x-16-3-4-white-125-box-8-boxes-carton-303362>: HTTP status code is not handled or not allowed 2026-01-24 23:14:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/410kas6orj-safety-padlock-1-12h-shackle-orange-6set>: HTTP status code is not handled or not allowed 2026-01-24 23:14:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/410kas3grn-safety-padlock-1-12h-shackle-green-3set>: HTTP status code is not handled or not allowed 2026-01-24 23:14:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/distribution-container-with-hinged-lid-24-1-2x15x13-3-4-red>: HTTP status code is not handled or not allowed 2026-01-24 23:14:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/plumbing/plumbing_insulation>: HTTP status code is not handled or not allowed 2026-01-24 23:14:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/hats_headwear already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:14:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classic-standard-wicking-t-shirt-with-pocket-ansi-hi-viz-yellow-2xl>: HTTP status code is not handled or not allowed 2026-01-24 23:14:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/cash_register_pos_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:14:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-l-x-36-w-production-bench-esd-safety-edge-blue-1>: HTTP status code is not handled or not allowed 2026-01-24 23:14:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96-l-x-36-w-production-bench-esd-square-edge-blue-2>: HTTP status code is not handled or not allowed 2026-01-24 23:14:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jazz-band-banded-hearingprotector-50-per-cs-6506>: HTTP status code is not handled or not allowed 2026-01-24 23:14:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/hats_headwear>: HTTP status code is not handled or not allowed 2026-01-24 23:14:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classic-cotton-high-visibility-t-shirt-orange-medium>: HTTP status code is not handled or not allowed 2026-01-24 23:14:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/paper_stationery/cash_register_pos_rolls landed on page that is not a product page. 2026-01-24 23:14:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:14:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sensors-44-1487-sco11-0540-21>: HTTP status code is not handled or not allowed 2026-01-24 23:14:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bracket-3520b-282>: HTTP status code is not handled or not allowed 2026-01-24 23:14:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/big-and-tall-mesh-ergonomic-drafting-stool-black>: HTTP status code is not handled or not allowed 2026-01-24 23:14:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chain-plastic-2-links-red-and-white-50-feet-trade-size-8>: HTTP status code is not handled or not allowed 2026-01-24 23:14:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards>: HTTP status code is not handled or not allowed 2026-01-24 23:14:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caps-28-1828-ham990044000>: HTTP status code is not handled or not allowed 2026-01-24 23:14:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s1107ylw-aluminum-safety-padlock-1-12w-x-3h-shackle-yellow>: HTTP status code is not handled or not allowed 2026-01-24 23:14:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/machinery_parts_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:14:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s1-safety-quick-coupling-3-8-automotive-profile-with-1-4-mnpt-connection>: HTTP status code is not handled or not allowed 2026-01-24 23:14:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/italian-ornamental-tools-trowel-square-3-4>: HTTP status code is not handled or not allowed 2026-01-24 23:14:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-safety-quick-release-coupling-1-4-body-iso-6150-profile-3-8-hose-barb-cxn>: HTTP status code is not handled or not allowed 2026-01-24 23:14:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-595-062-vl-enclosure-wall-mount-computer-workstation>: HTTP status code is not handled or not allowed 2026-01-24 23:14:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/machinery_parts_accessories landed on page that is not a product page. 2026-01-24 23:15:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-premium-locker-single-tier-15x15x72-3-door-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:15:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-14w-x-20l-6-mil-clear-250pack>: HTTP status code is not handled or not allowed 2026-01-24 23:15:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gaskets-32-1188-ham990035600>: HTTP status code is not handled or not allowed 2026-01-24 23:15:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/lathe_chucks_accessories/lathe_chuck_jaws already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:15:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/push-to-connect-fittings-3-8-npt>: HTTP status code is not handled or not allowed 2026-01-24 23:15:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-fnpt-filter>: HTTP status code is not handled or not allowed 2026-01-24 23:15:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5c-collet-chuck-steel-body-4-inch-plain-back-3974>: HTTP status code is not handled or not allowed 2026-01-24 23:15:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 35144 pages (at 98 pages/min), scraped 15798 items (at 60 items/min) 2026-01-24 23:15:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-x-19-inch-adjustable-height-steel-work-platform-9inh-to-14inh-mcos48199>: HTTP status code is not handled or not allowed 2026-01-24 23:15:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/key-6x50-ghb1340-107>: HTTP status code is not handled or not allowed 2026-01-24 23:15:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/air_compressors/compressed_air_treatment_dryers_frls/compressed_air_regulators already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:15:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/lathe_chucks_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:15:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11-300-quick-point-snap-off-blades-9-mm-3-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:15:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-step-24in-w-10in-d-top-step-steel-rolling-ladder-perforated-tread-no-handrail-kdsr004246>: HTTP status code is not handled or not allowed 2026-01-24 23:15:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reel-rack-starter-unit-36-w-x-36-d-x-96-h>: HTTP status code is not handled or not allowed 2026-01-24 23:15:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-unit-5-tier-chrome-24w-x-30l-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 23:15:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/atptw-touch-up-paint-traffic-white-12-oz-spray-can>: HTTP status code is not handled or not allowed 2026-01-24 23:15:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/lathe_chucks_accessories/lathe_chucks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:15:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/minigrip-slider-grip-reclosable-poly-bags-9w-x-12l-27-mil-clear-250pack>: HTTP status code is not handled or not allowed 2026-01-24 23:15:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powerful-magnet-tool-holders-amc24plc>: HTTP status code is not handled or not allowed 2026-01-24 23:15:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-unit-5-tier-poly-z-brite-21w-x-36l-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 23:15:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/air_compressors/compressed_air_treatment_dryers_frls/compressed_air_regulators landed on page that is not a product page. 2026-01-24 23:15:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/lathe_chucks_accessories landed on page that is not a product page. 2026-01-24 23:15:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-mm-od-nylon-tubing-natural-color-100-roll-160-500-psi>: HTTP status code is not handled or not allowed 2026-01-24 23:15:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/lathe_chucks_accessories/lathe_chucks>: HTTP status code is not handled or not allowed 2026-01-24 23:15:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-long-stacking-box-48-x-15-x-10-4-5-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:15:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-step-steel-easy-turn-rolling-ladder-serrated-tread-safety-angle-kdad114242>: HTTP status code is not handled or not allowed 2026-01-24 23:15:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-phase-drum-switch-momentary>: HTTP status code is not handled or not allowed 2026-01-24 23:15:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-x-19-inch-adjustable-height-steel-work-platform-9inh-to-14inh-mcos24199>: HTTP status code is not handled or not allowed 2026-01-24 23:15:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-nexelon-21w-x-24l-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 23:15:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-5w-x-9l-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 23:15:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/hand_tools_1/chisels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:15:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/open-end-static-shielding-bags-11w-x-14l-28-mil-transparent-100pack>: HTTP status code is not handled or not allowed 2026-01-24 23:15:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/propane-gas-infrared-heater-091030lps-straight-configuration-30-l-100000-btu>: HTTP status code is not handled or not allowed 2026-01-24 23:15:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/propane-gas-infrared-heater-094550lps-straight-configuration-50-l-200000-btu>: HTTP status code is not handled or not allowed 2026-01-24 23:15:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-48x12x72-with-4-shelves-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 23:15:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lmk23-mtg-brackets-qty-2-fits-230mm-steelzinc>: HTTP status code is not handled or not allowed 2026-01-24 23:15:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/repeat-cycle-timer-0-60-min-spdt-120-vac>: HTTP status code is not handled or not allowed 2026-01-24 23:15:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tra706t-heavy-duty-narrow-crown-staples-3-8-inch-1000-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:15:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/hand_tools_1/chisels>: HTTP status code is not handled or not allowed 2026-01-24 23:15:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/propane-gas-infrared-heater-093520lpu-u-configuration-20-l-150000-btu>: HTTP status code is not handled or not allowed 2026-01-24 23:15:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ck12-310-contactor-3-pole-460v>: HTTP status code is not handled or not allowed 2026-01-24 23:15:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-polypropylene-coverall-elastic-wrists-ankles-white-large-25-case>: HTTP status code is not handled or not allowed 2026-01-24 23:15:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shift-lever-ghb-1340-4109>: HTTP status code is not handled or not allowed 2026-01-24 23:15:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-pin-text-150031>: HTTP status code is not handled or not allowed 2026-01-24 23:16:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foot-pound-ratchet-head-torque-wrenches-6014c>: HTTP status code is not handled or not allowed 2026-01-24 23:16:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unslotted-steel-pallet-rack-beam-144inl-x-6inh-5702-lb-cap-set-of-2>: HTTP status code is not handled or not allowed 2026-01-24 23:16:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/model-no-12-straight-pipe-wrenches-12-2-pipe-capacity>: HTTP status code is not handled or not allowed 2026-01-24 23:16:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nut-ghb-1340-4208>: HTTP status code is not handled or not allowed 2026-01-24 23:16:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-2w-x-3l-8-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 23:16:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/model-no-824-aluminum-straight-pipe-wrench-24>: HTTP status code is not handled or not allowed 2026-01-24 23:16:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-rack-starter-w-steel-decking-96-w-x-36-d-x-144-h>: HTTP status code is not handled or not allowed 2026-01-24 23:16:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crowd-control-barrier-powder-coated-yellow-102-l-x-40-h-x-1-5-8-dia>: HTTP status code is not handled or not allowed 2026-01-24 23:16:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/screw-rod-mdl-ghbbdb-1340a-321718>: HTTP status code is not handled or not allowed 2026-01-24 23:16:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 35236 pages (at 92 pages/min), scraped 15847 items (at 49 items/min) 2026-01-24 23:16:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-w-floor-rack-with-12-hangers-black-2>: HTTP status code is not handled or not allowed 2026-01-24 23:16:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-cylindrical-weights-nist-class-f-0-2-lbs>: HTTP status code is not handled or not allowed 2026-01-24 23:16:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spac-457-1-2-hp-115v-37ft-shutoff-48-gpm-10ft-cord-sump-pump>: HTTP status code is not handled or not allowed 2026-01-24 23:16:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ts199-thermo-glove-anti-fatigue-lrg-open-finger-relief-from-strain-and-fatigue-rsi>: HTTP status code is not handled or not allowed 2026-01-24 23:16:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gusseted-poly-bags-16w-x-14d-x-36h-4-mil-clear-100pack>: HTTP status code is not handled or not allowed 2026-01-24 23:16:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-w-x-36-d-production-workbench-maple-butcher-block-square-edge-with-drawers-shelf-tan>: HTTP status code is not handled or not allowed 2026-01-24 23:16:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v2g-readers-eyewear-2-0-clear-lens-with-black-strap-temples>: HTTP status code is not handled or not allowed 2026-01-24 23:16:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprng-jsm-403-40329>: HTTP status code is not handled or not allowed 2026-01-24 23:16:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-cabinets-w-3674ds-b-wardrobe-closet-assembled-36x24x72-black>: HTTP status code is not handled or not allowed 2026-01-24 23:16:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-w-x-30-d-production-workbench-plastic-laminate-square-edge-with-drawers-shelf-tan>: HTTP status code is not handled or not allowed 2026-01-24 23:16:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yellow-cap-style-4-point-snap-lock-suspension-hard-hat>: HTTP status code is not handled or not allowed 2026-01-24 23:16:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/food_beverages/coffee_beverages/sports_drinks_mixes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:16:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pointer-4200-5051741>: HTTP status code is not handled or not allowed 2026-01-24 23:16:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mom-s-magical-goop-stain-removers-16-fl-oz-squeeze-bottle>: HTTP status code is not handled or not allowed 2026-01-24 23:16:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hon-310-series-4-drawer-vertical-file-26-d-letter-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:16:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-hanford-triple-post-with-support-brace-fluted-base-e1-economy-mailbox>: HTTP status code is not handled or not allowed 2026-01-24 23:16:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/food_beverages/coffee_beverages/sports_drinks_mixes>: HTTP status code is not handled or not allowed 2026-01-24 23:16:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washer-ghb-1340-6242>: HTTP status code is not handled or not allowed 2026-01-24 23:16:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/full_height_storage_cabinets/stationary_storage_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:16:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/building_construction_tape/masking_tape already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:16:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acroprint-atr480-totalizing-time-clock>: HTTP status code is not handled or not allowed 2026-01-24 23:16:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-precision-ansi-roller-chain-100-1r-1-1-4-pitch-10ft-box>: HTTP status code is not handled or not allowed 2026-01-24 23:16:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/office_furniture/literature_mail_sorters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:16:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/250-060r-inspection-tags-pack-of-20>: HTTP status code is not handled or not allowed 2026-01-24 23:16:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/segment-5051331>: HTTP status code is not handled or not allowed 2026-01-24 23:16:50 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/cabinets/full_height_storage_cabinets/stationary_storage_cabinets landed on page that is not a product page. 2026-01-24 23:16:51 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/building_construction_tape/masking_tape landed on page that is not a product page. 2026-01-24 23:16:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fibre-disc-787c-tn-quick-change-787c-5-80-25-per-inner-100-per-case>: HTTP status code is not handled or not allowed 2026-01-24 23:16:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-stacking-bin-6-x-9-1-4-x-5-yellow>: HTTP status code is not handled or not allowed 2026-01-24 23:16:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-activation-button-assembly-side-front-for-761218>: HTTP status code is not handled or not allowed 2026-01-24 23:16:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polypropylene-bags-7l-x-5w-15-mil-clear-5000pack>: HTTP status code is not handled or not allowed 2026-01-24 23:16:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-precision-ansi-nickel-plated-roller-chain-50-1np-5-8-pitch-100ft-reel>: HTTP status code is not handled or not allowed 2026-01-24 23:16:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/office_furniture/literature_mail_sorters landed on page that is not a product page. 2026-01-24 23:17:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-electronic-packaging-workbench-maple-butcher-block-safety-edge-60-x-30-w-lower-shelf-kit>: HTTP status code is not handled or not allowed 2026-01-24 23:17:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wood-crate-with-1-polyethylene-foam-lining-four-way-entry-72-l-x-48-w-x-55-12-h>: HTTP status code is not handled or not allowed 2026-01-24 23:17:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-9w-x-14l-4-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 23:17:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:17:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/41911-6mm-ceramic-nozzle-kit-3-nozzles-and-orings>: HTTP status code is not handled or not allowed 2026-01-24 23:17:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 35327 pages (at 91 pages/min), scraped 15906 items (at 59 items/min) 2026-01-24 23:17:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-step-75-incline-ladder-24-inch-w-perforated>: HTTP status code is not handled or not allowed 2026-01-24 23:17:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-step-turn-platform-24-inch-w-perforated>: HTTP status code is not handled or not allowed 2026-01-24 23:17:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/halux-universal-led-treatment-light-n50-3-p-sx-gooseneck-floor-stand>: HTTP status code is not handled or not allowed 2026-01-24 23:17:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-tpi-fan-head-non-oscillating-1>: HTTP status code is not handled or not allowed 2026-01-24 23:17:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/minigrip174-reclosable-poly-bags-13w-x-18l-4-mil-clear-500pack>: HTTP status code is not handled or not allowed 2026-01-24 23:17:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits>: HTTP status code is not handled or not allowed 2026-01-24 23:17:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/41001-40-lb-portable-deadman-blaster-steel>: HTTP status code is not handled or not allowed 2026-01-24 23:17:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cantilever-rack-straight-arm-with-2-lip-15>: HTTP status code is not handled or not allowed 2026-01-24 23:17:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-step-steel-rolling-ladder-grip-strut>: HTTP status code is not handled or not allowed 2026-01-24 23:17:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/compact_fluorescent_light_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:17:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gondola-free-standing-slider-system-with-28-variety-bins-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:17:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wla-led-workstation-light-5000-850-t-120-277v-63w-59>: HTTP status code is not handled or not allowed 2026-01-24 23:17:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/value-2-unit-midnight-blue-2-41-gal-bottles-cans-waste-2042-2>: HTTP status code is not handled or not allowed 2026-01-24 23:17:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40356-1-8-id-nozzles-3-pack-ceramic>: HTTP status code is not handled or not allowed 2026-01-24 23:17:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40072-1-4-id-ceramic-nozzle-80cfm-at-80psi>: HTTP status code is not handled or not allowed 2026-01-24 23:17:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bib12-bench-in-a-box-technical-workbench-60-w-x-30-d-esd-laminate-top-blue>: HTTP status code is not handled or not allowed 2026-01-24 23:17:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speedpress-76-ultimate-steel-safety-ruler>: HTTP status code is not handled or not allowed 2026-01-24 23:17:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/compact_fluorescent_light_bulbs landed on page that is not a product page. 2026-01-24 23:17:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/table-lock-plunger-jtm-1050-c-46>: HTTP status code is not handled or not allowed 2026-01-24 23:17:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40375-air-hose-kit-rubber>: HTTP status code is not handled or not allowed 2026-01-24 23:17:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gusseted-polypropylene-bags-6l-x-4w-x-15h--15-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 23:17:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-lid-44-gallon-rubbermaid-round-brute-container>: HTTP status code is not handled or not allowed 2026-01-24 23:17:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/350195-tpfa-y-table-powerfeed>: HTTP status code is not handled or not allowed 2026-01-24 23:17:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-size-flat-solid-cover-f-steam-table-pan-24-ga>: HTTP status code is not handled or not allowed 2026-01-24 23:17:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-20w-x-20l-4-mil-clear-250pack>: HTTP status code is not handled or not allowed 2026-01-24 23:17:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/resealable-polypropylene-bags-13l-x-10w-15-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-24 23:17:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speakman-anystream-alexandria-hand-held-showerhead-polished-chrome-finish-2-5-gpm>: HTTP status code is not handled or not allowed 2026-01-24 23:17:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spindle-nut-mdl-648hd-2648002>: HTTP status code is not handled or not allowed 2026-01-24 23:17:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washer-pt2748w-61>: HTTP status code is not handled or not allowed 2026-01-24 23:17:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rear-end-plate-jsm3330-14>: HTTP status code is not handled or not allowed 2026-01-24 23:17:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/squeeze-bottle-8-oz-clear-12-pk>: HTTP status code is not handled or not allowed 2026-01-24 23:17:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-step-steel-double-entry-mobile-platform-24inl-x-24inw-42in-handrails-wlwp152424slc-y>: HTTP status code is not handled or not allowed 2026-01-24 23:17:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bin-nestable-8-3-8-w-x-11-5-8-d-x-6-h-yellow>: HTTP status code is not handled or not allowed 2026-01-24 23:17:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-athletic-sneakers-steel-toe-boots-size-105m-black>: HTTP status code is not handled or not allowed 2026-01-24 23:17:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bin-divider-fits-4-1-8-w-x-6-h-bins>: HTTP status code is not handled or not allowed 2026-01-24 23:17:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gusseted-poly-bags-12w-x-8d-x-24h-6-mil-clear-200pack>: HTTP status code is not handled or not allowed 2026-01-24 23:17:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ccs2rbb15lg-rotational-base-bracket-15-degree-fits-45x60mm-alumltgray>: HTTP status code is not handled or not allowed 2026-01-24 23:17:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powr-flite-1-4-hp-axial-floor-dryer>: HTTP status code is not handled or not allowed 2026-01-24 23:17:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a24n20clp-medium-type-1-enclosure-2400x2000x1062-steelgray>: HTTP status code is not handled or not allowed 2026-01-24 23:17:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-block-reclosable-poly-bags-18w-x-20l-2-mil-clear-500pack>: HTTP status code is not handled or not allowed 2026-01-24 23:18:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-x-36-4-drawer-with-plastic-top-workbench>: HTTP status code is not handled or not allowed 2026-01-24 23:18:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-packing-workbench-plastic-safety-edge-with-electric-72-x-36-with-lower-shelf-kit>: HTTP status code is not handled or not allowed 2026-01-24 23:18:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rx-power-workstation-hutch-84w-x-30d-x-22-14h-blue-with-black-handle>: HTTP status code is not handled or not allowed 2026-01-24 23:18:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8inl-row-spacer>: HTTP status code is not handled or not allowed 2026-01-24 23:18:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/folding-door-gate-48-w-x-59-h-986581>: HTTP status code is not handled or not allowed 2026-01-24 23:18:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manual-water-closet-flushometer-valve-4>: HTTP status code is not handled or not allowed 2026-01-24 23:18:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mapa-solo-green-977-disposable-nitrile-gloves-powder-free-100-box-size-9-977019>: HTTP status code is not handled or not allowed 2026-01-24 23:18:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 35425 pages (at 98 pages/min), scraped 15958 items (at 52 items/min) 2026-01-24 23:18:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jbc-revolution-28-reflective-traffic-cone-chain-kit-sky-blue-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-24 23:18:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/work-benches/workbenches_worktables/cabinet_pedestal_workbenches/stationary_cabinet_pedestal_workbenches already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:18:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-ss-304-cast-welding-spud-1-npt-female>: HTTP status code is not handled or not allowed 2026-01-24 23:18:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nut-6296113>: HTTP status code is not handled or not allowed 2026-01-24 23:18:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/packaging/envelopes_mailers/shipping_envelopes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:18:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waterhog-fashion-mat-med-gray-3-x-5>: HTTP status code is not handled or not allowed 2026-01-24 23:18:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700001-general-business-first-aid-cabinet-refill-for-1001-fae-0103>: HTTP status code is not handled or not allowed 2026-01-24 23:18:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/work-benches/workbenches_worktables/cabinet_pedestal_workbenches/stationary_cabinet_pedestal_workbenches>: HTTP status code is not handled or not allowed 2026-01-24 23:18:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speedi-grille-white-floor-vent-grille-vent-cover-6-x-12>: HTTP status code is not handled or not allowed 2026-01-24 23:18:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-solid-stainless-steel-shelving-add-on-unit-5-tier-48w-x-18d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 23:18:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-1-tubing-x-1-mnpt>: HTTP status code is not handled or not allowed 2026-01-24 23:18:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22w-adjustable-height-laptop-cart-cool-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:18:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/retaining-ring-992522>: HTTP status code is not handled or not allowed 2026-01-24 23:18:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/packaging/envelopes_mailers/shipping_envelopes landed on page that is not a product page. 2026-01-24 23:18:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96-w-x-30-d-production-workbench-maple-butcher-block-square-edge-drawe-upright-shelf-tan>: HTTP status code is not handled or not allowed 2026-01-24 23:18:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6inx4in-2-mil-recloseable-poly-bag-clear-pkg-qty-1000>: HTTP status code is not handled or not allowed 2026-01-24 23:18:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/propane-regulator-5-to-60-psi-14-npt-pol-inlet-spud-and-nut>: HTTP status code is not handled or not allowed 2026-01-24 23:18:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keg-transport-skate-11-3-16-16-1-4-usable-diameter-200-lb-capacity-black>: HTTP status code is not handled or not allowed 2026-01-24 23:18:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anm5-clip-nut-pkg-for-rack-a-m5-steelzinc>: HTTP status code is not handled or not allowed 2026-01-24 23:18:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-line-part-replacement-for-kerosene-heater-3341-0038>: HTTP status code is not handled or not allowed 2026-01-24 23:18:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-solid-galvanized-steel-shelving-add-on-unit-5-tier-72w-x-18d-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 23:18:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/table-insert-64a-6290694>: HTTP status code is not handled or not allowed 2026-01-24 23:18:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elite-wall-desk-single-size-manual-close-keyed-lock-cherry>: HTTP status code is not handled or not allowed 2026-01-24 23:18:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bollard-safety-net-4-foot-l-by-22-foot-w-fits-3-1-2-inches-to-4-1-2-inches-dia-brsn-422-5>: HTTP status code is not handled or not allowed 2026-01-24 23:18:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/armorultra-job-on-pallet-3-layer-epoxy-system-1000-sq-ft-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:18:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bollard-safety-net-4-foot-l-by-22-foot-w-fits-3-5-8-inches-to-5-1-2-inches-dia-brsn-422-6>: HTTP status code is not handled or not allowed 2026-01-24 23:18:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delta-aluminum-220-series-portable-chest-57-inch-l>: HTTP status code is not handled or not allowed 2026-01-24 23:18:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/armorultra-job-on-pallet-3-layer-epoxy-system-1000-sq-ft-desert-tan>: HTTP status code is not handled or not allowed 2026-01-24 23:18:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/epoxy-mortar-patch-kit-bulk-size>: HTTP status code is not handled or not allowed 2026-01-24 23:18:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/armorclad-2-layer-epoxy-floor-kit-medium-flake-broadcast-600-sq-ft-racing-blue>: HTTP status code is not handled or not allowed 2026-01-24 23:18:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:18:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/weathershield-commercial-canopy-18w-x-50l-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:18:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/krylon-industrial-work-day-enamel-paint-red-primer>: HTTP status code is not handled or not allowed 2026-01-24 23:18:52 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-24 23:18:53 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:16 2026-01-24 23:18:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelving-starter-unit-5-tier-42w-x-21d-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 23:18:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:18:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classical-b-banded-v-belt-3-b158>: HTTP status code is not handled or not allowed 2026-01-24 23:18:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_lockout_identification_tags already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:18:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 23:18:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regulated-labels-health-flammability-reactivity-2l-x-3w-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-24 23:19:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classical-a-v-belt-a36>: HTTP status code is not handled or not allowed 2026-01-24 23:19:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:19:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ht303-high-traffic-universal-roll30w-x-300lmedium-weight>: HTTP status code is not handled or not allowed 2026-01-24 23:19:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-24 23:19:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_lockout_identification_tags>: HTTP status code is not handled or not allowed 2026-01-24 23:19:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/80076-do-not-enter-sign-high-intensity-prismatic-reflective-sign-aluminum-24w-x-24h>: HTTP status code is not handled or not allowed 2026-01-24 23:19:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 35521 pages (at 96 pages/min), scraped 16015 items (at 57 items/min) 2026-01-24 23:19:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grade-70-transport-binder-chain-5-16-dia-4700-lb-cap-16-box-bag>: HTTP status code is not handled or not allowed 2026-01-24 23:19:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-24 23:19:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heat-seal-bubble-mailers-5-10-12w-x-16l-kraft-25pack>: HTTP status code is not handled or not allowed 2026-01-24 23:19:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cast-acrylic-bar-48-l-x-2-w-x-3-16-thick-green>: HTTP status code is not handled or not allowed 2026-01-24 23:19:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelate-silver-epoxy-wire-shelving-starter-unit-5-tier-30w-x-24d-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 23:19:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/121453-hazardous-materials-identification-guide-color-bar-labels-10pkg-3w-x-5h>: HTTP status code is not handled or not allowed 2026-01-24 23:19:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/parking_traffic_signs/parking_permits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:19:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grade-43-high-test-chain-3-8-dia-5400-lb-cap-10-box>: HTTP status code is not handled or not allowed 2026-01-24 23:19:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cast-acrylic-bar-48-l-x-4-w-x-3-16-thick-red>: HTTP status code is not handled or not allowed 2026-01-24 23:19:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-x-9-x-12-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 23:19:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:19:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sko-20-oil-only-labpack-spill-kit20-gallon-drum>: HTTP status code is not handled or not allowed 2026-01-24 23:19:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-cush-n-tuf-cold-forged-4x4-12-swivel-8x2-12-ace-tuf-ball-450lb>: HTTP status code is not handled or not allowed 2026-01-24 23:19:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/parking_traffic_signs/parking_permits landed on page that is not a product page. 2026-01-24 23:19:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelving-add-on-unit-5-tier-42w-x-24d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 23:19:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stop-plate-jds-12bpds-12cs-612115>: HTTP status code is not handled or not allowed 2026-01-24 23:19:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-metal-wheel-10x3-1-roller>: HTTP status code is not handled or not allowed 2026-01-24 23:19:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:19:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/retain-ring-intl-rtw35-15-planer-6284716>: HTTP status code is not handled or not allowed 2026-01-24 23:19:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs landed on page that is not a product page. 2026-01-24 23:19:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/switch-plate-model-201planer-6012112>: HTTP status code is not handled or not allowed 2026-01-24 23:19:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/104316-toughstripe-floor-marking-tape-polyester-2w-x-100l-orange>: HTTP status code is not handled or not allowed 2026-01-24 23:19:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-x-18-x-48-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 23:19:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700431pur-hshaped-pegboard-display-wwheel-base-16w-x-60h-pl-1-pc>: HTTP status code is not handled or not allowed 2026-01-24 23:19:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:19:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 23:19:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelite-vented-plastic-shelving-starter-unit-5-tier-42w-x-24d-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 23:19:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_lockout_identification_tags already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:19:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22-x-12-x-10-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 23:19:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:19:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-nut-m20x25p-model-201-plnr-6012293>: HTTP status code is not handled or not allowed 2026-01-24 23:19:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:19:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-x-12-x-8-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-24 23:19:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700355pnk-pegboard-powerwing-display-135w-x-22h-pk-1-pc>: HTTP status code is not handled or not allowed 2026-01-24 23:19:51 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 23:19:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelon-wire-shelving-starter-unit-1-dunnage-shelf-24w-x-18d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 23:19:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-seal-bubble-mailers-2-8-12w-x-12l-white-25pack>: HTTP status code is not handled or not allowed 2026-01-24 23:19:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 23:19:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700354gre-pegboard-powerwing-display-14w-x-60h-gn-1-pc>: HTTP status code is not handled or not allowed 2026-01-24 23:19:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelite-vented-plastic-shelving-starter-unit-5-tier-72w-x-18d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 23:19:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-24 23:19:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reinforced-solid-shelf-20-inch-48-inch>: HTTP status code is not handled or not allowed 2026-01-24 23:19:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-x-5-x-40-full-over-lap-container>: HTTP status code is not handled or not allowed 2026-01-24 23:20:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-dunnage-rack-20-inch-w-36-inch-l-8-inch-h>: HTTP status code is not handled or not allowed 2026-01-24 23:20:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_lockout_identification_tags landed on page that is not a product page. 2026-01-24 23:20:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/food_beverages/snacks_candies/cookies_pastries already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:20:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700420blu-pinwheel-pegboard-display-16w-x-60h-bl-1-pc>: HTTP status code is not handled or not allowed 2026-01-24 23:20:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5ea-b05-washer-sprng-82mmx154mm-6285911>: HTTP status code is not handled or not allowed 2026-01-24 23:20:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interion-174-48-wide-desk-rustic-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:20:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-x-10-x-9-insulated-container>: HTTP status code is not handled or not allowed 2026-01-24 23:20:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labels-with-fragile-do-not-drop-print-5l-x-3w-fluorescent-yellow-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-24 23:20:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labels-with-lay-flat-print-5l-x-3w-fluorescent-green-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-24 23:20:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-foam-epdm-strip-120-l-x-1-w-x-3-4-thick-black-acrylic-adhesive>: HTTP status code is not handled or not allowed 2026-01-24 23:20:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crowd-control-stanchion-40-in-stainless-steel-post-red-belt-7-1-2-ft-l>: HTTP status code is not handled or not allowed 2026-01-24 23:20:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-x-4-clevis-pin-low-carbon-steel-zinc-clear-clpz-0500-4000>: HTTP status code is not handled or not allowed 2026-01-24 23:20:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 35612 pages (at 91 pages/min), scraped 16059 items (at 44 items/min) 2026-01-24 23:20:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/food_beverages/snacks_candies/cookies_pastries>: HTTP status code is not handled or not allowed 2026-01-24 23:20:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-magnetic-retractable-belt-barrier-black-yellow-mount-black-yellow-30-ft-l>: HTTP status code is not handled or not allowed 2026-01-24 23:20:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/610109blk-flat-revolving-display-base-075h-x-9dia-bk-10pack>: HTTP status code is not handled or not allowed 2026-01-24 23:20:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-14-1-2x20-self-seal-kra-bubble-mailers-with-opening-tear-strip-retail>: HTTP status code is not handled or not allowed 2026-01-24 23:20:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/610199blk-flat-revolving-display-base-05h-x-9dia-bk-10pack>: HTTP status code is not handled or not allowed 2026-01-24 23:20:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guard-dog-low-profile-5-ch-orange-lidblack-base>: HTTP status code is not handled or not allowed 2026-01-24 23:20:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_angle_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:20:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aerobic-bar-vertical-storage-rack-with-16-versabars>: HTTP status code is not handled or not allowed 2026-01-24 23:20:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conntek-14455-15-15-50-amp-rv-extension-cord-with-nema-14-50p-to-nema-ss2-50r>: HTTP status code is not handled or not allowed 2026-01-24 23:20:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/716-14-hex-nut-plain-6517001>: HTTP status code is not handled or not allowed 2026-01-24 23:20:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluke-789-processmeterwith-250-ohm-hart-resistor>: HTTP status code is not handled or not allowed 2026-01-24 23:20:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_angle_stock landed on page that is not a product page. 2026-01-24 23:20:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/80a-4-washer-6289200>: HTTP status code is not handled or not allowed 2026-01-24 23:20:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gp-professional-series-citrus-3l-heavy-duty-industrial-hand-cleaner-4-case-44627>: HTTP status code is not handled or not allowed 2026-01-24 23:20:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-plain-redi-seal-envelopes-w-security-tint-4-18w-x-9-12l-white-2500pak>: HTTP status code is not handled or not allowed 2026-01-24 23:20:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/towing_accessories/hitch_covers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:20:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/desktop-direct-thermal-labels-4w-x-5l-1-core-4od-white-12pack>: HTTP status code is not handled or not allowed 2026-01-24 23:20:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/oil_fluid_management/motorcycle_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:20:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-60-l-x-3-4-w-x-3-4-thick-white>: HTTP status code is not handled or not allowed 2026-01-24 23:20:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unweighted-tuning-fork-2048-cps>: HTTP status code is not handled or not allowed 2026-01-24 23:20:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/grill_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:20:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-office-partition-panel-w-partial-window-and-pass-thru-cable-36-1-4wx101-1-2h-tan-gl>: HTTP status code is not handled or not allowed 2026-01-24 23:20:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/oil_fluid_management/motorcycle_mats>: HTTP status code is not handled or not allowed 2026-01-24 23:20:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/hand_tools_1/ratchets_drive_tools/torque_wrenches_multipliers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:20:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/foam/foam_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:20:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-x-24-x-6-4-2-1-jumbo-bookfolds>: HTTP status code is not handled or not allowed 2026-01-24 23:20:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/towing_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:20:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-door-enclosed-bulletin-board-walnut-frame-96w-x-48h>: HTTP status code is not handled or not allowed 2026-01-24 23:20:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:20:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-x-8-25-pk-self-seal-white-stayflat-mailers>: HTTP status code is not handled or not allowed 2026-01-24 23:20:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/hand_tools_1/ratchets_drive_tools/torque_wrenches_multipliers>: HTTP status code is not handled or not allowed 2026-01-24 23:20:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/retail_fixtures_displays_equipment/retail_signs_sign_holders/letter_boards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:20:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/foam/foam_sheets landed on page that is not a product page. 2026-01-24 23:20:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-socket-3-8-drive-15mm-deep>: HTTP status code is not handled or not allowed 2026-01-24 23:21:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-30w-solid-end-panel-white>: HTTP status code is not handled or not allowed 2026-01-24 23:21:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kdd726-steering-wheel-message-cover-vinyl-20-diameter>: HTTP status code is not handled or not allowed 2026-01-24 23:21:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/towing_accessories landed on page that is not a product page. 2026-01-24 23:21:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-socket-3-8-drive-22mm-deep>: HTTP status code is not handled or not allowed 2026-01-24 23:21:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/impact-socket-1-2-drive-flex-9-16-6-point-flex-short>: HTTP status code is not handled or not allowed 2026-01-24 23:21:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-36-l-x-3-w-x-1-8-thick-black-antistatic>: HTTP status code is not handled or not allowed 2026-01-24 23:21:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/retail_fixtures_displays_equipment/retail_signs_sign_holders/letter_boards landed on page that is not a product page. 2026-01-24 23:21:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:21:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-socket-3-8-drive-12-point-13-16-deep>: HTTP status code is not handled or not allowed 2026-01-24 23:21:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:21:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 35700 pages (at 88 pages/min), scraped 16113 items (at 54 items/min) 2026-01-24 23:21:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-door-enclosed-vinyl-tackboard-silver-w-silver-frame-72w-x-48h>: HTTP status code is not handled or not allowed 2026-01-24 23:21:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-foam-epdm-roll-120-l-x-54-w-x-1-thick-black>: HTTP status code is not handled or not allowed 2026-01-24 23:21:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/floor_mats landed on page that is not a product page. 2026-01-24 23:21:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plus-cart-pkg-red-double-sided-danger>: HTTP status code is not handled or not allowed 2026-01-24 23:21:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-steel-hand-truck-8-ace-tuf-wheels-20-58x46>: HTTP status code is not handled or not allowed 2026-01-24 23:21:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 23:21:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-tilt-truck-30x48-solid-wood-moldon-rubber-wheels-2200>: HTTP status code is not handled or not allowed 2026-01-24 23:21:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks landed on page that is not a product page. 2026-01-24 23:21:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/msmk133vp-danger-sign-no-smoking-14w-x-10h-plastic>: HTTP status code is not handled or not allowed 2026-01-24 23:21:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/decor_linens/rugs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:21:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/decor_linens/rugs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:21:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/34x25-liter-under-the-counter-base-cabinet-35w-x-22d-x-35-1-2h>: HTTP status code is not handled or not allowed 2026-01-24 23:21:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_microscopes/digital_microscopes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:21:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-4-tube-o-d-zinc-plated-u-bolt-cushion-pipe-clamp>: HTTP status code is not handled or not allowed 2026-01-24 23:21:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-polypropylene-rail-mount-beta-smoothie-clamp>: HTTP status code is not handled or not allowed 2026-01-24 23:21:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/decor_linens already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:21:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/decor_linens already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:21:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-sheet-24-l-x-12-w-x-3-16-thick-white>: HTTP status code is not handled or not allowed 2026-01-24 23:21:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-8-inches-buckle-for-stainless-steel-100-per-case>: HTTP status code is not handled or not allowed 2026-01-24 23:21:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4flexo-pet-general-purpose-sleeving-dia-200-orange>: HTTP status code is not handled or not allowed 2026-01-24 23:21:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_microscopes/digital_microscopes landed on page that is not a product page. 2026-01-24 23:21:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pm1800b-3-18-bandsaw-5-hp-3-phase-460v>: HTTP status code is not handled or not allowed 2026-01-24 23:21:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/decor_linens landed on page that is not a product page. 2026-01-24 23:21:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/decor_linens landed on page that is not a product page. 2026-01-24 23:21:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/male-single-swivel-socket-member-3>: HTTP status code is not handled or not allowed 2026-01-24 23:21:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4000-indoor-led-message-sign-full-color-4080rgb>: HTTP status code is not handled or not allowed 2026-01-24 23:21:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_drawers_keyboard_trays already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:21:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-socket-1-4-drive-9mm-deep>: HTTP status code is not handled or not allowed 2026-01-24 23:21:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speco-4mp-h265-ai-ip-dome-camera-w-ir--junction-box-28-12mm-motorized-lens-white-housing>: HTTP status code is not handled or not allowed 2026-01-24 23:21:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/z-line-additional-perforated-steel-shelf-level-with-clips-42-w-x-18-d-x-1-5-16-h>: HTTP status code is not handled or not allowed 2026-01-24 23:21:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/collar-30366176>: HTTP status code is not handled or not allowed 2026-01-24 23:21:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:21:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/impact-socket-1-2-drive-3-4-6-point-deep>: HTTP status code is not handled or not allowed 2026-01-24 23:21:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cpvc-bar-48-l-x-1-1-2-w-x-1-8-thick-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:21:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:21:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_drawers_keyboard_trays>: HTTP status code is not handled or not allowed 2026-01-24 23:21:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-x-60-heavy-duty-pallet-scale-10000-lb-x-2-lb>: HTTP status code is not handled or not allowed 2026-01-24 23:21:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bookcases_hutches already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:21:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-weigh-beam-system-5000-lb-x-1-lb>: HTTP status code is not handled or not allowed 2026-01-24 23:21:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/duo-sweep-medium-duty-angle-broom-w-12-flare-head-only-8-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:21:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-24 23:22:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 23:22:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crossover-coupling-30366288>: HTTP status code is not handled or not allowed 2026-01-24 23:22:04 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bookcases_hutches landed on page that is not a product page. 2026-01-24 23:22:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mgnf956vs-think-sign-safety-first-10w-x-7h-adhesive-vinyl>: HTTP status code is not handled or not allowed 2026-01-24 23:22:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-bowl-mop-12-white>: HTTP status code is not handled or not allowed 2026-01-24 23:22:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 35792 pages (at 92 pages/min), scraped 16169 items (at 56 items/min) 2026-01-24 23:22:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-inch-lightweight-super-duty-impact-w-6-ext-anvil>: HTTP status code is not handled or not allowed 2026-01-24 23:22:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-bar-48-l-x-3-4-w-x-3-32-thick-clear>: HTTP status code is not handled or not allowed 2026-01-24 23:22:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-bar-24-l-x-4-w-x-3-8-thick-clear-scratch-and-uv-resistant>: HTTP status code is not handled or not allowed 2026-01-24 23:22:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hurricane-360-hoist-and-trolley-combo-geared-20-ton-capacity-30-lift>: HTTP status code is not handled or not allowed 2026-01-24 23:22:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-x-8-open-end-static-shielding-bags>: HTTP status code is not handled or not allowed 2026-01-24 23:22:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/jobsite_portable_lighting/job_site_work_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:22:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/bar_restaurant_equipment/dispensers_organizers/beverage_dispensers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:22:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4x15-4-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-24 23:22:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3x3-4-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-24 23:22:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-supply-charger-msp-070-3>: HTTP status code is not handled or not allowed 2026-01-24 23:22:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crystallizing-dishes-100x50mm-case-of-18>: HTTP status code is not handled or not allowed 2026-01-24 23:22:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brushless-hepa-backpack-vacuum-kit-with-40v-max-xgt-li-ion-batteries-15-gal-capacity>: HTTP status code is not handled or not allowed 2026-01-24 23:22:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/jobsite_portable_lighting/job_site_work_lights>: HTTP status code is not handled or not allowed 2026-01-24 23:22:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7x10-3-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-24 23:22:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/bar_restaurant_equipment/dispensers_organizers/beverage_dispensers landed on page that is not a product page. 2026-01-24 23:22:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:22:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/freestanding-3-panel-corner-room-divider-with-partial-window-48-x-72-blue>: HTTP status code is not handled or not allowed 2026-01-24 23:22:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speakman-flw-flow-switch>: HTTP status code is not handled or not allowed 2026-01-24 23:22:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:22:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dax-reflective-decals-red>: HTTP status code is not handled or not allowed 2026-01-24 23:22:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rs22-3-semi-automatic-random-side-drive-case-sealer-with-3-tape-head>: HTTP status code is not handled or not allowed 2026-01-24 23:22:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cw-premium-portable-electric-pressure-washer-4000-psi-15-hp-39-gpm-3-phase-230v>: HTTP status code is not handled or not allowed 2026-01-24 23:22:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-24 23:22:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 23:22:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:22:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cth-1520-fire-retardant-tarp-14x14-weave-15-x-20>: HTTP status code is not handled or not allowed 2026-01-24 23:22:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-1-2-x-24-kra-mailing-tubes-with-caps-case>: HTTP status code is not handled or not allowed 2026-01-24 23:22:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelate-wire-shelving-30485ep-48w-x-30d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 23:22:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-cohesive-air-bubble-dispenser-pack-12w-x-175l-x-316-bubble-clear>: HTTP status code is not handled or not allowed 2026-01-24 23:22:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats landed on page that is not a product page. 2026-01-24 23:22:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-rod-60-l-x-1-1-8-dia-black>: HTTP status code is not handled or not allowed 2026-01-24 23:22:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/39-51-degree-baume-hydrometer>: HTTP status code is not handled or not allowed 2026-01-24 23:23:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:23:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:23:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluke-tl223-electrical-test-lead-set-ac220-tp1-tl224>: HTTP status code is not handled or not allowed 2026-01-24 23:23:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anti-static-flush-cut-foam-pouches-3w-x-4l-x-18-thick-pink-500pack>: HTTP status code is not handled or not allowed 2026-01-24 23:23:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:23:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-pmp-gel-staining-box-with-plastic-cover-5-x-5-x-2>: HTTP status code is not handled or not allowed 2026-01-24 23:23:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 35884 pages (at 92 pages/min), scraped 16231 items (at 62 items/min) 2026-01-24 23:23:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock landed on page that is not a product page. 2026-01-24 23:23:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-cu-yd-gray-medium-duty-self-dumping-hopper-986764>: HTTP status code is not handled or not allowed 2026-01-24 23:23:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 23:23:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-cu-yd-gray-extra-heavy-duty-self-dumping-hopper-986780>: HTTP status code is not handled or not allowed 2026-01-24 23:23:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:23:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/snow_melting_heated_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:23:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 23:23:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-edge-4w-20l-1-ply-endless-poly-flat-web-sling>: HTTP status code is not handled or not allowed 2026-01-24 23:23:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:23:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lewisbins-conductive-hanging-stacking-part-bin-8-1316w-x-6-58d-x-2-2932h>: HTTP status code is not handled or not allowed 2026-01-24 23:23:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polaris-rzr-170cc-utv-battery-2016-2018-10-amps-12v-b-terminals>: HTTP status code is not handled or not allowed 2026-01-24 23:23:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock landed on page that is not a product page. 2026-01-24 23:23:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interlocking-storage-cabinet-11-3-4w-11-3-8d-x-11h-8-red-drawers>: HTTP status code is not handled or not allowed 2026-01-24 23:23:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interlocking-storage-cabinet-11-3-4w-11-3-8d-x-11h-10-blue-drawers>: HTTP status code is not handled or not allowed 2026-01-24 23:23:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock landed on page that is not a product page. 2026-01-24 23:23:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roundup-1w-12l-endless-poly-roundsling-green>: HTTP status code is not handled or not allowed 2026-01-24 23:23:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/floor_mats/snow_melting_heated_mats landed on page that is not a product page. 2026-01-24 23:23:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-epoxy-wire-shelving-add-on-14486gz-48w-x-14d-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 23:23:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:23:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interlocking-storage-cabinet-11-3-4w-11-3-8d-x-11h-8-blue-drawers>: HTTP status code is not handled or not allowed 2026-01-24 23:23:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ghs-labeled-safety-vented-dichloromethane-wash-bottles-500ml-16oz-pe-w-ylw-pp-cap-4pk>: HTTP status code is not handled or not allowed 2026-01-24 23:23:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-edge-2w-10l-2-ply-eye-eye-poly-web-sling-twisted-eyes>: HTTP status code is not handled or not allowed 2026-01-24 23:23:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-edge-4w-12l-2-ply-eye-eye-poly-web-sling-flat-eyes>: HTTP status code is not handled or not allowed 2026-01-24 23:23:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock landed on page that is not a product page. 2026-01-24 23:23:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-plastic-bin-16-1-4l-x-11-3-4w-x-22h-red>: HTTP status code is not handled or not allowed 2026-01-24 23:23:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/retail_fixtures_displays_equipment/displays_display_racks/garment_accessory_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:23:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ghs-labeled-safety-vented-methanol-wash-bottles-500ml-16oz-polyethylene-w-green-cap-4pk>: HTTP status code is not handled or not allowed 2026-01-24 23:23:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:23:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-foam-ptfe-tube-72-l-x-3-4-id-x-1-1-2-od-white>: HTTP status code is not handled or not allowed 2026-01-24 23:23:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wooden-wet-floor-sign-walnut-finish-with-nickel-plate>: HTTP status code is not handled or not allowed 2026-01-24 23:23:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-rack-24x36x60-with-88-gray-drawers>: HTTP status code is not handled or not allowed 2026-01-24 23:23:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:23:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-ceiling-mounting-bracket-for-plasma-tv-36in-55in>: HTTP status code is not handled or not allowed 2026-01-24 23:23:50 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/retail_fixtures_displays_equipment/displays_display_racks/garment_accessory_racks landed on page that is not a product page. 2026-01-24 23:23:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storage-container-with-lid-32-1-2-x-23-1-2-x-20-blue>: HTTP status code is not handled or not allowed 2026-01-24 23:23:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock landed on page that is not a product page. 2026-01-24 23:23:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:23:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-x-10-resealable-clear-face-document-envelopes>: HTTP status code is not handled or not allowed 2026-01-24 23:23:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:23:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buyers-aluminum-gull-wing-cross-box-23x27x71>: HTTP status code is not handled or not allowed 2026-01-24 23:23:56 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers landed on page that is not a product page. 2026-01-24 23:23:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-hdpe-sheet-96-l-x-48-w-x-1-thick-black-marine>: HTTP status code is not handled or not allowed 2026-01-24 23:24:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:24:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paint-marker-oil-based-medium-white-ink>: HTTP status code is not handled or not allowed 2026-01-24 23:24:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers landed on page that is not a product page. 2026-01-24 23:24:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers landed on page that is not a product page. 2026-01-24 23:24:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:24:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buyers-aluminum-underbody-box-w-t-handle-18x18x30>: HTTP status code is not handled or not allowed 2026-01-24 23:24:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:24:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simplehuman-174-profile-open-can-2-3-5-gallon-brushed-ss>: HTTP status code is not handled or not allowed 2026-01-24 23:24:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:24:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 35968 pages (at 84 pages/min), scraped 16277 items (at 46 items/min) 2026-01-24 23:24:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:24:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-24 23:24:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-24 23:24:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:24:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers>: HTTP status code is not handled or not allowed 2026-01-24 23:24:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-ptfe-sheet-36-l-x-12-w-x-1-1-2-thick-black>: HTTP status code is not handled or not allowed 2026-01-24 23:24:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock landed on page that is not a product page. 2026-01-24 23:24:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 23:24:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/itm-30-piece-tamper-resistant-set>: HTTP status code is not handled or not allowed 2026-01-24 23:24:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers landed on page that is not a product page. 2026-01-24 23:24:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-wire-shelf-stem-caster-truck-2-with-brakes-poly-green-24-w-x-60-l-x-69-h>: HTTP status code is not handled or not allowed 2026-01-24 23:24:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-x-200-yds-waxed-30-basis-weight-vci-paper>: HTTP status code is not handled or not allowed 2026-01-24 23:24:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-24-l-x-4-w-x-2-1-2-thick-white>: HTTP status code is not handled or not allowed 2026-01-24 23:24:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-48-l-x-4-w-x-2-3-4-thick-white>: HTTP status code is not handled or not allowed 2026-01-24 23:24:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-wire-shelves-utility-cart-chrome-2-casters-w-brakes-24-w-x-42-l-x-39-h>: HTTP status code is not handled or not allowed 2026-01-24 23:24:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyethylene-brackets-for-40-horizontal-secador-desiccator-pack-of-2-pairs>: HTTP status code is not handled or not allowed 2026-01-24 23:24:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:24:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plus-calibrated-liquid-in-glass-thermometer-10-to-225c-total-immersion>: HTTP status code is not handled or not allowed 2026-01-24 23:24:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:24:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:24:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bumpon-protective-product-sj5202-hexagon-0-433-w-x-0-063-l-lt-brown-pkg-of-3000>: HTTP status code is not handled or not allowed 2026-01-24 23:24:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-36-l-x-5-w-x-1-thick-white>: HTTP status code is not handled or not allowed 2026-01-24 23:24:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-spigot-for-autoclavable-polypropylene-carboys-f11846-0025-and-f11846-0050>: HTTP status code is not handled or not allowed 2026-01-24 23:24:42 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-24 23:24:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commercial-recess-mount-ceiling-heater-240v-3000w>: HTTP status code is not handled or not allowed 2026-01-24 23:24:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 23:24:44 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 23:24:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:24:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stacked-chair-dolly-for-narrow-chairs>: HTTP status code is not handled or not allowed 2026-01-24 23:24:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leviton-8300-hr-20-amp-125-volt-heavy-duty-hospital-grade-duplex-receptacle-red>: HTTP status code is not handled or not allowed 2026-01-24 23:24:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_storage_bench_protection/laboratory_bench_racks_stands already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:24:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-voltage-wall-mounted-thermostats-rk130eaa>: HTTP status code is not handled or not allowed 2026-01-24 23:24:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/briskheat-silicone-rubber-55-gallon-steel-drum-heater-120v>: HTTP status code is not handled or not allowed 2026-01-24 23:24:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-60w-x-14d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 23:24:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:24:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 23:24:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gez-100es-2rs-spherical-plain-bearing-inch-sealed>: HTTP status code is not handled or not allowed 2026-01-24 23:24:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-48w-x-36d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 23:24:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_storage_bench_protection/laboratory_bench_racks_stands landed on page that is not a product page. 2026-01-24 23:24:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xt1-91-foam-lined-frame-safety-glasses-io-anti-fog-lens-black-frame>: HTTP status code is not handled or not allowed 2026-01-24 23:25:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-72w-x-36d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 23:25:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/liberty-check-and-form-boxes-9-3-4w-x-23-3-4d-x-6-1-4h-white-blue>: HTTP status code is not handled or not allowed 2026-01-24 23:25:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 23:25:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sv4-class-2-5-pt-breakaway-solid-vest-hi-viz-green-2xl>: HTTP status code is not handled or not allowed 2026-01-24 23:25:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-5-tier-42w-x-14d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 23:25:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:25:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/secador-amber-30-auto-desiccator-cabinet-120v-16-cu-ft>: HTTP status code is not handled or not allowed 2026-01-24 23:25:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 36055 pages (at 87 pages/min), scraped 16325 items (at 48 items/min) 2026-01-24 23:25:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/type-500-small-outdoor-waterproof-case-w-sponge-insert-foam-8-3-4-l-x-7-w-x-3-1-2h-black>: HTTP status code is not handled or not allowed 2026-01-24 23:25:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:25:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-acetal-bar-48-l-x-1-2-w-x-3-8-thick-off-white-antistatic>: HTTP status code is not handled or not allowed 2026-01-24 23:25:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:25:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tz-extra-strength-adhesive-tapes-laminated-black-on-yellow-1-1-2w>: HTTP status code is not handled or not allowed 2026-01-24 23:25:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 23:25:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/label_tapes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:25:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/bins_totes_containers/organizers_caddies/small_parts_organizers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:25:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u-shaped-rectangular-tubing-hangrail-matte-white>: HTTP status code is not handled or not allowed 2026-01-24 23:25:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9600-15ex-16-inch-diameter-statically-conductive-ducting-15-length>: HTTP status code is not handled or not allowed 2026-01-24 23:25:21 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-24 23:25:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-ptfe-rod-24-l-x-5-8-dia-off-white-glass-filled>: HTTP status code is not handled or not allowed 2026-01-24 23:25:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 23:25:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:25:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-neoprene-roll-600-l-x-12-w-x-1-32-thick-black-40a>: HTTP status code is not handled or not allowed 2026-01-24 23:25:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/label_tapes landed on page that is not a product page. 2026-01-24 23:25:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/bins_totes_containers/organizers_caddies/small_parts_organizers landed on page that is not a product page. 2026-01-24 23:25:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_storage_bench_protection/laboratory_bench_racks_stands already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:25:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tape-head-kleen-pad-individually-sealed-pads-80-box>: HTTP status code is not handled or not allowed 2026-01-24 23:25:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-epoxy-wire-shelving-add-on-21548b-54w-x-21d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 23:25:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spinbar-teflon-micro-flea-magnetic-stirring-bar-5-x-2mm-blue>: HTTP status code is not handled or not allowed 2026-01-24 23:25:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 23:25:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-54w-x-21d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 23:25:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:25:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_storage_bench_protection/laboratory_bench_racks_stands landed on page that is not a product page. 2026-01-24 23:25:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cable-truck-step-perforated-step-12-inch-wx9-inch-h-black-5230912>: HTTP status code is not handled or not allowed 2026-01-24 23:25:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:25:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/competitor-ventilated-three-tier-locker-adder-1-wide-12w-x-12d-x-24h-unassembled-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:25:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-double-tier-locker-starter-1-wide-12w-x-18d-x-30h-unassembled-almond>: HTTP status code is not handled or not allowed 2026-01-24 23:25:45 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 23:25:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/140059-screw-driver-t20-8l-x-6w-x-4h>: HTTP status code is not handled or not allowed 2026-01-24 23:25:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:25:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22-x-28-double-bulletin-sign-holder-w-rectangular-tubing-base-chrome>: HTTP status code is not handled or not allowed 2026-01-24 23:25:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-deluxe-hook-semi-gloss-white>: HTTP status code is not handled or not allowed 2026-01-24 23:25:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 23:25:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lcd43909blk-full-motion-tv-wall-mount-for-32-60-monitors-blacks>: HTTP status code is not handled or not allowed 2026-01-24 23:25:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/omega-supreme-esd-vacuum>: HTTP status code is not handled or not allowed 2026-01-24 23:25:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_storage_bench_protection/laboratory_bench_racks_stands already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:25:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 23:25:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24w-x-12d-straight-shelf-epoxy-chrome>: HTTP status code is not handled or not allowed 2026-01-24 23:26:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-silicone-sheet-12-l-x-12-w-x-3-32-thick-red-food-70a>: HTTP status code is not handled or not allowed 2026-01-24 23:26:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:26:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20d-x-24w-x-38h-register-stand-maple>: HTTP status code is not handled or not allowed 2026-01-24 23:26:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_storage_bench_protection/laboratory_bench_racks_stands landed on page that is not a product page. 2026-01-24 23:26:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_supplies_dispensers/baby_changing_tables already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:26:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-ride-on-floor-scrubber-40-cleaning-path>: HTTP status code is not handled or not allowed 2026-01-24 23:26:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 36143 pages (at 88 pages/min), scraped 16380 items (at 55 items/min) 2026-01-24 23:26:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_flasks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:26:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-knife-bracket-chrome>: HTTP status code is not handled or not allowed 2026-01-24 23:26:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sewer-tape-1-2-x-40>: HTTP status code is not handled or not allowed 2026-01-24 23:26:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 23:26:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-drum-auger-1-4-x-25>: HTTP status code is not handled or not allowed 2026-01-24 23:26:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oscillated-wound-strapping-dispenser-for-3-4and-1-1-4w>: HTTP status code is not handled or not allowed 2026-01-24 23:26:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:26:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compact-bedside-cart-1-locking-drawer-white-with-black>: HTTP status code is not handled or not allowed 2026-01-24 23:26:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_flasks landed on page that is not a product page. 2026-01-24 23:26:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:26:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-level-60x48-wood-deck>: HTTP status code is not handled or not allowed 2026-01-24 23:26:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_supplies_dispensers/baby_changing_tables landed on page that is not a product page. 2026-01-24 23:26:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gasket-cutter-kit-ax6010-style-1-cap-to-13>: HTTP status code is not handled or not allowed 2026-01-24 23:26:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-w-extra-heavy-duty-towel-bar-stainless-steel-9066-180000>: HTTP status code is not handled or not allowed 2026-01-24 23:26:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls>: HTTP status code is not handled or not allowed 2026-01-24 23:26:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-buna-n-sheet-12-l-x-12-w-x-3-32-thick-black-high-strength-80a-acrylic-adhesive-bulk-rs-h80-49>: HTTP status code is not handled or not allowed 2026-01-24 23:26:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-24 23:26:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-868strc-aq-surface-mount-horn-strobe-for-outdoor-use-24v-acdc-clear>: HTTP status code is not handled or not allowed 2026-01-24 23:26:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:26:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24w-x-15d-straight-shelf-w-front-lip-semi-gloss-white>: HTTP status code is not handled or not allowed 2026-01-24 23:26:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mount-anywhere-basket-24-6-6-b>: HTTP status code is not handled or not allowed 2026-01-24 23:26:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-125ledfa120a-125-led-flash-amber-120vac>: HTTP status code is not handled or not allowed 2026-01-24 23:26:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/die_cut_labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:26:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-cleaner>: HTTP status code is not handled or not allowed 2026-01-24 23:26:44 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-24 23:26:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pool-pump-motor-2-hp-3450-rpm-230-115v-odp-y56j-frame>: HTTP status code is not handled or not allowed 2026-01-24 23:26:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cl3515-2-hp-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-24 23:26:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:26:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:26:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-hinged-top-basket-truck-18-bu-gray-vinyl-wood-base-all-swivel>: HTTP status code is not handled or not allowed 2026-01-24 23:26:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:26:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-270mda240a-70-mm-magnetodynamic-sounder-gray-240v-ac>: HTTP status code is not handled or not allowed 2026-01-24 23:26:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-hinged-top-basket-truck-6-bu-blue-vinyl-wood-base-all-swivel>: HTTP status code is not handled or not allowed 2026-01-24 23:26:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs landed on page that is not a product page. 2026-01-24 23:27:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs>: HTTP status code is not handled or not allowed 2026-01-24 23:27:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels landed on page that is not a product page. 2026-01-24 23:27:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-single-phase-odp-motor-1-3-hp-1725-rpm-115-230v-odp-60-herts>: HTTP status code is not handled or not allowed 2026-01-24 23:27:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unit-heater-motor-1-3-hp-1140-rpm-115-208-230v-teao-f48y-frame>: HTTP status code is not handled or not allowed 2026-01-24 23:27:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rapidguard-2-lift-off-welded-wire-panel-3-w-x-8-h-panel>: HTTP status code is not handled or not allowed 2026-01-24 23:27:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honeywell-redlink-internet-gateway>: HTTP status code is not handled or not allowed 2026-01-24 23:27:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 36233 pages (at 90 pages/min), scraped 16438 items (at 58 items/min) 2026-01-24 23:27:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/everyday-glossy-photo-paper-4in-x-6in-white-100-sheets-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:27:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-union-y-5mm-tube-1-3-8-l>: HTTP status code is not handled or not allowed 2026-01-24 23:27:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/photo_presentation_paper already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:27:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jsh-series-electric-chain-hoist-1-4-ton-15-ft-lift-1-phase-115v>: HTTP status code is not handled or not allowed 2026-01-24 23:27:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-rivet-boltless-shelving-48-inch-w-x-24-inch-d-x-84-inch-h-6-levels-add-on-w-o-decking-tan>: HTTP status code is not handled or not allowed 2026-01-24 23:27:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cewdm3558t-2-hp-1755-rpm>: HTTP status code is not handled or not allowed 2026-01-24 23:27:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/paper_stationery/photo_presentation_paper>: HTTP status code is not handled or not allowed 2026-01-24 23:27:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pre-configured-6x8-partitioned-office-72-h-starter-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:27:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cem3558-2-hp-1755-rpm>: HTTP status code is not handled or not allowed 2026-01-24 23:27:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cewdm3613t-5-hp-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-24 23:27:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/men-shrt-slv-deluxe-western-style-shirt-xl-sc24-sc24lbssxl>: HTTP status code is not handled or not allowed 2026-01-24 23:27:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-straight-male-10mm-tube-x-3-8-bspp-1-l>: HTTP status code is not handled or not allowed 2026-01-24 23:27:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/toolbox-z400-white-1-4-fold-60sheets-bundle-12-bundles-case-5010001>: HTTP status code is not handled or not allowed 2026-01-24 23:27:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-union-elbow-1-2-tube-1-1-5-l>: HTTP status code is not handled or not allowed 2026-01-24 23:27:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-needle-valve-knob-adjustment-10-32-fm-x-10-32-ml-3-4-l>: HTTP status code is not handled or not allowed 2026-01-24 23:27:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-5-tier-starter-unit-30-w-x-18-d-x-74-h>: HTTP status code is not handled or not allowed 2026-01-24 23:27:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cabinet-with-electronic-lock-and-card-reader-36-w-x-24-d-x-75-h-blue>: HTTP status code is not handled or not allowed 2026-01-24 23:27:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cart-utility-3shlf-pm>: HTTP status code is not handled or not allowed 2026-01-24 23:27:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-door-outdoor-message-center-45-x-36-cedar>: HTTP status code is not handled or not allowed 2026-01-24 23:27:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-jm3542-75-hp-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-24 23:27:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-boltless-shelving-60-inch-w-x-36-inch-d-1-level-w-wood-decking-black>: HTTP status code is not handled or not allowed 2026-01-24 23:28:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/filter-multi-pleat-xl10-20-w-x-20-h-x-2-d-12-ctn>: HTTP status code is not handled or not allowed 2026-01-24 23:28:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-m1558t-5-1-3-hp-1725-850-rpm>: HTTP status code is not handled or not allowed 2026-01-24 23:28:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bassinet-31-l-x-17-3-4-w-x-38-1-2-h-w-2-shelves-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 23:28:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/appliances/cooking_equipment/ovens already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:28:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/warming-cabinet-30-w-x-74-1-2-h-x-26-5-8-d-freestanding-1-glass-door-3-adj-shelves>: HTTP status code is not handled or not allowed 2026-01-24 23:28:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 36326 pages (at 93 pages/min), scraped 16506 items (at 68 items/min) 2026-01-24 23:28:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/padded-blue-quilted-fabric-appliance-truck-cover>: HTTP status code is not handled or not allowed 2026-01-24 23:28:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-station-windsor-scrub-sink-wall-mounted-with-elbow-action-control-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 23:28:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mri-safe-double-basin-solution-stand-w-h-brace-ss-29-wx15-dx33-h-8-5-qt-cap>: HTTP status code is not handled or not allowed 2026-01-24 23:28:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/appliances/cooking_equipment/ovens landed on page that is not a product page. 2026-01-24 23:28:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-needle-valve-screw-adjustment-1-8-fm-nptf-x-1-8-swift-fit-universal-thread-0-l>: HTTP status code is not handled or not allowed 2026-01-24 23:28:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-lanyard-swivel-hook-blue>: HTTP status code is not handled or not allowed 2026-01-24 23:28:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mesh-back-task-stool-red>: HTTP status code is not handled or not allowed 2026-01-24 23:28:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/i-display-fixture-with-spinner-base-and-casters-24-w-x-24-d-x-55-h-white>: HTTP status code is not handled or not allowed 2026-01-24 23:28:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/def-tote-warmer-with-adjustabel-temperature-control-and-frame-330-gallon>: HTTP status code is not handled or not allowed 2026-01-24 23:28:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:28:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jacket-size-tall-men-s-lt-type-r-class-3-fluorescent-yellow-green-black>: HTTP status code is not handled or not allowed 2026-01-24 23:28:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-em3583t-1-5-hp-3500-rpm>: HTTP status code is not handled or not allowed 2026-01-24 23:28:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/71580-24-inch-commercial-pedestal-fan-oscillating-1-4-hp-4400-cfm>: HTTP status code is not handled or not allowed 2026-01-24 23:28:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_labels landed on page that is not a product page. 2026-01-24 23:28:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-flow-control-flow-knob-adjustment-6mm-tube-x-m5-unf-4-5-l>: HTTP status code is not handled or not allowed 2026-01-24 23:28:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xt-traction-overshoe-medium-oil-resistant-black>: HTTP status code is not handled or not allowed 2026-01-24 23:28:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-em3314t-15-hp-3525-rpm>: HTTP status code is not handled or not allowed 2026-01-24 23:28:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:28:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kraft-deluxe-literature-mailer-9-x-6-1-4-x-2-200lb-ect-32-b-50-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:28:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-dock-board-with-lock-pins-72-w-x-84-l-15-000-lb-cap>: HTTP status code is not handled or not allowed 2026-01-24 23:28:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-buckle-ice-traction-12-overshoe-mens-sz-7-waterproof-vulcanized-rubber-blk>: HTTP status code is not handled or not allowed 2026-01-24 23:28:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats landed on page that is not a product page. 2026-01-24 23:28:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kraft-paper-roll-50lb-36-x-1000-feet-red>: HTTP status code is not handled or not allowed 2026-01-24 23:28:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-x-4-x-3-corrugated-boxes-200lb-test-ect-32>: HTTP status code is not handled or not allowed 2026-01-24 23:28:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-gas-heater-infrared-vacuum-u-tube-150000-btu-40l>: HTTP status code is not handled or not allowed 2026-01-24 23:29:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-gas-heater-infrared-positive-pressure-straight-tube-tough-guy-200000-btu-50l>: HTTP status code is not handled or not allowed 2026-01-24 23:29:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-9in-square-spot-light-with-9-led-1492134>: HTTP status code is not handled or not allowed 2026-01-24 23:29:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/moisture-barrier-bag-emirfi-15-x-20-65mm-100-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:29:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arti-glass-digital-scale-rio-red>: HTTP status code is not handled or not allowed 2026-01-24 23:29:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 36422 pages (at 96 pages/min), scraped 16576 items (at 70 items/min) 2026-01-24 23:29:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oil-safe-mini-spout-lid-grey>: HTTP status code is not handled or not allowed 2026-01-24 23:29:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/plumbing_parts_accessories/pipe_hose_tubing_accessories/pipe_markers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:29:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magliner-174-gemini-senior-2-1-convertible-hand-truck-microcellular-gmk81uae>: HTTP status code is not handled or not allowed 2026-01-24 23:29:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/push-plate-4-x-16-satin>: HTTP status code is not handled or not allowed 2026-01-24 23:29:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/door-stop-plunger-style-spring-loaded-aluminum-painted>: HTTP status code is not handled or not allowed 2026-01-24 23:29:21 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/plumbing/plumbing_parts_accessories/pipe_hose_tubing_accessories/pipe_markers landed on page that is not a product page. 2026-01-24 23:29:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precoiled-strap-on-pipe-marker-domestic-cold-water-supply-2>: HTTP status code is not handled or not allowed 2026-01-24 23:29:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hemco-le-fume-hood-wexplosion-proof-light-only-60-x-32-x-48>: HTTP status code is not handled or not allowed 2026-01-24 23:29:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-rl1304a277-5-hp-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-24 23:29:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/furniture_parts_accessories/footrests already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:29:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pressure-sensitive-pipe-marker-direction-arrow-green-3>: HTTP status code is not handled or not allowed 2026-01-24 23:29:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6210b-skirtslack-hanger-pinch-grip-wsoft-pads-10l-plastic-bk-pkg-qty-200>: HTTP status code is not handled or not allowed 2026-01-24 23:29:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/nails_pins_brads/staples_1 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:29:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-rl1324a-75-hp-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-24 23:29:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24v-relay-kit-eclipse-compact-infrared-tube-heaters>: HTTP status code is not handled or not allowed 2026-01-24 23:29:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-vbm3116-1-hp-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-24 23:29:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-8482-6500-watts-portable-generator-gasoline-electric-recoil-start-120-240v>: HTTP status code is not handled or not allowed 2026-01-24 23:29:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tray-fast-food-10x14-navy-blue>: HTTP status code is not handled or not allowed 2026-01-24 23:29:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-1-2-x-099-ring-shank-15-degree-coil-framing-nails-galvanized-3600-qty>: HTTP status code is not handled or not allowed 2026-01-24 23:29:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-calcium-lime-rust-remover-28-oz-bottle-6-bottles-pack-352250>: HTTP status code is not handled or not allowed 2026-01-24 23:29:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/furniture_parts_accessories/footrests landed on page that is not a product page. 2026-01-24 23:29:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/divider-for-30-d-x-12-h-stacking-bin-sold-per-each-269943>: HTTP status code is not handled or not allowed 2026-01-24 23:29:44 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/nails_pins_brads/staples_1 landed on page that is not a product page. 2026-01-24 23:29:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-x-21-sds-max-speed-clean-dust-extraction-bit>: HTTP status code is not handled or not allowed 2026-01-24 23:29:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-x-3-welded-pipe-nipple-schedule-40-304-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 23:29:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:29:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electrical-rated-gloves-black-14-unlined-smooth-finish-beaded-class-0-8>: HTTP status code is not handled or not allowed 2026-01-24 23:29:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/cubicle_partition_panels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:29:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-vejmm3313t-10-hp-1770-rpm>: HTTP status code is not handled or not allowed 2026-01-24 23:29:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/cubicle_partition_panels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:29:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-vewdm3611t-3-hp-1760-rpm>: HTTP status code is not handled or not allowed 2026-01-24 23:29:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-m3452-17-hp-1140-rpm>: HTTP status code is not handled or not allowed 2026-01-24 23:30:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2004-w-6-all-plastic-crowd-control-barricade-w-rubber-feet-interlocking-white>: HTTP status code is not handled or not allowed 2026-01-24 23:30:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks landed on page that is not a product page. 2026-01-24 23:30:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lcm0150-02-manual-lifting-lift-magnet-w-telescoping-pole-150-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-24 23:30:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/partitions_dividers/cubicle_partition_panels landed on page that is not a product page. 2026-01-24 23:30:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pkb-platinum-portable-unit-208v-15kw-3-phase>: HTTP status code is not handled or not allowed 2026-01-24 23:30:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/partitions_dividers/cubicle_partition_panels landed on page that is not a product page. 2026-01-24 23:30:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-vinyl-sign-warning-keep-all-cylinders-chained>: HTTP status code is not handled or not allowed 2026-01-24 23:30:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:30:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 36516 pages (at 94 pages/min), scraped 16636 items (at 60 items/min) 2026-01-24 23:30:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/csp-sg-4kit-all-plastic-manhole-guard-nonconductive-yellow>: HTTP status code is not handled or not allowed 2026-01-24 23:30:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-vwdm3538-5-hp-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-24 23:30:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zurn-5-x-5-square-floor-drain-wscrews-nickel>: HTTP status code is not handled or not allowed 2026-01-24 23:30:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-zdvsm2333t-15-hp-5400-rpm>: HTTP status code is not handled or not allowed 2026-01-24 23:30:18 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs landed on page that is not a product page. 2026-01-24 23:30:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/little-giant-wwc2-3672-6phfl-mobile-heavy-duty-cabinet-workbench-steel-top-center-shelf>: HTTP status code is not handled or not allowed 2026-01-24 23:30:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:30:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-zdm4117t-30te-hp-3900-rpm>: HTTP status code is not handled or not allowed 2026-01-24 23:30:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cam-follower-heavy-sealed-hex-head-3-4-l-stud-04375-w-roller-0625-roller-dia>: HTTP status code is not handled or not allowed 2026-01-24 23:30:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-5-tier-wire-shelving-starter-unit-36w-x-14d-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 23:30:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mh-2d-2448-fl-48-w-mobile-bench-cabinets-hardboard-top-2-doors>: HTTP status code is not handled or not allowed 2026-01-24 23:30:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs landed on page that is not a product page. 2026-01-24 23:30:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jat-410-1-8-straight-pencil-die-grinder>: HTTP status code is not handled or not allowed 2026-01-24 23:30:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-esd-18w-x-54l-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 23:30:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/watts-filter-housing-wrenches-500-4000-4200-7000-v-i-h-series>: HTTP status code is not handled or not allowed 2026-01-24 23:30:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:30:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/r-ah-arrow-head-starting-drill-w-r-cables>: HTTP status code is not handled or not allowed 2026-01-24 23:30:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_components/pallet_rack_upright_frames already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:30:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sk3000-structural-channel-pallet-rack-12-inch-row-spacer-4-inch-frame>: HTTP status code is not handled or not allowed 2026-01-24 23:30:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:30:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelf-truck-esd-24w-x-54l-x-80h-polyurethane-4-swivel-2-brake-casters>: HTTP status code is not handled or not allowed 2026-01-24 23:30:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs>: HTTP status code is not handled or not allowed 2026-01-24 23:30:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-battery-cable-with-5-16-ring-terminals-1-0-awg-4-ft-red>: HTTP status code is not handled or not allowed 2026-01-24 23:30:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/appliances/refrigerators_freezers/refrigerator_freezer_combos/undercounter_refrigerator_freezer_combos already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:30:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caution-sign-14x10-rigid-plastic-hearing-protection-required>: HTTP status code is not handled or not allowed 2026-01-24 23:30:51 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs landed on page that is not a product page. 2026-01-24 23:30:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caution-sign-14x10-rigid-plastic-do-not-operate-without-guards>: HTTP status code is not handled or not allowed 2026-01-24 23:30:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_components/pallet_rack_upright_frames landed on page that is not a product page. 2026-01-24 23:30:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-solid-security-cage-truck-49-x-26-x-72-986992>: HTTP status code is not handled or not allowed 2026-01-24 23:30:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ch-276238-2-3-test-ball-plug-17-psi-40-ft>: HTTP status code is not handled or not allowed 2026-01-24 23:30:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/appliances/refrigerators_freezers/refrigerator_freezer_combos/undercounter_refrigerator_freezer_combos landed on page that is not a product page. 2026-01-24 23:30:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/danger-sign-14x10-aluminum-explosive-gas>: HTTP status code is not handled or not allowed 2026-01-24 23:31:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-622-hand-chain-hoist-1-ton-cap-10ft-lift>: HTTP status code is not handled or not allowed 2026-01-24 23:31:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-4-tier-esd-24w-x-72l-x-86h>: HTTP status code is not handled or not allowed 2026-01-24 23:31:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:31:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergency-sign-10x7-vinyl-first-aid-station>: HTTP status code is not handled or not allowed 2026-01-24 23:31:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/46630-satin-disc-wheel-1-8-mandrel-1-aluminum-oxide-coarse>: HTTP status code is not handled or not allowed 2026-01-24 23:31:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:31:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 36606 pages (at 90 pages/min), scraped 16696 items (at 60 items/min) 2026-01-24 23:31:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-shelf-truck-esd-3-shelf-24w-x-72l-x-40h-polyurethane-4-swivel-2-brake-casters>: HTTP status code is not handled or not allowed 2026-01-24 23:31:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:31:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs>: HTTP status code is not handled or not allowed 2026-01-24 23:31:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:31:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/db-k23bn-bath-waste-dblue-trim-for-sch-40-rough-in-kit-touchtoe-stp-bru-nik-finish-trim>: HTTP status code is not handled or not allowed 2026-01-24 23:31:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs landed on page that is not a product page. 2026-01-24 23:31:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:31:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4x24x4-1-2-open-top-bin-boxes>: HTTP status code is not handled or not allowed 2026-01-24 23:31:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24791-flap-wheel-mandrel-3-8-x-3-8-x-1-8-aluminum-oxide-medium>: HTTP status code is not handled or not allowed 2026-01-24 23:31:21 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs landed on page that is not a product page. 2026-01-24 23:31:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs landed on page that is not a product page. 2026-01-24 23:31:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-gallon-sharps-container-w-rotor-lid-color-red>: HTTP status code is not handled or not allowed 2026-01-24 23:31:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cloth-belt-82529-1-2-12-120-grit-ceramic>: HTTP status code is not handled or not allowed 2026-01-24 23:31:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/utility-cart-esd-2-shelf-24w-x-30l-polyurethane-4-swivel-2-brake-casters>: HTTP status code is not handled or not allowed 2026-01-24 23:31:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/outdoor_tools_power_equipment/chain_pole_saws already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:31:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/82468-psa-disc-5-dia-120-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-24 23:31:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs landed on page that is not a product page. 2026-01-24 23:31:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/individual-character-stencil-12-number-5>: HTTP status code is not handled or not allowed 2026-01-24 23:31:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/dunnage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:31:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electric-office-partition-panel-with-partial-window-48-14-w-x-64-h-tan>: HTTP status code is not handled or not allowed 2026-01-24 23:31:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/stands/printer_stands already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:31:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/outdoor_tools_power_equipment/chain_pole_saws>: HTTP status code is not handled or not allowed 2026-01-24 23:31:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-esd-18w-x-72l-x-63h>: HTTP status code is not handled or not allowed 2026-01-24 23:31:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/copy_printer_paper already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:31:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electric-office-partition-panel-48-14-w-x-46-h-tan>: HTTP status code is not handled or not allowed 2026-01-24 23:31:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/dunnage_racks landed on page that is not a product page. 2026-01-24 23:31:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-20x1-7-8-carriage-bolt-flat-head-1430CF>: HTTP status code is not handled or not allowed 2026-01-24 23:31:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buna-n-foam-no-adhesive-316-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 23:31:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/internally-lubricated-buna-n-o-ring-dash-116-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-24 23:31:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/paper_stationery/copy_printer_paper>: HTTP status code is not handled or not allowed 2026-01-24 23:31:51 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/stands/printer_stands landed on page that is not a product page. 2026-01-24 23:31:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-safety-sign-fire-alarm-vinyl-1>: HTTP status code is not handled or not allowed 2026-01-24 23:31:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securio-auto-feed-100c-100-sheet-stack-5-6-sheet-cross-cut-6-6-gal-capacity>: HTTP status code is not handled or not allowed 2026-01-24 23:32:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8x5-8-modified-truss-head-self-drilling-screws-0810KPM>: HTTP status code is not handled or not allowed 2026-01-24 23:32:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/asb1185-lawn-and-garden-tire-assembly-size-18-x-850-8>: HTTP status code is not handled or not allowed 2026-01-24 23:32:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:32:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thermal-receipt-roll-paper-30270-2-14-x-300-white-1-roll>: HTTP status code is not handled or not allowed 2026-01-24 23:32:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:32:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-flammable-storage-freezer-30-cu-ft-ff301www-0mgp>: HTTP status code is not handled or not allowed 2026-01-24 23:32:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brown-viton-o-ring-2-5mm-wide-9mm-id-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-24 23:32:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 36691 pages (at 85 pages/min), scraped 16750 items (at 54 items/min) 2026-01-24 23:32:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:32:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/453-1-1-2-plastic-valve-extension>: HTTP status code is not handled or not allowed 2026-01-24 23:32:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:32:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:32:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:32:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs>: HTTP status code is not handled or not allowed 2026-01-24 23:32:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards landed on page that is not a product page. 2026-01-24 23:32:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/riverdale-queen-tufted-upholstered-platform-bed-dark-gray-pocket-spring-mattress>: HTTP status code is not handled or not allowed 2026-01-24 23:32:18 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards landed on page that is not a product page. 2026-01-24 23:32:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-40x1-4-oval-head-machine-screws-0404MPOB>: HTTP status code is not handled or not allowed 2026-01-24 23:32:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards landed on page that is not a product page. 2026-01-24 23:32:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards>: HTTP status code is not handled or not allowed 2026-01-24 23:32:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tribeca-king-tufted-upholstered-platform-bed-black-pocket-spring-mattress>: HTTP status code is not handled or not allowed 2026-01-24 23:32:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-8482-diamond-plate-anti-fatigue-mat-9-16-thick-4-x-cut-to-75-blk-yellow>: HTTP status code is not handled or not allowed 2026-01-24 23:32:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silicone-foam-with-high-temp-adhesive-116-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 23:32:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards landed on page that is not a product page. 2026-01-24 23:32:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/snow_ice_removal/ice_melt_rock_salt already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:32:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/heaters/stoves_fireplaces_log_sets_fire_pits/fireplaces_log_sets_fireboxes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:32:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/column-protector-12-column-opening-yellow>: HTTP status code is not handled or not allowed 2026-01-24 23:32:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sr-aluminum-2-in-1-hand-truck-pneumatic-wheels-987052>: HTTP status code is not handled or not allowed 2026-01-24 23:32:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:32:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:32:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grocery-paper-bags-3-9-16in-x-4-3-4in-brown-500-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:32:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:32:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/snow_ice_removal/ice_melt_rock_salt landed on page that is not a product page. 2026-01-24 23:32:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/heaters/stoves_fireplaces_log_sets_fire_pits/fireplaces_log_sets_fireboxes>: HTTP status code is not handled or not allowed 2026-01-24 23:32:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/pens already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:32:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/monthly-planner-9-x-11-winestone-2015-aag7026050>: HTTP status code is not handled or not allowed 2026-01-24 23:32:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/building_construction_tape/marking_warning_tape already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:32:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-25-11110-30-inhg-vac-psi-14-lm-plastic>: HTTP status code is not handled or not allowed 2026-01-24 23:32:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges landed on page that is not a product page. 2026-01-24 23:32:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges landed on page that is not a product page. 2026-01-24 23:32:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges>: HTTP status code is not handled or not allowed 2026-01-24 23:32:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/pencils already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:32:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-x-40-flat-top-pallet-4-way-entry>: HTTP status code is not handled or not allowed 2026-01-24 23:32:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waterfed-hand-brush>: HTTP status code is not handled or not allowed 2026-01-24 23:32:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/pens landed on page that is not a product page. 2026-01-24 23:32:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dot-matrix-printer-address-labels-1-across-15-16-x-3-1-2-white-5000-box-4013>: HTTP status code is not handled or not allowed 2026-01-24 23:32:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/building_construction_tape/marking_warning_tape landed on page that is not a product page. 2026-01-24 23:32:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glow-sign-rigid-plastic-stairway>: HTTP status code is not handled or not allowed 2026-01-24 23:32:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-8-u-s-s-flat-washers-62WUSSG>: HTTP status code is not handled or not allowed 2026-01-24 23:32:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/self_luminous_emergency_exit_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:32:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/non-illuminated-push-button-operator-22mm-green-flush-style>: HTTP status code is not handled or not allowed 2026-01-24 23:32:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-labels-with-trueblock-technology-3-1-3-x-4-white-600-box-8464>: HTTP status code is not handled or not allowed 2026-01-24 23:33:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/pencils already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:33:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-16-18x3-3-4-open-eye-bolt-machine-screws-thread-3160IO>: HTTP status code is not handled or not allowed 2026-01-24 23:33:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/highlighters_markers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:33:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/highlighters_markers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:33:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/pencils landed on page that is not a product page. 2026-01-24 23:33:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6204zznr-double-shielded-wsnap-ring-20mm-bore-47mm-od>: HTTP status code is not handled or not allowed 2026-01-24 23:33:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/pencils already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:33:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/self_luminous_emergency_exit_signs landed on page that is not a product page. 2026-01-24 23:33:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/easy-peel-inkjet-address-labels-1-1-3-x-4-white-1400-box-8462>: HTTP status code is not handled or not allowed 2026-01-24 23:33:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/self_luminous_emergency_exit_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:33:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 36769 pages (at 78 pages/min), scraped 16792 items (at 42 items/min) 2026-01-24 23:33:09 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/pencils landed on page that is not a product page. 2026-01-24 23:33:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glow-sign-12x9-fire-extinguisher>: HTTP status code is not handled or not allowed 2026-01-24 23:33:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/highlighters_markers landed on page that is not a product page. 2026-01-24 23:33:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-fixed-spreader-beam-yellow-40000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 23:33:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/highlighters_markers landed on page that is not a product page. 2026-01-24 23:33:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glow-sign-vinyl-fire-extinguisher-right-arrow>: HTTP status code is not handled or not allowed 2026-01-24 23:33:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/pencils landed on page that is not a product page. 2026-01-24 23:33:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/558-professional-rolling-trimmer-51in-cutting-length>: HTTP status code is not handled or not allowed 2026-01-24 23:33:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/self_luminous_emergency_exit_signs landed on page that is not a product page. 2026-01-24 23:33:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sfl1450-1-4-refrigerant-safety-locking-caps-package-of-50>: HTTP status code is not handled or not allowed 2026-01-24 23:33:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6006zz-double-shielded-30mm-bore-55mm-od>: HTTP status code is not handled or not allowed 2026-01-24 23:33:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6002zz-double-shielded-15mm-bore-32mm-od>: HTTP status code is not handled or not allowed 2026-01-24 23:33:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tall-isolation-cart-3-drawers-standard-key-lock-37-1-2-wx22-lx42-63-100-h-mauve>: HTTP status code is not handled or not allowed 2026-01-24 23:33:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-96-economy-h-beam-adjustable-length-yellow-18-l-x-40-w-x-108-h-215lbs-1000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 23:33:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rain-parka-jacket-ansi-class-3-300d-oxfordpu-coating-limeblack-m>: HTTP status code is not handled or not allowed 2026-01-24 23:33:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-hopper-grinder-storage-system-hopper-rack-mhg-4-position>: HTTP status code is not handled or not allowed 2026-01-24 23:33:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-dielectric-union-copper>: HTTP status code is not handled or not allowed 2026-01-24 23:33:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6004-2rsnr-double-sealed-wsnap-ring-20mm-bore-42mm-od>: HTTP status code is not handled or not allowed 2026-01-24 23:33:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/divider-for-small-drawer-clear-40716>: HTTP status code is not handled or not allowed 2026-01-24 23:33:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jumbo-premium-light-weight-inspection-gloves-unhemmed-men-97-500j>: HTTP status code is not handled or not allowed 2026-01-24 23:33:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluke-87v-e2-kit-industrial-true-rms-multimeter-87-5-e2-kit>: HTTP status code is not handled or not allowed 2026-01-24 23:33:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/fluorescent_linear_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:33:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/eye_protection/safety_glasses/frameless_safety_glasses already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:33:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/330-volt-start-capacitor-108-130-mfd-108d>: HTTP status code is not handled or not allowed 2026-01-24 23:33:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-spread-basket-lifting-beam-yellow-1000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 23:33:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6017zz-double-shielded-85mm-bore-130mm-od>: HTTP status code is not handled or not allowed 2026-01-24 23:33:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dot-placard-flammable-3-blank-1>: HTTP status code is not handled or not allowed 2026-01-24 23:33:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/fluorescent_linear_bulbs landed on page that is not a product page. 2026-01-24 23:33:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/eye_protection/safety_glasses/frameless_safety_glasses landed on page that is not a product page. 2026-01-24 23:33:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swf12wuu-34-id-round-flange-doublewide-type-linear-bearing-wseals-steel>: HTTP status code is not handled or not allowed 2026-01-24 23:33:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/special_purpose_carts_trucks/shopping_baskets_carts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:33:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-acrylic-sealer-brown-gloss-finish-5-gallon-pail-1-case-cp-1542>: HTTP status code is not handled or not allowed 2026-01-24 23:34:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6806-open-30mm-bore-42mm-od>: HTTP status code is not handled or not allowed 2026-01-24 23:34:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6809-open-45mm-bore-58mm-od>: HTTP status code is not handled or not allowed 2026-01-24 23:34:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/machinery_parts_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:34:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flashing-light-led-120-240vac-hazardous-location-green-191xl-120-240g>: HTTP status code is not handled or not allowed 2026-01-24 23:34:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/drum_barrel_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:34:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 36864 pages (at 95 pages/min), scraped 16854 items (at 62 items/min) 2026-01-24 23:34:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lead-screw-bracket-for-jet-band-saws>: HTTP status code is not handled or not allowed 2026-01-24 23:34:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poly-z-brite-2-shelf-container-rack-bk183052z-30w-x-18d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 23:34:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/strobe-12-24vdc-pipe-surface-mount-amber-fb2pst-012-024a>: HTTP status code is not handled or not allowed 2026-01-24 23:34:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a36h3008gqrlp-wallmount-type-4x-encl-3625x3025x800-fiberglass>: HTTP status code is not handled or not allowed 2026-01-24 23:34:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/machinery_parts_accessories landed on page that is not a product page. 2026-01-24 23:34:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/utility_racks/drum_barrel_racks>: HTTP status code is not handled or not allowed 2026-01-24 23:34:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/oil_fluid_management/oil_fluid_transfer already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:34:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/attachment-kit-321291>: HTTP status code is not handled or not allowed 2026-01-24 23:34:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/oil_fluid_management/oil_fluid_transfer already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:34:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1000-181-l-ext-length-pipette-tip-rack-empty-rack-wafer-included-non-sterile-20pk>: HTTP status code is not handled or not allowed 2026-01-24 23:34:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drylin-t-polymer-lined-profile-guide-size-25>: HTTP status code is not handled or not allowed 2026-01-24 23:34:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-plastic-2-tray-black-shelf-maintenance-utility-cart-44-x-25-1-2-8-pneumatic>: HTTP status code is not handled or not allowed 2026-01-24 23:34:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bridges_returns already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:34:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/walk-on-floor-sign-emergency-eye-wash>: HTTP status code is not handled or not allowed 2026-01-24 23:34:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/special_purpose_carts_trucks/shopping_baskets_carts landed on page that is not a product page. 2026-01-24 23:34:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fda-approved-yellow-x-large-tote-combo-w-4wheels-47-1-4l-x-51-1-2w-x-33h>: HTTP status code is not handled or not allowed 2026-01-24 23:34:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/vehicle-fleet-maintenance/oil_fluid_management/oil_fluid_transfer landed on page that is not a product page. 2026-01-24 23:34:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/vehicle-fleet-maintenance/oil_fluid_management/oil_fluid_transfer landed on page that is not a product page. 2026-01-24 23:34:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/facility-visi-sign-lock-out-center-1>: HTTP status code is not handled or not allowed 2026-01-24 23:34:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_components/pallet_rack_enclosures already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:34:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bridges_returns>: HTTP status code is not handled or not allowed 2026-01-24 23:34:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tempered-glass-chair-mat-for-hard-floors-and-carpets-50-inch-length-by-44-inch-width>: HTTP status code is not handled or not allowed 2026-01-24 23:34:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-poly-drum-funnel-cover>: HTTP status code is not handled or not allowed 2026-01-24 23:34:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/walldog-screw-anchor-light-duty-oval-head-short-zinc>: HTTP status code is not handled or not allowed 2026-01-24 23:34:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_components/pallet_rack_enclosures landed on page that is not a product page. 2026-01-24 23:34:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/d0346-oem-motor-for-penn-vent-1-4-hp-1550-rpm-115v-open>: HTTP status code is not handled or not allowed 2026-01-24 23:34:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/511-rotunda-graphics-case-trade-show-case-10l-x-10w-x-44h-black>: HTTP status code is not handled or not allowed 2026-01-24 23:34:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/emergency_vehicle_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:34:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powerstud-wedge-expansion-anchor-sd1-34-x-414>: HTTP status code is not handled or not allowed 2026-01-24 23:34:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/54363-7hp-extended-right-angle-die-grinder-18000-rpm-geared-rear-exhaust>: HTTP status code is not handled or not allowed 2026-01-24 23:34:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/strain-relief-grips-3-4-npt-0-520-to-0-730-dia>: HTTP status code is not handled or not allowed 2026-01-24 23:34:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polly-products-econo-mizer-4-ft-backed-bench-with-arms-cedar-bench-brown-frame>: HTTP status code is not handled or not allowed 2026-01-24 23:34:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylon-plastic-adjustable-lever-stainless-steel-14-20-78-stud-177l>: HTTP status code is not handled or not allowed 2026-01-24 23:34:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precut-strips-thin-18-x-1-5-x-1-12-sonic-silver-10-pieces-pk>: HTTP status code is not handled or not allowed 2026-01-24 23:35:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbi-sala-lad-saf-vert-sfety-telescoping-single-pt-anchor-ext-galv-4-user-6116618>: HTTP status code is not handled or not allowed 2026-01-24 23:35:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/emergency_vehicle_lights landed on page that is not a product page. 2026-01-24 23:35:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-lug-style-butterfly-valve-w-epdm-seals-and-dbl-acting-pneum-actuator>: HTTP status code is not handled or not allowed 2026-01-24 23:35:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/graphic-machine-labels-safety-first-read-directions-before-using-this-machine>: HTTP status code is not handled or not allowed 2026-01-24 23:35:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machine-labels-warning-cylinders-must-be-chained-at-all-times>: HTTP status code is not handled or not allowed 2026-01-24 23:35:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-54-limit-switch-36-1-gear-ratio-w-4-contact-blocks>: HTTP status code is not handled or not allowed 2026-01-24 23:35:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:35:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/surface-mounted-end-wall-90-degrees-corners-2-ft-ft-wings-12-ft-h-w-caps-6-inch-w-silkworm>: HTTP status code is not handled or not allowed 2026-01-24 23:35:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polycarbonate-surface-mounted-90-corner-guard-3-4-inch-wing-4-foot-h-clear-taped>: HTTP status code is not handled or not allowed 2026-01-24 23:35:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 36959 pages (at 95 pages/min), scraped 16913 items (at 59 items/min) 2026-01-24 23:35:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-267-dn4-non-drip-quick-coupling-2-4-5-l>: HTTP status code is not handled or not allowed 2026-01-24 23:35:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-232-dn6-3-robust-screw-to-connect-coupling-2-9-5-l>: HTTP status code is not handled or not allowed 2026-01-24 23:35:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs landed on page that is not a product page. 2026-01-24 23:35:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-807-tlx-flat-face-twist-lock-nipple>: HTTP status code is not handled or not allowed 2026-01-24 23:35:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-chamber-and-plunger-kit-for-ph50-ph55-series-ph50acmk>: HTTP status code is not handled or not allowed 2026-01-24 23:35:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machine-labels-caution-do-not-operate-without-guards-place>: HTTP status code is not handled or not allowed 2026-01-24 23:35:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unistrut-1-58-90-fitting-p1499eg-1-hole-1-slot-electro-galvanized-6-78>: HTTP status code is not handled or not allowed 2026-01-24 23:35:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-ton-forged-eye-w-3in-id-plain-finish-b16140>: HTTP status code is not handled or not allowed 2026-01-24 23:35:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-lab-coat-navy-polycombed-cotton-regular-46>: HTTP status code is not handled or not allowed 2026-01-24 23:35:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-axle-for-6hbw20-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-24 23:35:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dripless-caulk-guns-cr200-10-oz-hd-cradle-gun-ratchet-drive>: HTTP status code is not handled or not allowed 2026-01-24 23:35:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unisex-concealed-snap-front-lab-coat-royal-blue-nomexaramid-l>: HTTP status code is not handled or not allowed 2026-01-24 23:35:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-bar-1-14-thick-x-2-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 23:35:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:35:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/frost-wall-mount-manual-vertical-liquid-soap-dispenser-stainless>: HTTP status code is not handled or not allowed 2026-01-24 23:35:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/odp-w-rigid-base-364ts-frame-60-hp-1800-rpm>: HTTP status code is not handled or not allowed 2026-01-24 23:35:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/composites/composite_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:35:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36x36x86-chrome-wire-shelving-with-8-36-d-hopper-bins-blue>: HTTP status code is not handled or not allowed 2026-01-24 23:35:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-lab-coat-white-polycombed-cotton-regular-60>: HTTP status code is not handled or not allowed 2026-01-24 23:35:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/17in-white-polishing-pad-5-per-case>: HTTP status code is not handled or not allowed 2026-01-24 23:35:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 23:35:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-built-in-combo-lock-for-horizontal-latch-box-locker-green-dial-rh>: HTTP status code is not handled or not allowed 2026-01-24 23:35:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:35:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/foam/foam_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:35:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/composites/composite_sheet_stock landed on page that is not a product page. 2026-01-24 23:35:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-general-security-combo-padlock-grey-dial>: HTTP status code is not handled or not allowed 2026-01-24 23:35:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:35:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/desktop-stapler-value-pack-black-12-case>: HTTP status code is not handled or not allowed 2026-01-24 23:35:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/adhesives_sealants_tape/adhesives_glue/general_purpose_glue already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:35:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-built-in-combination-lock-wrap-around-latch-technology-metal-dial>: HTTP status code is not handled or not allowed 2026-01-24 23:35:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-hooks-value-pack-large-5lb-cap-white-3-hooks-and-6-strips-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:35:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits landed on page that is not a product page. 2026-01-24 23:35:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-assembly-for-wpt45-rev-c-jan-2013-pallet-trucks-bj-1120-133000-10-xl>: HTTP status code is not handled or not allowed 2026-01-24 23:35:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/foam/foam_rolls landed on page that is not a product page. 2026-01-24 23:35:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:35:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-built-in-combination-lock-blue-metal-dial-right-hinged>: HTTP status code is not handled or not allowed 2026-01-24 23:35:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-24 23:35:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/adhesives_sealants_tape/adhesives_glue/general_purpose_glue landed on page that is not a product page. 2026-01-24 23:35:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/desk_accessories/desk_organizers_letter_trays already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:35:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caster-wheel-assembly-for-smx-45-walkie-pallet-truck>: HTTP status code is not handled or not allowed 2026-01-24 23:36:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/dock_bumpers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:36:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/dock_bumpers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:36:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-cubic-foot-salt-storage-box-yellow-48-l-x-33-1-4-w-x-27-3-4-h-960-capacity-lbs-nest>: HTTP status code is not handled or not allowed 2026-01-24 23:36:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caster-assembly-for-111-112-113-114-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-24 23:36:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 23:36:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/work-height-platform-truck-with-6-deep-deck-24-x-48-adj-height-polyurethane-retaining-lip>: HTTP status code is not handled or not allowed 2026-01-24 23:36:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/respiratory_protection/supplied_air_respirator_systems already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:36:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microflex-174-lifestar-ec-153-lse-104-nitrile-gloves-powder-free-size-xxl-100-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:36:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 37045 pages (at 86 pages/min), scraped 16962 items (at 49 items/min) 2026-01-24 23:36:09 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/desk_accessories/desk_organizers_letter_trays landed on page that is not a product page. 2026-01-24 23:36:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/dock_bumpers>: HTTP status code is not handled or not allowed 2026-01-24 23:36:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/dock_bumpers landed on page that is not a product page. 2026-01-24 23:36:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microflex-174-supreno-174-ec-sec-375-nitrile-gloves-powder-free-size-m-100-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:36:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/koch-filter-merv-11-high-capacity-xl11-pleated-panel-extended-surface-20-inch-w-x-20-inch-h-x-1-inch-d>: HTTP status code is not handled or not allowed 2026-01-24 23:36:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/respiratory_protection/supplied_air_respirator_systems landed on page that is not a product page. 2026-01-24 23:36:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/koch-filter-merv-11-standard-capacity-xl11-pleated-panel-ext-surface-20-inch-w-x-20-inch-h-x-4-inch-d>: HTTP status code is not handled or not allowed 2026-01-24 23:36:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:36:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-assembly-for-sp-36-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-24 23:36:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slb-lever-hoists-3300-lb-capacity-10-ft-lift-15-1116-head-room>: HTTP status code is not handled or not allowed 2026-01-24 23:36:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-cf-hand-chain-hoist-1-10-lift>: HTTP status code is not handled or not allowed 2026-01-24 23:36:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:36:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-single-arm-shelf-bracket-green-pair>: HTTP status code is not handled or not allowed 2026-01-24 23:36:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-plastic-bar-w-lse-acrylic-adhesive-18-thick-x-12-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-24 23:36:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/top-chest-w-4-drawers-34-w-x-25-1-4-d-x-27-h-red>: HTTP status code is not handled or not allowed 2026-01-24 23:36:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x36x74-chrome-wire-shelving-with-30-quickpick-double-open-bins-red-1>: HTTP status code is not handled or not allowed 2026-01-24 23:36:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-ner-dual-speed-electric-chain-hoist-12-15-1525>: HTTP status code is not handled or not allowed 2026-01-24 23:36:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drive-tire-for-w-40-xl-w-40-xt-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-24 23:36:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 23:36:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 23:36:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sos-outdoor-storage-container-72-x-36-x-36-36-cu-ft-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:36:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/cart_truck_accessories/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:36:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12u-linier-fixed-wall-mount-cabinet-solid-door>: HTTP status code is not handled or not allowed 2026-01-24 23:36:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:36:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nps-8ft-mobile-cafeteria-stool-unit-with-mdf-core-top-protect-edge-gray-top-gray-stool>: HTTP status code is not handled or not allowed 2026-01-24 23:36:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pivot-pin-for-smx-45-walkie-pallet-truck>: HTTP status code is not handled or not allowed 2026-01-24 23:36:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyurethane-1-channel-hose-bridges-set-2-channel-black-eb12>: HTTP status code is not handled or not allowed 2026-01-24 23:36:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/composites/composite_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:36:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-sner-electric-chain-hoist-motor-trolley-1-20-7>: HTTP status code is not handled or not allowed 2026-01-24 23:36:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-cabinet-exhaust-fan-with-shutters>: HTTP status code is not handled or not allowed 2026-01-24 23:36:45 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/cart_truck_accessories/pallet_truck_parts_accessories/pallet_truck_wheel_kits landed on page that is not a product page. 2026-01-24 23:36:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-24 23:36:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cobbler-apron-navy-polyestercotton-m>: HTTP status code is not handled or not allowed 2026-01-24 23:36:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:36:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12x36x74-chrome-wire-shelving-with-77-4-h-shelf-bins-black>: HTTP status code is not handled or not allowed 2026-01-24 23:36:51 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/composites/composite_sheet_stock landed on page that is not a product page. 2026-01-24 23:36:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-assembly-for-pe-3000-pallet-trucks-cr-77086-a>: HTTP status code is not handled or not allowed 2026-01-24 23:36:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caster-assembly-for-w-40z-b218-45z-c215-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-24 23:36:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:36:55 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-24 23:36:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ballymore-11-step-perforated-cantilever-ladder-28-overhang>: HTTP status code is not handled or not allowed 2026-01-24 23:36:56 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:17 2026-01-24 23:36:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:36:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:36:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rcl-550gc>: HTTP status code is not handled or not allowed 2026-01-24 23:37:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-24 23:37:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24x36x74-chrome-wire-shelving-with-33-4-h-shelf-bins-ivory>: HTTP status code is not handled or not allowed 2026-01-24 23:37:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 23:37:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/collarless-butcher-wrap-wo-pockets-white-polyestercombed-cotton-xl>: HTTP status code is not handled or not allowed 2026-01-24 23:37:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/composites/composite_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:37:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/retail_fixtures_displays_equipment/garment_supplies/sewing_machines already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:37:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-bracket-for-vostermans-50-in-circulator-fans-cmfbracket72d>: HTTP status code is not handled or not allowed 2026-01-24 23:37:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-24 23:37:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uhmw-polyethylene-plastic-bar-14-thick-x-6-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 23:37:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 37134 pages (at 89 pages/min), scraped 17010 items (at 48 items/min) 2026-01-24 23:37:09 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 23:37:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-x-30-bar-height-breakroom-table-black>: HTTP status code is not handled or not allowed 2026-01-24 23:37:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rolling-ladder-7-step-aluminum-perforated-14-deep-top-step-with-handrails-24-w-step>: HTTP status code is not handled or not allowed 2026-01-24 23:37:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/composites/composite_rod_stock landed on page that is not a product page. 2026-01-24 23:37:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/retail_fixtures_displays_equipment/garment_supplies/sewing_machines landed on page that is not a product page. 2026-01-24 23:37:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/collarless-butcher-wrap-winterior-pockets-white-polyestercombed-cotton-l>: HTTP status code is not handled or not allowed 2026-01-24 23:37:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/office_furniture/literature_mail_sorters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:37:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/46-square-picnic-table-with-backrests-expanded-metal-blue>: HTTP status code is not handled or not allowed 2026-01-24 23:37:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-kit-for-electric-pallet-jack-truck-gwk-w80z-lw-fits-hyster-model-w80z>: HTTP status code is not handled or not allowed 2026-01-24 23:37:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-dc-control-bc154-r-dc-scr-control-115-230v-1-50-2-hp-nema-4x>: HTTP status code is not handled or not allowed 2026-01-24 23:37:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/semi-clear-fda-silicone-rubber-sheet-whigh-temp-adhesive-60a-18-thick-x-12w-x-12l>: HTTP status code is not handled or not allowed 2026-01-24 23:37:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/rainwear already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:37:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/office_furniture/literature_mail_sorters>: HTTP status code is not handled or not allowed 2026-01-24 23:37:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/monitor_risers_stands already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:37:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sample-tube-2ml-external-threads-polypropylene-round-bottom-self-standing-1000-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:37:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ridgeline-vented-cap-style-hard-hat-blue-4-point-ratchet-suspension>: HTTP status code is not handled or not allowed 2026-01-24 23:37:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/rainwear already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:37:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:37:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24x60x63-chrome-wire-shelving-with-36-3-h-grid-container-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:37:34 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/rainwear landed on page that is not a product page. 2026-01-24 23:37:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/monitor_risers_stands>: HTTP status code is not handled or not allowed 2026-01-24 23:37:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-bar-34-thick-x-1-14-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-24 23:37:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/rainwear landed on page that is not a product page. 2026-01-24 23:37:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-24 23:37:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leadhead-auto-darkening-welding-helmet-manual-98x87mm-american-eagle>: HTTP status code is not handled or not allowed 2026-01-24 23:37:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/power_tools/power_shears_nibblers_cutters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:37:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-deluxe-office-partition-panel-24-1-4w-x-43-1-2h-blue>: HTTP status code is not handled or not allowed 2026-01-24 23:37:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-34-thick-x-5-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-24 23:37:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3832-adjustable-deck-rail-bracket-steel-575-x-975-x-175-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:37:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/block-grip-vinyl-floor-mat-2mm-thick-4-x-15-brown>: HTTP status code is not handled or not allowed 2026-01-24 23:37:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:37:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-deluxe-office-partition-panel-60-1-4w-x-43-1-2h-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:37:56 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/power_tools/power_shears_nibblers_cutters landed on page that is not a product page. 2026-01-24 23:37:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-1-12-thick-x-2-12-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-24 23:37:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-allsteel-triple-brace-double-hinge-folding-chair-black>: HTTP status code is not handled or not allowed 2026-01-24 23:38:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paramount-counter-height-cabinet-assembled-36x18x42-gray-269871gy>: HTTP status code is not handled or not allowed 2026-01-24 23:38:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-sheet-14-thick-x-39-wide-x-78-long>: HTTP status code is not handled or not allowed 2026-01-24 23:38:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/omega-ii-cap-with-accessory-slots-and-6-point-slide-lock-suspension-silver>: HTTP status code is not handled or not allowed 2026-01-24 23:38:04 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-24 23:38:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:38:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tach-it-micro-s-tagging-guns>: HTTP status code is not handled or not allowed 2026-01-24 23:38:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/built-in-combo-lock-ada-compliant-vertical-dead-bolt-2-user-keys>: HTTP status code is not handled or not allowed 2026-01-24 23:38:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 37223 pages (at 89 pages/min), scraped 17065 items (at 55 items/min) 2026-01-24 23:38:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:38:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tube-holder-use-with-gtr-hd-series-tube-rotators-48-place-for-15ml-microcentrifuge-tubes>: HTTP status code is not handled or not allowed 2026-01-24 23:38:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:38:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-24 23:38:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/head_face_protection/head_protection_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:38:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:38:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-drawer-counter-high-24-w-cabinet-beige-2>: HTTP status code is not handled or not allowed 2026-01-24 23:38:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/500g-deicing-sprayer-55hp-200p-pump-75-12-hose-manual-reel-7-pro-boom>: HTTP status code is not handled or not allowed 2026-01-24 23:38:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents landed on page that is not a product page. 2026-01-24 23:38:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents landed on page that is not a product page. 2026-01-24 23:38:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jmd-45vspft-var-spd-geared-hd-sq-col-mill-drill-pwr-downfeed-dp500-x-axis-pwrfeed>: HTTP status code is not handled or not allowed 2026-01-24 23:38:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/head_face_protection/head_protection_accessories>: HTTP status code is not handled or not allowed 2026-01-24 23:38:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-24 23:38:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-stainless-steel-utility-sink-w-faucet-24-x-24-x-14-deep-non-nsf>: HTTP status code is not handled or not allowed 2026-01-24 23:38:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wp3403-iv-keystone-1-gang-3-port-wall-plate-ivory>: HTTP status code is not handled or not allowed 2026-01-24 23:38:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ac2m02-bk-2m-6-6-ft-high-speed-hdmi-cable-with-ethernet>: HTTP status code is not handled or not allowed 2026-01-24 23:38:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/centrifuge-tube-10ml-attached-yellow-polypropylene-screw-cap-polypropylene-sterile-1000-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:38:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50in-camo-series-cree-led-light-bar-300w-20-000lm-while-supplies-last>: HTTP status code is not handled or not allowed 2026-01-24 23:38:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-compliance-cover-for-hd-single-row-10in-led-light-bar>: HTTP status code is not handled or not allowed 2026-01-24 23:38:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/31-deluxe-triple-brush-push-power-sweeper-132-gal-cap-bg-477>: HTTP status code is not handled or not allowed 2026-01-24 23:38:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pipette-diamond-pro-fixed-volume-50ul-blue>: HTTP status code is not handled or not allowed 2026-01-24 23:38:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=jtm-1254rvs-with-acu-rite-203-3x-q-dro-x-y-z-powerfeeds already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:38:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tru-ray-construction-paper-9-x-12-bright-assorted50-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:38:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durham-steel-compartment-box-rack-bearing-20-x-15-3-4-x-15-with-4-of-20-compartment-boxes>: HTTP status code is not handled or not allowed 2026-01-24 23:38:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wireless-tube-rack-16mm-full-size-72-place-nylon-white>: HTTP status code is not handled or not allowed 2026-01-24 23:38:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transfer-pipet-65ml-narrow-stem-155mm-5000-pk>: HTTP status code is not handled or not allowed 2026-01-24 23:38:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=jtm-1254rvs-with-acu-rite-203-3x-q-dro-x-y-z-powerfeeds>: HTTP status code is not handled or not allowed 2026-01-24 23:38:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-camwear-perforated-spoon-clear>: HTTP status code is not handled or not allowed 2026-01-24 23:38:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transfer-pipet-70ml-general-purpose-standard-155mm-5000-pk>: HTTP status code is not handled or not allowed 2026-01-24 23:38:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=etm-949-mill-3-axis-acu-rite-203-knee-dro-x-axis-jet-powerfeed-usa-pwrd-draw-bar already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:38:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-welded-gear-locker-with-door-legs-24x24x72-red>: HTTP status code is not handled or not allowed 2026-01-24 23:38:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aspirating-pipette-2ml-polystyrene-standard-tip-275mm-sterile-no-printing-pp-800-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:38:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-corrugated-mailer-4-x-3-x-1->: HTTP status code is not handled or not allowed 2026-01-24 23:39:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=etm-949-mill-2-axis-acu-rite-203-dro-x-y-axis-jet-powerfeeds-usa-powered-draw-bar already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:39:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=etm-949-mill-3-axis-acu-rite-203-knee-dro-x-axis-jet-powerfeed-usa-pwrd-draw-bar landed on page that is not a product page. 2026-01-24 23:39:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/market-tray-10x30-black>: HTTP status code is not handled or not allowed 2026-01-24 23:39:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evergreen-tissue-paper-20-x-30>: HTTP status code is not handled or not allowed 2026-01-24 23:39:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=evs-949-mill-3-axis-acu-rite-303-quill-dro-x-y-z-axis-jet-pwrfds-usa-pwrd-drw-br already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:39:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/market-tray-8x30-blue>: HTTP status code is not handled or not allowed 2026-01-24 23:39:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3891-97-1-1-2-inch-self-priming-bronze-centrifugal-pump-100gpm-75psi-viton-seal>: HTTP status code is not handled or not allowed 2026-01-24 23:39:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 37311 pages (at 88 pages/min), scraped 17121 items (at 56 items/min) 2026-01-24 23:39:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=etm-949-mill-2-axis-newall-dp700-dro-servo-x-y-z-axis-pwrfds-usa-air-pwrd-draw-bar already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:39:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=etm-949-mill-2-axis-acu-rite-203-dro-x-y-axis-jet-powerfeeds-usa-powered-draw-bar landed on page that is not a product page. 2026-01-24 23:39:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camrack-base-rack-full-size-4-inch-inside-stack-height-soft-gray-nsf>: HTTP status code is not handled or not allowed 2026-01-24 23:39:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kraft-literature-mailer-12-x-8-x-3>: HTTP status code is not handled or not allowed 2026-01-24 23:39:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/combination_metalworking_machines already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:39:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=evs-949-mill-3-axis-acu-rite-303-quill-dro-x-y-z-axis-jet-pwrfds-usa-pwrd-drw-br landed on page that is not a product page. 2026-01-24 23:39:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camrack-base-rack-4-1-4-inch-inside-stack-height-beige-nsf>: HTTP status code is not handled or not allowed 2026-01-24 23:39:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bottle-wide-mouth-ldpe-attached-polypropylene-screw-cap-30ml-12-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:39:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1853-sd-3l6-3-8-x-1-4-double-cut-ball-shaped-bur>: HTTP status code is not handled or not allowed 2026-01-24 23:39:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-step-stock-picking-ladder-perforated-tread-spl-7-p>: HTTP status code is not handled or not allowed 2026-01-24 23:39:21 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=etm-949-mill-2-axis-newall-dp700-dro-servo-x-y-z-axis-pwrfds-usa-air-pwrd-draw-bar landed on page that is not a product page. 2026-01-24 23:39:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/combination_metalworking_machines landed on page that is not a product page. 2026-01-24 23:39:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pipette-diamond-fixed-volume-5000ul-5ml-light-green>: HTTP status code is not handled or not allowed 2026-01-24 23:39:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/media-flap-parflp006000-for-portacool-jetstream-260>: HTTP status code is not handled or not allowed 2026-01-24 23:39:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securall-self-latch-mobile-counter-high-industrial-cabinet-red>: HTTP status code is not handled or not allowed 2026-01-24 23:39:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m6-1-0-x-60mm-din-7985-phillips-drive-pan-head-zinc-finish-steel-machine-screw-pkg-100>: HTTP status code is not handled or not allowed 2026-01-24 23:39:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/telescoping-outer-boxes-16-1-2-x-16-1-2-x-40>: HTTP status code is not handled or not allowed 2026-01-24 23:39:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/btl-rack-15-double-tier-integral-keyhole-s-s>: HTTP status code is not handled or not allowed 2026-01-24 23:39:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/binders_covers/clipboards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:39:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inoculation-loop-rigid-1ul-with-needle-with-calibration-certificate-sterile-blue-500-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:39:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/graduated-cylinder-polypropylene-printed-graduations-2000ml>: HTTP status code is not handled or not allowed 2026-01-24 23:39:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/filter-pipette-tip-01-20ul-certified-universal-low-retention-45mm-natural-sterile-960pk>: HTTP status code is not handled or not allowed 2026-01-24 23:39:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/supco-therm-o-disc-commercial-refrigeration-defrost-thermostat-55-35>: HTTP status code is not handled or not allowed 2026-01-24 23:39:50 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/binders_covers/clipboards landed on page that is not a product page. 2026-01-24 23:39:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/450l100-standard-timing-belt-l-1-x-45-t120-trapezoidal>: HTTP status code is not handled or not allowed 2026-01-24 23:39:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-1-4-x-77-1-2-h-deluxe-office-partition-panel-non-electric-w-raceway-gray-277557gy>: HTTP status code is not handled or not allowed 2026-01-24 23:39:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/169l050-standard-timing-belt-l-1-2-x-16-7-8-t45-trapezoidal>: HTTP status code is not handled or not allowed 2026-01-24 23:39:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/supco-compact-three-phase-motor-protector>: HTTP status code is not handled or not allowed 2026-01-24 23:39:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dissipative-dual-layer-rubber-roll-light-blue-0060in-x-36in-x-50ft>: HTTP status code is not handled or not allowed 2026-01-24 23:40:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/355h100-standard-timing-belt-h-1-x-35-1-2-t71-trapezoidal>: HTTP status code is not handled or not allowed 2026-01-24 23:40:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/centrifuge-tube-15ml-attached-red-screw-cap-pp-sterile-500-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:40:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/electrical_power_testing/electrical_power_testing_parts_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:40:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-angle-beam-48-w-x-48-d-x-84-h-4-level-starter-no-deck-tan>: HTTP status code is not handled or not allowed 2026-01-24 23:40:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 37411 pages (at 100 pages/min), scraped 17191 items (at 70 items/min) 2026-01-24 23:40:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/graduated-cylinder-polypropylene-molded-graduations-100ml>: HTTP status code is not handled or not allowed 2026-01-24 23:40:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esd-drawer-with-label-3-5-8-inch-w-x-15-3-4-inch-d-x-3-1-4-inch-h-price-for-pkg-qty-30>: HTTP status code is not handled or not allowed 2026-01-24 23:40:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/testing_inspection_instruments/electrical_power_testing/electrical_power_testing_parts_accessories>: HTTP status code is not handled or not allowed 2026-01-24 23:40:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/extra_heavy_boltless_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:40:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6333560-mfb-cabinet-u-handles-nylon-plastic-with-tapped-inserts>: HTTP status code is not handled or not allowed 2026-01-24 23:40:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/office_furniture/literature_mail_sorters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:40:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tuffblok-xl-tube-rack-4-way-polypropylene-orange-5-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:40:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securall-sump-liner-b1600>: HTTP status code is not handled or not allowed 2026-01-24 23:40:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/extra_heavy_boltless_shelves>: HTTP status code is not handled or not allowed 2026-01-24 23:40:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securall-personnel-door-36wx80h>: HTTP status code is not handled or not allowed 2026-01-24 23:40:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/superthrive-vi30148-gardening-vitamin-solution-plant-food-4-oz>: HTTP status code is not handled or not allowed 2026-01-24 23:40:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3400-squids-174-grabber-dual-clip-mount-granite>: HTTP status code is not handled or not allowed 2026-01-24 23:40:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/office_furniture/literature_mail_sorters landed on page that is not a product page. 2026-01-24 23:40:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/funnel-powder-polypropylene-180mm-5-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:40:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-o-smith-single-phase-jet-pump-motor-115-230-volts-3450-rpm-1-2hp-2>: HTTP status code is not handled or not allowed 2026-01-24 23:40:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-o-smith-5-5-8-split-capacitor-condenser-fan-motor-208-230-volts-1075-rpm-13>: HTTP status code is not handled or not allowed 2026-01-24 23:40:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-vis-lined-bib-overall-class-e-3xl-yellow>: HTTP status code is not handled or not allowed 2026-01-24 23:40:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cap-snap-12-13mm-pe-for-vacuum-and-test-tubes-green-1000-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:40:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/test-tube-13-x-75mm-5ml-polystyrene-2000-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:40:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-packing-list-envelopes-7-l-x-5-1-2-w-clear-1000-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:40:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-o-smith-full-rated-pool-filter-motor-115-230-volts-3450-rpm-1>: HTTP status code is not handled or not allowed 2026-01-24 23:40:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-bank-15a-onboard-battery-charger>: HTTP status code is not handled or not allowed 2026-01-24 23:40:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compact-boutique-folding-crib-with-oversized-casters-foam-mattress-white>: HTTP status code is not handled or not allowed 2026-01-24 23:40:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/trash_recycling/recycling_bins_receptacles/recycling_cans already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:40:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/funnel-powder-polypropylene-150mm-10-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:40:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hazardous-waste-california-6-6-yellow-red-black>: HTTP status code is not handled or not allowed 2026-01-24 23:40:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wireless-tube-rack-with-rfid-for-captrack-ct1-and-ct2-models-half-size-13mm-36-place-magenta>: HTTP status code is not handled or not allowed 2026-01-24 23:41:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/trash_recycling/recycling_bins_receptacles/recycling_cans landed on page that is not a product page. 2026-01-24 23:41:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/number-3-hydraulic-adjustable-height-packing-workbench-laminate-square-edge-96-x-30-black>: HTTP status code is not handled or not allowed 2026-01-24 23:41:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_batteries/vehicle_battery_accessories/vehicle_battery_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:41:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beaker-with-handle-polypropylene-molded-graduations-5000ml-12-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:41:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-3-3-shaded-pole-open-motor-115-volts-1500-rpm-18>: HTTP status code is not handled or not allowed 2026-01-24 23:41:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 37505 pages (at 94 pages/min), scraped 17258 items (at 67 items/min) 2026-01-24 23:41:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oxidizer-4-4-yellow-black>: HTTP status code is not handled or not allowed 2026-01-24 23:41:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_batteries/vehicle_battery_accessories/vehicle_battery_chargers landed on page that is not a product page. 2026-01-24 23:41:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10a-12v-fully-automatic-battery-charger>: HTTP status code is not handled or not allowed 2026-01-24 23:41:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transport-tube-5ml-polypropylene-ss-separate-green-screwcap-mg-1000-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:41:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-cart-for-signs-shadow-boards-up-to-68-h-x-30-w-anodized-aluminum-frame>: HTTP status code is not handled or not allowed 2026-01-24 23:41:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_tools/cleaning_supply_dispensers_organizers/broom_holders_shadow_boards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:41:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluorescent-orange-2-dia-discs>: HTTP status code is not handled or not allowed 2026-01-24 23:41:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/hvac_electric_motors/unit_bearing_fan_motors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:41:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slim-trash-can-23-gallon-red>: HTTP status code is not handled or not allowed 2026-01-24 23:41:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/to-go-rack-chrome-24w-x-30l-x-63h-4-wire-shelves-dividers-ledges>: HTTP status code is not handled or not allowed 2026-01-24 23:41:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_tools/cleaning_supply_dispensers_organizers/broom_holders_shadow_boards>: HTTP status code is not handled or not allowed 2026-01-24 23:41:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-2-mr-clip-blue-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-24 23:41:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/non-ansi-multi-color-long-sleeve-safety-t-shirt-with-black-bottom-blue-xl>: HTTP status code is not handled or not allowed 2026-01-24 23:41:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-2-mr-clip-safety-pink-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-24 23:41:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:41:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/non-ansi-multi-color-long-sleeve-safety-t-shirt-with-black-bottom-blue-5xl>: HTTP status code is not handled or not allowed 2026-01-24 23:41:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/hvac_electric_motors/unit_bearing_fan_motors landed on page that is not a product page. 2026-01-24 23:41:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-seal-bubble-mailers-1-7-1-4-x-12-gold-100-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:41:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-3-3-shaded-pole-dra-inducer-blower-115-volts-3000-rpm-10>: HTTP status code is not handled or not allowed 2026-01-24 23:41:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-multi-color-vest-cert-green-2xl-3xl>: HTTP status code is not handled or not allowed 2026-01-24 23:41:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-24 23:41:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-3-night-glow-sherpa-line-heavy-weight-sierra-jacket-xl>: HTTP status code is not handled or not allowed 2026-01-24 23:41:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/sneeze_guards_distancing_barriers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:41:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/origami-5-tier-large-heavy-duty-rack-78-w-x-36-w-x-21-h-white>: HTTP status code is not handled or not allowed 2026-01-24 23:41:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mantis-uplight-max-36-wall-sconce-36w-commercial-fly-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:41:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/material-handling/lift_tables_work_positioners/pallet_carousels_positioners already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:41:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/690-12-12-inch-1-4-fnpt-dual-head-air-chuck>: HTTP status code is not handled or not allowed 2026-01-24 23:41:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-16bs19hx42-metric-1-pitch-42mm-finished-bore-19-teeth>: HTTP status code is not handled or not allowed 2026-01-24 23:41:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/desktop_dividers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:41:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80bs34x21516-1-pitch-2-1516-finished-bore-34-teeth>: HTTP status code is not handled or not allowed 2026-01-24 23:41:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-120a15hx114-a-plate-1-12-pitch-1-14-bore-15-teeth>: HTTP status code is not handled or not allowed 2026-01-24 23:42:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/material-handling/lift_tables_work_positioners/pallet_carousels_positioners>: HTTP status code is not handled or not allowed 2026-01-24 23:42:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/csi-kit-professional-bed-bug-detection>: HTTP status code is not handled or not allowed 2026-01-24 23:42:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-plastic-garbage-can-with-lid-dolly-32-gal-brown>: HTTP status code is not handled or not allowed 2026-01-24 23:42:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/partitions_dividers/desktop_dividers landed on page that is not a product page. 2026-01-24 23:42:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 37595 pages (at 90 pages/min), scraped 17319 items (at 61 items/min) 2026-01-24 23:42:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-wd14aa2pz7-1-4-hp-1-phase-1700-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:42:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-5456-1-8-hp-1-phase-825-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:42:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/fans_blowers/exhaust_fans/bathroom_exhaust_fans already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:42:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-t32c2j14-1-1-2-hp-1-phase-1725-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:42:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/power_tools/drills_drivers_impact_wrenches/power_drills already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:42:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=us-motor-5456-1-8-hp-1-phase-825-rpm landed on page that is not a product page. 2026-01-24 23:42:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/fans_blowers/exhaust_fans/bathroom_exhaust_fans landed on page that is not a product page. 2026-01-24 23:42:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=us-motor-wd14aa2pz7-1-4-hp-1-phase-1700-rpm landed on page that is not a product page. 2026-01-24 23:42:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-t3c2p21-3-hp-1-phase-1730-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:42:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=us-motor-t32c2j14-1-1-2-hp-1-phase-1725-rpm landed on page that is not a product page. 2026-01-24 23:42:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/power_tools/drills_drivers_impact_wrenches already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:42:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/power_tools/drills_drivers_impact_wrenches landed on page that is not a product page. 2026-01-24 23:42:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=us-motor-t3c2p21-3-hp-1-phase-1730-rpm landed on page that is not a product page. 2026-01-24 23:42:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-agh10fl2-1-1-8-hp-1-phase-3450-1725-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:42:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-ee516-1-hp-3-phase-3450-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:42:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/general_purpose_electric_motors/three_phase_motors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:42:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:42:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=us-motor-agh10fl2-1-1-8-hp-1-phase-3450-1725-rpm landed on page that is not a product page. 2026-01-24 23:43:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=us-motor-ee516-1-hp-3-phase-3450-rpm landed on page that is not a product page. 2026-01-24 23:43:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:43:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:43:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:43:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves landed on page that is not a product page. 2026-01-24 23:43:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/general_purpose_electric_motors/three_phase_motors landed on page that is not a product page. 2026-01-24 23:43:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-c200p2c-200-hp-3-phase-1785-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:43:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 37681 pages (at 86 pages/min), scraped 17395 items (at 76 items/min) 2026-01-24 23:43:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves landed on page that is not a product page. 2026-01-24 23:43:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-8726-1-3-hp-1-phase-1625-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:43:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves landed on page that is not a product page. 2026-01-24 23:43:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves landed on page that is not a product page. 2026-01-24 23:43:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=us-motor-c200p2c-200-hp-3-phase-1785-rpm landed on page that is not a product page. 2026-01-24 23:43:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=us-motor-8726-1-3-hp-1-phase-1625-rpm landed on page that is not a product page. 2026-01-24 23:43:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/decor_linens/bathroom_accessories/grab_bars already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:43:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/decor_linens/bathroom_accessories/grab_bars landed on page that is not a product page. 2026-01-24 23:44:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 37772 pages (at 91 pages/min), scraped 17480 items (at 85 items/min) 2026-01-24 23:44:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/door_frames already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/door_frames landed on page that is not a product page. 2026-01-24 23:44:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/doors_windows/doors landed on page that is not a product page. 2026-01-24 23:44:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/doors_windows/doors landed on page that is not a product page. 2026-01-24 23:44:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/doors_windows/doors landed on page that is not a product page. 2026-01-24 23:44:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/doors_windows/doors landed on page that is not a product page. 2026-01-24 23:44:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/doors_windows/doors landed on page that is not a product page. 2026-01-24 23:44:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/doors_windows/doors landed on page that is not a product page. 2026-01-24 23:44:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:34 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/doors_windows/doors landed on page that is not a product page. 2026-01-24 23:44:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/doors_windows/doors landed on page that is not a product page. 2026-01-24 23:44:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_plates_braces already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/doors_windows/doors landed on page that is not a product page. 2026-01-24 23:44:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/doors_windows/doors landed on page that is not a product page. 2026-01-24 23:44:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_plates_braces already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/doors_windows/doors landed on page that is not a product page. 2026-01-24 23:44:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:44 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/doors_windows/doors landed on page that is not a product page. 2026-01-24 23:44:44 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_plates_braces landed on page that is not a product page. 2026-01-24 23:44:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/doors_windows/doors landed on page that is not a product page. 2026-01-24 23:44:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/doors_windows/doors landed on page that is not a product page. 2026-01-24 23:44:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/doors_windows/doors landed on page that is not a product page. 2026-01-24 23:44:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_plates_braces landed on page that is not a product page. 2026-01-24 23:44:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:50 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 23:44:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/doors_windows/doors landed on page that is not a product page. 2026-01-24 23:44:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/foodservice_disposables/disposable_lids already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/foodservice_disposables/disposable_plates_bowls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:44:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-24 23:45:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/foodservice_disposables/disposable_lids landed on page that is not a product page. 2026-01-24 23:45:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/foodservice_disposables/disposable_plates_bowls landed on page that is not a product page. 2026-01-24 23:45:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/wheelchairs_physical_aids/walkers_rollators already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:45:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 37832 pages (at 60 pages/min), scraped 17517 items (at 37 items/min) 2026-01-24 23:45:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/exam_patient_room_supplies/exam_room_consumables/alcohol_prep_pads already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:45:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/wheelchairs_physical_aids/walkers_rollators landed on page that is not a product page. 2026-01-24 23:45:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/physical_therapy/exercise_fitness_equipment/exercise_weights_weight_training already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:45:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:45:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/exam_patient_room_supplies/exam_room_consumables/alcohol_prep_pads landed on page that is not a product page. 2026-01-24 23:45:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/seating/footrests_ottomans already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:45:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/power_tools/drills_drivers_impact_wrenches/power_drill_combo_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:45:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/tool_storage/chests_roller_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:45:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/physical_therapy/exercise_fitness_equipment/exercise_weights_weight_training landed on page that is not a product page. 2026-01-24 23:45:34 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks landed on page that is not a product page. 2026-01-24 23:45:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/seating/footrests_ottomans landed on page that is not a product page. 2026-01-24 23:45:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/material-handling/cranes_hoists/crane_parts_accessories/lifting_magnets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:45:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/air_conditioners_chillers/ductless_air_conditioners/ductless_air_handlers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:45:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/air_conditioners_chillers/ductless_air_conditioners/ductless_air_handlers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:45:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/power_tools/drills_drivers_impact_wrenches/power_drill_combo_kits landed on page that is not a product page. 2026-01-24 23:45:42 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/tool_storage/chests_roller_cabinets landed on page that is not a product page. 2026-01-24 23:45:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/material-handling/cranes_hoists/crane_parts_accessories/lifting_magnets landed on page that is not a product page. 2026-01-24 23:45:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/air_conditioners_chillers/ductless_air_conditioners/ductless_air_handlers landed on page that is not a product page. 2026-01-24 23:45:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/air_conditioners_chillers/ductless_air_conditioners/ductless_air_handlers landed on page that is not a product page. 2026-01-24 23:46:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 37925 pages (at 93 pages/min), scraped 17599 items (at 82 items/min) 2026-01-24 23:46:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/air_purifiers/hepa_media_air_purifiers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:46:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commercial-grade-cargo-lifting-net-7000-lb-capacity-10-opening-72l-x-72w-orange>: HTTP status code is not handled or not allowed 2026-01-24 23:46:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/secure-click-organic-vapor-cartridge-filter-p100-d80921>: HTTP status code is not handled or not allowed 2026-01-24 23:46:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/medical_devices_instruments/vital_signs_monitors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:46:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lr6-led-3-tier-signal-tower-light-no-flashing-w-buzzer-24v-redambergreen>: HTTP status code is not handled or not allowed 2026-01-24 23:46:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/light-duty-conveyor-netting-1146-lb-capacity-600l-x-72w-black>: HTTP status code is not handled or not allowed 2026-01-24 23:46:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/patch-seal-spray-14-oz-capacity-clear-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-24 23:46:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/upright-controlled-room-temperature-cabinet-15-cuft-capacity-solid-door>: HTTP status code is not handled or not allowed 2026-01-24 23:46:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/medical_devices_instruments/vital_signs_monitors>: HTTP status code is not handled or not allowed 2026-01-24 23:46:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80bs09hx1-1-pitch-1-finished-bore-9-teeth>: HTTP status code is not handled or not allowed 2026-01-24 23:46:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-bath-tissue-septic-safe-2-ply-wht-3-3-4-inch-x-4-inch-625-sheets-roll-48-rolls-ctn>: HTTP status code is not handled or not allowed 2026-01-24 23:46:45 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/air_purifiers/hepa_media_air_purifiers landed on page that is not a product page. 2026-01-24 23:46:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-40sh26h-qd-bushed-12-pitch-26-teeth>: HTTP status code is not handled or not allowed 2026-01-24 23:46:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:46:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:46:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:46:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/red-dot-ecg-monitoring-electrodes-0-9-inch-x-0-7-inch-neonatal-limb-band-3-bg-100-bg-cs>: HTTP status code is not handled or not allowed 2026-01-24 23:46:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:46:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steri-drape-cesarean-section-sheet-with-aperture-pouch-7966-101-inch-x-122-inch-5-each-case>: HTTP status code is not handled or not allowed 2026-01-24 23:46:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:46:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:46:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zig-elusion-heritage-work-men-s-low-cut-sneaker-composite-toe-size-10-5m-gray-black>: HTTP status code is not handled or not allowed 2026-01-24 23:46:56 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-24 23:46:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-24 23:46:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zig-elusion-heritage-work-men-s-low-cut-sneaker-composite-toe-size-9w-black-red>: HTTP status code is not handled or not allowed 2026-01-24 23:46:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80bs27x134-1-pitch-1-34-finished-bore-27-teeth>: HTTP status code is not handled or not allowed 2026-01-24 23:47:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-24 23:47:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 23:47:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/attest-sterilization-record-envelope-9-1-2-inch-x-11-1-2-inch-100-pk-5-pk-cs>: HTTP status code is not handled or not allowed 2026-01-24 23:47:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 23:47:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 23:47:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:47:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vitals-skate-inspired-work-shoes-composite-toe-size-12m-rinsed-black-tower-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:47:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:47:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:47:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-square-shape-weigh-boat-45-x-45-mm-20-ml-polystyrene-black-500-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:47:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:47:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 38011 pages (at 86 pages/min), scraped 17661 items (at 62 items/min) 2026-01-24 23:47:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/truflex-fly-skylar-sneaker-composite-toe-size-12m-black>: HTTP status code is not handled or not allowed 2026-01-24 23:47:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-propette-le-single-channel-pipette-1-10ul>: HTTP status code is not handled or not allowed 2026-01-24 23:47:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 23:47:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-24 23:47:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 23:47:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-24 23:47:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:47:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-100a54x138-a-plate-1-14-pitch-1-38-bore-54-teeth>: HTTP status code is not handled or not allowed 2026-01-24 23:47:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:47:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/true-skate-inspired-work-shoes-composite-toe-size-13w-navy>: HTTP status code is not handled or not allowed 2026-01-24 23:47:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80bb12h-idler-1-pitch-34-bore-12-teeth>: HTTP status code is not handled or not allowed 2026-01-24 23:47:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:47:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35bs34hx118-38-pitch-1-18-finished-bore-34-teeth>: HTTP status code is not handled or not allowed 2026-01-24 23:47:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sealed-lead-acid---agm---vrla-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 23:47:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 23:47:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 23:47:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dsr-psj4424-jump-starter-replacement-battery-75ah-12v-it>: HTTP status code is not handled or not allowed 2026-01-24 23:47:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stone-skate-inspired-work-shoes-composite-toe-size-7m-black-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:47:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:47:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 23:47:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stone-skate-inspired-work-shoes-composite-toe-size-13m-blue-navy>: HTTP status code is not handled or not allowed 2026-01-24 23:47:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:47:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nanoflex-tr-athletic-work-shoes-composite-toe-size-8m-navy-light-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:47:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-24 23:47:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 23:47:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galion-150p-industrial-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 23:47:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caterpillar-d5-industrial-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 23:47:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50bs22hx1316-58-pitch-1-316-finished-bore-22-teeth>: HTTP status code is not handled or not allowed 2026-01-24 23:47:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nanoflex-tr-athletic-work-shoes-composite-toe-size-9m-black>: HTTP status code is not handled or not allowed 2026-01-24 23:47:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-08bs22hx38-metric-12-pitch-38mm-finished-bore-22-teeth>: HTTP status code is not handled or not allowed 2026-01-24 23:48:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nano-x1-adventure-athletic-work-shoes-composite-toe-size-11-5m-black>: HTTP status code is not handled or not allowed 2026-01-24 23:48:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80bs18hx1316-1-pitch-1-316-finished-bore-18-teeth>: HTTP status code is not handled or not allowed 2026-01-24 23:48:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35b14hx12mpb-38-pitch-12-plain-bore-14-teeth>: HTTP status code is not handled or not allowed 2026-01-24 23:48:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 38104 pages (at 93 pages/min), scraped 17726 items (at 65 items/min) 2026-01-24 23:48:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80bs23x1-1-pitch-1-finished-bore-23-teeth>: HTTP status code is not handled or not allowed 2026-01-24 23:48:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-20bs13hx30-metric-1-14-pitch-30mm-finished-bore-13-teeth>: HTTP status code is not handled or not allowed 2026-01-24 23:48:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nano-x1-adventure-athletic-work-shoes-composite-toe-size-10-5m-silver-gray-clay-black>: HTTP status code is not handled or not allowed 2026-01-24 23:48:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:48:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/03029-food-container-airtight-wlid-pp-26-qt-13-size-colorclip-transparent>: HTTP status code is not handled or not allowed 2026-01-24 23:48:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:48:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:48:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/toro-56027-lawn-and-garden-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 23:48:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-softshell-jacket-black-20f-comfort-rating-s>: HTTP status code is not handled or not allowed 2026-01-24 23:48:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:48:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elan-el7-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 23:48:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 23:48:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/floatride-energy-3-adventure-men-s-athletic-work-shoes-composite-toe-size-10-5m-black>: HTTP status code is not handled or not allowed 2026-01-24 23:48:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/irontuff-jackoat-black-hivis-lime-50-comfort-rating-3xl-regular>: HTTP status code is not handled or not allowed 2026-01-24 23:48:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-tapered-roller-bearing-e30203j-17mm-bore-40mm-od>: HTTP status code is not handled or not allowed 2026-01-24 23:48:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 23:48:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:48:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evolve-skate-inspired-work-shoes-composite-toe-size-12m-navy>: HTTP status code is not handled or not allowed 2026-01-24 23:48:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:48:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 23:48:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 23:48:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-u-handle-441l-083-dia-black-m6-14-socket-cap-screw>: HTTP status code is not handled or not allowed 2026-01-24 23:48:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:48:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-ddc-l-gy-drawer-wlock-16w-x-14d-x-11h-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:48:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:48:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 23:48:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-24 23:48:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/cable_management/cable_protectors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:48:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/comfortie-casual-work-shoes-steel-toe-size-7w-black-white>: HTTP status code is not handled or not allowed 2026-01-24 23:48:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/cable_management/cable_protectors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:48:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 23:48:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/can-am-outlander-570cc-atv-replacement-battery-2016-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 23:48:44 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 23:48:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:48:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:48:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cc-pump-motor-wwe5-36-184jm-tefc-rigid-c-3-ph-184jm-5-hp-3600-rpm>: HTTP status code is not handled or not allowed 2026-01-24 23:48:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/general_purpose_electric_motors/three_phase_motors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:48:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:48:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/cable_management/cable_protectors>: HTTP status code is not handled or not allowed 2026-01-24 23:48:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/cable_management/cable_protectors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:48:51 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/cable_management/cable_protectors landed on page that is not a product page. 2026-01-24 23:48:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tecc-electric-scooter-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 23:48:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-24 23:48:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-24 23:48:56 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/general_purpose_electric_motors/three_phase_motors landed on page that is not a product page. 2026-01-24 23:48:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-24 23:48:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/cable_management/cable_protectors landed on page that is not a product page. 2026-01-24 23:48:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arctic-cat-cf5-efi-500cc-snowmobile-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 23:49:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-instrument-table-with-lower-shelf-33x18x34-987206>: HTTP status code is not handled or not allowed 2026-01-24 23:49:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-foot-guards-rubber-strap-wtension-buckle-aluminum-alloy-4-12w-1-pair>: HTTP status code is not handled or not allowed 2026-01-24 23:49:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medical-data-z-turbo-medical-replacement-battery-9ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 23:49:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 38190 pages (at 86 pages/min), scraped 17776 items (at 50 items/min) 2026-01-24 23:49:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rj11-modular-coupler-straight-through-inline-coupler-f-f>: HTTP status code is not handled or not allowed 2026-01-24 23:49:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usb-30-superspeed-sdxc-memory-card-media-reader-writer-5gbps>: HTTP status code is not handled or not allowed 2026-01-24 23:49:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/greenline-656-hand-truck-w-continuous-handle-semi-pneumatic-wheels-600-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-24 23:49:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/fans_blowers/exhaust_fans/panel_exhaust_fans already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:49:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/products-350wsb-unassembled-direct-drive-cement-mixer>: HTTP status code is not handled or not allowed 2026-01-24 23:49:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ready-built-ii153-3-tier-9-doors-locker-36w-x-18d-x-84h-black-assembled>: HTTP status code is not handled or not allowed 2026-01-24 23:49:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-level-drum-alarm-polypropylene-threaded-plug-5-working-depth-w-feed-pipe-1>: HTTP status code is not handled or not allowed 2026-01-24 23:49:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/fans_blowers/exhaust_fans/panel_exhaust_fans landed on page that is not a product page. 2026-01-24 23:49:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-tech-open-shelving-8-shelves-starter-unit-800-lb-cap-36w-x-24d-x-87h-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:49:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bb-bpl20-12-sealed-lead-acid-replacement-battery-18ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 23:49:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/calm90-opf-output-flange-for-calm-series-90mm-aluminum-worm-gear-reducer>: HTTP status code is not handled or not allowed 2026-01-24 23:49:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-transformer-tray-09603-48-x-48-56-gallon-capacity-black>: HTTP status code is not handled or not allowed 2026-01-24 23:49:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mighty-products-mn-tm-0650-6-x-50-premium-tennis-court-wind-screen-mesh>: HTTP status code is not handled or not allowed 2026-01-24 23:49:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e-track-end-cap-heavy-duty-plastic-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-24 23:49:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/boardwalk-bellmans-luggage-cart-w-black-carpet-8-wheels-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 23:49:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spark-resistant-manual-chain-hoist-5-ton-capacity-30-lift>: HTTP status code is not handled or not allowed 2026-01-24 23:49:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/centrifuge-tube-rack-set-sterile-50ml-capacity-clear-pack-of-500>: HTTP status code is not handled or not allowed 2026-01-24 23:49:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/plumbing_fittings/pipe_clamps_fittings/black_malleable_pipe_fittings already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:49:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/plumbing_fittings/pipe_clamps_fittings/black_malleable_pipe_fittings already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:49:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-in-close-black-steel-pipe-nipple-150-psi-lead-free>: HTTP status code is not handled or not allowed 2026-01-24 23:49:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/plumbing_fittings/pipe_clamps_fittings/galvanized_malleable_pipe_fittings already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:49:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/bay_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:49:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/altronix-smp5pmctxpd8cb-alarm-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 23:50:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/plumbing/plumbing_fittings/pipe_clamps_fittings/black_malleable_pipe_fittings>: HTTP status code is not handled or not allowed 2026-01-24 23:50:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/plumbing/plumbing_fittings/pipe_clamps_fittings/black_malleable_pipe_fittings landed on page that is not a product page. 2026-01-24 23:50:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/plumbing/plumbing_fittings/pipe_clamps_fittings/galvanized_malleable_pipe_fittings>: HTTP status code is not handled or not allowed 2026-01-24 23:50:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/bay_lights landed on page that is not a product page. 2026-01-24 23:50:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glacier-basic-acoustical-panels-shadowline-edge-24l-x-24w-white>: HTTP status code is not handled or not allowed 2026-01-24 23:50:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 38286 pages (at 96 pages/min), scraped 17849 items (at 73 items/min) 2026-01-24 23:50:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/laboratory-drench-shower-ceiling-mount-stainless-steel-pipe-20-gpm>: HTTP status code is not handled or not allowed 2026-01-24 23:50:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/622052rs-deep-groove-ball-bearing-wide-width-double-sealed-25mm-bore-52mm-od>: HTTP status code is not handled or not allowed 2026-01-24 23:50:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ortho-kinetics-lark-wheelchair-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 23:50:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quickie-p500-22nf-agm-wheelchair-replacement-battery-55ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 23:50:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-electric-vibrator-mve-30906-1200rpm-3-phase-60hz-230460v-6pole>: HTTP status code is not handled or not allowed 2026-01-24 23:50:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60162rsnr-deep-groove-ball-bearing-double-sealed-snap-ring-80mm-bore-125mm-od>: HTTP status code is not handled or not allowed 2026-01-24 23:50:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shoprider-te889dx4-4-wheelchair-replacement-battery-55ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 23:50:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/material-handling/ladders_personnel_lifts/step_ladders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:50:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelf-truck-54l-x-21w-x-60h-polyurethane-brake-casters>: HTTP status code is not handled or not allowed 2026-01-24 23:50:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:50:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ticonderoga-woodcase-hb-2-pencil-with-latexfree-eraser-96pack>: HTTP status code is not handled or not allowed 2026-01-24 23:50:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelf-truck-60l-x-18w-x-60h-polyurethane-brake-casters>: HTTP status code is not handled or not allowed 2026-01-24 23:50:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/westco-20l-lawn-and-garden-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 23:50:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/material-handling/ladders_personnel_lifts/step_ladders landed on page that is not a product page. 2026-01-24 23:50:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:50:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:50:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelf-truck-42l-x-14w-x-80h-polyurethane-brake-casters>: HTTP status code is not handled or not allowed 2026-01-24 23:50:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-24 23:50:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:50:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/columbia-mtd-760-lawn-and-garden-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 23:50:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/highlighters_markers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:50:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:50:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/core-medical-lab-cart-51351-6-drawers-white>: HTTP status code is not handled or not allowed 2026-01-24 23:50:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/pens already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:50:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers>: HTTP status code is not handled or not allowed 2026-01-24 23:50:49 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-24 23:50:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/flashlights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:50:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/accent-pen-style-liquid-highlighter-chisel-tip-fluorescent-green-ink>: HTTP status code is not handled or not allowed 2026-01-24 23:50:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-24 23:50:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/highlighters_markers>: HTTP status code is not handled or not allowed 2026-01-24 23:50:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/grinders_buffers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:50:56 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/pens landed on page that is not a product page. 2026-01-24 23:50:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/power_tools/power_grinder/die_grinders/corded_die_grinders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:50:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bl1850-2-18v-lxt-lithium-ion-battery-50-ah-2pk>: HTTP status code is not handled or not allowed 2026-01-24 23:50:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-24 23:50:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/flashlights landed on page that is not a product page. 2026-01-24 23:51:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-powered-fda-compliant-pump-act-fda-air>: HTTP status code is not handled or not allowed 2026-01-24 23:51:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/technologies-motorized-stock-cart-mcs-1772-243615-54-1500-lb-cap-36x24x54>: HTTP status code is not handled or not allowed 2026-01-24 23:51:04 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/grinders_buffers landed on page that is not a product page. 2026-01-24 23:51:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-lsci-emergency-light-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 23:51:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/vehicle_flood_utility_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:51:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sd-t60-air-sample-tube-60>: HTTP status code is not handled or not allowed 2026-01-24 23:51:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 38373 pages (at 87 pages/min), scraped 17906 items (at 57 items/min) 2026-01-24 23:51:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/teledyne-2sc6s8-emergency-light-replacement-battery-10ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 23:51:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/power_tools/power_grinder/die_grinders/corded_die_grinders landed on page that is not a product page. 2026-01-24 23:51:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/optional-fork-pockets-ost-fp-modern-equipment-meco-open-mesh-steel-security-trucks>: HTTP status code is not handled or not allowed 2026-01-24 23:51:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/vehicle_flood_utility_lights landed on page that is not a product page. 2026-01-24 23:51:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergi-lite-m3-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 23:51:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/full_height_storage_cabinets/stationary_storage_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:51:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/utility-reflector-light-with-porcelain-socket-sl302b3-silver-3pk>: HTTP status code is not handled or not allowed 2026-01-24 23:51:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clip-finished-end-panel-12-inch-d-x-87-inch-h-jet-black>: HTTP status code is not handled or not allowed 2026-01-24 23:51:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hf-acid-eater-safety-spill-kit-5-gallons-2901-005>: HTTP status code is not handled or not allowed 2026-01-24 23:51:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/cabinets/full_height_storage_cabinets/stationary_storage_cabinets landed on page that is not a product page. 2026-01-24 23:51:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/maintenance-pro153-4-drawer-chest-29w-x-20d-x-16-12h-blue>: HTTP status code is not handled or not allowed 2026-01-24 23:51:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/umbrellas_awnings_shades/outdoor_umbrellas_bases/umbrellas already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:51:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wave-lever-keyed-entry-polished-bronze>: HTTP status code is not handled or not allowed 2026-01-24 23:51:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/athens-square-base-w-wheels-for-libra-cast-aluminum-108-lb-weight-artic-white>: HTTP status code is not handled or not allowed 2026-01-24 23:51:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nerm003l-l-fg-10-ner-food-grade-hoist-with-motorized-trolley-1-4-ton-capacity>: HTTP status code is not handled or not allowed 2026-01-24 23:51:45 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/outdoor_furniture/umbrellas_awnings_shades/outdoor_umbrellas_bases/umbrellas landed on page that is not a product page. 2026-01-24 23:51:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esd-3-4-length-jacket-snap-cuff-teal-small>: HTTP status code is not handled or not allowed 2026-01-24 23:51:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3087-double-tnut-112l>: HTTP status code is not handled or not allowed 2026-01-24 23:51:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3793-double-end-fastener>: HTTP status code is not handled or not allowed 2026-01-24 23:51:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dsc-rb712-12v-72ah-alarm-battery>: HTTP status code is not handled or not allowed 2026-01-24 23:51:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cannondale-cannibal-440cc-atv-replacement-battery-2002-2003-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 23:51:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/parking_traffic_signs/parking_traffic_signs_1 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:52:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dhl-pickupdhl-no-pickup-sign-double-sided-plastic-10-x-14-yellowred>: HTTP status code is not handled or not allowed 2026-01-24 23:52:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slow-traffic-sign-egp-reflective-aluminum-24-x-24-blackyellow>: HTTP status code is not handled or not allowed 2026-01-24 23:52:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nerp030c-20-ner-electric-chain-hoist-push-trolley-3-ton-20-lift-17-ftmin-208v>: HTTP status code is not handled or not allowed 2026-01-24 23:52:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:52:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/parking_traffic_signs/parking_traffic_signs_1 landed on page that is not a product page. 2026-01-24 23:52:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ner005ld-15-ner-dual-speed-electric-chain-hoist-12-ton-15-lift-1525-ftmin-230v>: HTTP status code is not handled or not allowed 2026-01-24 23:52:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 38466 pages (at 93 pages/min), scraped 17975 items (at 69 items/min) 2026-01-24 23:52:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chamberlain-wd962kev-garage-door-replacement-battery-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 23:52:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cagiva-raptor-650cc-motorcycle-replacement-battery-2001-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 23:52:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-24 23:52:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ner030cd-10-ner-dual-speed-electric-chain-hoist-3-ton-10-lift-173-ftmin-230v>: HTTP status code is not handled or not allowed 2026-01-24 23:52:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-gtr-1500cc-personal-watercraft-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 23:52:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dolphin-scooter-ride-on-toys-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 23:52:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:52:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pailpro-5-gallon-can-tipper-75-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-24 23:52:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/peugeot-sv-100cc-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 23:52:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kb-unit-heater-with-bracket-thermostat-20-kw-480v-1-or-3-phase>: HTTP status code is not handled or not allowed 2026-01-24 23:52:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ms-h-s-circular-analog-hotplate-stirrer-81131102-110v-60hz-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 23:52:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs>: HTTP status code is not handled or not allowed 2026-01-24 23:52:31 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-24 23:52:32 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:18 2026-01-24 23:52:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ceramic-magnetic-hook-mhhh14-14-lbs-pull-white-enamel-paint>: HTTP status code is not handled or not allowed 2026-01-24 23:52:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anti-scratch-frameless-safety-glasses-yellow-lens-amber-temple-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-24 23:52:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quickie-p190-mobility-scooter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 23:52:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foodpro-purofort-multi-grip-safety-boots-cleated-outsole-steel-toe-size-12-15h-blue>: HTTP status code is not handled or not allowed 2026-01-24 23:52:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96-x-48-slatwall-panel-maple-veneer-without-inserts-pkg-qty-4>: HTTP status code is not handled or not allowed 2026-01-24 23:52:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/600a-food-wrapping-machine-115v-26l-x-2212w-x-9h>: HTTP status code is not handled or not allowed 2026-01-24 23:52:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/80b-beater-paddle-for-m802-l800-l800d-80-qt-mixer>: HTTP status code is not handled or not allowed 2026-01-24 23:52:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-side-printable-business-card-2-inch-x-3-1-2-inch-white-1000-cards-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:52:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stackable-shelf-bin-8-1-4w-x-12d-x-7h-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:53:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelf-unit-36w-x-14d-x-74h-with-28-ssb443-stackable-shelf-bins-ivory>: HTTP status code is not handled or not allowed 2026-01-24 23:53:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/airborne-life-support-systems-infant-transport-incubator-20h-medical-battery-26ah-12v>: HTTP status code is not handled or not allowed 2026-01-24 23:53:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rigid-conduit-locknut-carbon-steel-1-14-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 23:53:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/bins_totes_containers/protective_equipment_cases/waterproof_equipment_cases already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:53:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:53:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merits-p320-junior-power-chair-wheelchair-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 23:53:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 38563 pages (at 97 pages/min), scraped 18048 items (at 73 items/min) 2026-01-24 23:53:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/904-0008-904-case-102l-x-79w-x-45h-blue>: HTTP status code is not handled or not allowed 2026-01-24 23:53:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pride-mobility-sidekick-tri-wheeler-wheelchair-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 23:53:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/bins_totes_containers/protective_equipment_cases/waterproof_equipment_cases>: HTTP status code is not handled or not allowed 2026-01-24 23:53:18 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats landed on page that is not a product page. 2026-01-24 23:53:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-ft-type-1aa-fiberglass-manhole-ladder-m7112-1>: HTTP status code is not handled or not allowed 2026-01-24 23:53:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/invacare-lynx-sx-3-atm-take-along-wheelchair-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 23:53:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insect-control-air-curtain-36-inch-w-door-120v-3-4hp-1-ph-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-24 23:53:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=dwa-dcs380p1-20v-max-lithium-ion-reciprocating-saw-kit-50-ah already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:53:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pride-mobility-go-go-sc54hd-elite-traveller-plus-wheelchair-battery-35ah-12v>: HTTP status code is not handled or not allowed 2026-01-24 23:53:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:53:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:53:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobcat-218es-zero-turn-radius-rider-lawn-and-garden-replacement-battery-18ah-12v>: HTTP status code is not handled or not allowed 2026-01-24 23:53:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triple-grip-anchor-kit-w-combo-pan-head-screw-12-size-101-pieces>: HTTP status code is not handled or not allowed 2026-01-24 23:53:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=dwa-dcs380p1-20v-max-lithium-ion-reciprocating-saw-kit-50-ah landed on page that is not a product page. 2026-01-24 23:53:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1504-premium-class-2-fall-protection-mesh-6-pockets-safety-vest-orange-3xl>: HTTP status code is not handled or not allowed 2026-01-24 23:53:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-24 23:53:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-24 23:53:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/phillips-and-square-flat-head-machine-screw-6-32-x-2-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 23:53:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/machine_mounts_pads_vibration_dampers/vibration_isolating_machine_levelers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:53:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slotted-flat-head-machine-screw-6-32-x-1-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 23:53:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/69000-series-hutch-with-doors-48-inchw-x-17-inchd-x-36-inchh-mahogany>: HTTP status code is not handled or not allowed 2026-01-24 23:53:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ibt-technologies-bt75-12uxl-emergency-light-replacement-battery-75ah-12v-it>: HTTP status code is not handled or not allowed 2026-01-24 23:53:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/machine_mounts_pads_vibration_dampers/vibration_isolating_machine_levelers landed on page that is not a product page. 2026-01-24 23:53:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microfiber-towel-12-x-12-blue-12-towels-pack-2512-b-dz>: HTTP status code is not handled or not allowed 2026-01-24 23:53:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48h-x-48w-slatwall-panelgray-laminate-with-aluminum-inserts-968343GI>: HTTP status code is not handled or not allowed 2026-01-24 23:53:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/wall_mount_belts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:53:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ion-audio-block-rocker-am-fm-ipa16-speaker-replacement-battery-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 23:54:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/staplers_staples/staplers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:54:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-head-cap-screw-grade-5-carbon-steel-12-13-x-1-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 23:54:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/wall_mount_belts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:54:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrf206101l56c-cast-iron-right-angle-worm-gear-reducer-101-ratio-56c-frame>: HTTP status code is not handled or not allowed 2026-01-24 23:54:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/punches_1 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:54:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrf133201r56c-cast-iron-right-angle-worm-gear-reducer-201-ratio-56c-frame>: HTTP status code is not handled or not allowed 2026-01-24 23:54:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 38654 pages (at 91 pages/min), scraped 18114 items (at 66 items/min) 2026-01-24 23:54:09 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/staplers_staples/staplers landed on page that is not a product page. 2026-01-24 23:54:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bombardier-traxter-500-4x4-auto-xt-500cc-atv-replacement-battery-2001-2005-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 23:54:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/wall_mount_belts landed on page that is not a product page. 2026-01-24 23:54:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/wall_mount_belts landed on page that is not a product page. 2026-01-24 23:54:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/account-book-record-ruled-8-5-8-x-14-1-8-black-cover-300-pages-pad>: HTTP status code is not handled or not allowed 2026-01-24 23:54:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/punches_1 landed on page that is not a product page. 2026-01-24 23:54:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-staples-55-sheet-capacity-3-8-leg-length-1000-box>: HTTP status code is not handled or not allowed 2026-01-24 23:54:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrf262501de56c-cast-iron-right-angle-worm-gear-reducer-501-ratio-56c-frame>: HTTP status code is not handled or not allowed 2026-01-24 23:54:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-head-lag-screw-carbon-steel-38-x-4-12-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 23:54:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/automatic-opaque-label-counter-for-up-to-6-w-x-13-diameter-3-to-3-core-rolls>: HTTP status code is not handled or not allowed 2026-01-24 23:54:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/packaging/labels_barcode_equipment/label_dispensers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:54:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/staplers_staples/staplers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:54:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lock-washer-18-8-stainless-steel-58-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-24 23:54:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-purpose-container-placard-label-holder-astp5-with-place-label-here-6-x-8-white>: HTTP status code is not handled or not allowed 2026-01-24 23:54:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machine-screw-star-pin-drive-button-head-10-32-x-12-size-100-pk>: HTTP status code is not handled or not allowed 2026-01-24 23:54:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/packaging/labels_barcode_equipment/label_dispensers landed on page that is not a product page. 2026-01-24 23:54:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/staplers_staples/staplers landed on page that is not a product page. 2026-01-24 23:54:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/victory-magnum-x-1-stealth-edition-1731cc-motorcycle-replacement-battery-2016-12v>: HTTP status code is not handled or not allowed 2026-01-24 23:54:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/strain-relief-connector-steel-1-dia-purple-grommet>: HTTP status code is not handled or not allowed 2026-01-24 23:54:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/easy-off-fume-free-oven-cleaner-lemon-145oz-aerosol-12-case-rac87977ct>: HTTP status code is not handled or not allowed 2026-01-24 23:54:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kawasaki-jt1500-stx-1500cc-personal-watercraft-replacement-battery-2004-2013-12v>: HTTP status code is not handled or not allowed 2026-01-24 23:54:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-ucf206-20-flange-mount-bearing-4-bolt-flange-set-screw-locking-bore-31-75-mm>: HTTP status code is not handled or not allowed 2026-01-24 23:54:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rascal-265le-candyapple-mobility-scooter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 23:54:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-ucf212-38-flange-mount-bearing-4-bolt-flange-set-screw-locking-bore-60-325-mm>: HTTP status code is not handled or not allowed 2026-01-24 23:54:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-row-spherical-roller-bearing-22230exw33c3-150mm-bore-270mm-od>: HTTP status code is not handled or not allowed 2026-01-24 23:55:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-row-spherical-roller-bearing-23032ew33kc3-160mm-bore-240mm-od-tapered-bore>: HTTP status code is not handled or not allowed 2026-01-24 23:55:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-row-cylindrical-roller-bearing-nu310myc3-50mm-bore-110mm-od>: HTTP status code is not handled or not allowed 2026-01-24 23:55:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flammable-cabinet-manual-close-double-door-22-gallon-capacity-35w-x-22d-x-35h>: HTTP status code is not handled or not allowed 2026-01-24 23:55:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 38752 pages (at 98 pages/min), scraped 18186 items (at 72 items/min) 2026-01-24 23:55:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8293-class-2-hooded-sweatshirt-wblack-front-limeblack-s>: HTTP status code is not handled or not allowed 2026-01-24 23:55:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/klip-it-closed-starter-csk85-1236-6-36-inch-w-x-12-inch-d-x-85-inch-h-6-shelves-20-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:55:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-roof-for-global-industrial-utility-vehicle-615162>: HTTP status code is not handled or not allowed 2026-01-24 23:55:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-4-in-x-4-1-2-in-lead-free-seamless-red-brass-pipe-nipple-140-psi-sch-40-import>: HTTP status code is not handled or not allowed 2026-01-24 23:55:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-chrome-wire-shelving-unit-add-on-60w-x-12d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 23:55:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-bolt-open-starter-ost75-1248-5x-48-inch-w-x-12-inch-d-x-75-inch-h-5-shelves-18-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:55:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss60022rs-deep-groove-ball-bearing-stainless-steel-double-sealed-od-32mm-bore-15mmmetric>: HTTP status code is not handled or not allowed 2026-01-24 23:55:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pre-inked-message-stamp-received-1-5-8-x-1-2-red>: HTTP status code is not handled or not allowed 2026-01-24 23:55:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/printable-self-adhesive-tabs-1-3-4-width-assorted-80-tabs-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:55:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/120lb-cable-ties-natural-6100-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:55:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-stainless-steel-wire-shelving-unit-add-on-24w-x-24d-x-34h>: HTTP status code is not handled or not allowed 2026-01-24 23:55:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/forms already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:55:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yamaha-vmax-600-deluxe-593cc-snowmobile-replacement-battery-2000-2001-12v-i>: HTTP status code is not handled or not allowed 2026-01-24 23:55:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-mailers-6-x-6-x-2-200lb-test-ect-32-kraft-50-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:55:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mk-5007k-7-14-circular-saw>: HTTP status code is not handled or not allowed 2026-01-24 23:55:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-motor-for-30-evaporative-cooler-model-600543-292221>: HTTP status code is not handled or not allowed 2026-01-24 23:56:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-8713-containment-berm-rapid-rise-12-x-50-copolymer-2000>: HTTP status code is not handled or not allowed 2026-01-24 23:56:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/paper_stationery/forms landed on page that is not a product page. 2026-01-24 23:56:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-gsx-ltd-600-ho-sdi-594cc-snowmobile-replacement-battery-2007-2008-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 23:56:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-plastic-locker-11-921212720-double-tier-12-x-12-x-36-1-wide-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:56:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 38848 pages (at 96 pages/min), scraped 18263 items (at 77 items/min) 2026-01-24 23:56:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-ski-doo-gsx-ltd-800-ho-ptek-800cc-snowmobile-replacement-battery-2007-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 23:56:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/queuepro-300-retractable-belt-barrier-40-silver-post-16-blue-belt-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-24 23:56:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-plus-phenolic-locker-11-811518601-single-tier-15-x-18-x-60-silver-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:56:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-plus-phenolic-locker-11-821218721-double-tier-12x18x36-graphite-grafix>: HTTP status code is not handled or not allowed 2026-01-24 23:56:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arctic-cat-cougar-mountain-cat-550cc-snowmobile-replacement-battery-1995-1998-12v>: HTTP status code is not handled or not allowed 2026-01-24 23:56:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/766-striped-vinyl-tape-2-x-36-yds-5-mil-black-yellow-2-pack>: HTTP status code is not handled or not allowed 2026-01-24 23:56:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fao-rubber-tourniquet-1-x-18-1-box>: HTTP status code is not handled or not allowed 2026-01-24 23:56:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stone-for-lil-luigi-and-le-peppe-1-2mm>: HTTP status code is not handled or not allowed 2026-01-24 23:56:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-335ss-butterfly-valve-with-epdm-seals--10-position-handle-6-wafer>: HTTP status code is not handled or not allowed 2026-01-24 23:56:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pace-tech-almax-systems-ii-medical-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 23:56:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-367ss-butterfly-valve-with-epdm-seals--10-position-handle-12-lug>: HTTP status code is not handled or not allowed 2026-01-24 23:56:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_alarms_signals/forklift_alert_sensors_systems already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:56:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u-channel-sign-post-10l-2-lbsft-galvanized-post-with-holes-top-to-bottom>: HTTP status code is not handled or not allowed 2026-01-24 23:56:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/see-a-360-degree-steel-full-dome-mirror-indoor-24-diameter-pvs24-360>: HTTP status code is not handled or not allowed 2026-01-24 23:56:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xb-series-explosion-proof-convection-double-unit-heater-cw-tstat-2000w-480v-1-phase>: HTTP status code is not handled or not allowed 2026-01-24 23:56:50 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_alarms_signals/forklift_alert_sensors_systems landed on page that is not a product page. 2026-01-24 23:56:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-gal-sat-alum-ash-trash-recept-w-sand-cover-c1531sa>: HTTP status code is not handled or not allowed 2026-01-24 23:56:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mercury-topaz-84864-sealed-lead-acid-replacement-battery-26ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 23:56:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-cal-coin-grip-anti-slip-rubber-mat-2mm-x-4-x-25-rolled-rubber-black>: HTTP status code is not handled or not allowed 2026-01-24 23:57:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mm3-2d-2448flpb-48w-x-24d-mobile-workbench-center-shelf-non-slip-vinyl-top>: HTTP status code is not handled or not allowed 2026-01-24 23:57:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sunstone-power-spt12-5-sealed-lead-acid-replacement-battery-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 23:57:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ironguard-steel-faced-dock-bumper-22-6040-13-w-x-5-1-4-d-x-20-h>: HTTP status code is not handled or not allowed 2026-01-24 23:57:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 38946 pages (at 98 pages/min), scraped 18340 items (at 77 items/min) 2026-01-24 23:57:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/energy-drink-original-83-oz-24carton>: HTTP status code is not handled or not allowed 2026-01-24 23:57:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdpe-55-gal-conical-bottom-tank-w-cover-22dia-x-44h-3-16wall-white>: HTTP status code is not handled or not allowed 2026-01-24 23:57:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-nexelate-silver-epoxy-wire-shelving-unit-add-on-42w-x-24d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 23:57:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-nexelate-silver-epoxy-wire-shelving-unit-add-on-48w-x-30d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 23:57:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-wall-clock-15-34>: HTTP status code is not handled or not allowed 2026-01-24 23:57:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ge-caddx-networx-nx-8-12v-7ah-alarm-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-24 23:57:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oily-waste-can-10-gallon-black-935-flbk>: HTTP status code is not handled or not allowed 2026-01-24 23:57:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fallte-7073lx-foreman-3-d-full-body-harness-3-d-rings-size-l-xl>: HTTP status code is not handled or not allowed 2026-01-24 23:57:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-nexelate-silver-epoxy-wire-shelving-unit-starter-36w-x-14d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 23:57:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-nexelate-silver-epoxy-wire-shelving-unit-starter-72w-x-36d-x-74h>: HTTP status code is not handled or not allowed 2026-01-24 23:57:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/small-fixed-hub-fan-blade-12-inch-dia-25-pitch-ccw-5-16-inch-bore-2-inch-blade-depth-5-blade>: HTTP status code is not handled or not allowed 2026-01-24 23:57:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zipr-mobility-4-traveler-wheelchair-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-24 23:57:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-inlet-blower-wheel-6-25-64-inch-dia-ccw-2050-rpm-1-2-inch-bore-7-inch-w-galvanized>: HTTP status code is not handled or not allowed 2026-01-24 23:57:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simplicity-16gth-lawn-and-garden-replacement-battery-18ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 23:57:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mtd-2248f-walk-behind-lawn-and-garden-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 23:57:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-stainless-steel-solid-shelving-unit-starter-48w-x-24d-x-54h>: HTTP status code is not handled or not allowed 2026-01-24 23:57:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-inlet-blower-wheel-5-3-4-inch-dia-cw-3450-rpm-1-2-inch-bore-4-inch-w-galvanized>: HTTP status code is not handled or not allowed 2026-01-24 23:57:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/startsystem-aed-case-and-oxygen-system-life-02-101>: HTTP status code is not handled or not allowed 2026-01-24 23:57:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/hand_tools_1/screwdrivers/multi_bit_screwdrivers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:57:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pk-2061-solar-mc3-mc4-crimping-tool-kit>: HTTP status code is not handled or not allowed 2026-01-24 23:58:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smart-label-printer-620-228-labels-276-second>: HTTP status code is not handled or not allowed 2026-01-24 23:58:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/hand_tools_1/screwdrivers/multi_bit_screwdrivers>: HTTP status code is not handled or not allowed 2026-01-24 23:58:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 39043 pages (at 97 pages/min), scraped 18416 items (at 76 items/min) 2026-01-24 23:58:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sewn-vinyl-business-card-file-binder>: HTTP status code is not handled or not allowed 2026-01-24 23:58:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-flange-shelf-30-inch-l-x-36-inch-w-20-gauge-300-lb-capacity-dark-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:58:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simplex-20013072-olt-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 23:58:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/york-wide-light-r2e3-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-24 23:58:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/palmgren-9612301-drill-press-vise-3>: HTTP status code is not handled or not allowed 2026-01-24 23:58:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/exam_patient_room_supplies/disposable_glove_dispensers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:58:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/medical_chart_file_holders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:58:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wm-cascade-10-pocket-magazine-rack-light-oak>: HTTP status code is not handled or not allowed 2026-01-24 23:58:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/medical_chart_file_holders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:58:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-step-folding-step-stool-with-rubber-hand-grip-22-12l-x-18-12w-x-40-34h-black>: HTTP status code is not handled or not allowed 2026-01-24 23:58:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bh2760vc-brute-breaker-hammer-tool-only>: HTTP status code is not handled or not allowed 2026-01-24 23:58:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/exam_patient_room_supplies/disposable_glove_dispensers landed on page that is not a product page. 2026-01-24 23:58:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/imported-tpg-321-c-6-tin-coated-carbide-insert>: HTTP status code is not handled or not allowed 2026-01-24 23:58:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/medical_chart_file_holders landed on page that is not a product page. 2026-01-24 23:58:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speedcell-kit-with-medium-columns-for-96w-x-42d-bay-white>: HTTP status code is not handled or not allowed 2026-01-24 23:58:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/awsw-partition-88x7-p-clear>: HTTP status code is not handled or not allowed 2026-01-24 23:58:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/medical_chart_file_holders landed on page that is not a product page. 2026-01-24 23:58:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regenerative-blower-ab-100-3-phase-1-stage-067-hp>: HTTP status code is not handled or not allowed 2026-01-24 23:58:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/room_dividers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:58:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/five-shelf-extra-hd-boltless-shelving-add-on-36inw-x-12ind-x-60inh-laminate-deck-usa>: HTTP status code is not handled or not allowed 2026-01-24 23:58:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/awrd-partition-80x84-f-cloud-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:58:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/melamine-whiteboard-48-x-36>: HTTP status code is not handled or not allowed 2026-01-24 23:58:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/partitions_dividers/room_dividers landed on page that is not a product page. 2026-01-24 23:58:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qvtrk-3-108est-double-unit-track-assembly-kit-3-tracks-108>: HTTP status code is not handled or not allowed 2026-01-24 23:58:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/acoustical_partition_panels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:58:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m16-metric-eyebolt-shoulder-pattern-m16-x-200>: HTTP status code is not handled or not allowed 2026-01-24 23:58:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/915-case-w-padded-divider-15-3-8l-x-12-1-8w-x-6-13-16h-black>: HTTP status code is not handled or not allowed 2026-01-24 23:58:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-steel-mobile-storage-bin-msb12-2036-95-12-openings-36w-x-24d-x-49-1-4h>: HTTP status code is not handled or not allowed 2026-01-24 23:58:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/heaters/electric_heaters/electric_unit_heaters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:58:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/partitions_dividers/acoustical_partition_panels landed on page that is not a product page. 2026-01-24 23:58:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/ink_toner/laser_toner_cartridges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:58:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bsm-inc-ada-ss-sink-3-station-with-plumbing-skirt--electronic-faucet-72l-x-20w-x-6d-sink>: HTTP status code is not handled or not allowed 2026-01-24 23:59:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polaris-high-ball-1731cc-motorcycle-replacement-battery-2011-2014-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 23:59:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/heaters/electric_heaters/electric_unit_heaters>: HTTP status code is not handled or not allowed 2026-01-24 23:59:09 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/ink_toner/laser_toner_cartridges landed on page that is not a product page. 2026-01-24 23:59:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 39136 pages (at 93 pages/min), scraped 18480 items (at 64 items/min) 2026-01-24 23:59:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gssafe-5007-class-3-hi-viz-moisture-wicking-birdseye-short-sleeve-t-shirt-lime-3xl>: HTTP status code is not handled or not allowed 2026-01-24 23:59:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/explorer153-insulated-overboots-threaded-outsole-3xl-13h-black>: HTTP status code is not handled or not allowed 2026-01-24 23:59:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-cabinet-workbench-with-2-shelves-72w-x-36d-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:59:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/imported-5-c-expanding-collet-size-3>: HTTP status code is not handled or not allowed 2026-01-24 23:59:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motofino-mf50qt-2-50cc-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 23:59:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/voyager-nylon-all-season-waterproof-overboots-threaded-outsole-l-11h-black>: HTTP status code is not handled or not allowed 2026-01-24 23:59:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/die-stock-handle-for-2-1-2inch-diameter-round-die>: HTTP status code is not handled or not allowed 2026-01-24 23:59:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xtreme-power-xa-750-electric-scooter-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-24 23:59:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-nexelate-36w-x-18d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 23:59:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/air_conditioners_chillers/window_air_conditioners already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-24 23:59:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-ration-2400-kcal-30-box-dx2400f>: HTTP status code is not handled or not allowed 2026-01-24 23:59:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelate-48w-x-14d-x-69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 23:59:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/non-locking-latch-spira-m002nll-for-wall-mount-mailboxes>: HTTP status code is not handled or not allowed 2026-01-24 23:59:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelate-42w-x-18d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 23:59:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/air_conditioners_chillers/window_air_conditioners landed on page that is not a product page. 2026-01-24 23:59:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelate-48w-x-36d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 23:59:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u1449-ns-cantilever-rack-single-sided-upright-49w-x-14h>: HTTP status code is not handled or not allowed 2026-01-24 23:59:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-renegade-1200-x-1200cc-snowmobile-replacement-battery-2010-12v-b>: HTTP status code is not handled or not allowed 2026-01-24 23:59:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-nexelate-60w-x-36d-x-69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-24 23:59:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelate-60w-x-14d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-24 23:59:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/janitorial-cabinet-jan6618dh-02-welded-36w-x-18d-x-64h-medium-gray>: HTTP status code is not handled or not allowed 2026-01-24 23:59:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/cube_storage already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:00:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/SVT-9630-96-W-x-30-D-18-ga-T-430-Stainless-Steel-Workbench>: HTTP status code is not handled or not allowed 2026-01-25 00:00:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/computers_electronics/cables_connectors/cable_connectors_adapters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:00:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-black-epoxy-42w-x-18d-x-60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 00:00:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-black-epoxy-54w-x-21d-x-69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 00:00:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/cabinets/cube_storage landed on page that is not a product page. 2026-01-25 00:00:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 39231 pages (at 95 pages/min), scraped 18552 items (at 72 items/min) 2026-01-25 00:00:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vestil-pallet-truck-109316l-x-27w-4000-lb-capacity->: HTTP status code is not handled or not allowed 2026-01-25 00:00:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/computers_electronics/cables_connectors/cable_connectors_adapters landed on page that is not a product page. 2026-01-25 00:00:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wireworks-4-tier-3-cup-dispensers-w-1-lid-organizer-2-side-panels-c8504wfd>: HTTP status code is not handled or not allowed 2026-01-25 00:00:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-black-epoxy-36w-x-14d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 00:00:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-black-epoxy-48w-x-14d-x-69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 00:00:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/97000-wine-glass-1-liter-decanter-12-pack>: HTTP status code is not handled or not allowed 2026-01-25 00:00:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-phase-motor-em3770t-5-1770-hp-75-rpm-213t-frame-c-face-mount-tefc-575-volts>: HTTP status code is not handled or not allowed 2026-01-25 00:00:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-phase-motor-em3550t-5-3500-hp-15-rpm-143t-frame-foot-mount-tefc-575-volts>: HTTP status code is not handled or not allowed 2026-01-25 00:00:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_microscopes/stereo_microscopes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:00:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/coat_racks_umbrella_stands/umbrella_stands_holders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:00:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hardcap-aerolite-lightweight-baseball-style-bump-cap-hdpe-liner-and-adj-back-short-brim-navy>: HTTP status code is not handled or not allowed 2026-01-25 00:00:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evo-6161-ascend-full-brim-safety-helmet-hdpe-shell-6-pt-suspension-wheel-ratchet-adjustment-black>: HTTP status code is not handled or not allowed 2026-01-25 00:00:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nt22-7gw-window-entry-awning-7-3-8-w-x-2-9-16-h-x-2-d-gray-white>: HTTP status code is not handled or not allowed 2026-01-25 00:00:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_microscopes/stereo_microscopes landed on page that is not a product page. 2026-01-25 00:00:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/coat_racks_umbrella_stands/umbrella_stands_holders>: HTTP status code is not handled or not allowed 2026-01-25 00:00:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dynamic-dom-cap-style-dome-hard-hat-hdpe-shell-6-pt-suspension-rachet-adjustment-red>: HTTP status code is not handled or not allowed 2026-01-25 00:00:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washdown-motor-cesswdm3550t-3-ph-15-hp-3500-rpm-208-230-460-v-tenv-143tc-fr>: HTTP status code is not handled or not allowed 2026-01-25 00:00:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/two_way_radios_accessories/two_way_radios already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:00:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelon-60w-x-21d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 00:00:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tube-rack-for-benchmixer-xl-xlq-50-x-13mm>: HTTP status code is not handled or not allowed 2026-01-25 00:00:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-poly-green-30w-x-24d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 00:00:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/two_way_radios_accessories/two_way_radios landed on page that is not a product page. 2026-01-25 00:00:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hvac-motor-ehm3558t-5-3-ph-2-hp-575-v-1755-rpm-tefc-145t-frame>: HTTP status code is not handled or not allowed 2026-01-25 00:00:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hvac-motor-em2332t-g-3-ph-10-hp-230-460-v-1200-rpm-tefc-256t-frame>: HTTP status code is not handled or not allowed 2026-01-25 00:01:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-poly-green-42w-x-14d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 00:01:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-step-steel-roll-ladd-rear-exit-gate-24-30-plat-perf-rwsr108246-xr>: HTTP status code is not handled or not allowed 2026-01-25 00:01:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-step-steel-roll-ladd-rear-exit-gate-24-30-plat-serr-rwsr115242-xr>: HTTP status code is not handled or not allowed 2026-01-25 00:01:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 39326 pages (at 95 pages/min), scraped 18623 items (at 71 items/min) 2026-01-25 00:01:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-level-steel-shelf-truck-with-5-shelves-36-x-24-800-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 00:01:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-iec-motor-emm3710-3ph-208-230-460v-1770rpm-55-75-kw-hp-60hz-d132s>: HTTP status code is not handled or not allowed 2026-01-25 00:01:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultratech-gloves-latex-cut-a4-impact-1-abrasion-4-puncture-5-gray-blue-2xl>: HTTP status code is not handled or not allowed 2026-01-25 00:01:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-poly-green-60w-x-24d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 00:01:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/encoder-tachometer-mounting-kits-w-o-tach-417708-27-dpg-dpfv-tenv-frames-c280-c400>: HTTP status code is not handled or not allowed 2026-01-25 00:01:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelon-54w-x-14d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 00:01:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-acting-hydraulic-cylinder-55-ton-2-stroke>: HTTP status code is not handled or not allowed 2026-01-25 00:01:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelon-48w-x-14d-x-60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 00:01:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-ac-motor-c-face-kit-field-conversion-10-1304gld-tefc-10-324-6uc-tc-frame>: HTTP status code is not handled or not allowed 2026-01-25 00:01:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelon-42w-x-21d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 00:01:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/outdoor-slatted-steel-trash-can-with-ashtray-lid-access-door-36-gallon-black>: HTTP status code is not handled or not allowed 2026-01-25 00:01:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qswb2000-wall-bracket-33-12l-x-25-14w-x-18-18h>: HTTP status code is not handled or not allowed 2026-01-25 00:01:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/celltreat-scientific-products-250l-pipette-mla-dtipper-purple>: HTTP status code is not handled or not allowed 2026-01-25 00:01:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/celltreat-scientific-products-250l-pipette-mla-dtipper-pink>: HTTP status code is not handled or not allowed 2026-01-25 00:01:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/fall_protection/harnesses already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:01:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-x-27-winch-strap-with-41766-18-flat-hook>: HTTP status code is not handled or not allowed 2026-01-25 00:01:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14000-btu-portable-air-conditioner-heater-with-3m-silvershield-filter-arc-148mhp>: HTTP status code is not handled or not allowed 2026-01-25 00:01:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cl-focus-ii-disc-26-walk-behind-midsize-scrubber-disc-05391a>: HTTP status code is not handled or not allowed 2026-01-25 00:02:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/vacuums/backpack_vacuums already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:02:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/topoge-series-180-handhole-gasket-7-x-9-x-34-black-rubber-elliptical-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 00:02:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/fall_protection/harnesses landed on page that is not a product page. 2026-01-25 00:02:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/heaters/heater_accessories_components/heater_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:02:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cep-6450m-50-63-81-sow-power-cord>: HTTP status code is not handled or not allowed 2026-01-25 00:02:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/achim-sterling-self-adhesive-vinyl-floor-tile-12-x-12-gray-speckled-granite-20-pack>: HTTP status code is not handled or not allowed 2026-01-25 00:02:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 39421 pages (at 95 pages/min), scraped 18697 items (at 74 items/min) 2026-01-25 00:02:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/vacuums/backpack_vacuums landed on page that is not a product page. 2026-01-25 00:02:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9oz-30-sateen-cotton-jacket-navy-4xl>: HTTP status code is not handled or not allowed 2026-01-25 00:02:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durham-dividers-for-compartment-boxes-fits-box-5671700>: HTTP status code is not handled or not allowed 2026-01-25 00:02:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/euroclean-dust-magnet-starterkit>: HTTP status code is not handled or not allowed 2026-01-25 00:02:18 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/heaters/heater_accessories_components/heater_accessories landed on page that is not a product page. 2026-01-25 00:02:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/modular_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:02:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sa-mirella-conference-table-42-round-sanddune>: HTTP status code is not handled or not allowed 2026-01-25 00:02:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/storage_panels/slatwalls_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:02:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/food_beverages/coffee_beverages/coffee already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:02:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hot-cocoa-mix-regular-1-oz-50-box>: HTTP status code is not handled or not allowed 2026-01-25 00:02:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/modular_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:02:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/directional-control-valve-hv11bgood0-1-spool-4-way-spool-action>: HTTP status code is not handled or not allowed 2026-01-25 00:02:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:02:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:02:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/floor_mats/modular_mats>: HTTP status code is not handled or not allowed 2026-01-25 00:02:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:02:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/storage_panels/slatwalls_accessories landed on page that is not a product page. 2026-01-25 00:02:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/food_beverages/coffee_beverages/coffee landed on page that is not a product page. 2026-01-25 00:02:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:02:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/floor_mats/modular_mats>: HTTP status code is not handled or not allowed 2026-01-25 00:02:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:02:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-30-maple-square-edge-mobile-pedestal-workbench-black-318651BK>: HTTP status code is not handled or not allowed 2026-01-25 00:02:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 00:02:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:02:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1971900-horizontal-storage-shed>: HTTP status code is not handled or not allowed 2026-01-25 00:02:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:02:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts landed on page that is not a product page. 2026-01-25 00:02:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts landed on page that is not a product page. 2026-01-25 00:02:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/full-zip-hooded-sweater-indura-flame-resistant-11-oz-4xl-navy>: HTTP status code is not handled or not allowed 2026-01-25 00:02:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:02:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 00:02:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/desk_accessories/document_holders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:02:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats landed on page that is not a product page. 2026-01-25 00:02:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-38w-x-28l-navy>: HTTP status code is not handled or not allowed 2026-01-25 00:02:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats landed on page that is not a product page. 2026-01-25 00:02:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats landed on page that is not a product page. 2026-01-25 00:02:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/laptop_tablet_stands already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:02:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/measurement_layout_tools/protractors_squares already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:02:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/laptop_tablet_stands already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:02:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/desktop-curved-copy-holder>: HTTP status code is not handled or not allowed 2026-01-25 00:02:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:02:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts landed on page that is not a product page. 2026-01-25 00:02:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:02:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esd-modular-work-bench-adder-c2-spa-e3072-75ggb-c2-series-30d-x-72w-x-75h-grey-laminate>: HTTP status code is not handled or not allowed 2026-01-25 00:02:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:02:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:02:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/octo-glove-box-holder-305308>: HTTP status code is not handled or not allowed 2026-01-25 00:02:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:02:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/laptop_tablet_stands landed on page that is not a product page. 2026-01-25 00:02:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/measurement_layout_tools/protractors_squares landed on page that is not a product page. 2026-01-25 00:02:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/laptop_tablet_stands>: HTTP status code is not handled or not allowed 2026-01-25 00:03:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-42w-x-33l-navy>: HTTP status code is not handled or not allowed 2026-01-25 00:03:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/desk_accessories/document_holders landed on page that is not a product page. 2026-01-25 00:03:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 00:03:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:03:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 00:03:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 00:03:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 00:03:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-36w-x-34l-navy>: HTTP status code is not handled or not allowed 2026-01-25 00:03:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 39495 pages (at 74 pages/min), scraped 18734 items (at 37 items/min) 2026-01-25 00:03:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:03:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 00:03:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vented-plastic-shelving-54x24x86-nexelon-finish-331792N>: HTTP status code is not handled or not allowed 2026-01-25 00:03:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/special_purpose_carts_trucks/bin_tote_carts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:03:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 00:03:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/md-vertical-unit-heater-v42s01-42000-btu-950-cfm-115v>: HTTP status code is not handled or not allowed 2026-01-25 00:03:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:03:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/def-hand-lever-pump-with-discharge-hose>: HTTP status code is not handled or not allowed 2026-01-25 00:03:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ball-screw-support-bearing-40tab09up4-single-flush-ground-40mm-bore-90mm-od>: HTTP status code is not handled or not allowed 2026-01-25 00:03:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:03:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:03:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/carts-trucks/special_purpose_carts_trucks/bin_tote_carts>: HTTP status code is not handled or not allowed 2026-01-25 00:03:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:03:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:03:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-double-wall-cardboard-corrugated-box-36-x-36-x-12-275-lb-ect-48-5-pack>: HTTP status code is not handled or not allowed 2026-01-25 00:03:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 00:03:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 00:03:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:03:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cardboard-single-wall-corrugated-box-36-x-24-x-18-200lb-test-ect-32-kraft-10-pack>: HTTP status code is not handled or not allowed 2026-01-25 00:03:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:03:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 00:03:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-inch-drive-impact-socket-1-1-2-inch-6-point-2-15-64-inch-l>: HTTP status code is not handled or not allowed 2026-01-25 00:03:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/sockets_bits/impact_sockets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:03:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 00:03:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-34w-x-30l-tan>: HTTP status code is not handled or not allowed 2026-01-25 00:03:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 00:03:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 00:03:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:03:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 00:03:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manchester-stucco-locking-column-mailbox-slate-gray-w-decorative-fleur-de-lis-door-antique-copper>: HTTP status code is not handled or not allowed 2026-01-25 00:03:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:03:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:03:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 00:03:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/sockets_bits/impact_sockets landed on page that is not a product page. 2026-01-25 00:03:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-crayons-nontoxic-assorted-8-box>: HTTP status code is not handled or not allowed 2026-01-25 00:03:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:03:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/rulers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:03:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-40w-x-31l-black>: HTTP status code is not handled or not allowed 2026-01-25 00:03:44 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 00:03:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:03:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:03:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 00:03:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 00:03:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:03:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sparkling-crystals-glitter-16-oz-silver>: HTTP status code is not handled or not allowed 2026-01-25 00:03:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/educational_materials_books/instructional_materials/flash_cards_educational_games already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:03:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:03:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 00:03:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/rulers landed on page that is not a product page. 2026-01-25 00:03:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 00:03:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rde-01-collapsible-computer-desk-48-x-235-table-black>: HTTP status code is not handled or not allowed 2026-01-25 00:03:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economical-desktop-tape-dispenser-black-1-pack>: HTTP status code is not handled or not allowed 2026-01-25 00:03:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 00:03:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 00:03:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/educational_materials_books/instructional_materials/flash_cards_educational_games landed on page that is not a product page. 2026-01-25 00:03:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 00:04:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:04:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:04:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ctl-bandsaw-blade-11-foot-long-x-1-inch-wide-4-6-tpi-x-0035-thick>: HTTP status code is not handled or not allowed 2026-01-25 00:04:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/69000-series-rectangular-desk-shell-llr69902-48-w-x-30-d-x-29-1-2-mahogany>: HTTP status code is not handled or not allowed 2026-01-25 00:04:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:04:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/dollies/machine_skates_dollies already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:04:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 39568 pages (at 73 pages/min), scraped 18769 items (at 35 items/min) 2026-01-25 00:04:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:04:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-36w-x-34l-tan>: HTTP status code is not handled or not allowed 2026-01-25 00:04:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 00:04:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 00:04:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:04:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/83101-knife-guard-narrow-black-handle-6-1-8-l>: HTTP status code is not handled or not allowed 2026-01-25 00:04:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-38w-x-unhemmed-gray>: HTTP status code is not handled or not allowed 2026-01-25 00:04:18 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/dollies/machine_skates_dollies landed on page that is not a product page. 2026-01-25 00:04:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xleratoreco-174-hand-dryer-stainless-steel-110-120v-xl-sb-eco-110-120>: HTTP status code is not handled or not allowed 2026-01-25 00:04:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 00:04:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 00:04:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 00:04:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:04:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/linen-control-center-1-hanging-bar-no-shelf-cam-lock-silver-vein-w-cam-lock>: HTTP status code is not handled or not allowed 2026-01-25 00:04:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fmd-2-compression-mount-50-lbs-max-load-1-4-deflection>: HTTP status code is not handled or not allowed 2026-01-25 00:04:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:04:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-jeans-indura-denim-14-oz-28w-x-32l-flame-resistant-blue>: HTTP status code is not handled or not allowed 2026-01-25 00:04:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/velvet-6-black-rope-satin-ss-hinged-ends-1-12-diameter>: HTTP status code is not handled or not allowed 2026-01-25 00:04:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:04:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss12-supersorb-loose-sorbent-28-oz-shaker-12-per-pack>: HTTP status code is not handled or not allowed 2026-01-25 00:04:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:04:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:04:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 00:04:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-work-pants-reflective-material-flame-resistant-46w-x-32l-red>: HTTP status code is not handled or not allowed 2026-01-25 00:04:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 00:04:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:04:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:04:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-jeans-indura-denim-14-oz-56w-x-34l-flame-resistant-blue>: HTTP status code is not handled or not allowed 2026-01-25 00:04:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 00:04:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 00:04:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:04:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/indoor-outdoor-portable-evaporative-air-cooler-co48pm-100-pint>: HTTP status code is not handled or not allowed 2026-01-25 00:04:51 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 00:04:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-24w-x-14d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 00:04:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/computers_electronics/audio_video_equipment/audio_visual_mounts/monitor_mounts_stands already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:04:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/woodsman-solid-color-oil-siding-fence-wood-stain-white-gallon-591154>: HTTP status code is not handled or not allowed 2026-01-25 00:04:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 00:04:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rtl10439b-crutch-tips-fits-7-8-diameter-crutch-gray-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 00:04:56 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 00:04:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-12w-x-36d-x-75h-with-9-8h-plastic-shelf-bins-red>: HTTP status code is not handled or not allowed 2026-01-25 00:05:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manitowoc-ice-iy-0504a-indigo-series-ice-maker-cube-style-air-cooled-self-contained-condenser>: HTTP status code is not handled or not allowed 2026-01-25 00:05:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/computers_electronics/audio_video_equipment/audio_visual_mounts/monitor_mounts_stands>: HTTP status code is not handled or not allowed 2026-01-25 00:05:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drivable-power-stocker-li-18-blue-ps-12d>: HTTP status code is not handled or not allowed 2026-01-25 00:05:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 00:05:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nestable-plastic-shelf-bin-8-3-8w-x-11-5-8d-x-8h-blue>: HTTP status code is not handled or not allowed 2026-01-25 00:05:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-12a-1ph-start-stop-24-60v-4-2-5-7a>: HTTP status code is not handled or not allowed 2026-01-25 00:05:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-12a-1ph-remote-start-terminals-start-stop-100-250v-2-3-3-1a>: HTTP status code is not handled or not allowed 2026-01-25 00:05:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1500048-hook2loop-bungee-tether-10-pack>: HTTP status code is not handled or not allowed 2026-01-25 00:05:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-16a-3ph-remote-start-start-stop-250-500v-16-20a>: HTTP status code is not handled or not allowed 2026-01-25 00:05:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_roof_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:05:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 39651 pages (at 83 pages/min), scraped 18813 items (at 44 items/min) 2026-01-25 00:05:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-round-table-ladder-back-bar-ht-set-graphite-nebula-laminate-table-black-vinyl-chair>: HTTP status code is not handled or not allowed 2026-01-25 00:05:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deep-groove-ball-bearings-inch-1621-2rs-sealed-light-duty-05-inch-bore-1375-inch-od>: HTTP status code is not handled or not allowed 2026-01-25 00:05:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deep-groove-ball-bearings-metric-6307-2rs-2-rubber-seals-heavy-duty-35mm-bore-80mm-od>: HTTP status code is not handled or not allowed 2026-01-25 00:05:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:05:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-16a-1ph-remote-start-terminals-start-stop-100-250v-16-20a>: HTTP status code is not handled or not allowed 2026-01-25 00:05:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/211163-flat-socket-cap-screw-5-16-18-x-1-steel-black-oxide-ft-unc-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 00:05:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-26a-1ph-remote-start-reset-button-24-60v-13-16a>: HTTP status code is not handled or not allowed 2026-01-25 00:05:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_cabinets_drawers?token=null already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:05:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hms44alv-nfpa-classification-label-4-x-4-redyellowwhiteblue>: HTTP status code is not handled or not allowed 2026-01-25 00:05:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/nuts/lock_nuts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:05:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/392040-fender-washer-1-4-x-1-x-0-05-304-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 00:05:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hmk-hazard-identification-system-kit-redyellowwhiteblue>: HTTP status code is not handled or not allowed 2026-01-25 00:05:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/washers/split_lock_washers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:05:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/osha-sign-caution-personal-protective-equipment-required-beyond-this-point-10-x-14-ywblk-B974794>: HTTP status code is not handled or not allowed 2026-01-25 00:05:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_cabinets_drawers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:05:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/fasteners/nuts/lock_nuts>: HTTP status code is not handled or not allowed 2026-01-25 00:05:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precision-compression-spring-024-od-x-0032-x-1-music-wire-pkg-of-5>: HTTP status code is not handled or not allowed 2026-01-25 00:05:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-52a-1ph-remote-start-start-stop-24-60v-25-33a>: HTTP status code is not handled or not allowed 2026-01-25 00:05:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/washers/split_lock_washers landed on page that is not a product page. 2026-01-25 00:05:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-65a-3ph-separate-coil-voltage-start-stop-100-250v-50-60a>: HTTP status code is not handled or not allowed 2026-01-25 00:05:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-65a-3ph-remote-start-terminals-reset-button-100-250v-36-47a>: HTTP status code is not handled or not allowed 2026-01-25 00:05:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_cabinets_drawers>: HTTP status code is not handled or not allowed 2026-01-25 00:05:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c167p-osha-sign-caution-keep-electrical-panel-area-clear-10-x-7-yellowblack>: HTTP status code is not handled or not allowed 2026-01-25 00:05:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-80a-3ph-start-stop-24-60v-57-68a>: HTTP status code is not handled or not allowed 2026-01-25 00:05:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smoothmove-moving-storage-medium-box-18-1-8w-x-18-3-4d-x-16-5-8h-kraft>: HTTP status code is not handled or not allowed 2026-01-25 00:05:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tec-clean-a1-electronics-and-pcb-ultrasonic-cleaner-solution-10-8-ph-10-l>: HTTP status code is not handled or not allowed 2026-01-25 00:05:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/twist-tie-material-17-2000-red-2000-per-spool-red-12-pack>: HTTP status code is not handled or not allowed 2026-01-25 00:06:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paraclipse-insect-inn-ultra-two-system-120v-covers-4000-sq-ft-250602>: HTTP status code is not handled or not allowed 2026-01-25 00:06:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/locksets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:06:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/victorian-style-keyed-mortise-glass-knob-lock-set-classic-bronze-finish>: HTTP status code is not handled or not allowed 2026-01-25 00:06:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11-step-configurabl-forward-descent-roll-ladder-perforated-ukdec111246>: HTTP status code is not handled or not allowed 2026-01-25 00:06:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 39745 pages (at 94 pages/min), scraped 18883 items (at 70 items/min) 2026-01-25 00:06:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/fasteners/door_window_hardware/locksets>: HTTP status code is not handled or not allowed 2026-01-25 00:06:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keyless-rigid-coupling-c600e100-c600-series-steel-1d-x-2598d-1l-shaft>: HTTP status code is not handled or not allowed 2026-01-25 00:06:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-72-inch-w-x-18-inch-d-x-74-inch-h-2000-lbs-cap-4-open-grid-shelves-dunnage-stand>: HTTP status code is not handled or not allowed 2026-01-25 00:06:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-frame-folding-tank-alf-500-72l-x-72w-x-29h-500-gallon-cap-yellow>: HTTP status code is not handled or not allowed 2026-01-25 00:06:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-shaft-arbor-al4l-lefthand-type-d-214l-thread-fits-12-shaft>: HTTP status code is not handled or not allowed 2026-01-25 00:06:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-60-inch-w-x-21-inch-d-1000-lbs-cap-removable-solid-shelf-mat-polymer-gray>: HTTP status code is not handled or not allowed 2026-01-25 00:06:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-aire-wall-decontamination-pool-aadp-33-36l-x-12w-x-29h-60-gallon-cap-blue>: HTTP status code is not handled or not allowed 2026-01-25 00:06:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waste-mate-n267-submersible-sewage-pump-1-2-hp>: HTTP status code is not handled or not allowed 2026-01-25 00:06:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kni-cobra-water-pump-plier-w-polished-head-plastic-coated-handle-16-inch-l>: HTTP status code is not handled or not allowed 2026-01-25 00:06:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24x60x79-metromax-cart>: HTTP status code is not handled or not allowed 2026-01-25 00:06:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/florence-4b-horizontal-mailbox-140053a-28-h-14-mailboxes-front-loading-aluminum-usps>: HTTP status code is not handled or not allowed 2026-01-25 00:06:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/replacement_shelves_decks/wire_shelving_additional_shelves/chrome_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:06:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/krinkle-gauze-roll-non-sterile-4-12w-x-4-110-yards-100-pcs>: HTTP status code is not handled or not allowed 2026-01-25 00:06:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rb4016-9-m-sublite-athletic-work-shoe-steel-toe-men-s-size-9>: HTTP status code is not handled or not allowed 2026-01-25 00:06:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-head-stethoscope-50-pcs>: HTTP status code is not handled or not allowed 2026-01-25 00:06:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tm519j-traffic-sign-yield-here-arrow-to-pedestrian-24-x-24-white>: HTTP status code is not handled or not allowed 2026-01-25 00:06:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tm197g-traffic-sign-reserved-parking-van-accessible-18-x-12-white>: HTTP status code is not handled or not allowed 2026-01-25 00:06:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dynaride-s3-lite-wheelchair-flip-desk-arm-foot-rest-20w-seat>: HTTP status code is not handled or not allowed 2026-01-25 00:06:56 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/replacement_shelves_decks/wire_shelving_additional_shelves/chrome_shelves landed on page that is not a product page. 2026-01-25 00:06:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tm500k-traffic-sign-advance-turn-arrow-left-15-x-21-white>: HTTP status code is not handled or not allowed 2026-01-25 00:06:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forkliftable-order-picking-truck-48-x-30-3600-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-25 00:07:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dynarest-multi-zone-foam-pressure-mattress-80l-x-36w-x-6h>: HTTP status code is not handled or not allowed 2026-01-25 00:07:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/back-pain-relief-black-firm>: HTTP status code is not handled or not allowed 2026-01-25 00:07:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/amscope-binocular-stereo-boom-microscope-3-5x-90x-magnification-w-light-ring>: HTTP status code is not handled or not allowed 2026-01-25 00:07:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/welding_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:07:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 39844 pages (at 99 pages/min), scraped 18959 items (at 76 items/min) 2026-01-25 00:07:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tenactiv-cx-glove-blended-hppe-steel-micropore-nitrile-palm-a7-size-9>: HTTP status code is not handled or not allowed 2026-01-25 00:07:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/amscope-large-stand-ultimate-circuit-board-zoom-stereo-microscope-2x-225x-mag-w-1-3mp-camera>: HTTP status code is not handled or not allowed 2026-01-25 00:07:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anatomical-chart-skull-paper>: HTTP status code is not handled or not allowed 2026-01-25 00:07:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/floor_mats/welding_mats>: HTTP status code is not handled or not allowed 2026-01-25 00:07:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anatomical-model-knee-joint-with-removable-muscles-12-part>: HTTP status code is not handled or not allowed 2026-01-25 00:07:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/office-facility-signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:07:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ktools-tradesman-pro-modular-drill-pouch-with-belt-clip-black>: HTTP status code is not handled or not allowed 2026-01-25 00:07:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/cable_management/cable_protectors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:07:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eco-2-shelf-narrow-bookcase-black>: HTTP status code is not handled or not allowed 2026-01-25 00:07:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-10-tablet-stand-ktkts710-black>: HTTP status code is not handled or not allowed 2026-01-25 00:07:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/display-stand-mmmms80b-165w-x-1275d-x-588h-silver-black>: HTTP status code is not handled or not allowed 2026-01-25 00:07:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dvd-recordable-media-max639013-dvd-r-media-16x-speed-470-gb-capcity>: HTTP status code is not handled or not allowed 2026-01-25 00:07:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ct-usa-endurance-night-stand-laminate-top-21-inch-x-18-inch-stone-gray>: HTTP status code is not handled or not allowed 2026-01-25 00:07:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/data_storage_media already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:07:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/cable_management/cable_protectors>: HTTP status code is not handled or not allowed 2026-01-25 00:07:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/data_storage_media already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:07:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fl-casual-work-three-quarter-smooth-leather-cognac-7eee>: HTTP status code is not handled or not allowed 2026-01-25 00:07:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quikpik-48-guide-rail>: HTTP status code is not handled or not allowed 2026-01-25 00:07:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/office-facility-signs landed on page that is not a product page. 2026-01-25 00:07:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbk-athletic-work-shoe-dual-layer-stretch-mesh-upper-black-dark-grey-6-5m>: HTTP status code is not handled or not allowed 2026-01-25 00:07:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/data_storage_media landed on page that is not a product page. 2026-01-25 00:07:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/data_storage_media landed on page that is not a product page. 2026-01-25 00:07:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/floor-mounted-parallel-bars-height-width-adjustable-24l>: HTTP status code is not handled or not allowed 2026-01-25 00:07:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/loading-dock-safety-net-with-posts-4-feet-x-32-feet>: HTTP status code is not handled or not allowed 2026-01-25 00:07:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:07:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/horizontal-fixed-sign-frame-50-1130dh-cl-7-x-11-unslotted-chrome>: HTTP status code is not handled or not allowed 2026-01-25 00:07:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h18805-0000-poxygrid-labware-drain-stand-16-34l-x-6-12w-x-19h-14-supports-12-pins>: HTTP status code is not handled or not allowed 2026-01-25 00:07:49 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-25 00:07:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-heat-gun-1200-degrees-f>: HTTP status code is not handled or not allowed 2026-01-25 00:07:50 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:19 2026-01-25 00:07:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-25 00:07:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ironcat-30-irontex-flame-retardant-cotton-jacket-green-medium-all-cotton>: HTTP status code is not handled or not allowed 2026-01-25 00:07:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-chrome-3-tier-wire-shelving-add-on-unit-72w-x-21d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 00:08:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/gloves_sleeves/cut_resistant_gloves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:08:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-chrome-2-tier-wire-shelving-add-on-unit-60w-x-18d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 00:08:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/np-two-piece-biocooler-fits-box-18-inch-l-x-18-inch-w-x-18-inch-h-white>: HTTP status code is not handled or not allowed 2026-01-25 00:08:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/electric_motor_parts_accessories/electric_motor_power_supplies already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:08:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-gauge-staple-1-1-2-length-1-2-crown-304-stainless-steel-pkg-of-5000-made-in-usa>: HTTP status code is not handled or not allowed 2026-01-25 00:08:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/luvraflex-chair-black-poly-back-seat-chrome-frame>: HTTP status code is not handled or not allowed 2026-01-25 00:08:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 39935 pages (at 91 pages/min), scraped 19020 items (at 61 items/min) 2026-01-25 00:08:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/gloves_sleeves/cut_resistant_gloves landed on page that is not a product page. 2026-01-25 00:08:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbwp-chrome-front-load-laundry-cart-w-single-pole-rack>: HTTP status code is not handled or not allowed 2026-01-25 00:08:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/electric_motor_parts_accessories/electric_motor_power_supplies landed on page that is not a product page. 2026-01-25 00:08:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/bulk_container_trucks_tilt_trucks/basket_trucks/vinyl_basket_trucks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:08:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pistol-case-qf920gr-watertight-4-pistol-capacity-20-1-8-x16-1-8-x10-1-8-gray>: HTTP status code is not handled or not allowed 2026-01-25 00:08:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/bulk_container_trucks_tilt_trucks/basket_trucks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:08:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/bulk_container_trucks_tilt_trucks/basket_trucks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:08:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbwp-10-bushel-vinyl-bumper-truck-2-rigid-2-swivel-casters-forest-green>: HTTP status code is not handled or not allowed 2026-01-25 00:08:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tt-esd-rubber-matting-mt4548-48x50x0080-gray>: HTTP status code is not handled or not allowed 2026-01-25 00:08:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/bulk_container_trucks_tilt_trucks/basket_trucks/vinyl_basket_trucks landed on page that is not a product page. 2026-01-25 00:08:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tt-esd-rubber-matting-mt3060-30x60x0080-blue>: HTTP status code is not handled or not allowed 2026-01-25 00:08:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/bulk_container_trucks_tilt_trucks/basket_trucks landed on page that is not a product page. 2026-01-25 00:08:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/carts-trucks/bulk_container_trucks_tilt_trucks/basket_trucks>: HTTP status code is not handled or not allowed 2026-01-25 00:08:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-ft-outdoor-bench-with-back-vertical-steel-slat-green>: HTTP status code is not handled or not allowed 2026-01-25 00:08:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/td-1-28-gpf-bowl-toilet-w-rh-flush-17-3-16-inch-wx28-3-8-inch-dx30-1-8-inch-h-10-inch-rough-in-cotn>: HTTP status code is not handled or not allowed 2026-01-25 00:08:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-y-connector-global-industrial-153-gantry-cranes-1-pc>: HTTP status code is not handled or not allowed 2026-01-25 00:08:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chest-freezer-w-solid-top-21-cuft-capacity-white>: HTTP status code is not handled or not allowed 2026-01-25 00:08:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cha-anesthesia-cart-w-intermediate-height-autolock-blue>: HTTP status code is not handled or not allowed 2026-01-25 00:08:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/weathermaker-rooftop-gas-heat-electric-cool-unit-3-ton-1-ph>: HTTP status code is not handled or not allowed 2026-01-25 00:08:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-5-step-tilt-roll-ladder-expanded-metal-hl5nsx>: HTTP status code is not handled or not allowed 2026-01-25 00:08:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ajc-lithonia-elb-0604-6v-4-5ah-emergency-light-battery>: HTTP status code is not handled or not allowed 2026-01-25 00:08:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/152733-vertical-double-sided-stand-up-sign-holder-5-5-x-8-5-acrylic>: HTTP status code is not handled or not allowed 2026-01-25 00:08:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wern-proform-f3-climbing-harness-w-quick-connect-legs-m-l>: HTTP status code is not handled or not allowed 2026-01-25 00:08:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/13-wall-clock-black>: HTTP status code is not handled or not allowed 2026-01-25 00:09:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gm-patient-bib-13-1-2-inch-x-18-inch-2-ply-tissue-poly-mauve-500-case>: HTTP status code is not handled or not allowed 2026-01-25 00:09:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-drop-cord-switch-articulating-dock-and-door-fan-2-speed-silver>: HTTP status code is not handled or not allowed 2026-01-25 00:09:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22-204-3-1-2-twist-waste-drain-with-1-1-4-overflow-outlet-cap-included>: HTTP status code is not handled or not allowed 2026-01-25 00:09:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 40035 pages (at 100 pages/min), scraped 19095 items (at 75 items/min) 2026-01-25 00:09:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inplant-office-vinyl-cov-hdbd-10x16-2-wall-3-lgts-3-dplx-1-door-gray>: HTTP status code is not handled or not allowed 2026-01-25 00:09:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/igg-tri-point-internal-bore-gauge-ip54-0-315-0-40-inch-0-001mm-0-00005-inch-accuracy-0-00015-inch>: HTTP status code is not handled or not allowed 2026-01-25 00:09:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20v-max-4-tool-combo-kit-w-saws-21-1-2-l>: HTTP status code is not handled or not allowed 2026-01-25 00:09:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zp-a07259-zp-selig-good-stuff-aerosol-can-912501-20n18>: HTTP status code is not handled or not allowed 2026-01-25 00:09:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/infinity-3-tier-3-door-locker-15-in-w-x-15-in-d-x-24-in-h-gray-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 00:09:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-electric-office-partition-panel-with-partial-window-48-1-4w-x-77-1-2h-blue>: HTTP status code is not handled or not allowed 2026-01-25 00:09:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trayp548-spillage-tray-for-48w-ductless-fume-hood-48w-x-1h-x-22d>: HTTP status code is not handled or not allowed 2026-01-25 00:09:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-2-inch-light-duty-stowable-stanchion-41-inch-h-white-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 00:09:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polygon-spinbar-magnetic-stir-bar-371220025-teflon-ptfe-1l-x-1-4dia-white>: HTTP status code is not handled or not allowed 2026-01-25 00:09:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-series-rigid-plate-caster-406108939-mold-on-polyurethane-6dia-1200-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 00:09:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/color-coded-exercise-putty-xx-soft-tan-90cc>: HTTP status code is not handled or not allowed 2026-01-25 00:09:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/helene-ceiling-medallion-cm11he-1178od-x-358id-x-78d>: HTTP status code is not handled or not allowed 2026-01-25 00:09:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/physical_therapy/exercise_fitness_equipment/exercise_machines already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:09:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-wall-sconce-sco12x06x04tr-1234w-x-614d-x-478h>: HTTP status code is not handled or not allowed 2026-01-25 00:09:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/antimicrobial-exercise-putty-green-4-ounce-medium>: HTTP status code is not handled or not allowed 2026-01-25 00:09:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-wide-designer-wood-locker-23364-triple-tier-3-wide-15x24x24-cherry-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 00:09:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/physical_therapy/exercise_fitness_equipment/exercise_machines landed on page that is not a product page. 2026-01-25 00:09:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-unit-storage-shelving-chrome-48w-x-21d-x-86h-14-ft>: HTTP status code is not handled or not allowed 2026-01-25 00:09:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14031415-perfo-60-piece-tool-holder-accessory-kit-pl38>: HTTP status code is not handled or not allowed 2026-01-25 00:09:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/skate-board-rom-exerciser-exercise-skate-foam-padded-and-upholstered-large-12-x-6>: HTTP status code is not handled or not allowed 2026-01-25 00:10:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dsfcf180uvk-fontemagna-compact-countertop-water-dispenser-in-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 00:10:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/modular_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:10:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-panel-moulding-pml00x00tr-34h-x-38d-x-94l>: HTTP status code is not handled or not allowed 2026-01-25 00:10:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/plumbing_parts_accessories/toilet_urinal_carriers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:10:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jaw-jaw-turnbuckle-3-4-x-6>: HTTP status code is not handled or not allowed 2026-01-25 00:10:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 40128 pages (at 93 pages/min), scraped 19167 items (at 72 items/min) 2026-01-25 00:10:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9350m-e-v-blk-integra-esd-vinyl-upholstered-chair-medium-back-aluminum-base-black>: HTTP status code is not handled or not allowed 2026-01-25 00:10:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/floor_mats/modular_mats landed on page that is not a product page. 2026-01-25 00:10:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-20-x-5-8-security-machine-screw-button-torx-head-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 00:10:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/plumbing_parts_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:10:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sewn-liquid-bag-filter-polyester-multifilament-4-1-8-x-14-250-micron-steel-ring-pkg-50>: HTTP status code is not handled or not allowed 2026-01-25 00:10:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/liquid-bag-filter-polyester-felt-6-7-8-dia-x-34-l-200-micron-plastic-sure-seal-ring-pkg-qty-50>: HTTP status code is not handled or not allowed 2026-01-25 00:10:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-32-x-3-4-security-machine-screw-button-torx-head-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 00:10:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6mm-hex-t-handle-13368>: HTTP status code is not handled or not allowed 2026-01-25 00:10:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foil-fiberglass-duct-wrap-2625x-12x15-ft>: HTTP status code is not handled or not allowed 2026-01-25 00:10:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/task_desk_chairs/task_office_chairs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:10:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/needle-valve-metal-release-collet-5-32-tube-x-1-8-swift-fit-screw-adjustment>: HTTP status code is not handled or not allowed 2026-01-25 00:10:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/plumbing/plumbing_parts_accessories landed on page that is not a product page. 2026-01-25 00:10:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8npt-poly-muffler-75-micron-145-psi-5-to-212-f>: HTTP status code is not handled or not allowed 2026-01-25 00:10:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelving-36w-x-24d-x-54h-3-shelf-starter-chrome>: HTTP status code is not handled or not allowed 2026-01-25 00:10:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelving-72w-x-18d-x-34h-2-shelf-add-on-chrome>: HTTP status code is not handled or not allowed 2026-01-25 00:10:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/digital-stirrer-4-4-ceramic-top-100-120v>: HTTP status code is not handled or not allowed 2026-01-25 00:10:45 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/task_desk_chairs/task_office_chairs landed on page that is not a product page. 2026-01-25 00:10:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/student-round-hotplate-3-5-diameter-aluminum-top-plate-120v>: HTTP status code is not handled or not allowed 2026-01-25 00:10:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/karcher-1-286-908-0-3000psi-28amps-240volts-3-5gpm-electric-pressure-washer>: HTTP status code is not handled or not allowed 2026-01-25 00:10:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/garage_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:10:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/maxq-4450-incubated-shaker-without-cooling-coil-digital-120v>: HTTP status code is not handled or not allowed 2026-01-25 00:10:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-shelf-level-for-bulk-storage-rack-96-w-x-36-d-wood-deck-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 00:10:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:11:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-value-max-locker-12-w-x-12-d-x-78-h-729-parchment-single-tier-1-wide-knock-down>: HTTP status code is not handled or not allowed 2026-01-25 00:11:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:11:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:11:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/cabinets/garage_cabinets landed on page that is not a product page. 2026-01-25 00:11:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:11:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/in-out-magnetic-dry-eraseletter-board-steelvinyl-surface-36w-x-24h-ga0387830>: HTTP status code is not handled or not allowed 2026-01-25 00:11:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:11:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-series-wheel-5000054592-ws-5-x-2-performa-rubber-1-2-straight-roller-bearing>: HTTP status code is not handled or not allowed 2026-01-25 00:11:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 40217 pages (at 89 pages/min), scraped 19232 items (at 65 items/min) 2026-01-25 00:11:09 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks landed on page that is not a product page. 2026-01-25 00:11:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/westcott-13140-kids-scissors-5-blunt-pack-of-12-assorted>: HTTP status code is not handled or not allowed 2026-01-25 00:11:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:11:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks landed on page that is not a product page. 2026-01-25 00:11:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/avery-6692-laser-cddvd-labels-matte-white-30pack>: HTTP status code is not handled or not allowed 2026-01-25 00:11:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks landed on page that is not a product page. 2026-01-25 00:11:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/221147-cold-handle-for-chicago-faucets>: HTTP status code is not handled or not allowed 2026-01-25 00:11:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks landed on page that is not a product page. 2026-01-25 00:11:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dx340-double-lock-folder-knife>: HTTP status code is not handled or not allowed 2026-01-25 00:11:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks landed on page that is not a product page. 2026-01-25 00:11:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/air_conditioners_chillers/industrial_commercial_portable_air_conditioners already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:11:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks>: HTTP status code is not handled or not allowed 2026-01-25 00:11:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-kit-for-precision-high-performance-incubator-pr205075m-and-pr205075g>: HTTP status code is not handled or not allowed 2026-01-25 00:11:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oregon-state-flag-4x6-nylon>: HTTP status code is not handled or not allowed 2026-01-25 00:11:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premier-pharmacy-vaccine-undercounter-refrigerator-built-in-solid-door-4-6-cu-ft-ph-abt-hc-ucbi-0404>: HTTP status code is not handled or not allowed 2026-01-25 00:11:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/abrasives/grinding_cutoff_wheels/cut_off_wheels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:11:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premier-pharmacy-vaccine-compact-refrigerator-glass-door-10-5-cu-ft-ph-abt-hc-10pg>: HTTP status code is not handled or not allowed 2026-01-25 00:11:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/air_conditioners_chillers/industrial_commercial_portable_air_conditioners landed on page that is not a product page. 2026-01-25 00:11:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-pharmacy-vaccine-refrigerator-triple-swing-solid-doors-72-cu-ft-ph-abt-s72s>: HTTP status code is not handled or not allowed 2026-01-25 00:11:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1506080-dispenser-condiment-stacker-for-san-jamar>: HTTP status code is not handled or not allowed 2026-01-25 00:11:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/abrasives/grinding_cutoff_wheels/cut_off_wheels landed on page that is not a product page. 2026-01-25 00:11:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mcr-safety-v201r-orange-safety-vest-2-reflective-strips-polyester-side-straps-one-size>: HTTP status code is not handled or not allowed 2026-01-25 00:11:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/first-aid-only-740010-ansi-compliant-first-aid-kit-refill-for-10-unit-first-aid-kits>: HTTP status code is not handled or not allowed 2026-01-25 00:11:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-ad830s-stainless-steel-divider-30d-x-8h-for-wire-shelves>: HTTP status code is not handled or not allowed 2026-01-25 00:11:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-ah18n-nexelon-utility-cart-handle-18w>: HTTP status code is not handled or not allowed 2026-01-25 00:11:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-al436s-stainless-steel-wire-ledge-36l-x-4h>: HTTP status code is not handled or not allowed 2026-01-25 00:11:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/wire_cable/electrical_wire_cable already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:11:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/wiring_devices/wire_clips_clamps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:11:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/163204-fryer-screen-11-x-14-for-prince-castle>: HTTP status code is not handled or not allowed 2026-01-25 00:11:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fm-pro-series-steel-industrial-mobile-sanitzation-cart-41w-x-20d-x-43h-black>: HTTP status code is not handled or not allowed 2026-01-25 00:11:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tubing-with-cuff-exerciser-35-circumference-silver>: HTTP status code is not handled or not allowed 2026-01-25 00:11:56 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/wire_cable/electrical_wire_cable landed on page that is not a product page. 2026-01-25 00:11:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/wiring_devices/wire_clips_clamps landed on page that is not a product page. 2026-01-25 00:11:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uv-black-heavy-duty-cable-ties-502221-100-80-100-pcs>: HTTP status code is not handled or not allowed 2026-01-25 00:11:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/retail_fixtures_displays_equipment/cash_handling_equipment/cash_handling_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:11:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/cable_management/cable_ties_wraps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:12:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-canister-filter-for-ufo-103h-ufo-103h1-ufo-105d-dust-collector>: HTTP status code is not handled or not allowed 2026-01-25 00:12:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/wiring_devices/wire_clips_clamps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:12:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fireplace-grate-bg5-215em-21w-1-2-steel-5-bars-ember-retainer>: HTTP status code is not handled or not allowed 2026-01-25 00:12:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hager-780-157-standard-duty-full-surface-hinge-fire-rated>: HTTP status code is not handled or not allowed 2026-01-25 00:12:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/surge_protectors_extension_cords/extension_cords already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:12:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/retail_fixtures_displays_equipment/cash_handling_equipment/cash_handling_accessories landed on page that is not a product page. 2026-01-25 00:12:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/cable_management/cable_ties_wraps>: HTTP status code is not handled or not allowed 2026-01-25 00:12:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 40310 pages (at 93 pages/min), scraped 19292 items (at 60 items/min) 2026-01-25 00:12:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_closers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:12:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/69724-3-stainless-steel-scraper-red>: HTTP status code is not handled or not allowed 2026-01-25 00:12:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_hinges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:12:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/wiring_devices/wire_clips_clamps landed on page that is not a product page. 2026-01-25 00:12:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-wall-rail-stainless-steel-84-long-3-rails-basic-set>: HTTP status code is not handled or not allowed 2026-01-25 00:12:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/surge_protectors_extension_cords/extension_cords landed on page that is not a product page. 2026-01-25 00:12:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tote-box-heat-resistant-40-to-300-degrees-8>: HTTP status code is not handled or not allowed 2026-01-25 00:12:18 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_closers landed on page that is not a product page. 2026-01-25 00:12:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_hinges landed on page that is not a product page. 2026-01-25 00:12:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-600-series-recessed-single-tissue-dispenser-bright-polished>: HTTP status code is not handled or not allowed 2026-01-25 00:12:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/kitchen_tools/hand_tools/replacement_handles already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:12:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/56871-3-gallon-bucket-lid-pink>: HTTP status code is not handled or not allowed 2026-01-25 00:12:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/62507-lobby-dustpan-w-broom-orange>: HTTP status code is not handled or not allowed 2026-01-25 00:12:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-scissor-li-table-1650-lb-capacity-1>: HTTP status code is not handled or not allowed 2026-01-25 00:12:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/platform_trucks/flatbed_platform_trucks/steel_flatbed_platform_trucks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:12:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-w-x-30-d-production-workbench-esd-laminate-square-edge-with-drawers-shelf-blue>: HTTP status code is not handled or not allowed 2026-01-25 00:12:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/kitchen_tools/hand_tools/replacement_handles landed on page that is not a product page. 2026-01-25 00:12:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/bollards/safety_bollards_posts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:12:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-x-18-x-60-phenolic-locker-single-tier-graphite-grafix>: HTTP status code is not handled or not allowed 2026-01-25 00:12:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/44-h-x-48-w-so-nylon-column-protector-yellow-cover-white-tapes>: HTTP status code is not handled or not allowed 2026-01-25 00:12:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/platform_trucks/flatbed_platform_trucks/steel_flatbed_platform_trucks landed on page that is not a product page. 2026-01-25 00:12:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/easy-weld153-stick-welder-machine-120230-v>: HTTP status code is not handled or not allowed 2026-01-25 00:12:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/bollards/safety_bollards_posts landed on page that is not a product page. 2026-01-25 00:12:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drop-box-5-1-2-w-x-6-1-4-h-x12-d-black>: HTTP status code is not handled or not allowed 2026-01-25 00:12:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1292173-filter-grease-25-inch-x-20-inch-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 00:12:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-ladder-extension>: HTTP status code is not handled or not allowed 2026-01-25 00:12:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fir-plywood-premium-grade-crate-premium42-34-29-collapsible-inside-42l-x-34w-x-29>: HTTP status code is not handled or not allowed 2026-01-25 00:13:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/wardrobe_combination_cabinets/combination_cabients already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:13:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dinex-white-embossed-traycover-w-scalloped-edge-round-corner-12-3-4-x-16-5-8-white-dx5999i00102>: HTTP status code is not handled or not allowed 2026-01-25 00:13:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dinex-flat-tray-15-x-20-12-cs-burgundy-dx1089m61>: HTTP status code is not handled or not allowed 2026-01-25 00:13:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:13:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/cabinets/wardrobe_combination_cabinets/combination_cabients>: HTTP status code is not handled or not allowed 2026-01-25 00:13:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 40400 pages (at 90 pages/min), scraped 19357 items (at 65 items/min) 2026-01-25 00:13:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bolted-teardrop-pallet-rack-add-on-96inw-x-48ind-x-96inh-5030-lbs-shelf-cap>: HTTP status code is not handled or not allowed 2026-01-25 00:13:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-174-bathroom-tissue-paper-500-sheets-roll-96-rolls-case>: HTTP status code is not handled or not allowed 2026-01-25 00:13:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits landed on page that is not a product page. 2026-01-25 00:13:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2221033-blade-for-waring-products>: HTTP status code is not handled or not allowed 2026-01-25 00:13:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-1797-11-16-inch-x-7-8-inch-replacement-brass-burner-cap>: HTTP status code is not handled or not allowed 2026-01-25 00:13:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rl-360-6-dual-ramp-portable-bariatric-wheelchair-scale-1000-lb-x-0-2-lb>: HTTP status code is not handled or not allowed 2026-01-25 00:13:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-storage-cabinet-36x22x72-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 00:13:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010467-gasket-center-door-for-blodgett-oven>: HTTP status code is not handled or not allowed 2026-01-25 00:13:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/dollies/open_deck_dollies already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:13:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/261672-press-in-needle11-16>: HTTP status code is not handled or not allowed 2026-01-25 00:13:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industries-ball-tee-for-15-tubing-polished-brass>: HTTP status code is not handled or not allowed 2026-01-25 00:13:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insulated-safety-bib-overall-s-orange>: HTTP status code is not handled or not allowed 2026-01-25 00:13:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/dollies/open_deck_dollies landed on page that is not a product page. 2026-01-25 00:13:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-cube-ice-maker-approx-505-lb-production-half-size-cube>: HTTP status code is not handled or not allowed 2026-01-25 00:13:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insulated-safety-bib-overall-m-black>: HTTP status code is not handled or not allowed 2026-01-25 00:13:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-in-ceiling-mount-fan-8402-cfm>: HTTP status code is not handled or not allowed 2026-01-25 00:13:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/safety_railings_fittings/architectural_railing_components already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:13:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelite-vented-plastic-mat-shelf-truck-48l-x-18w-x-80h-polyurethane-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 00:13:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bmj04020-socket-cap-screw>: HTTP status code is not handled or not allowed 2026-01-25 00:13:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/safety_railings_fittings/architectural_railing_components>: HTTP status code is not handled or not allowed 2026-01-25 00:13:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/folders_filing/wall_files already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:14:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/tool_storage/chests_roller_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:14:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/342005-element-kit-208v-for-blodgett-oven>: HTTP status code is not handled or not allowed 2026-01-25 00:14:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/file_folders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:14:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/file_folders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:14:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-high-security-bolt-seal-green-50-pack>: HTTP status code is not handled or not allowed 2026-01-25 00:14:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/classification_folders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:14:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/54-h-nexel-chrome-wire-shelving-30-w-x-18-d>: HTTP status code is not handled or not allowed 2026-01-25 00:14:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/folders_filing/wall_files landed on page that is not a product page. 2026-01-25 00:14:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 40490 pages (at 90 pages/min), scraped 19424 items (at 67 items/min) 2026-01-25 00:14:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/through-thread-plastic-hand-knobs-1-4-20-thread-1-1-4-inch-knob-dia-13-16-inch-knob-height>: HTTP status code is not handled or not allowed 2026-01-25 00:14:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/file_folders landed on page that is not a product page. 2026-01-25 00:14:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/file_folders>: HTTP status code is not handled or not allowed 2026-01-25 00:14:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/tool_storage/chests_roller_cabinets landed on page that is not a product page. 2026-01-25 00:14:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/classification_folders landed on page that is not a product page. 2026-01-25 00:14:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smead-jan-dec-open-accordion-expanding-file-12-pocket-legal-leather-like-redrope>: HTTP status code is not handled or not allowed 2026-01-25 00:14:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/file_folders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:14:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/classification_folders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:14:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hubbel-commercial-16-3-40>: HTTP status code is not handled or not allowed 2026-01-25 00:14:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/relaxed-fit-twill-safety-cargo-utility-pants-30l-inseam-x-34-waist-black>: HTTP status code is not handled or not allowed 2026-01-25 00:14:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011279-hinge-set-for-glasspro>: HTTP status code is not handled or not allowed 2026-01-25 00:14:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/crowd_control/crowd_control_barricades already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:14:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/file_folders landed on page that is not a product page. 2026-01-25 00:14:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/classification_folders>: HTTP status code is not handled or not allowed 2026-01-25 00:14:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/expanding_file_folders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:14:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-ring-buoy-white-uscg-approved-10-205-whi>: HTTP status code is not handled or not allowed 2026-01-25 00:14:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stage-pie-unit-hardboard-48w-x-8h-stage-units>: HTTP status code is not handled or not allowed 2026-01-25 00:14:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/crowd_control/crowd_control_barricades landed on page that is not a product page. 2026-01-25 00:14:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/63-h-nexel-chrome-wire-shelving-add-on-60-w-x-24-d>: HTTP status code is not handled or not allowed 2026-01-25 00:14:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/file_folders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:14:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-125mm-bore-x-360mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 00:14:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2941065-rod-torque-7-16hex-end-revers-for-anthony-international-california>: HTTP status code is not handled or not allowed 2026-01-25 00:14:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/expanding_file_folders landed on page that is not a product page. 2026-01-25 00:14:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-2-box-bottom-pressboard-hanging-folder-letter-standard-green-25box>: HTTP status code is not handled or not allowed 2026-01-25 00:14:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/86-h-nexel-chrome-wire-shelving-36-w-x-18-d>: HTTP status code is not handled or not allowed 2026-01-25 00:14:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14awg-2c-solid-fire-alarm-cable-plenum-fplr-1000-ft-spool-red>: HTTP status code is not handled or not allowed 2026-01-25 00:14:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-box-slide-24-d-shelves-pair>: HTTP status code is not handled or not allowed 2026-01-25 00:14:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/base-bar-mount-kit-3--1-4-bore-nfpa-cylinder>: HTTP status code is not handled or not allowed 2026-01-25 00:14:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/file_folders landed on page that is not a product page. 2026-01-25 00:14:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-stage-carpet-96l-x-36w-x-24h-red>: HTTP status code is not handled or not allowed 2026-01-25 00:14:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spherical-rod-eye-cs-20-for-iso-cylinders-m8x1-25-threads>: HTTP status code is not handled or not allowed 2026-01-25 00:14:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-safety-parka-jacket-lt-navy>: HTTP status code is not handled or not allowed 2026-01-25 00:14:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-poly-oxford-lined-safety-parka-jacket-xxs-fluorescent-green>: HTTP status code is not handled or not allowed 2026-01-25 00:14:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-50mm-bore-x-320mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 00:15:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/34-1226-toaster-element-116v-328w-5-1-2-inch-x-5-1-8-inch>: HTTP status code is not handled or not allowed 2026-01-25 00:15:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/wire_shelving/poly_zbrite_wire_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:15:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-40mm-bore-x-350mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 00:15:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sandwich-size-reclosable-bag-6-x-6-12-mil>: HTTP status code is not handled or not allowed 2026-01-25 00:15:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 40586 pages (at 96 pages/min), scraped 19489 items (at 65 items/min) 2026-01-25 00:15:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/moist-heat-pack-cover-foam-filled-terry-oversize-with-pocket-24x-30>: HTTP status code is not handled or not allowed 2026-01-25 00:15:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1711100-roller-butter-spreader-for-prince-castle>: HTTP status code is not handled or not allowed 2026-01-25 00:15:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/wire_shelving/poly_zbrite_wire_shelves landed on page that is not a product page. 2026-01-25 00:15:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1132-inch-id-x-1316-inch-od-x-0020-inch-steel-arbor-shim-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 00:15:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-heavy-duty-shelving-48x24x60-with-5-shelves-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 00:15:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-38-inch-id-x-1-78-inch-od-x-0005-inch-steel-arbor-shim-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 00:15:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-heavy-duty-shelving-48x18x72-with-5-shelves-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 00:15:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/switches_receptacles_sensors/plugs_receptacles/straight_blade_plugs_receptacles already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:15:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4mm-square-metric-keystock-gold-dichromate-finish-12-inch-length-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 00:15:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1341144-lock-cylinder-kit-detex-value>: HTTP status code is not handled or not allowed 2026-01-25 00:15:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-piece-steel-shim-stock-assortment-6-inch-x-50-inch-rolls>: HTTP status code is not handled or not allowed 2026-01-25 00:15:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/indoor-air-quality-meter-3000mah-rechargeable-li-ion-battery-black>: HTTP status code is not handled or not allowed 2026-01-25 00:15:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/switches_receptacles_sensors/plugs_receptacles/straight_blade_plugs_receptacles landed on page that is not a product page. 2026-01-25 00:15:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-56-tefc-1ph-1725rpm-c6k17fk21ll>: HTTP status code is not handled or not allowed 2026-01-25 00:15:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-20-ga-48-wx24-dx85-h-open-clip-style-7-shelf>: HTTP status code is not handled or not allowed 2026-01-25 00:15:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-56h-dp-1ph-3450rpm-p6k34dk18aa>: HTTP status code is not handled or not allowed 2026-01-25 00:15:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/parcel-protector-vault-mailbox-and-parcel-drop-dvu0050-rear-access-sand>: HTTP status code is not handled or not allowed 2026-01-25 00:15:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-elbow-85110-04-32-14-tube-x-1032-male-unf-thread-composite-body>: HTTP status code is not handled or not allowed 2026-01-25 00:15:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conductive-bin-accessory-bin-divider-6>: HTTP status code is not handled or not allowed 2026-01-25 00:15:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smartcompliance-nitrile-lightweight-gloves-one-size-2-pairsbox>: HTTP status code is not handled or not allowed 2026-01-25 00:15:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd80sn-916-inch-band-heavy-duty-4-pc-hose-clamp-2-12-inch-5-716-inch-clmpg-diam-10-pcs>: HTTP status code is not handled or not allowed 2026-01-25 00:16:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-20-ga-48-wx12-dx73-h-closed-clip-style-5-shelf-starter>: HTTP status code is not handled or not allowed 2026-01-25 00:16:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-20-ga-48-wx12-d-x85-h-closed-clip-style-5-shelf-add-on>: HTTP status code is not handled or not allowed 2026-01-25 00:16:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b24hs-all-ss-worm-gear-hose-clamp-1-116-inch-2-inch-clamping-diameter-box-of-10>: HTTP status code is not handled or not allowed 2026-01-25 00:16:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-x-4-x-96-single-weathered-wood-post-hardware>: HTTP status code is not handled or not allowed 2026-01-25 00:16:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 40683 pages (at 97 pages/min), scraped 19562 items (at 73 items/min) 2026-01-25 00:16:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-cu-yd-gray-self-dumping-forkli-hopper-with-heavy-gauge-base>: HTTP status code is not handled or not allowed 2026-01-25 00:16:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/patient_support_immobilization/splints_supports already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:16:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2721266-burner-lower-inch-h-inch-for-nieco-corp>: HTTP status code is not handled or not allowed 2026-01-25 00:16:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-pro-right-angle-drill-attachment-3-32-1-2-chuck-capacity>: HTTP status code is not handled or not allowed 2026-01-25 00:16:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-56-tefc-1ph-3450rpm-c6c34fc14kk>: HTTP status code is not handled or not allowed 2026-01-25 00:16:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/patient_support_immobilization/splints_supports landed on page that is not a product page. 2026-01-25 00:16:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-flame-resistant-helix-work-shirt-2xl-tall-dark-khaki>: HTTP status code is not handled or not allowed 2026-01-25 00:16:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s9639-165w-led-br40-reflector-103-beam-spread-medium-base-3000k-dimmable>: HTTP status code is not handled or not allowed 2026-01-25 00:16:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-inch-x-1-inch-x-0025-inch-white-plastic-color-coded-slotted-shim-pack-of-20>: HTTP status code is not handled or not allowed 2026-01-25 00:16:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/utility-knives-irwin-2089100>: HTTP status code is not handled or not allowed 2026-01-25 00:16:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1421402-handle-wood-54-inch-tapered-end-for-carlisle-foodservice-products>: HTTP status code is not handled or not allowed 2026-01-25 00:16:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-1059-1-4-inch-20-x-3-4-inch-stainless-steel-hex-head-cap-screw>: HTTP status code is not handled or not allowed 2026-01-25 00:16:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-display-rack-black-4-shelves>: HTTP status code is not handled or not allowed 2026-01-25 00:16:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/metals/stainless_steel_tool_wrap already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:16:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/computers_electronics/audio_video_equipment/audio_visual_mounts/monitor_mounts_stands already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:16:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-inch-plastic-classroom-chair-set-of-five-orange>: HTTP status code is not handled or not allowed 2026-01-25 00:16:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-dock-board-with-steel-curbs-3>: HTTP status code is not handled or not allowed 2026-01-25 00:16:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/lathes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:16:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/metals/stainless_steel_tool_wrap>: HTTP status code is not handled or not allowed 2026-01-25 00:16:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/room_dividers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:16:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/computers_electronics/audio_video_equipment/audio_visual_mounts/monitor_mounts_stands landed on page that is not a product page. 2026-01-25 00:16:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lyon-welded-mesh-side-storage-locker-24-x21-x78>: HTTP status code is not handled or not allowed 2026-01-25 00:17:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lyon-full-height-modular-storage-drawer-cabinet-8>: HTTP status code is not handled or not allowed 2026-01-25 00:17:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/lathes landed on page that is not a product page. 2026-01-25 00:17:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/partitions_dividers/room_dividers landed on page that is not a product page. 2026-01-25 00:17:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-2373-3-inch-swivel-plate-caster-with-brake-200-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 00:17:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/46-round-bar-height-table-with-4-seats-in-ground-mount-perforated-pattern-blue>: HTTP status code is not handled or not allowed 2026-01-25 00:17:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 40775 pages (at 92 pages/min), scraped 19632 items (at 70 items/min) 2026-01-25 00:17:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/furniture_parts_accessories/partition_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:17:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/push-to-connect-tube-fitting-nylon-90-degree-elbow-adapter-10mm-tube-od-x-38-npt-male>: HTTP status code is not handled or not allowed 2026-01-25 00:17:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/seating/reception_sofas_sectionals already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:17:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/push-to-connect-tube-fitting-nylon-plastic-straight-adapter-38-tube-od-x-38-npt-female>: HTTP status code is not handled or not allowed 2026-01-25 00:17:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/push-to-connect-tube-fitting-nylon-90-degree-elbow-adapter-18-tube-od-x-18-npt-male>: HTTP status code is not handled or not allowed 2026-01-25 00:17:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-flat-washer-sae-extra-thick-9-32-i-d-steel-yellow-zinc-grade-8-pkg-of-50>: HTTP status code is not handled or not allowed 2026-01-25 00:17:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/furniture_parts_accessories/partition_accessories landed on page that is not a product page. 2026-01-25 00:17:21 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/seating/reception_sofas_sectionals landed on page that is not a product page. 2026-01-25 00:17:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washdown-motor-56-tefc-3ph-1725rpm-c6t17wk1jj>: HTTP status code is not handled or not allowed 2026-01-25 00:17:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/push-to-connect-tube-fitting-nylon-plastic-straight-reducer-10mm-tube-od-x-6mm-tube-od>: HTTP status code is not handled or not allowed 2026-01-25 00:17:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-ft-cord-set-18-2-spt-1-clear-silver>: HTTP status code is not handled or not allowed 2026-01-25 00:17:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/general_purpose_electric_motors/metric_three_phase_motors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:17:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wear-ever-classic-rolled-edge-stock-pots-4305-8-gauge-20-quart-capacity>: HTTP status code is not handled or not allowed 2026-01-25 00:17:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15ml-centrifuge-tube-paperboard-rack-sterile-500-case>: HTTP status code is not handled or not allowed 2026-01-25 00:17:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/cookware_bakeware/pots_pans/frying_pans_skillets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:17:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fda-epdm-camlock-gasket-for-1-12-hose-coupling-pack-of-5>: HTTP status code is not handled or not allowed 2026-01-25 00:17:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8014150-hood-filter-type6-stainless-steel-16x25>: HTTP status code is not handled or not allowed 2026-01-25 00:17:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/general_purpose_electric_motors/metric_three_phase_motors landed on page that is not a product page. 2026-01-25 00:17:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/orbit-side-car-with-fans-beige-1>: HTTP status code is not handled or not allowed 2026-01-25 00:17:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/soft-fda-silicone-rubber-cord-0275-cross-section-10-ft-length>: HTTP status code is not handled or not allowed 2026-01-25 00:17:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-x-30-esd-square-edge-work-bench-adjustable-height-1-1-4-top>: HTTP status code is not handled or not allowed 2026-01-25 00:17:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/cookware_bakeware/pots_pans/frying_pans_skillets landed on page that is not a product page. 2026-01-25 00:17:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-red-pin-heavy-duty-dock-boards-33>: HTTP status code is not handled or not allowed 2026-01-25 00:17:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-in-coat-towel-rack-shelf-silver>: HTTP status code is not handled or not allowed 2026-01-25 00:17:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slotted-hook-hangers-100-pack-charcoal-gray>: HTTP status code is not handled or not allowed 2026-01-25 00:18:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/lathes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:18:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-round-table-with-4-imme-armless-chairs-navy-seatdesigner-white-top>: HTTP status code is not handled or not allowed 2026-01-25 00:18:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/lathes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:18:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/350090-rb-6-p-riser-block-for-jtm-124vs>: HTTP status code is not handled or not allowed 2026-01-25 00:18:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wear-ever-tapered-sauce-pan-z434312-8-3-4-bottom-diameter-4-1-2-quart-capacity>: HTTP status code is not handled or not allowed 2026-01-25 00:18:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 40869 pages (at 94 pages/min), scraped 19702 items (at 70 items/min) 2026-01-25 00:18:09 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/lathes landed on page that is not a product page. 2026-01-25 00:18:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-round-table-with-4-imme-armless-chairs-sky-blue-seatdesigner-white-top>: HTTP status code is not handled or not allowed 2026-01-25 00:18:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/lathes landed on page that is not a product page. 2026-01-25 00:18:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/optio-brazier-3819-19-quart-capacity-15-3-4-diameter>: HTTP status code is not handled or not allowed 2026-01-25 00:18:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/cookware_bakeware/pots_pans/stock_pots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:18:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-workmate-stand-ahwh2436>: HTTP status code is not handled or not allowed 2026-01-25 00:18:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-butcher-meat-saw-25-inches>: HTTP status code is not handled or not allowed 2026-01-25 00:18:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-level-72x24-high-capacity-rack-steel-deck-gray>: HTTP status code is not handled or not allowed 2026-01-25 00:18:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/cookware_bakeware/pots_pans/stock_pots landed on page that is not a product page. 2026-01-25 00:18:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cable-tie-11-75-lb-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 00:18:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_chair_table_sets/conversation_sets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:18:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/norton-66261194393-blaze-r920-cartridge-roll-1-2-inch-x-1-1-2-inch-x-1-8-inch-60-grit-seeded-gel-ceramic>: HTTP status code is not handled or not allowed 2026-01-25 00:18:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blue-print-pivot-mobile-rack-with-12-pivot-hangers-12-36-hanging-clamps>: HTTP status code is not handled or not allowed 2026-01-25 00:18:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-cable-tie-18-175-lb-50-pk>: HTTP status code is not handled or not allowed 2026-01-25 00:18:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grid-wall-bin-stackable-24>: HTTP status code is not handled or not allowed 2026-01-25 00:18:42 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_chair_table_sets/conversation_sets landed on page that is not a product page. 2026-01-25 00:18:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011036-infrared-burner>: HTTP status code is not handled or not allowed 2026-01-25 00:18:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2371065-gasket-milk-cooler-24-ft-for-beverage-air>: HTTP status code is not handled or not allowed 2026-01-25 00:18:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/guest_chairs/lounge_chairs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:18:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/reel_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:18:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-3457-tubular-steel-burner-23-1-4-inch-x-6-1-2-inch>: HTTP status code is not handled or not allowed 2026-01-25 00:18:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rw10-3s1y-lightweight-rain-jacket-hi-viz-lime-xl>: HTTP status code is not handled or not allowed 2026-01-25 00:18:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/appliances/food_holding_warming_equipment/food_warmers_hot_food_display_cases already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:18:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/guest_chairs/lounge_chairs>: HTTP status code is not handled or not allowed 2026-01-25 00:18:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/341029-urn-heater208v-4000w>: HTTP status code is not handled or not allowed 2026-01-25 00:19:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8014112-hood-filter-type6-alum-16x25-w-lock-handles>: HTTP status code is not handled or not allowed 2026-01-25 00:19:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:19:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/appliances/food_holding_warming_equipment/food_warmers_hot_food_display_cases>: HTTP status code is not handled or not allowed 2026-01-25 00:19:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/reel_racks landed on page that is not a product page. 2026-01-25 00:19:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/305d-wire-pull-4-us26d>: HTTP status code is not handled or not allowed 2026-01-25 00:19:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_hinges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:19:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 40956 pages (at 87 pages/min), scraped 19765 items (at 63 items/min) 2026-01-25 00:19:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/insulation_film_weather_stripping already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:19:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks>: HTTP status code is not handled or not allowed 2026-01-25 00:19:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/insulation_film_weather_stripping already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:19:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/281464-kason-401sa6065184-curtain-strip-easimnt-for-kason>: HTTP status code is not handled or not allowed 2026-01-25 00:19:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_hinges>: HTTP status code is not handled or not allowed 2026-01-25 00:19:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_cabinets_pedestals already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:19:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/door_window_hardware/insulation_film_weather_stripping landed on page that is not a product page. 2026-01-25 00:19:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10x15x2-expandable-ship-lite-envelopes>: HTTP status code is not handled or not allowed 2026-01-25 00:19:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/door_window_hardware/insulation_film_weather_stripping landed on page that is not a product page. 2026-01-25 00:19:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hand-dryer-w-noise-reduction-nozzle-white-thermoset-fiberglass-hepa-110-120v>: HTTP status code is not handled or not allowed 2026-01-25 00:19:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blickman-18-dia-round-medical-hamper>: HTTP status code is not handled or not allowed 2026-01-25 00:19:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ptp231-tough-mark-strip>: HTTP status code is not handled or not allowed 2026-01-25 00:19:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/68-1264-1-3-hp-fryer-filter-pump-motor-with-gasket-240v>: HTTP status code is not handled or not allowed 2026-01-25 00:19:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_cabinets_pedestals landed on page that is not a product page. 2026-01-25 00:19:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blickman-howard-instrument-shelf-table-30l-x-16w-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 00:19:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tape-silicone-self-sealing-10>: HTTP status code is not handled or not allowed 2026-01-25 00:19:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5581018-breaker-20a>: HTTP status code is not handled or not allowed 2026-01-25 00:19:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/plastic_laminate_bathroom_partitions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:19:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16x12x8-double-wall-boxes>: HTTP status code is not handled or not allowed 2026-01-25 00:19:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012179-thermostat-for-star-manufacturing>: HTTP status code is not handled or not allowed 2026-01-25 00:19:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/wiring_devices/relays_sequencers/relays already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:19:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-retention-load-pin-kit-for-3-4-weblok-assembly>: HTTP status code is not handled or not allowed 2026-01-25 00:19:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/plastic_laminate_bathroom_partitions>: HTTP status code is not handled or not allowed 2026-01-25 00:19:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24x22x60-wardrobe-boxes>: HTTP status code is not handled or not allowed 2026-01-25 00:19:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/wiring_devices/relays_sequencers/relays landed on page that is not a product page. 2026-01-25 00:19:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-14-x-60-grey-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 00:19:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-24-x-42-black-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 00:19:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/abrasives/grinding_cutoff_wheels/cut_off_wheels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:19:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/power_tools/power_saws_blades/band_saws already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:20:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-shelf-heavy-duty-boltless-shelving-add-on-42in-w-x-30in-d-x-36in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-25 00:20:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1381181-thermometer-c22-t-type-for-comark-instruments>: HTTP status code is not handled or not allowed 2026-01-25 00:20:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-heavy-duty-boltless-shelving-add-on-48in-w-x-12in-d-x-96in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-25 00:20:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enclosure-kit-slide-door-18-x-36-x-13-black>: HTTP status code is not handled or not allowed 2026-01-25 00:20:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 41049 pages (at 93 pages/min), scraped 19832 items (at 67 items/min) 2026-01-25 00:20:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/shelving_accessories/shelving_enclosure_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:20:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/shelving_accessories/shelving_enclosure_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:20:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cta-cap-torque-tester-100inlb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 00:20:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/abrasives/grinding_cutoff_wheels/cut_off_wheels landed on page that is not a product page. 2026-01-25 00:20:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/power_tools/power_saws_blades/band_saws landed on page that is not a product page. 2026-01-25 00:20:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xtreme-1-2-gallon-variable-speed-blender-dial-control-bpa-free-copolyester>: HTTP status code is not handled or not allowed 2026-01-25 00:20:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-x2150-2-mil-poly-tubing>: HTTP status code is not handled or not allowed 2026-01-25 00:20:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/shelving_accessories/shelving_enclosure_kits landed on page that is not a product page. 2026-01-25 00:20:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/shelving_accessories/shelving_enclosure_kits landed on page that is not a product page. 2026-01-25 00:20:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-starter-48inw-x-48ind-x-60inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 00:20:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-ventilation-fan-12-in-diameter>: HTTP status code is not handled or not allowed 2026-01-25 00:20:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-open-cntr-dbl-solenoid-valve-pilot-spring-cntred-1-2-nptf-24v-dc-2w-coil-led-conn>: HTTP status code is not handled or not allowed 2026-01-25 00:20:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/prime-flame-resistant-long-pants-m-gray-df2-cm-762-plp-rg-md>: HTTP status code is not handled or not allowed 2026-01-25 00:20:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kit2cv113x12-12-cal-cm2-ultrasoft-arc-flash-kit-with-fr-coverall-3xl-glove-size-12>: HTTP status code is not handled or not allowed 2026-01-25 00:20:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-2-rubber-voltage-glove-kit-black-size-11-kitgc211>: HTTP status code is not handled or not allowed 2026-01-25 00:20:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-high-capacity-bulk-rack-with-steel-decking-60x24x96-starter>: HTTP status code is not handled or not allowed 2026-01-25 00:20:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-high-capacity-bulk-rack-with-steel-decking-96x24x96-starter>: HTTP status code is not handled or not allowed 2026-01-25 00:20:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-titanium-non-stick-bent-scissors>: HTTP status code is not handled or not allowed 2026-01-25 00:20:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/translucent-liner-18-x-54>: HTTP status code is not handled or not allowed 2026-01-25 00:20:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-g-1-4-double-solenoid-valve-pilot-24v-dc-2w-coil-led-connection>: HTTP status code is not handled or not allowed 2026-01-25 00:20:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-16-18-x-3-1-2-hex-head-cap-screw-steel-plain-unc-grade-8-usa-50-pack-452096>: HTTP status code is not handled or not allowed 2026-01-25 00:20:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kevlar-surgeon-style-coat-l-yellow-c28kvlg36>: HTTP status code is not handled or not allowed 2026-01-25 00:21:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-long-sleeve-t-shirt-m-gray-c541ngelsmd>: HTTP status code is not handled or not allowed 2026-01-25 00:21:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-2-double-solenoid-valve-pilot-1-8-nptf-220v-ac-5va-coil-led-connection>: HTTP status code is not handled or not allowed 2026-01-25 00:21:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012223-switch-on-off-15a-24vdcv1b1bv0c-arc00-for-cleveland-range>: HTTP status code is not handled or not allowed 2026-01-25 00:21:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-1-8-x-9-1-4-x-4-kraft-literature-mailers>: HTTP status code is not handled or not allowed 2026-01-25 00:21:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 41144 pages (at 95 pages/min), scraped 19902 items (at 70 items/min) 2026-01-25 00:21:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-value-mobile-coat-rack-hangers-sold-separately>: HTTP status code is not handled or not allowed 2026-01-25 00:21:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/bar_restaurant_equipment/bartending_supplies/bar_mats_shelf_liners already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:21:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kit2cv11bl09-12-cal-ultrasoft-arc-flash-kit-fr-coverall-balaclava-l-glove-sz-09>: HTTP status code is not handled or not allowed 2026-01-25 00:21:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-32-x-1-2-phillips-flat-head-machine-screw-pkg-of-100-dwa88016>: HTTP status code is not handled or not allowed 2026-01-25 00:21:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kit2cv11bxl11-12-cal-ultrasoft-arc-flash-kit-fr-coverall-balaclava-xl-sz-11>: HTTP status code is not handled or not allowed 2026-01-25 00:21:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/bar_restaurant_equipment/bartending_supplies/bar_mats_shelf_liners landed on page that is not a product page. 2026-01-25 00:21:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/structural-plastic-shelving-36x24x72-adjustable>: HTTP status code is not handled or not allowed 2026-01-25 00:21:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/621163-stick-m-ups-thermometer>: HTTP status code is not handled or not allowed 2026-01-25 00:21:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/power_tools/impact_drivers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:21:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-hose-leader-line-wye-430-2521511-2-1-2-nh-x-1-1-2-npsh-brass-2-piece>: HTTP status code is not handled or not allowed 2026-01-25 00:21:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-add-on-96inw-x-48ind-x-96inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 00:21:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-starter-96inw-x-24ind-x-72inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 00:21:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2631053-washer-lock-stainless-steel-5-16-inch-id-for-hobart>: HTTP status code is not handled or not allowed 2026-01-25 00:21:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-starter-96inw-x-48ind-x-84inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 00:21:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/power_tools/impact_drivers landed on page that is not a product page. 2026-01-25 00:21:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-sign-cotton-candy>: HTTP status code is not handled or not allowed 2026-01-25 00:21:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-24w-15-step-steel-rolling-ladder-21d-top-step-wcal-osha-handrail-ca>: HTTP status code is not handled or not allowed 2026-01-25 00:21:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-starter-60inw-x-36ind-x-84inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 00:21:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010836-bearing-roll-pinassembly-for-middleby>: HTTP status code is not handled or not allowed 2026-01-25 00:21:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/265890-ret-ring-knife-shaft-for-hobart>: HTTP status code is not handled or not allowed 2026-01-25 00:21:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/retail_fixtures_displays_equipment/garment_supplies/hangers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:21:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-add-on-72inw-x-36ind-x-84inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 00:21:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-heat-knit-hood-in-nomex-osfm-white-h31nk>: HTTP status code is not handled or not allowed 2026-01-25 00:21:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-open-1-8-nptf-single-solenoid-valve-pilot-spr-return-110v-ac-5va-coil-led-conn>: HTTP status code is not handled or not allowed 2026-01-25 00:21:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/retail_fixtures_displays_equipment/garment_supplies/hangers landed on page that is not a product page. 2026-01-25 00:22:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-locker-accessory-closed-front-base-12w-x-6h-725-gray>: HTTP status code is not handled or not allowed 2026-01-25 00:22:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-kevlar-knit-sleeve-yellow-s00kn18>: HTTP status code is not handled or not allowed 2026-01-25 00:22:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-2-double-solenoid-valve-pilot-1-2-nptf-220v-ac-5va-coil-led-connection>: HTTP status code is not handled or not allowed 2026-01-25 00:22:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-add-on-72inw-x-24ind-x-60inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 00:22:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 41238 pages (at 94 pages/min), scraped 19971 items (at 69 items/min) 2026-01-25 00:22:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-16-x-3-4-security-self-drilling-screw-pan-torx-head-2-drill-pt-410-ss-ft-usa-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 00:22:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-24-x-3-4-security-machine-screw-button-torx-head-steel-zinc-ft-unc-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 00:22:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pegboard-garden-tool-board-organizer-green-white-48-x-32-x-9>: HTTP status code is not handled or not allowed 2026-01-25 00:22:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/laser-ink-jet-insert-letter-pref-1-in-2-5-8-in-1800-per-package>: HTTP status code is not handled or not allowed 2026-01-25 00:22:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pegboard-standard-tool-storage-kit-blue-red-48-x-32-x-9>: HTTP status code is not handled or not allowed 2026-01-25 00:22:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silicone-foam-sheet-no-adhesive-1-4-x-36-x-36>: HTTP status code is not handled or not allowed 2026-01-25 00:22:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-add-on-60inw-x-36ind-x-84inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 00:22:30 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-25 00:22:31 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:20 2026-01-25 00:22:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/horizon-womens-safety-work-boots-alloy-toe-8h-size-11-carbon-black>: HTTP status code is not handled or not allowed 2026-01-25 00:22:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roller-tarp-kit-for-8-foot-bed-polymer-pickup-truck-dump-inserts>: HTTP status code is not handled or not allowed 2026-01-25 00:22:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011050-limit-bracket-for-henny-penny>: HTTP status code is not handled or not allowed 2026-01-25 00:22:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/faultless-rigid-plate-caster-6-tpr-wheel>: HTTP status code is not handled or not allowed 2026-01-25 00:22:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/emergency_lighting_signage/emergency_lighting already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:22:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-king-sk3000-pallet-rack-108w-x-48d-x-144h-4040-lbs-cap-per-level-add-on-unit>: HTTP status code is not handled or not allowed 2026-01-25 00:22:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/emergency_lighting_signage/emergency_lighting already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:22:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:22:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-steel-emergency-light-6v-54w>: HTTP status code is not handled or not allowed 2026-01-25 00:22:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:22:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/64-inch-width-x-19-1-2-inch-height-utv-snow-plow-with-hitch>: HTTP status code is not handled or not allowed 2026-01-25 00:22:51 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/emergency_lighting_signage/emergency_lighting landed on page that is not a product page. 2026-01-25 00:22:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2200va-ups-smart-pro-network-tower-line-interactive-6-outlets>: HTTP status code is not handled or not allowed 2026-01-25 00:22:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/emergency_lighting_signage/emergency_lighting landed on page that is not a product page. 2026-01-25 00:22:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits landed on page that is not a product page. 2026-01-25 00:22:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-32-x-1-1-4-security-machine-screw-flat-torx-plus-head-18-8-stainless-steel-ft-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 00:22:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits landed on page that is not a product page. 2026-01-25 00:22:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-edge-work-bench-top-plastic-60-w-x-30-d-x-1-5-8-thick>: HTTP status code is not handled or not allowed 2026-01-25 00:23:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/282427-kason-400sa00001-2-cleat-strip-curtain>: HTTP status code is not handled or not allowed 2026-01-25 00:23:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/omega-womens-safety-work-boots-steel-toe-6h-size-85w-bourbon-brown>: HTTP status code is not handled or not allowed 2026-01-25 00:23:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gravity-womens-safety-work-boots-steel-toe-4h-size-10w-vegan-blackout>: HTTP status code is not handled or not allowed 2026-01-25 00:23:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 41332 pages (at 94 pages/min), scraped 20041 items (at 70 items/min) 2026-01-25 00:23:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-inch-polypropylene-hose-strainer>: HTTP status code is not handled or not allowed 2026-01-25 00:23:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2631023-curtain-dishwasher-225x-205-for-hobart>: HTTP status code is not handled or not allowed 2026-01-25 00:23:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:23:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-15-16-dia-zinc-plated-steel-skatewheel-50-lb-cap->: HTTP status code is not handled or not allowed 2026-01-25 00:23:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-4964-boxwood-1-light-outdoor-wall-clear-beveled-white-9-875w-x-19h>: HTTP status code is not handled or not allowed 2026-01-25 00:23:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:23:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pulsar-knee-boot-plain-toe-chevron-plus-174-outsole-15-h-blk-gray-size-4>: HTTP status code is not handled or not allowed 2026-01-25 00:23:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-25 00:23:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aware-wear-ansi-class-3-hooded-pull-over-sweatshirt-61544-lime>: HTTP status code is not handled or not allowed 2026-01-25 00:23:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/bollards/bollard_covers_post_sleeves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:23:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-box-spreader-bar-black>: HTTP status code is not handled or not allowed 2026-01-25 00:23:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011599-kason-11808fm0000-led-light-fixture-for-kason>: HTTP status code is not handled or not allowed 2026-01-25 00:23:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6in-row-spacer-w-hardware-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-25 00:23:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-stem-caster-4-dia-nomadic-derlin-bearing-with-2-h-stem>: HTTP status code is not handled or not allowed 2026-01-25 00:23:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hss-tmx-t-shank-gp-drills-118-2-13-16-inch-diameter-x-17-25-64-inch-oal-number-5-mt>: HTTP status code is not handled or not allowed 2026-01-25 00:23:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/282406-hopper-door-kit-for-prince-castle>: HTTP status code is not handled or not allowed 2026-01-25 00:23:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-25 00:23:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adder-unit-letter-25-inch-30-3-4-inch-92-1-2-inch-8-opening-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 00:23:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/science-lab-table-30h-with-black-phenolic-resin-top-48-24>: HTTP status code is not handled or not allowed 2026-01-25 00:23:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/bollards/bollard_covers_post_sleeves landed on page that is not a product page. 2026-01-25 00:23:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foam-nitrile-palm-coated-nylon-gloves-posigrip-713snfl>: HTTP status code is not handled or not allowed 2026-01-25 00:23:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/decorative_lighting/chandeliers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:23:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-x-6-portable-wire-grid-wall-panel-chrome>: HTTP status code is not handled or not allowed 2026-01-25 00:23:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/letter-file-storage-drawer-locking-black>: HTTP status code is not handled or not allowed 2026-01-25 00:23:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/bar_restaurant_equipment/bartending_supplies/bar_mats_shelf_liners already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:23:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-bollard-with-galvanized-steel-cap-wg-24>: HTTP status code is not handled or not allowed 2026-01-25 00:23:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/decorative_lighting/chandeliers landed on page that is not a product page. 2026-01-25 00:23:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-adjustable-flip-top-table-crank-handle-gray>: HTTP status code is not handled or not allowed 2026-01-25 00:23:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/decorative_lighting/pendant_lighting already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:23:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hss-tmx-plain-teeth-side-milling-cutter-5-inch-dia-x-1-inch-face-x-1-1-4-inch-hole-x-24-teeth>: HTTP status code is not handled or not allowed 2026-01-25 00:23:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/decorative_lighting/vanity_bath_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:23:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/bar_restaurant_equipment/bartending_supplies/bar_mats_shelf_liners landed on page that is not a product page. 2026-01-25 00:23:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-x-20-maroon-x-stripping-pad-10-per-case>: HTTP status code is not handled or not allowed 2026-01-25 00:23:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/decorative_lighting/vanity_bath_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:23:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/abrasives/sanding_discs/flap_sanding_discs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:23:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/decorative_lighting/pendant_lighting>: HTTP status code is not handled or not allowed 2026-01-25 00:24:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qp-pro-two-stage-air-compressor-15-hp-120-gallon-horizontal-230v-3-phase>: HTTP status code is not handled or not allowed 2026-01-25 00:24:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/decorative_lighting/vanity_bath_lights landed on page that is not a product page. 2026-01-25 00:24:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traffic-barrel-drum-base-30-lb-recycled-rubber-base-round-03-732-30>: HTTP status code is not handled or not allowed 2026-01-25 00:24:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/decorative_lighting/vanity_bath_lights landed on page that is not a product page. 2026-01-25 00:24:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyurethane-mold-on-wheel-e-808-sa-8-dia-2w-1260-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 00:24:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/abrasives/sanding_discs/flap_sanding_discs landed on page that is not a product page. 2026-01-25 00:24:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 41424 pages (at 92 pages/min), scraped 20099 items (at 58 items/min) 2026-01-25 00:24:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-boltless-shelving-starter-36inw-x-18ind-x-84inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 00:24:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-boltless-shelving-add-on-36inw-x-18ind-x-48inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 00:24:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-24-w-4-step-steel-rolling-ladder-10-d-top-step-kdsr104246>: HTTP status code is not handled or not allowed 2026-01-25 00:24:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/basic-mesh-task-chair-with-arms>: HTTP status code is not handled or not allowed 2026-01-25 00:24:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m12-cordless-compact-lithium-ion-battery>: HTTP status code is not handled or not allowed 2026-01-25 00:24:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-boltless-shelving-add-on-48inw-x-12ind-x-72inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 00:24:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-cabinet-with-removable-bins-57>: HTTP status code is not handled or not allowed 2026-01-25 00:24:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/solid-blue-curtain-wall-partition-24-x-8>: HTTP status code is not handled or not allowed 2026-01-25 00:24:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-cabinet-with-removable-bins-38>: HTTP status code is not handled or not allowed 2026-01-25 00:24:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-rectangular-outdoor-table-expanded-metal-blue>: HTTP status code is not handled or not allowed 2026-01-25 00:24:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/incandescent_light_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:24:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pivot-boom-forklift-jib-lift-crane-pb-jlt436-4000-lb-capacity-36-on-center-fork-pockets>: HTTP status code is not handled or not allowed 2026-01-25 00:24:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-safety-view-locker-six-tier-12x12x12-6-doors-unassembled-tan>: HTTP status code is not handled or not allowed 2026-01-25 00:24:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commercial-fibc-bulk-bags-duffel-top-flat-bottom-2205-lbs-pp-35-x-35-x-40-pkg-qty-1>: HTTP status code is not handled or not allowed 2026-01-25 00:24:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/incandescent_light_bulbs landed on page that is not a product page. 2026-01-25 00:24:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-68-inch-high-post-black-epoxy-4-pack>: HTTP status code is not handled or not allowed 2026-01-25 00:24:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/halogen_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:24:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/halogen_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:24:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/500par56q-nsp-500w-sealed-beam-w-mogul-end-prong-base>: HTTP status code is not handled or not allowed 2026-01-25 00:24:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torque-limiting-socket-1-2-drive-21mm-110-ft-lbs>: HTTP status code is not handled or not allowed 2026-01-25 00:24:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/halogen_bulbs landed on page that is not a product page. 2026-01-25 00:24:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torque-limiting-extension-1-2-drive-55-ft-lbs>: HTTP status code is not handled or not allowed 2026-01-25 00:24:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/halogen_bulbs landed on page that is not a product page. 2026-01-25 00:24:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/266243-hinge-cam-lift-1-1-8-inch-19-5-8>: HTTP status code is not handled or not allowed 2026-01-25 00:25:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ram-screw-type-10-ton-15-l-compressed>: HTTP status code is not handled or not allowed 2026-01-25 00:25:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wheel-polyurethane-5x2-with-1-2-axle>: HTTP status code is not handled or not allowed 2026-01-25 00:25:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/trash_recycling/trash_recycling_accessories/trash_can_lids_dollies already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:25:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/342306-element-cal-rod-88-inch-200w120v>: HTTP status code is not handled or not allowed 2026-01-25 00:25:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1241401-switch-defrost-term-cooler-for-russell>: HTTP status code is not handled or not allowed 2026-01-25 00:25:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 41516 pages (at 92 pages/min), scraped 20167 items (at 68 items/min) 2026-01-25 00:25:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/461734-control-box-assembly>: HTTP status code is not handled or not allowed 2026-01-25 00:25:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/trash_recycling/trash_recycling_accessories/trash_can_lids_dollies landed on page that is not a product page. 2026-01-25 00:25:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cft32w-4p-830-32w-compact-fluorescent-w-gx24vq-3-base-warm>: HTTP status code is not handled or not allowed 2026-01-25 00:25:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shop-press-50-ton-super-duty>: HTTP status code is not handled or not allowed 2026-01-25 00:25:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/compact_fluorescent_light_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:25:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010375-thermometer-for-groen>: HTTP status code is not handled or not allowed 2026-01-25 00:25:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fc8t9-ww-rs-22w-fluorescent-w-4-pin-base-warm-white>: HTTP status code is not handled or not allowed 2026-01-25 00:25:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/compact_fluorescent_light_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:25:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/combination_metalworking_machines already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:25:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/compact_fluorescent_light_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:25:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a24h2408allp-wall-mount-encl-type-4-2400x2400x800>: HTTP status code is not handled or not allowed 2026-01-25 00:25:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_seating/outdoor_chaise_lounges_daybeds already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:25:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/compact_fluorescent_light_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:25:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dspbl-dust-head-48x5-cttnsyn-12>: HTTP status code is not handled or not allowed 2026-01-25 00:25:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/combination_metalworking_machines already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:25:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/75br30-b-75w-incandescent-w-medium-base>: HTTP status code is not handled or not allowed 2026-01-25 00:25:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/compact_fluorescent_light_bulbs landed on page that is not a product page. 2026-01-25 00:25:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/compact_fluorescent_light_bulbs landed on page that is not a product page. 2026-01-25 00:25:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/combination_metalworking_machines landed on page that is not a product page. 2026-01-25 00:25:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/311-premium-face-shield-headgear-6-12l-x-19-12w-x-116-thick-clear>: HTTP status code is not handled or not allowed 2026-01-25 00:25:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/compact_fluorescent_light_bulbs landed on page that is not a product page. 2026-01-25 00:25:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_seating/outdoor_chaise_lounges_daybeds>: HTTP status code is not handled or not allowed 2026-01-25 00:25:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/combination_metalworking_machines landed on page that is not a product page. 2026-01-25 00:25:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e1003-12-03w-incandescent-w-sc-bay-base>: HTTP status code is not handled or not allowed 2026-01-25 00:25:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/compact_fluorescent_light_bulbs landed on page that is not a product page. 2026-01-25 00:25:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dsc-210-recessed-access-door-for-tile-lock-24wx24h-210p2424l>: HTTP status code is not handled or not allowed 2026-01-25 00:25:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/cable_management/cable_clamps_clips_hardware already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:25:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-rod-3-diameter-x-6-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 00:25:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/cable_management/cable_clamps_clips_hardware already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:25:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-cantilever-rack-double-sided-add-on-heavy-duty-48-w-x-60-d-x-8-h-with-lip>: HTTP status code is not handled or not allowed 2026-01-25 00:25:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/heaters/electric_heaters/overhead_infrared_heaters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:25:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/incandescent_light_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:25:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rlp1ns-long-sleeve-pullover-t-shirt-with-uv-protection-2xl-hi-vis-lime>: HTTP status code is not handled or not allowed 2026-01-25 00:25:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/cable_management/cable_clamps_clips_hardware>: HTTP status code is not handled or not allowed 2026-01-25 00:25:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-1-thick-x-3-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 00:25:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-step-stairway-slope-work-platform-w-handrails-36in-x-36in-platform>: HTTP status code is not handled or not allowed 2026-01-25 00:25:56 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/heaters/electric_heaters/overhead_infrared_heaters landed on page that is not a product page. 2026-01-25 00:25:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/incandescent_light_bulbs landed on page that is not a product page. 2026-01-25 00:25:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/cable_management/cable_clamps_clips_hardware landed on page that is not a product page. 2026-01-25 00:25:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rlp1-long-sleeve-pullover-t-shirt-with-uv-protection-class-3-xl-hi-vis-lime>: HTTP status code is not handled or not allowed 2026-01-25 00:26:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uhmw-polyethylene-plastic-rod-1-1-2-diameter-x-3-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 00:26:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:26:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ibc-2000i-73l-x-80-1-2w-x-29-1-2-x-29-1-2h>: HTTP status code is not handled or not allowed 2026-01-25 00:26:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/fluorescent_linear_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:26:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/prestige-plus-l-shaped-workstation-with-pedestal-white-chocolate-and-antigua>: HTTP status code is not handled or not allowed 2026-01-25 00:26:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 41603 pages (at 87 pages/min), scraped 20221 items (at 54 items/min) 2026-01-25 00:26:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock>: HTTP status code is not handled or not allowed 2026-01-25 00:26:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-sport-cone-yellow>: HTTP status code is not handled or not allowed 2026-01-25 00:26:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/fluorescent_linear_bulbs landed on page that is not a product page. 2026-01-25 00:26:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pld-u-bend-fluorescent-bulb-g24q2-base-18w-1080-lumens-2700k-warm-white-pk-of-10>: HTTP status code is not handled or not allowed 2026-01-25 00:26:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2531027-receptacle-locking-l15-30r>: HTTP status code is not handled or not allowed 2026-01-25 00:26:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:26:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ushio-egm-jcv120v-1000wbp28>: HTTP status code is not handled or not allowed 2026-01-25 00:26:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/decorative_lighting/flush_mount_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:26:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-welded-steel-container-blue-14>: HTTP status code is not handled or not allowed 2026-01-25 00:26:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/gloves_sleeves/cut_resistant_gloves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:26:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/verishield-153-behind-the-neck-ear-muffs-28-db-black>: HTTP status code is not handled or not allowed 2026-01-25 00:26:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 00:26:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/specialty_light_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:26:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/specialty_light_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:26:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-b11-ctc-light-bulb-e12-base-45w-300-lumens-2700k-warm-white-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 00:26:34 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/decorative_lighting/flush_mount_lights landed on page that is not a product page. 2026-01-25 00:26:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-container-forkli-able-with-double-wall-gray-1>: HTTP status code is not handled or not allowed 2026-01-25 00:26:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/gloves_sleeves/cut_resistant_gloves landed on page that is not a product page. 2026-01-25 00:26:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/881318-filter-drier-liquid-line-for-parker-hannifin>: HTTP status code is not handled or not allowed 2026-01-25 00:26:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/specialty_light_bulbs landed on page that is not a product page. 2026-01-25 00:26:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/specialty_light_bulbs landed on page that is not a product page. 2026-01-25 00:26:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-g1sc-150-b>: HTTP status code is not handled or not allowed 2026-01-25 00:26:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1381267-thermometer-digital-green>: HTTP status code is not handled or not allowed 2026-01-25 00:26:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:26:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-inch-w-push-broom-head-with-green-polypropylene-bristles-and-steel-frame>: HTTP status code is not handled or not allowed 2026-01-25 00:26:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/wiring_devices/energy_management_contactor_panels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:26:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-s14-string-filament-light-bulb-e26-base-2w-200-lumens-2200k-amber-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 00:26:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sofia-modern-pendant-light-fixture-e26-base-40w-brushed-nickel>: HTTP status code is not handled or not allowed 2026-01-25 00:26:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents landed on page that is not a product page. 2026-01-25 00:26:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elkay-ezh2o-dual-hands-free-bi-level-stainless-steel-refrigerated-water-bottle-filling-station>: HTTP status code is not handled or not allowed 2026-01-25 00:26:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/wiring_devices/energy_management_contactor_panels landed on page that is not a product page. 2026-01-25 00:27:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-g2sc-262-ss>: HTTP status code is not handled or not allowed 2026-01-25 00:27:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/761426-ql2-ow200l-system-for-everpure-filter>: HTTP status code is not handled or not allowed 2026-01-25 00:27:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-gsc-262-ss>: HTTP status code is not handled or not allowed 2026-01-25 00:27:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/garage_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:27:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/made-in-usa-a-2-air-hardening-flat-stock-1-2x3-16x18>: HTTP status code is not handled or not allowed 2026-01-25 00:27:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 41692 pages (at 89 pages/min), scraped 20283 items (at 62 items/min) 2026-01-25 00:27:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/56-1354-15-psi-bronze-steam-safety-relief-valve-3-4-inch-npt-450-lb-hour>: HTTP status code is not handled or not allowed 2026-01-25 00:27:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/38-1321-junction-box-light-fixture-glass-globe>: HTTP status code is not handled or not allowed 2026-01-25 00:27:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/umbrellas_awnings_shades/outdoor_umbrellas_bases/umbrellas already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:27:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/cabinets/garage_cabinets>: HTTP status code is not handled or not allowed 2026-01-25 00:27:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/venture-patio-umbrella-with-silver-pole-108w-olefin-lemon-fabric>: HTTP status code is not handled or not allowed 2026-01-25 00:27:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/solid-galvanized-shelf-60x18>: HTTP status code is not handled or not allowed 2026-01-25 00:27:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/outdoor_furniture/umbrellas_awnings_shades/outdoor_umbrellas_bases/umbrellas>: HTTP status code is not handled or not allowed 2026-01-25 00:27:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-step-steel-stock-picking-ladder-perforated-tread-w-cal-osha-handrail-cal-spl-9-p>: HTTP status code is not handled or not allowed 2026-01-25 00:27:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-channel-cable-guard>: HTTP status code is not handled or not allowed 2026-01-25 00:27:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-valve-5-port-2-pos-mech-valve-1-8-nptf-port-roller-plunger-actr-air-spring-ret>: HTTP status code is not handled or not allowed 2026-01-25 00:27:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-step-steel-stock-picking-ladder-abrasive-tread-w-cal-osha-handrail-cal-spl-6-r>: HTTP status code is not handled or not allowed 2026-01-25 00:27:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/newport-patio-umbrella-w-silver-pole-108w-sunbrella-1a-antique-beige-fabric>: HTTP status code is not handled or not allowed 2026-01-25 00:27:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/926-atg-adhesive-transfer-tape-1-2-x-18-yds-5-mil-clear>: HTTP status code is not handled or not allowed 2026-01-25 00:27:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bussing-tub-22-l-x-17-w-x-7-h-ribbed-bottom-bus-box-polypropylene-black>: HTTP status code is not handled or not allowed 2026-01-25 00:27:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/cable_management/cable_ties_wraps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:27:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/umbrellas_awnings_shades/outdoor_umbrellas_bases/umbrellas already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:27:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zip-a-duct-28in-gray-inlet-section>: HTTP status code is not handled or not allowed 2026-01-25 00:27:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-24-w-14-step-steel-rolling-ladder-21-d-top-step-w-handrail-lock-step-orange-w-cal-osha-handrail-cal-fs144021p-o>: HTTP status code is not handled or not allowed 2026-01-25 00:27:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/designer-series-deluxe-bath-bench-with-back-white>: HTTP status code is not handled or not allowed 2026-01-25 00:27:51 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/cable_management/cable_ties_wraps landed on page that is not a product page. 2026-01-25 00:27:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/outdoor_furniture/umbrellas_awnings_shades/outdoor_umbrellas_bases/umbrellas landed on page that is not a product page. 2026-01-25 00:27:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11-step-stock-picking-ladder-perf-step>: HTTP status code is not handled or not allowed 2026-01-25 00:27:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aah-36rbmag2-n-storage-cabinet-all-welded-36-wx18-dx78-h-navy>: HTTP status code is not handled or not allowed 2026-01-25 00:27:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-step-16-wx65-d-stainless-steel-rolling-safety-ladder-perforated-tread-w-cal-osha-handrail-cal-ss072421p>: HTTP status code is not handled or not allowed 2026-01-25 00:28:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-unit-plastic-box-with-gasket-empty>: HTTP status code is not handled or not allowed 2026-01-25 00:28:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/envy174-foaming-disinfectant-cleaner-lavender-scent-19-oz-aerosol-spray-12case>: HTTP status code is not handled or not allowed 2026-01-25 00:28:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oxygen-cylinder-rack-use-with-28-cylinder-e-d-c-or-m9-rack>: HTTP status code is not handled or not allowed 2026-01-25 00:28:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 41787 pages (at 95 pages/min), scraped 20354 items (at 71 items/min) 2026-01-25 00:28:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-fold-portable-wheelchair-scooter-ramp-w-carry-handle-4-foot-30-inch>: HTTP status code is not handled or not allowed 2026-01-25 00:28:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dmr-digital-handheld-radio-4-watts-uhf-400-470-mhz-ip65-approved-1000-channels>: HTTP status code is not handled or not allowed 2026-01-25 00:28:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/material-handling/cranes_hoists/crane_parts_accessories/lifting_slings already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:28:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/task_desk_chairs/task_office_chairs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:28:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/plastic_laminate_bathroom_partitions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:28:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-closed-linear-bearing-wseals-sw4uu-14id-075l>: HTTP status code is not handled or not allowed 2026-01-25 00:28:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drum-truck-with-four-wheels-30-55-gallon-drums-2>: HTTP status code is not handled or not allowed 2026-01-25 00:28:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plain-magnetic-roll-stock-100l-x-2-14h-x-003-thick-black>: HTTP status code is not handled or not allowed 2026-01-25 00:28:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/material-handling/cranes_hoists/crane_parts_accessories/lifting_slings landed on page that is not a product page. 2026-01-25 00:28:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/plastic_laminate_bathroom_partitions>: HTTP status code is not handled or not allowed 2026-01-25 00:28:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/task_desk_chairs/task_office_chairs landed on page that is not a product page. 2026-01-25 00:28:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-pilaster-w-shoe-6w-x-82h-charcoal>: HTTP status code is not handled or not allowed 2026-01-25 00:28:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bishamon-mobilift-battery-powered-scissor-lift-table-1760lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 00:28:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-pilaster-w-shoe-12w-x-82h-khaki>: HTTP status code is not handled or not allowed 2026-01-25 00:28:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-squeegee-front-3-16-flat-ure-for-minuteman-172162>: HTTP status code is not handled or not allowed 2026-01-25 00:28:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/120-inch-racetrack-conference-table-in-espresso-executive-modular-furniture>: HTTP status code is not handled or not allowed 2026-01-25 00:28:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/driving-defensively-employee-booklet>: HTTP status code is not handled or not allowed 2026-01-25 00:28:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-signs-caution-wear-safety-shoes-rigid-plastic-10-h-x-14-w>: HTTP status code is not handled or not allowed 2026-01-25 00:28:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1815-30-hss-heavy-duty-steam-oxide-135-double-end-drill>: HTTP status code is not handled or not allowed 2026-01-25 00:28:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacment-brush-mount-for-nobles-tennant-9014664>: HTTP status code is not handled or not allowed 2026-01-25 00:28:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b151136-6-double-leg-chain-sling-w-sling-hook>: HTTP status code is not handled or not allowed 2026-01-25 00:28:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1889-5-8-6in-oal-hss-heavy-duty-bright-118-point-fast-helix-carbide-tipped-masonry-drill>: HTTP status code is not handled or not allowed 2026-01-25 00:28:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s152053-4-poly-web-sling-endless-2500-3200-6400-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-25 00:28:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-squeegee-front-for-ice-8311534>: HTTP status code is not handled or not allowed 2026-01-25 00:28:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/narrow-mouth-rtk-wash-bottles-500ml-isopropanol-24-case>: HTTP status code is not handled or not allowed 2026-01-25 00:28:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-lightweight-long-sleeve-flame-resistant-safety-shirt-3xl-navy-blue>: HTTP status code is not handled or not allowed 2026-01-25 00:28:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/guest_chairs/guest_reception_chairs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:29:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacment-cellulose-fiber-dust-panel-filter-for-nobles-tennant-1037206am>: HTTP status code is not handled or not allowed 2026-01-25 00:29:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/tables/laboratory_tables already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:29:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-vis-flame-resistant-safety-cargo-pant-36-waist-x-32l-inseam-navy>: HTTP status code is not handled or not allowed 2026-01-25 00:29:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-flame-resistant-button-up-safety-shirt-cottonnylon-l-gray>: HTTP status code is not handled or not allowed 2026-01-25 00:29:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/guest_chairs/guest_reception_chairs landed on page that is not a product page. 2026-01-25 00:29:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 41881 pages (at 94 pages/min), scraped 20421 items (at 67 items/min) 2026-01-25 00:29:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ppb-1r-pour-w-red-spout-lid-1-qt>: HTTP status code is not handled or not allowed 2026-01-25 00:29:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/psb-12r-squeeze-bottles-12-oz>: HTTP status code is not handled or not allowed 2026-01-25 00:29:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/51083-thor-safety-glasses-black-frameinoutdoor-lens>: HTTP status code is not handled or not allowed 2026-01-25 00:29:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/tables/laboratory_tables landed on page that is not a product page. 2026-01-25 00:29:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/abc-industries-thermohose-1399-heater-duct-8-dia-x-16l-x-4-pitch-yellowblack>: HTTP status code is not handled or not allowed 2026-01-25 00:29:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40201-2-unv-multi-purpose-life-vest-blue-universal-adult>: HTTP status code is not handled or not allowed 2026-01-25 00:29:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/led_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:29:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/psw-12r-wide-mouth-squeeze-bottles-12-oz>: HTTP status code is not handled or not allowed 2026-01-25 00:29:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/w632-beverage-server-32-oz-stainless-steel-5-1-2h>: HTTP status code is not handled or not allowed 2026-01-25 00:29:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o-ir-higher-performance-polyolefin-cf-shrink-film-100-ga-16w-x-2620l-clear-1-roll>: HTTP status code is not handled or not allowed 2026-01-25 00:29:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/under-sink-mounted-utility-sink-pump-system>: HTTP status code is not handled or not allowed 2026-01-25 00:29:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-x-16-x-16-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-25 00:29:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-canvas-ce-phenolic-rod-0-375-dia-x-48-000-inch>: HTTP status code is not handled or not allowed 2026-01-25 00:29:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/led_bulbs landed on page that is not a product page. 2026-01-25 00:29:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-x-10-x-8-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-25 00:29:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-climax-2c-218-z>: HTTP status code is not handled or not allowed 2026-01-25 00:29:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-oil-filled-cast-nylon-sheet-2-000-x-24-000-x-48-000>: HTTP status code is not handled or not allowed 2026-01-25 00:29:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-uno-pressure-gauge-1-4-npt-dry-0-3000-psi-lower-mount-501d-uno-204p>: HTTP status code is not handled or not allowed 2026-01-25 00:29:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-x-14-x-6-4-2-multi-depth-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-25 00:29:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/63874-aware-wear-hi-vis-1-4-zipper-sweatshirt-class-3-hi-vis-lime-black-3xl>: HTTP status code is not handled or not allowed 2026-01-25 00:29:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_tools/brooms_dust_pans/dusters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:29:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/plastic_laminate_bathroom_partitions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:29:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/patriot-fully-framed-ta-50-locker-wlouvered-doors-36wx24dx78h-blueall-welded>: HTTP status code is not handled or not allowed 2026-01-25 00:29:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/air_conditioners_chillers/ductless_air_conditioners/ductless_full_systems already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:29:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/plastic_laminate_bathroom_partitions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:30:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/integra-office-chair-high-back-no-tilt-18-23h-seat-black>: HTTP status code is not handled or not allowed 2026-01-25 00:30:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/040-0445rp-two-stage-compressor-pump-inline-twin-cylinder-75-rhp>: HTTP status code is not handled or not allowed 2026-01-25 00:30:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/plastic_laminate_bathroom_partitions>: HTTP status code is not handled or not allowed 2026-01-25 00:30:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-linen-phenolic-sheet-2-000-thick-36-x-48>: HTTP status code is not handled or not allowed 2026-01-25 00:30:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/air_conditioners_chillers/ductless_air_conditioners/ductless_full_systems>: HTTP status code is not handled or not allowed 2026-01-25 00:30:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/plastic_laminate_bathroom_partitions landed on page that is not a product page. 2026-01-25 00:30:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 41976 pages (at 95 pages/min), scraped 20490 items (at 69 items/min) 2026-01-25 00:30:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-x-18-black-mailing-tubes-with-caps-case>: HTTP status code is not handled or not allowed 2026-01-25 00:30:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stretch-wrap-film-for-hand-dispenser-blown-60-gauge-15w-x-2000l-clear-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-25 00:30:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-paper-phenolic-sheet-0-500-thick-36-x-48>: HTTP status code is not handled or not allowed 2026-01-25 00:30:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_tools/brooms_dust_pans/dusters landed on page that is not a product page. 2026-01-25 00:30:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/patriot-fully-framed-ta-50-locker-wsolid-doors-sides-42wx24dx78h-redunassembled>: HTTP status code is not handled or not allowed 2026-01-25 00:30:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelfsecurity-boxvented24wx21dx72hrdall-welded>: HTTP status code is not handled or not allowed 2026-01-25 00:30:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3m-red-buffer-pad-5100-12-in-5-case>: HTTP status code is not handled or not allowed 2026-01-25 00:30:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3m-topline-speed-burnish-pad-3200-20-in-5-case>: HTTP status code is not handled or not allowed 2026-01-25 00:30:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qf-clamp-with-pin-4-dia-304-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 00:30:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qf-hose-adapter-10-dia-galvanized-steel>: HTTP status code is not handled or not allowed 2026-01-25 00:30:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0386r-iron-tuff-enhanced-visibility-high-bib-overall-regular-navy-5xl>: HTTP status code is not handled or not allowed 2026-01-25 00:30:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/integra-vinyl-task-chair-high-back-articulating-tilt-25-35h-seat-black>: HTTP status code is not handled or not allowed 2026-01-25 00:30:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sit-stand-stool-black-brody-series>: HTTP status code is not handled or not allowed 2026-01-25 00:30:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelf-24wx24dx72h-red-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 00:30:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelf-14x54-nexelon>: HTTP status code is not handled or not allowed 2026-01-25 00:30:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-climax-1c-012-z>: HTTP status code is not handled or not allowed 2026-01-25 00:30:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0051r-3-hole-mask-black>: HTTP status code is not handled or not allowed 2026-01-25 00:30:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-fr-breathing-tube-cover>: HTTP status code is not handled or not allowed 2026-01-25 00:30:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:30:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelf-footlocker-vented-24wx24dx72hgrall-welded>: HTTP status code is not handled or not allowed 2026-01-25 00:30:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=jtm-4vs-1-mill-with-3-axis-acu-rite-g-2-millpwr-cnc already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:30:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelf-21x42-nexelon>: HTTP status code is not handled or not allowed 2026-01-25 00:30:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 00:30:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-coupling-climax-rc-125-s>: HTTP status code is not handled or not allowed 2026-01-25 00:30:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=jtm-4vs-1-mill-with-3-axis-acu-rite-g-2-millpwr-cnc landed on page that is not a product page. 2026-01-25 00:30:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interion-174-outdoor-caf-steel-mesh-stacking-armchair-gray-4-pack>: HTTP status code is not handled or not allowed 2026-01-25 00:30:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelate-silver-epoxy-wire-shelf-72x24>: HTTP status code is not handled or not allowed 2026-01-25 00:30:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0033-11-oxford-table-fork>: HTTP status code is not handled or not allowed 2026-01-25 00:30:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-one-piece-industry-standard-clamping-couplings-w-keyway-climax-miscc-20-20-kw>: HTTP status code is not handled or not allowed 2026-01-25 00:31:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/industrial_cleaners_degreasers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:31:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0323-iron-tuff-womans-coat-regular-navy-large>: HTTP status code is not handled or not allowed 2026-01-25 00:31:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powerquad153-vacuum-kit-with-2-inlet-30-gallon-capacity>: HTTP status code is not handled or not allowed 2026-01-25 00:31:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vinyl-top-folding-table-48-x-20-x-29-inch-beige>: HTTP status code is not handled or not allowed 2026-01-25 00:31:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 42073 pages (at 97 pages/min), scraped 20557 items (at 67 items/min) 2026-01-25 00:31:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-zlink-rear-cape-clip>: HTTP status code is not handled or not allowed 2026-01-25 00:31:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0322r-iron-tuff-polar-jacket-regular-navy-xl>: HTTP status code is not handled or not allowed 2026-01-25 00:31:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/industrial_cleaners_degreasers landed on page that is not a product page. 2026-01-25 00:31:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/azar-700913-bowl-for-pegboard-slatwall-12-w-x-13-1-4-d-x-7-1-2-h>: HTTP status code is not handled or not allowed 2026-01-25 00:31:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:31:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0358r-iron-tuff-siberian-jacket-tall-navy-xl>: HTTP status code is not handled or not allowed 2026-01-25 00:31:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lifting-manual-chain-hoist-w-overload-protection-1-ton-capacity-30-lift>: HTTP status code is not handled or not allowed 2026-01-25 00:31:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-recessed-screw-climax-h1c-287-s>: HTTP status code is not handled or not allowed 2026-01-25 00:31:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats>: HTTP status code is not handled or not allowed 2026-01-25 00:31:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cks-13-cake-stand>: HTTP status code is not handled or not allowed 2026-01-25 00:31:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dxs2153-metric-standard-impact-socket-22mm-drive-black>: HTTP status code is not handled or not allowed 2026-01-25 00:31:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/surge_protectors_extension_cords/extension_cords already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:31:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-sjt-power-supply-replacement-cord-16awg-13a-125v-gray>: HTTP status code is not handled or not allowed 2026-01-25 00:31:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-steel-bookcase-3-openings-36-w-x-12-d-x-42-h-black>: HTTP status code is not handled or not allowed 2026-01-25 00:31:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/lathes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:31:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1145-2-mini-heavy-duty-regulator-18-npt-250-psi-gauge-not-included>: HTTP status code is not handled or not allowed 2026-01-25 00:31:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/landscaping/planters_pots/planters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:31:44 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/surge_protectors_extension_cords/extension_cords landed on page that is not a product page. 2026-01-25 00:31:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-strip-with-acrylic-adhesive-60a-14-thick-x-2-wide-x-5-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 00:31:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:31:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-tier-shelving-starter-unit-solid-galvanized-steel-42w-x-18d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 00:31:49 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/lathes landed on page that is not a product page. 2026-01-25 00:31:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-green-2-tier-wire-shelving-starter-unit-36-w-x-18-d-x-34-h>: HTTP status code is not handled or not allowed 2026-01-25 00:31:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips>: HTTP status code is not handled or not allowed 2026-01-25 00:31:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/landscaping/planters_pots/planters landed on page that is not a product page. 2026-01-25 00:31:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-green-2-tier-wire-shelving-add-on-unit-72-w-x-21-d-x-34-h>: HTTP status code is not handled or not allowed 2026-01-25 00:32:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-tier-shelving-starter-unit-solid-galvanized-steel-42w-x-24d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 00:32:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kcl-2-oyster-clam-knife>: HTTP status code is not handled or not allowed 2026-01-25 00:32:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:32:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mxb-300q-mixing-bowl-3-qt-9-1-2d>: HTTP status code is not handled or not allowed 2026-01-25 00:32:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mesh-task-chair-fabric-seat-white-frame>: HTTP status code is not handled or not allowed 2026-01-25 00:32:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 42166 pages (at 93 pages/min), scraped 20625 items (at 68 items/min) 2026-01-25 00:32:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-left-side-upright-for-rubbermaid-janitor-cart>: HTTP status code is not handled or not allowed 2026-01-25 00:32:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:32:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vinyl-replacement-liner-16-bushel-green-2>: HTTP status code is not handled or not allowed 2026-01-25 00:32:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:32:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compact-efficient-heavy-duty-power-cord-reel-12-ga>: HTTP status code is not handled or not allowed 2026-01-25 00:32:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:32:18 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-25 00:32:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:32:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conductive-tray-17-3-4x11-3-4x1>: HTTP status code is not handled or not allowed 2026-01-25 00:32:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips landed on page that is not a product page. 2026-01-25 00:32:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips>: HTTP status code is not handled or not allowed 2026-01-25 00:32:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20076-depressed-center-wheel-t27-a24r-6-inch-x-1-4-inch-x-5-8-11-inch-24-grit-alum-oxide>: HTTP status code is not handled or not allowed 2026-01-25 00:32:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-25 00:32:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-roll-no-adhesive-50a-12-thick-x-36-wide-x-10-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 00:32:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-25 00:32:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-bv-6i-bench-vise-with-pipe-jaws-6-12-jaw-width-3-1316-throat-depth>: HTTP status code is not handled or not allowed 2026-01-25 00:32:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/suit-hanger-rubber-white-50-pack>: HTTP status code is not handled or not allowed 2026-01-25 00:32:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_seating/bleachers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:32:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:32:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50161-3a-fiber-disc-4-1-2-inch-x-7-8-inch-60-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-25 00:32:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/scales/portion_price_computing_scales already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:32:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/binders_covers/binders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:32:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adapter-for-pz-750-and-758c-enabled-scales>: HTTP status code is not handled or not allowed 2026-01-25 00:32:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-strip-no-adhesive-60a-18-thick-x-2-wide-x-10-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 00:32:44 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_seating/bleachers landed on page that is not a product page. 2026-01-25 00:32:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/binders_covers/binders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:32:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips landed on page that is not a product page. 2026-01-25 00:32:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/scales/portion_price_computing_scales>: HTTP status code is not handled or not allowed 2026-01-25 00:32:50 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/binders_covers/binders landed on page that is not a product page. 2026-01-25 00:32:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-sheet-no-adhesive-70a-116-thick-x-36-wide-x-36-long>: HTTP status code is not handled or not allowed 2026-01-25 00:32:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:32:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:32:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/binders_covers/binders>: HTTP status code is not handled or not allowed 2026-01-25 00:32:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:32:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powerquad153-510-gallon-wire-cage-coalescing-filter-black>: HTTP status code is not handled or not allowed 2026-01-25 00:33:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:33:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets>: HTTP status code is not handled or not allowed 2026-01-25 00:33:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:33:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-25 00:33:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-wheel-belt-grinder-2-w-x-48-l-single-phase-110v-bg-248-2>: HTTP status code is not handled or not allowed 2026-01-25 00:33:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:33:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-25 00:33:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n-7342-closet-door-pull-with-1-38-inch-solid-brass-satin-nickelpack-of-2>: HTTP status code is not handled or not allowed 2026-01-25 00:33:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=10-industrial-bench-grinder-single-phase-115v-igb-10 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:33:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 42250 pages (at 84 pages/min), scraped 20681 items (at 56 items/min) 2026-01-25 00:33:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets>: HTTP status code is not handled or not allowed 2026-01-25 00:33:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-25 00:33:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/three-wheel-single-speed-belt-grinder-2-w-x-48-l-single-phase-115v-bg-248s>: HTTP status code is not handled or not allowed 2026-01-25 00:33:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-25 00:33:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:33:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-tarp-system-with-mesh-tarp-7-1-2-ft-x-20-ft-dtr7520>: HTTP status code is not handled or not allowed 2026-01-25 00:33:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-lid-114001>: HTTP status code is not handled or not allowed 2026-01-25 00:33:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=10-industrial-bench-grinder-single-phase-115v-igb-10 landed on page that is not a product page. 2026-01-25 00:33:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:33:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pem-aluminum-door-sweep-w-nylon-insert-36-clear-anodized-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 00:33:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:33:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips>: HTTP status code is not handled or not allowed 2026-01-25 00:33:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-mesh-panel-3x10>: HTTP status code is not handled or not allowed 2026-01-25 00:33:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leeson-3-phase-explosion-proof-motor-2hp-1800rpm-145t-epfc-230-460v-60hz-40c-1-15sf-rigid>: HTTP status code is not handled or not allowed 2026-01-25 00:33:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls>: HTTP status code is not handled or not allowed 2026-01-25 00:33:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets>: HTTP status code is not handled or not allowed 2026-01-25 00:33:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pem-aluminum-saddle-threshold-36-l-x-8-w-1-4-h-mill-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 00:33:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/notchers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:33:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/313-carton-sealing-tape-2-x-55-yds-25-mil-clear-6pack>: HTTP status code is not handled or not allowed 2026-01-25 00:33:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:33:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=manually-operated-non-mitering-pipe-notcher-for-1-1-2-2-schedule-40-pipe already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:33:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:33:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-garment-mailers-28-34l-x-16w-x-5h-kraft-25pack>: HTTP status code is not handled or not allowed 2026-01-25 00:33:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/anchors_hangers/expansion_anchors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:33:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wagon-truck-lip-deck-36-x-60-pneumatic-wheels-3000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 00:33:44 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/notchers landed on page that is not a product page. 2026-01-25 00:33:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets>: HTTP status code is not handled or not allowed 2026-01-25 00:33:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=manually-operated-non-mitering-pipe-notcher-for-1-1-2-2-schedule-40-pipe landed on page that is not a product page. 2026-01-25 00:33:49 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips landed on page that is not a product page. 2026-01-25 00:33:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:33:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carton-sealing-tape-keep-refrigerated-3-x-110-yds-22-mil-redwhite-6pack>: HTTP status code is not handled or not allowed 2026-01-25 00:33:51 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/anchors_hangers/expansion_anchors landed on page that is not a product page. 2026-01-25 00:33:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/lockers/locker_accessories/locker_locks_keys already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:33:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-strip-no-adhesive-60a-38-thick-x-2-wide-x-5-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 00:33:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hx1-18-c45-steel-black-oxide-split-taper-bushing-1-18-inch-bore>: HTTP status code is not handled or not allowed 2026-01-25 00:33:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:33:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-25 00:33:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss3236-stainless-steel-36-tooth-1-125-pitch-spur-gear>: HTTP status code is not handled or not allowed 2026-01-25 00:34:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/lockers/locker_accessories/locker_locks_keys landed on page that is not a product page. 2026-01-25 00:34:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mid-size-stainless-steel-spray-nozzle-black>: HTTP status code is not handled or not allowed 2026-01-25 00:34:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/file_cabinets/blueprint_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:34:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96-x-30-extra-long-bench-maple-top-9>: HTTP status code is not handled or not allowed 2026-01-25 00:34:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:34:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-25 00:34:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluidity-03f-hung-diaphragm-solenoid-valve-2-2-nc-epdm-seal-3-4-nptf-18-mm-12v-dc>: HTTP status code is not handled or not allowed 2026-01-25 00:34:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 42336 pages (at 86 pages/min), scraped 20735 items (at 54 items/min) 2026-01-25 00:34:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vinyl-heavy-duty-ribbed-stair-tread-square-nose-125-x-42-black>: HTTP status code is not handled or not allowed 2026-01-25 00:34:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/cabinets/file_cabinets/blueprint_cabinets landed on page that is not a product page. 2026-01-25 00:34:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips>: HTTP status code is not handled or not allowed 2026-01-25 00:34:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thermoplastic-rubber-wall-base-25-in-x-48-in-black-brown>: HTTP status code is not handled or not allowed 2026-01-25 00:34:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluidity-03f-hung-diaphragm-solenoid-valve-2-2-nc-epdm-seal-1-nptf-25-mm-12v-dc>: HTTP status code is not handled or not allowed 2026-01-25 00:34:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-410-nipple-11-mm-x-16-mm-streamline-connection-eurostandard-10-4>: HTTP status code is not handled or not allowed 2026-01-25 00:34:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-sheet-no-adhesive-50a-132-thick-x-36-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 00:34:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/advantage-pro-max-pallet-jack-scale-truck-27w-x-62l-4400-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 00:34:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/j53129-comfortbrite-jacket-fluorescent-orange-sm>: HTTP status code is not handled or not allowed 2026-01-25 00:34:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/j31107-webdri-storm-fly-front-hooded-jacket-yellow-sm>: HTTP status code is not handled or not allowed 2026-01-25 00:34:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-tag-pre-strung-4-4-14l-x-2-18w-white-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 00:34:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slatwall-counter-top-cube-display-fixture-birch-w-spinner-base>: HTTP status code is not handled or not allowed 2026-01-25 00:34:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/packaging/labels_barcode_equipment/tags/blank_shipping_tags already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:34:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6x1x1-a36-o-v-bench-wheel-1-pk>: HTTP status code is not handled or not allowed 2026-01-25 00:34:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12x11-2x5-71-2x1-2-wa46-j-v-white-aluminum-oxi>: HTTP status code is not handled or not allowed 2026-01-25 00:34:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/abrasives/sanding_discs/flap_sanding_discs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:34:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tyvek174-shipping-tag-2-34l-x-1-38w-white-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 00:34:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s75522-class-3-long-sleeve-tshirt-fluorescent-yellowgreen-sm>: HTTP status code is not handled or not allowed 2026-01-25 00:34:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/packaging/labels_barcode_equipment/tags/blank_shipping_tags landed on page that is not a product page. 2026-01-25 00:34:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/58a-black-original-laserjet-toner-cartridge-3000-page-yield>: HTTP status code is not handled or not allowed 2026-01-25 00:34:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3m-s-b-6x6x1-8s-med048011-18278>: HTTP status code is not handled or not allowed 2026-01-25 00:34:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-tag-12-8l-x-4w-orange-500pack>: HTTP status code is not handled or not allowed 2026-01-25 00:34:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/abrasives/sanding_discs/flap_sanding_discs landed on page that is not a product page. 2026-01-25 00:35:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-level-60x36-high-capacity-rack-steel-deck>: HTTP status code is not handled or not allowed 2026-01-25 00:35:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-cabinet-2-door-36-inch-w-x-18-inch-d-x-35-inch-h>: HTTP status code is not handled or not allowed 2026-01-25 00:35:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-sheet-no-adhesive-70a-14-thick-x-36-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 00:35:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/j22201-iron-eagle-storm-fly-front-jacket-blue-hood-snaps-lg>: HTTP status code is not handled or not allowed 2026-01-25 00:35:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 42429 pages (at 93 pages/min), scraped 20803 items (at 68 items/min) 2026-01-25 00:35:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:35:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/printed-secure-tape-roll-opened-2-x-60-yds-25-mil-blue>: HTTP status code is not handled or not allowed 2026-01-25 00:35:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arrow-pneumatics-ar-40-a-non-cycling-refrigerated-air-dryer-40-cfm-1-ph-115-230v>: HTTP status code is not handled or not allowed 2026-01-25 00:35:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/del-serrated-nozzle-8-l-yellow>: HTTP status code is not handled or not allowed 2026-01-25 00:35:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls>: HTTP status code is not handled or not allowed 2026-01-25 00:35:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-his-sheet-0-187-thick-x-48-w-x-96-l>: HTTP status code is not handled or not allowed 2026-01-25 00:35:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reverse-tuck-folding-cartons-2-12l-x-1-34w-x-4h-white-500pack>: HTTP status code is not handled or not allowed 2026-01-25 00:35:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s62217-tuffenuff-plus-2-pc-suit-yellow-sm>: HTTP status code is not handled or not allowed 2026-01-25 00:35:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/l1001002-man-rope-hll-system-with-cross-arm-strap-and-rope-tensioner-100-feet-long>: HTTP status code is not handled or not allowed 2026-01-25 00:35:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/work-benches/workbenches_worktables/open_base_workbenches/stationary_open_base_workbenches/heavy_duty_stationary_workbenches already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:35:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spring-lift-platform-20-bushel-plastic-box-truck>: HTTP status code is not handled or not allowed 2026-01-25 00:35:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_shelving already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:35:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/red-gpo-3-sheet-0-500-thick-x-48-000-w-x-96-000-l>: HTTP status code is not handled or not allowed 2026-01-25 00:35:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hang-stack-bins-w-two-12-rails-six-bins-4-1-8w-x-7-3-8d-x-3h-ivory>: HTTP status code is not handled or not allowed 2026-01-25 00:35:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/work-benches/workbenches_worktables/open_base_workbenches/stationary_open_base_workbenches/heavy_duty_stationary_workbenches landed on page that is not a product page. 2026-01-25 00:35:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reverse-tuck-folding-cartons-4-1316l-x-1-14w-x-4-1316h-kraft-250pack>: HTTP status code is not handled or not allowed 2026-01-25 00:35:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/walls_ceilings/ceiling_tiles already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:35:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_shelving>: HTTP status code is not handled or not allowed 2026-01-25 00:35:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/truck_boxes_utility_boxes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:35:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-89-2309-cobalt-drill-bit-3-16>: HTTP status code is not handled or not allowed 2026-01-25 00:35:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/54h-nexel-chrome-wire-shelving-36wx21d>: HTTP status code is not handled or not allowed 2026-01-25 00:35:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/great-lakes-tin-decorative-metal-ceiling-tile-nails-in-copper-100-pack-tnu-11>: HTTP status code is not handled or not allowed 2026-01-25 00:35:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/truck_boxes_utility_boxes>: HTTP status code is not handled or not allowed 2026-01-25 00:35:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/walls_ceilings/ceiling_tiles landed on page that is not a product page. 2026-01-25 00:35:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-of-2-side-shelves-gray>: HTTP status code is not handled or not allowed 2026-01-25 00:36:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/walls_ceilings/ceiling_tiles already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:36:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-surface-cleaner-disinfectant-concentrated-pine-scent-14-oz-bottle>: HTTP status code is not handled or not allowed 2026-01-25 00:36:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/biwm2424-24-x-24-drywall-bead-flange-mortise-lock-prep>: HTTP status code is not handled or not allowed 2026-01-25 00:36:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/illinois-engineered-products-single-folding-gate-11w-to-12w-and-7h>: HTTP status code is not handled or not allowed 2026-01-25 00:36:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 42519 pages (at 90 pages/min), scraped 20870 items (at 67 items/min) 2026-01-25 00:36:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/wire_shelving/poly_zbrite_wire_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:36:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-carbon-steel-utility-blades-pack-of-20>: HTTP status code is not handled or not allowed 2026-01-25 00:36:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jamaica-beach-adj-sling-chaise-lounge-straw-sling-white-resin-frame-pack-of-16>: HTTP status code is not handled or not allowed 2026-01-25 00:36:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylon-brush-for-14-id-tubes-blue-pack-of-15>: HTTP status code is not handled or not allowed 2026-01-25 00:36:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/walls_ceilings/ceiling_tiles landed on page that is not a product page. 2026-01-25 00:36:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/panel-kit-72-w-workbench-with-18-w-36-w-pegboards-mounting-rail-gray>: HTTP status code is not handled or not allowed 2026-01-25 00:36:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/plumbing_fittings/pipe_clamps_fittings/brass_pipe_fittings already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:36:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/intensive-care-essential-healing-body-lotion-20-3-oz-pump-bottle>: HTTP status code is not handled or not allowed 2026-01-25 00:36:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/wire_shelving/poly_zbrite_wire_shelves landed on page that is not a product page. 2026-01-25 00:36:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3500-series-4b-vertical-mailbox-3-doors-recessed-mounted-bronze-usps-access>: HTTP status code is not handled or not allowed 2026-01-25 00:36:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/westport-16-square-resin-low-table-barn-red>: HTTP status code is not handled or not allowed 2026-01-25 00:36:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/needle-bearing-ta1015z-1230r-5235931>: HTTP status code is not handled or not allowed 2026-01-25 00:36:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/plumbing/plumbing_fittings/pipe_clamps_fittings/brass_pipe_fittings landed on page that is not a product page. 2026-01-25 00:36:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-bath-tissue-roll-with-opticore-septic-safe-white-1755-sheets-roll-36-case>: HTTP status code is not handled or not allowed 2026-01-25 00:36:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slip-on-platform-aluminum-w-li-attachment>: HTTP status code is not handled or not allowed 2026-01-25 00:36:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3400-bearing-bushing-5517511>: HTTP status code is not handled or not allowed 2026-01-25 00:36:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/access_doors_panels/roof_hatches already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:36:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flammable-cabinet-with-self-close-double-door-30-gallon>: HTTP status code is not handled or not allowed 2026-01-25 00:36:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-standard-particulate-filter-w-metal-bowl-float-drain>: HTTP status code is not handled or not allowed 2026-01-25 00:36:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vfswdnm3558t-e-ip69-2hp-1750rpm-3ph-60hz-145tc-3546m-tenv-f1>: HTTP status code is not handled or not allowed 2026-01-25 00:36:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/doors_windows/access_doors_panels/roof_hatches landed on page that is not a product page. 2026-01-25 00:36:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/switch-enclosure-5711521>: HTTP status code is not handled or not allowed 2026-01-25 00:36:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/lockers/bulk_storage_lockers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:36:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-for-dispatcher-locker-for-d1-series-in-the-2-mid-shelf-locations-36w-x-24d>: HTTP status code is not handled or not allowed 2026-01-25 00:36:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sd-600-1-utility-tailgate-drop-spreader-6-cu-ft>: HTTP status code is not handled or not allowed 2026-01-25 00:36:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-x-2-drywall-screw-phillips-bugle-head-steel-black-phosphate-sharp-point-pkg-of-1000>: HTTP status code is not handled or not allowed 2026-01-25 00:36:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/lockers/bulk_storage_lockers landed on page that is not a product page. 2026-01-25 00:37:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deep-woods-sportsmen-insect-repellent-1-oz-spray-bottle>: HTTP status code is not handled or not allowed 2026-01-25 00:37:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/indestructable-too-folding-table-30x96-bifold-charcoal-1200-series>: HTTP status code is not handled or not allowed 2026-01-25 00:37:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-vantage-multi-process-welder-566x-deutz-600a-60-hz-silver>: HTTP status code is not handled or not allowed 2026-01-25 00:37:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 42613 pages (at 94 pages/min), scraped 20938 items (at 68 items/min) 2026-01-25 00:37:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tennsco-capstone-boltless-shelving-add-on-unit-42-w-x-24-d-x-76-h-5-shelves-medium-grey>: HTTP status code is not handled or not allowed 2026-01-25 00:37:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/medical_laboratory_cold_storage/laboratory_cold_storage/laboratory_cold_storage_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:37:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/tables/cafeteria_restaurant_bar_tables already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:37:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8oz-natural-hdpe-wide-mouth-round-bottle-48-400-neck-finish-250-pack>: HTTP status code is not handled or not allowed 2026-01-25 00:37:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/38387-clamp-and-multimeter-greenorange>: HTTP status code is not handled or not allowed 2026-01-25 00:37:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/medical_laboratory_cold_storage/laboratory_cold_storage/laboratory_cold_storage_accessories>: HTTP status code is not handled or not allowed 2026-01-25 00:37:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/dollies/open_deck_dollies already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:37:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/tables/cafeteria_restaurant_bar_tables>: HTTP status code is not handled or not allowed 2026-01-25 00:37:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m12-fuel-12-volt-lithium-ion-brushless-cordless-1-4-hex-impact-driver-tool-only>: HTTP status code is not handled or not allowed 2026-01-25 00:37:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fan-assembly-kit-4-2200-cfm>: HTTP status code is not handled or not allowed 2026-01-25 00:37:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/dollies/open_deck_dollies landed on page that is not a product page. 2026-01-25 00:37:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8210hl-class-2-economy-vest-lime-s-m>: HTTP status code is not handled or not allowed 2026-01-25 00:37:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/retainer-5782841>: HTTP status code is not handled or not allowed 2026-01-25 00:37:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_bottles_jars already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:37:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/food-safe-vfswdl3506-ip69-75hp-3450rpm-1ph-60hz-56c-3516l-tefc-f1>: HTTP status code is not handled or not allowed 2026-01-25 00:37:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/food-safe-vfswdm3541-e-ip69-75hp-3500rpm-3ph-60hz-56c-3520m-tenv-f1>: HTTP status code is not handled or not allowed 2026-01-25 00:37:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_carboys_jerricans_jugs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:37:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-breathable-bib-pants-hi-viz-yellow-small>: HTTP status code is not handled or not allowed 2026-01-25 00:37:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:37:49 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_bottles_jars landed on page that is not a product page. 2026-01-25 00:37:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-7w-x-30l-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 00:37:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crew-sweatshirt-hi-vis-yellow-medium>: HTTP status code is not handled or not allowed 2026-01-25 00:37:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:37:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_bottles_jars already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:37:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-flame-resistant-pull-over-hoodie-hi-viz-yellow-large>: HTTP status code is not handled or not allowed 2026-01-25 00:37:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_carboys_jerricans_jugs landed on page that is not a product page. 2026-01-25 00:37:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions>: HTTP status code is not handled or not allowed 2026-01-25 00:37:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/emergency_vehicle_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:37:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-parka-hi-viz-yellow-large>: HTTP status code is not handled or not allowed 2026-01-25 00:38:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_carboys_jerricans_jugs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:38:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions landed on page that is not a product page. 2026-01-25 00:38:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_bottles_jars landed on page that is not a product page. 2026-01-25 00:38:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pulley-cover-assy-jdp-17mmffse-10909012a1>: HTTP status code is not handled or not allowed 2026-01-25 00:38:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_repair_maintenance/horns_horn_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:38:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lateral-file-letter-legal-size-fireproof-4-drawers-black-2>: HTTP status code is not handled or not allowed 2026-01-25 00:38:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ring-power-feeder-6288966>: HTTP status code is not handled or not allowed 2026-01-25 00:38:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/staplers_staples/staplers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:38:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 42695 pages (at 82 pages/min), scraped 20995 items (at 57 items/min) 2026-01-25 00:38:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_batteries/vehicle_battery_accessories/vehicle_battery_boxes_trays already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:38:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_carboys_jerricans_jugs>: HTTP status code is not handled or not allowed 2026-01-25 00:38:12 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-25 00:38:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/emergency_vehicle_lights landed on page that is not a product page. 2026-01-25 00:38:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_repair_maintenance/horns_horn_accessories>: HTTP status code is not handled or not allowed 2026-01-25 00:38:13 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:21 2026-01-25 00:38:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_pipettes_pipettors_syringe_tips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:38:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-14w-x-18l-1-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 00:38:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_batteries/vehicle_battery_accessories/vehicle_battery_boxes_trays landed on page that is not a product page. 2026-01-25 00:38:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/binders_covers/dividers_tabs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:38:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-16w-x-48l-2-mil-clear-250pack>: HTTP status code is not handled or not allowed 2026-01-25 00:38:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/staplers_staples/staplers landed on page that is not a product page. 2026-01-25 00:38:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-11w-x-24l-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 00:38:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/decor_linens/frames_posters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:38:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_pipettes_pipettors_syringe_tips landed on page that is not a product page. 2026-01-25 00:38:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/64oz-clear-jug-w-38-400-green-thermoset-f217-ptfe-cap-vacuum-6pk>: HTTP status code is not handled or not allowed 2026-01-25 00:38:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/stamps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:38:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stop-arm-7015-16-6>: HTTP status code is not handled or not allowed 2026-01-25 00:38:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/binders_covers/dividers_tabs landed on page that is not a product page. 2026-01-25 00:38:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/color_coding_labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:38:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rotary-paper-trimmer-26-cutting-length-15-sheet-capacity-gray>: HTTP status code is not handled or not allowed 2026-01-25 00:38:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/decor_linens/frames_posters landed on page that is not a product page. 2026-01-25 00:38:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/410kas3ylw-safety-padlock-1-12h-shackle-yellow-3set>: HTTP status code is not handled or not allowed 2026-01-25 00:38:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/stamps landed on page that is not a product page. 2026-01-25 00:38:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/notebooks_pads already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:38:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caution-wheels-must-be-chocked-before-10x14-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 00:38:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/address_shipping_labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:38:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/color_coding_labels landed on page that is not a product page. 2026-01-25 00:38:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ninja-x-15-guage-blk-nylon-spandex-shell-blk-bi-n9674m>: HTTP status code is not handled or not allowed 2026-01-25 00:38:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-8w-x-18l-3-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 00:38:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/paper_stationery/notebooks_pads>: HTTP status code is not handled or not allowed 2026-01-25 00:38:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/address_shipping_labels landed on page that is not a product page. 2026-01-25 00:38:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/miracool-bandana-assorted-colors-24-pack-w-header-card>: HTTP status code is not handled or not allowed 2026-01-25 00:38:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-3-arm-revers-quick-adj-set-75-ton-capacity-7-7-8-200-mm-open-23-5-8-600-mm-reach>: HTTP status code is not handled or not allowed 2026-01-25 00:38:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/woodstock-decorative-twin-size-headboard-metal-black>: HTTP status code is not handled or not allowed 2026-01-25 00:38:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gauges-62-1176-mar10-9268>: HTTP status code is not handled or not allowed 2026-01-25 00:39:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chain-plastic-1-1-2-links-on-a-reel-black-200-feet-trade-size-6>: HTTP status code is not handled or not allowed 2026-01-25 00:39:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-15w-x-18l-4-mil-clear-500pack>: HTTP status code is not handled or not allowed 2026-01-25 00:39:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hndl-lever-gh-1340w-gh1440w-06-10>: HTTP status code is not handled or not allowed 2026-01-25 00:39:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 42786 pages (at 91 pages/min), scraped 21057 items (at 62 items/min) 2026-01-25 00:39:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s1-safety-quick-coupling-3-8-high-flow-interchange-1-4-fnpt-connection>: HTTP status code is not handled or not allowed 2026-01-25 00:39:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30319235>: HTTP status code is not handled or not allowed 2026-01-25 00:39:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-premium-locker-triple-tier-12x15x24-9-door-gray>: HTTP status code is not handled or not allowed 2026-01-25 00:39:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/air_compressors/compressed_air_treatment_dryers_frls/compressed_air_filters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:39:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/55-515-wonder-bar-pry-bar-12-3-4-inch-long>: HTTP status code is not handled or not allowed 2026-01-25 00:39:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/end-mill-holder-nst-nmtb-40-shank-1-inch-type-7620-40-1-3-5-qc>: HTTP status code is not handled or not allowed 2026-01-25 00:39:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/penco-locker-single-tier-12x15x60-1-door-assembled-champagne-2>: HTTP status code is not handled or not allowed 2026-01-25 00:39:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/air_compressors/compressed_air_treatment_dryers_frls/compressed_air_filters landed on page that is not a product page. 2026-01-25 00:39:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/penco-locker-single-tier-15x18x72-1-door-assembled-gray-2>: HTTP status code is not handled or not allowed 2026-01-25 00:39:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-3w-x-12l-6-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 00:39:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vestil-pneumatic-tire-double-scissor-elevating-cart-cart-pn-600-d-600-lb>: HTTP status code is not handled or not allowed 2026-01-25 00:39:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/end-mill-holder-nst-nmtb-40-shank-1-2-inch-type-7620-40-1-2-2-qc>: HTTP status code is not handled or not allowed 2026-01-25 00:39:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/hand_tools_1/pliers/diagonal_cutting_pliers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:39:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-parts-bags-with-hang-hole-10w-x-12l-4-mil-clear-500pack>: HTTP status code is not handled or not allowed 2026-01-25 00:39:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0-470-10mm-x-0005-0-01mm-ids-digimatic-thickness-gage-deep-throat-type>: HTTP status code is not handled or not allowed 2026-01-25 00:39:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pesticide-cabinet-self-close-double-door-45-gallon>: HTTP status code is not handled or not allowed 2026-01-25 00:39:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/hand_tools_1/pliers/diagonal_cutting_pliers landed on page that is not a product page. 2026-01-25 00:39:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-unit-5-tier-chrome-18w-x-24l-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 00:39:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/balldriver-stubby-l-wrench-key-sets-16599>: HTTP status code is not handled or not allowed 2026-01-25 00:39:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/repeat-cycle-timer-0-60-sec-dpdt-120-vac>: HTTP status code is not handled or not allowed 2026-01-25 00:39:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-poly-z-brite-18w-x-36l-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 00:40:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-chrome-21w-x-36l-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 00:40:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spd-lever-ghd-2020pf-2401075>: HTTP status code is not handled or not allowed 2026-01-25 00:40:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/repeat-cycle-timer-0-6-sec-dpdt-120-vac>: HTTP status code is not handled or not allowed 2026-01-25 00:40:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/angle-flange-4-inch-diameter-galvanized>: HTTP status code is not handled or not allowed 2026-01-25 00:40:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 42883 pages (at 97 pages/min), scraped 21131 items (at 74 items/min) 2026-01-25 00:40:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-chrome-24w-x-36l-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 00:40:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microporous-coverall-elastic-wrists-ankles-white-medium-25-case>: HTTP status code is not handled or not allowed 2026-01-25 00:40:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/table-4-chair-set-navy-polypropylene-cafe-chairs-36w-x-29h-round-gray-nebula-table>: HTTP status code is not handled or not allowed 2026-01-25 00:40:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-desk-cherry-modular-office-furniture-1>: HTTP status code is not handled or not allowed 2026-01-25 00:40:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-oz-bottle-super-lube-air-tool-lubricant>: HTTP status code is not handled or not allowed 2026-01-25 00:40:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-oz-canister-super-lube-silicone-high-dielectric-vacuum-grease>: HTTP status code is not handled or not allowed 2026-01-25 00:40:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-parcelsentry-freestanding-mailparcel-box-wf-pb007-black>: HTTP status code is not handled or not allowed 2026-01-25 00:40:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cast-iron-grip-handle-weights-nist-class-f-25-kgs>: HTTP status code is not handled or not allowed 2026-01-25 00:40:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/industrial_lubricants already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:40:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-lb-pail-super-lube-synthetic-grease>: HTTP status code is not handled or not allowed 2026-01-25 00:40:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60in-fiberglass-handle-for-snap-on-wire-dust-frames-fgm146000000>: HTTP status code is not handled or not allowed 2026-01-25 00:40:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-x-4-anti-static-mat>: HTTP status code is not handled or not allowed 2026-01-25 00:40:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/industrial_lubricants landed on page that is not a product page. 2026-01-25 00:40:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v2-readers-eyewear-clear-3-0-lens-with-black-frame>: HTTP status code is not handled or not allowed 2026-01-25 00:40:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stone-panel-trash-sand-urn-gray-13-1-2-square-x-32-h>: HTTP status code is not handled or not allowed 2026-01-25 00:40:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/file_cabinets/vertical_file_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:40:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-hanford-twin-post-no-support-brace-with-ornate-base-e1-economy-mailbox>: HTTP status code is not handled or not allowed 2026-01-25 00:40:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washer-jsm-620-62037>: HTTP status code is not handled or not allowed 2026-01-25 00:40:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dot-traffic-cones-28-h-14-x-14-base-7-lbs-pvc-safety-green-6-pack>: HTTP status code is not handled or not allowed 2026-01-25 00:40:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/cabinets/file_cabinets/vertical_file_cabinets landed on page that is not a product page. 2026-01-25 00:41:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/offices-to-go-desk-shell-71x36-walnut>: HTTP status code is not handled or not allowed 2026-01-25 00:41:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/office_furniture/literature_mail_sorters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:41:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-pocket-oak-acrylic-wall-rack-light-oak>: HTTP status code is not handled or not allowed 2026-01-25 00:41:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-precision-ansi-roller-chain-25-1r-1-4-pitch-10ft-box>: HTTP status code is not handled or not allowed 2026-01-25 00:41:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 42976 pages (at 93 pages/min), scraped 21203 items (at 72 items/min) 2026-01-25 00:41:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-x-24-x-24-standard-shop-stand-maple-blue>: HTTP status code is not handled or not allowed 2026-01-25 00:41:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/office_furniture/literature_mail_sorters landed on page that is not a product page. 2026-01-25 00:41:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/243-1948-6h-hdmi-male-to-male-w-internet-1080p-30-awg-6-ft-gold-plated-black>: HTTP status code is not handled or not allowed 2026-01-25 00:41:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-x-24-x-36-standard-shop-stand-maple-gray>: HTTP status code is not handled or not allowed 2026-01-25 00:41:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/clips_fasteners/binder_clips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:41:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-stacking-bin-8-1-4-x-13-5-8-x-6-beige>: HTTP status code is not handled or not allowed 2026-01-25 00:41:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-precision-ansi-double-roller-chain-80-2r-1-pitch-offset-link>: HTTP status code is not handled or not allowed 2026-01-25 00:41:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:41:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kneeling-mat-6-pack>: HTTP status code is not handled or not allowed 2026-01-25 00:41:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/clips_fasteners/binder_clips landed on page that is not a product page. 2026-01-25 00:41:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-tarp-repair-tape-2-inch-x-35-foot>: HTTP status code is not handled or not allowed 2026-01-25 00:41:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/penco-recessed-handle-triple-tier-locker-12x15x24-assembled-1-wide-champagne>: HTTP status code is not handled or not allowed 2026-01-25 00:41:34 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats landed on page that is not a product page. 2026-01-25 00:41:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/throttle-valve-cap-jet-0t-y31002060>: HTTP status code is not handled or not allowed 2026-01-25 00:41:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bar-rack-horizontal-stacking-7>: HTTP status code is not handled or not allowed 2026-01-25 00:41:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:41:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/resealable-poly-bags-9w-x-12l-4-mil-clear-500pack>: HTTP status code is not handled or not allowed 2026-01-25 00:41:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wla-led-workstation-light-3000-850-t-120-277v-37w-354>: HTTP status code is not handled or not allowed 2026-01-25 00:41:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/value-1-stream-half-round-satin-black-16-gal-trash-t1899v>: HTTP status code is not handled or not allowed 2026-01-25 00:41:50 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits landed on page that is not a product page. 2026-01-25 00:41:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bib20-bench-in-a-box-cantilever-workbench-60-w-x-30-d-esd-laminate-top-dark-blue>: HTTP status code is not handled or not allowed 2026-01-25 00:41:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tarifold-vertical-hanging-pocket-5-pack>: HTTP status code is not handled or not allowed 2026-01-25 00:41:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speedpress-4-10-rhino-cutting-mat>: HTTP status code is not handled or not allowed 2026-01-25 00:41:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/icing-spatula-12-x-175>: HTTP status code is not handled or not allowed 2026-01-25 00:42:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rebuild-kit-rbk-3330>: HTTP status code is not handled or not allowed 2026-01-25 00:42:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-water-pipe-d-12-d-8-l-50-641245-floor-scrubber>: HTTP status code is not handled or not allowed 2026-01-25 00:42:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pivot-708020-107>: HTTP status code is not handled or not allowed 2026-01-25 00:42:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/wire_cable/electrical_wire_cable already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:42:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/wire_cable/electrical_wire_cable already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:42:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 43068 pages (at 92 pages/min), scraped 21271 items (at 68 items/min) 2026-01-25 00:42:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-conductor-shielded-plenum-cable-2-conductor-16-awg-19-0117-bc-white>: HTTP status code is not handled or not allowed 2026-01-25 00:42:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-block-reclosable-poly-bags-6w-x-9l-6-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 00:42:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/55-gallon-centrifugal-drum-pump-kit-polypropylene-110v>: HTTP status code is not handled or not allowed 2026-01-25 00:42:18 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/wire_cable/electrical_wire_cable landed on page that is not a product page. 2026-01-25 00:42:18 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/wire_cable/electrical_wire_cable landed on page that is not a product page. 2026-01-25 00:42:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stanco-lightweight-hrc-4-arc-clothing-kit-35-coat-overalls-hood-hard-hat-bag-ttk44-m>: HTTP status code is not handled or not allowed 2026-01-25 00:42:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stanco-standard-hrc-2-arc-clothing-kit-35-coat-bib-overalls-hood-hard-hat-bag-ttk11-3xl>: HTTP status code is not handled or not allowed 2026-01-25 00:42:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/packing-electric-workbench-maple-butcher-block-safety-edge-72-x-36-with-lower-shelf-kit>: HTTP status code is not handled or not allowed 2026-01-25 00:42:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kingline-chowder-bowl-167-oz-6-332white>: HTTP status code is not handled or not allowed 2026-01-25 00:42:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/madera-elongated-flushometer-valve-toilet-1>: HTTP status code is not handled or not allowed 2026-01-25 00:42:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/computer-cabinet-side-car-black-3>: HTTP status code is not handled or not allowed 2026-01-25 00:42:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jbc-revolution-28-reflective-traffic-cone-red-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 00:42:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/gloves_sleeves/chemical_resistant_gloves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:42:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h2o2-oxy-force-all-purpose-cleaner-light-citrus-scent-5-gallon-pail>: HTTP status code is not handled or not allowed 2026-01-25 00:42:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ingersoll-rand-261-air-impact-wrench-3-4>: HTTP status code is not handled or not allowed 2026-01-25 00:42:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ingersoll-rand-107xpa-3-8-ratchet>: HTTP status code is not handled or not allowed 2026-01-25 00:42:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pop-up-notes-super-sticky-3-x-3-canary-yellow-12-pk>: HTTP status code is not handled or not allowed 2026-01-25 00:42:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:42:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speedi-boot-straight-register-vent-boot-w-adj-hangers-for-hvac-duct-work-4-x-14-x-6>: HTTP status code is not handled or not allowed 2026-01-25 00:42:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/gloves_sleeves/chemical_resistant_gloves landed on page that is not a product page. 2026-01-25 00:42:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ccs2wjhlg-wall-joint-horizontal-p-arm-fits-45x60mm-alumltgray>: HTTP status code is not handled or not allowed 2026-01-25 00:43:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats>: HTTP status code is not handled or not allowed 2026-01-25 00:43:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/power_tools/lubrication_tools_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:43:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3m-duct-tape-3900-red-48-mm-x-54-8-m-7-7-mil-24-per-case>: HTTP status code is not handled or not allowed 2026-01-25 00:43:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-unit-5-tier-60w-x-24d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 00:43:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 43162 pages (at 94 pages/min), scraped 21343 items (at 72 items/min) 2026-01-25 00:43:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/emergency_vehicle_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:43:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/membrane-1632-6293265>: HTTP status code is not handled or not allowed 2026-01-25 00:43:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_alarms_signals/audible_alarms already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:43:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/power_tools/lubrication_tools_accessories>: HTTP status code is not handled or not allowed 2026-01-25 00:43:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/emergency_vehicle_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:43:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_alarms_signals/audible_alarms already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:43:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/pallet_stackers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:43:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-tote-dolly-3000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 00:43:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/emergency_vehicle_lights>: HTTP status code is not handled or not allowed 2026-01-25 00:43:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_alarms_signals/audible_alarms landed on page that is not a product page. 2026-01-25 00:43:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a12r126hm-3r-encl-horiz-mt-screw-cvr-1200x1200x600-galvanizedpaint>: HTTP status code is not handled or not allowed 2026-01-25 00:43:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/emergency_vehicle_lights landed on page that is not a product page. 2026-01-25 00:43:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/pallet_stackers landed on page that is not a product page. 2026-01-25 00:43:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-ke-thermaflex-flexible-hvac-duct-5-diameter-r8>: HTTP status code is not handled or not allowed 2026-01-25 00:43:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_alarms_signals/audible_alarms landed on page that is not a product page. 2026-01-25 00:43:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g-km-thermaflex-flexible-hvac-duct-9-diameter-r4-2>: HTTP status code is not handled or not allowed 2026-01-25 00:43:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modine-193000-btu-steam-or-hot-water-unit-heater-with-explosion-proof-motor-230-460-volt-3-phase>: HTTP status code is not handled or not allowed 2026-01-25 00:43:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bollard-safety-net-4-foot-l-by-20-foot-w-fits-3-1-2-inches-to-4-1-2-inches-dia-brsn-420-5>: HTTP status code is not handled or not allowed 2026-01-25 00:43:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/armorultra-job-on-pallet-2-layer-epoxy-system-5000-sq-ft-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 00:43:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/heaters/patio_heaters/gas_patio_heaters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:43:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/majestic-greenhouse-20w-x-60l-drop-down-w-film>: HTTP status code is not handled or not allowed 2026-01-25 00:43:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/foam/foam_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:43:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30x12x48-solar-star-greenhouse-w-poly-top-and-ends-drop-down-sides-prop-heater>: HTTP status code is not handled or not allowed 2026-01-25 00:43:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/narrow-3vx-cogged-v-belt-3vx800>: HTTP status code is not handled or not allowed 2026-01-25 00:43:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_lockout_identification_tags already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:43:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/heaters/patio_heaters/gas_patio_heaters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:43:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-spindle-assembly-30105000i>: HTTP status code is not handled or not allowed 2026-01-25 00:43:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:43:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/foam/foam_rolls>: HTTP status code is not handled or not allowed 2026-01-25 00:43:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:43:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/narrow-5vx-cogged-v-belt-5vx630>: HTTP status code is not handled or not allowed 2026-01-25 00:43:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/heaters/patio_heaters/gas_patio_heaters landed on page that is not a product page. 2026-01-25 00:43:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/heaters/patio_heaters/gas_patio_heaters>: HTTP status code is not handled or not allowed 2026-01-25 00:43:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_lockout_identification_tags landed on page that is not a product page. 2026-01-25 00:43:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:43:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 00:43:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:43:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/binders_covers/binders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:43:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/showcase-reference-view-binder-1-1-2-capacity-black>: HTTP status code is not handled or not allowed 2026-01-25 00:44:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bubble-lined-poly-mailers-9-12w-x-14-12l-330-mil-white-50pack>: HTTP status code is not handled or not allowed 2026-01-25 00:44:04 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-25 00:44:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cast-acrylic-sheet-12-l-x-12-w-x-1-8-thick-bronze>: HTTP status code is not handled or not allowed 2026-01-25 00:44:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents landed on page that is not a product page. 2026-01-25 00:44:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/locks_lockout_devices/lockout_tagout_devices/lockout_padlocks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:44:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/86456-self-laminating-caution-tag-w-clear-adhesive-overlay-polyester-3w-x-5-34h>: HTTP status code is not handled or not allowed 2026-01-25 00:44:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-25 00:44:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/binders_covers/binders landed on page that is not a product page. 2026-01-25 00:44:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/500-3-16-diameter-7x19-galvanized-aircraft-cable-2>: HTTP status code is not handled or not allowed 2026-01-25 00:44:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 43245 pages (at 83 pages/min), scraped 21393 items (at 50 items/min) 2026-01-25 00:44:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/key-8-8-35-5fk-g14-15planer-6284760>: HTTP status code is not handled or not allowed 2026-01-25 00:44:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:44:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/locks_lockout_devices/lockout_tagout_devices/lockout_padlocks landed on page that is not a product page. 2026-01-25 00:44:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelite-vented-plastic-shelving-starter-unit-5-tier-60w-x-18d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 00:44:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700228blk-multitower-pegboard-floor-display-20w-x-64h-x-20d-bk-1-pc>: HTTP status code is not handled or not allowed 2026-01-25 00:44:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:44:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_lockout_identification_tags already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:44:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48112-scaffold-status-holder-and-tags-plasticpolyester-3w-x-5-34h>: HTTP status code is not handled or not allowed 2026-01-25 00:44:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/225530-cosmetic-tray-w24-slots-1025w-x-25h-x-3d-clr-1-pc>: HTTP status code is not handled or not allowed 2026-01-25 00:44:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-25 00:44:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-aqualite-wheel-5x2-34-plain>: HTTP status code is not handled or not allowed 2026-01-25 00:44:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-z-brite-wire-shelving-starter-unit-1-dunnage-shelf-48w-x-18d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 00:44:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:44:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents>: HTTP status code is not handled or not allowed 2026-01-25 00:44:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_lockout_identification_tags landed on page that is not a product page. 2026-01-25 00:44:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/battery-doctor-die-cast-push-on-starter-switch>: HTTP status code is not handled or not allowed 2026-01-25 00:44:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-silicone-sheet-12-l-x-12-w-x-1-32-thick-blue-detectable-60a>: HTTP status code is not handled or not allowed 2026-01-25 00:44:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_lockout_identification_tags already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:44:34 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock landed on page that is not a product page. 2026-01-25 00:44:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-dia-round-paper-labels-with-wed-print-fluorescent-orange-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-25 00:44:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelon-wire-shelving-starter-unit-1-dunnage-shelf-36w-x-18d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 00:44:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-plated-ultra-heavy-duty-spring-mount-for-lighted-flagstaffs-over-8-ft>: HTTP status code is not handled or not allowed 2026-01-25 00:44:42 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_lockout_identification_tags landed on page that is not a product page. 2026-01-25 00:44:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelon-wire-shelving-starter-unit-1-dunnage-shelf-48w-x-24d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 00:44:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:44:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sani-adjustable-aluminum-tubular-shelf-20-inch-36-inch>: HTTP status code is not handled or not allowed 2026-01-25 00:44:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cooler-backroom-shelving-tubular-bar-style-24-inch-60-inch-bar-2-shelves>: HTTP status code is not handled or not allowed 2026-01-25 00:44:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-x-10-x-5-insulated-container>: HTTP status code is not handled or not allowed 2026-01-25 00:44:51 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock landed on page that is not a product page. 2026-01-25 00:44:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-wall-mounted-shelf-15-inch-24-inch-shelf>: HTTP status code is not handled or not allowed 2026-01-25 00:44:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-x-9-clevis-pin-low-carbon-steel-plain-clp-1000-9000>: HTTP status code is not handled or not allowed 2026-01-25 00:44:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-heavy-duty-quick-disconnect-warning-whip-w-o-light-16-x16-yellow-w-x-rectangle-flag>: HTTP status code is not handled or not allowed 2026-01-25 00:45:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-9-1-2x14-1-2-self-seal-kra-bubble-mailers-with-opening-tear-strip>: HTTP status code is not handled or not allowed 2026-01-25 00:45:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700432clr-hshaped-pegboard-display-32w-x-60h-clr-1-pc>: HTTP status code is not handled or not allowed 2026-01-25 00:45:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bollard-cover-4-1-2-in-dia-64-inch-h-red-yellow-tape>: HTTP status code is not handled or not allowed 2026-01-25 00:45:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-x-3-x-2-mailers>: HTTP status code is not handled or not allowed 2026-01-25 00:45:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 43336 pages (at 91 pages/min), scraped 21454 items (at 61 items/min) 2026-01-25 00:45:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:45:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-socket-3-8-drive-6-point-15-16-deep>: HTTP status code is not handled or not allowed 2026-01-25 00:45:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/80in-w-x-80in-d-reception-station-with-electric-raceway-cherry-counter-gray-panel>: HTTP status code is not handled or not allowed 2026-01-25 00:45:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/medical_devices_instruments/blood_pressure_monitors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:45:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/towing_accessories/hitch_covers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:45:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-1-8-x-9-1-8-x-1-2-1-kra-bookfold>: HTTP status code is not handled or not allowed 2026-01-25 00:45:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/towing_accessories/hitch_covers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:45:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/towing_accessories/hitch_covers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:45:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 00:45:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/000-4-x-8-kra-padded-mailer>: HTTP status code is not handled or not allowed 2026-01-25 00:45:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/medical_devices_instruments/blood_pressure_monitors landed on page that is not a product page. 2026-01-25 00:45:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/grill_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:45:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/towing_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:45:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/towing_accessories/hitch_covers>: HTTP status code is not handled or not allowed 2026-01-25 00:45:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/towing_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:45:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nhl-minnesota-wild-grill-mat-14238>: HTTP status code is not handled or not allowed 2026-01-25 00:45:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electric-office-partition-panel-48-1-4w-x-100h-tan-gl>: HTTP status code is not handled or not allowed 2026-01-25 00:45:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:45:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/outdoor_lighting_fixtures/wall_pack_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:45:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-socket-3-4-drive-6-point-1-short>: HTTP status code is not handled or not allowed 2026-01-25 00:45:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/towing_accessories landed on page that is not a product page. 2026-01-25 00:45:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spark-plug-socket-3-8-drive-5-8-extra-deep>: HTTP status code is not handled or not allowed 2026-01-25 00:45:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/towing_accessories landed on page that is not a product page. 2026-01-25 00:45:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-socket-1-4-drive-9-32-6-point-short>: HTTP status code is not handled or not allowed 2026-01-25 00:45:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/retail_fixtures_displays_equipment/retail_signs_sign_holders/letter_boards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:45:44 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/floor_mats landed on page that is not a product page. 2026-01-25 00:45:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/outdoor_lighting_fixtures/wall_pack_lights landed on page that is not a product page. 2026-01-25 00:45:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/impact-socket-1-2-drive-1-7-16-6-point-deep>: HTTP status code is not handled or not allowed 2026-01-25 00:45:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-foam-epdm-sheet-12-l-x-12-w-x-1-4-thick-black>: HTTP status code is not handled or not allowed 2026-01-25 00:45:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/foam/foam_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:45:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/retail_fixtures_displays_equipment/retail_signs_sign_holders/letter_boards landed on page that is not a product page. 2026-01-25 00:45:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/htg-trolley-1-2-ton-geared-wide-flange-width>: HTTP status code is not handled or not allowed 2026-01-25 00:45:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/composites/composite_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:45:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/07157-53169-ergonic-insulated-6-pc-set-slotted-phillips>: HTTP status code is not handled or not allowed 2026-01-25 00:46:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reserva-9-in-10-1-2-in-jbt-r3090tbk>: HTTP status code is not handled or not allowed 2026-01-25 00:46:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/foam/foam_rolls landed on page that is not a product page. 2026-01-25 00:46:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s153-sato-thermal-transfer-ribbons-4w-x-1345l-wax-black-24-rollscase>: HTTP status code is not handled or not allowed 2026-01-25 00:46:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:46:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/msmk132vp-danger-sign-no-smoking-10w-x-7h-plastic>: HTTP status code is not handled or not allowed 2026-01-25 00:46:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/composites/composite_bar_stock landed on page that is not a product page. 2026-01-25 00:46:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 43419 pages (at 83 pages/min), scraped 21511 items (at 57 items/min) 2026-01-25 00:46:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mcsp133va-danger-sign-confined-space-enter-by-permit-only-10w-x-7h-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 00:46:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lower-shelf-s-60-for-workbenches-60-w-x-14-d-x-3-h-medium-gray>: HTTP status code is not handled or not allowed 2026-01-25 00:46:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flexo-f6-split-wrappable-sleeve-125-dia-75-black>: HTTP status code is not handled or not allowed 2026-01-25 00:46:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-lumbar-back-support-cushion-14-x-13-burgundy-555-7300-0300>: HTTP status code is not handled or not allowed 2026-01-25 00:46:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 00:46:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_film_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:46:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leqm279xve-danger-keep-hands-clear-label-5w-x-3-12h1each>: HTTP status code is not handled or not allowed 2026-01-25 00:46:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-side-outlet-tee-2>: HTTP status code is not handled or not allowed 2026-01-25 00:46:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_film_stock>: HTTP status code is not handled or not allowed 2026-01-25 00:46:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:46:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:46:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/duraflex-woven-high-abrasion-sleeve-83-dia-100-black>: HTTP status code is not handled or not allowed 2026-01-25 00:46:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-13-16-x-8-5-16-x-2-1-2-lewisbins-divider-box-gray>: HTTP status code is not handled or not allowed 2026-01-25 00:46:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:46:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks>: HTTP status code is not handled or not allowed 2026-01-25 00:46:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:46:42 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 00:46:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cpvc-bar-48-l-x-2-1-2-w-x-1-16-thick-gray>: HTTP status code is not handled or not allowed 2026-01-25 00:46:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-bar-48-l-x-1-w-x-3-8-thick-clear-antistatic>: HTTP status code is not handled or not allowed 2026-01-25 00:46:49 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs landed on page that is not a product page. 2026-01-25 00:46:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/freestanding-2-panel-corner-room-divider-with-partial-window-36-x-60-gray>: HTTP status code is not handled or not allowed 2026-01-25 00:46:50 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock landed on page that is not a product page. 2026-01-25 00:46:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/150-cable-climbing-system-top-ext-38-dia-1x7-strands-galvanized-steel-cable>: HTTP status code is not handled or not allowed 2026-01-25 00:46:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:46:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14x16-1-5-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 00:46:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mecr622vs-caution-sign-low-headroom-14w-x-10h-adhesive-vinyl>: HTTP status code is not handled or not allowed 2026-01-25 00:46:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:46:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/refrigerated-truck-edge-of-dock-leveler-78-usable-w-100-overall-w-20-000-lb-986645>: HTTP status code is not handled or not allowed 2026-01-25 00:47:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:47:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hurricane-360-hoist-and-trolley-combo-standard-2-ton-capacity-30-lift>: HTTP status code is not handled or not allowed 2026-01-25 00:47:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 00:47:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/top-bracket-extension-assembly>: HTTP status code is not handled or not allowed 2026-01-25 00:47:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:47:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-25 00:47:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 43508 pages (at 89 pages/min), scraped 21575 items (at 64 items/min) 2026-01-25 00:47:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_film_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:47:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 00:47:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:47:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/35211-twist-lock-locking-liquid-dispenser-white-pump-top-round-bottle-4-oz>: HTTP status code is not handled or not allowed 2026-01-25 00:47:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-sheet-48-l-x-24-w-x-3-8-thick-clear-antistatic>: HTTP status code is not handled or not allowed 2026-01-25 00:47:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-25 00:47:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-bar-24-l-x-2-1-2-w-x-3-32-thick-clear>: HTTP status code is not handled or not allowed 2026-01-25 00:47:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 00:47:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-pull-type-water-cup-c3260tbr>: HTTP status code is not handled or not allowed 2026-01-25 00:47:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-nylon-6-6-bar-24-l-x-2-w-x-1-thick-black-mds-filled>: HTTP status code is not handled or not allowed 2026-01-25 00:47:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1400-1450-specific-gravity-astm-133h-hydrometer-for-heavy-liquids>: HTTP status code is not handled or not allowed 2026-01-25 00:47:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_film_stock landed on page that is not a product page. 2026-01-25 00:47:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:47:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classic-entrance-mat-waffle-pattern-3-8-thick-3-x-10-charcoal>: HTTP status code is not handled or not allowed 2026-01-25 00:47:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0990-1040-kg-l-draft-survey-hydrometer>: HTTP status code is not handled or not allowed 2026-01-25 00:47:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/electrical_power_testing/loop_process_calibrators already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:47:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-hdpe-sheet-6-l-x-6-w-x-1-thick-white-marine>: HTTP status code is not handled or not allowed 2026-01-25 00:47:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 00:47:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-ptfe-tube-72-l-x-1-3-16-id-x-1-9-16-od-white>: HTTP status code is not handled or not allowed 2026-01-25 00:47:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/electrical_power_testing/loop_process_calibrators already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:47:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/laboratory_heating_cooling/cryogenic_storage already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:47:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:47:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-rack-sump-with-drain-and-ball-valve-assembly>: HTTP status code is not handled or not allowed 2026-01-25 00:47:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:47:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/testing_inspection_instruments/electrical_power_testing/loop_process_calibrators landed on page that is not a product page. 2026-01-25 00:47:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelate-wire-shelving-add-on-a30368ep-36w-x-30d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 00:47:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/testing_inspection_instruments/electrical_power_testing/loop_process_calibrators landed on page that is not a product page. 2026-01-25 00:47:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/packing-peanut-loose-fill-dispenser-for-15ftsup3-bag-40l-x-40w-x-36w>: HTTP status code is not handled or not allowed 2026-01-25 00:47:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/laboratory_heating_cooling/cryogenic_storage landed on page that is not a product page. 2026-01-25 00:47:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers landed on page that is not a product page. 2026-01-25 00:47:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3500-3999-inventory-tag-2-part-carbon-style>: HTTP status code is not handled or not allowed 2026-01-25 00:47:50 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock landed on page that is not a product page. 2026-01-25 00:47:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/laboratory_heating_cooling/cryogenic_storage already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:47:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/work-benches/workbenches_worktables/automotive_garage_workbenches/mobile_automotive_workbenches already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:47:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-rod-36-l-x-2-3-4-dia-black>: HTTP status code is not handled or not allowed 2026-01-25 00:47:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/work-benches/workbenches_worktables/automotive_garage_workbenches/stationary_automotive_workbenches already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:47:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-rod-36-l-x-3-4-dia-white>: HTTP status code is not handled or not allowed 2026-01-25 00:47:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-air-foam-rolls-36w-x-2000l-x-132-thick-white-2-rolls>: HTTP status code is not handled or not allowed 2026-01-25 00:48:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/laboratory_heating_cooling/cryogenic_storage landed on page that is not a product page. 2026-01-25 00:48:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/work-benches/workbenches_worktables/automotive_garage_workbenches/mobile_automotive_workbenches landed on page that is not a product page. 2026-01-25 00:48:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crinkle-paper-10-lbs-white>: HTTP status code is not handled or not allowed 2026-01-25 00:48:04 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/work-benches/workbenches_worktables/automotive_garage_workbenches/stationary_automotive_workbenches landed on page that is not a product page. 2026-01-25 00:48:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-rod-60-l-x-5-8-dia-black>: HTTP status code is not handled or not allowed 2026-01-25 00:48:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roundup-2w-6l-endless-poly-roundsling-blue>: HTTP status code is not handled or not allowed 2026-01-25 00:48:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:48:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 43599 pages (at 91 pages/min), scraped 21633 items (at 58 items/min) 2026-01-25 00:48:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galileo-thermometer-18-to-26c-64-to-80f-5-spheres-11>: HTTP status code is not handled or not allowed 2026-01-25 00:48:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:48:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-edge-2w-4l-2-ply-eye-eye-poly-web-sling-flat-eyes>: HTTP status code is not handled or not allowed 2026-01-25 00:48:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:48:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-72-x-36-adj-height-workbench-w-drawer-black-esd-safety-edge-top>: HTTP status code is not handled or not allowed 2026-01-25 00:48:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock landed on page that is not a product page. 2026-01-25 00:48:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:48:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers>: HTTP status code is not handled or not allowed 2026-01-25 00:48:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-72-x-36-adj-height-workbench-w-drawer-riser-black-esd-square-edge-top>: HTTP status code is not handled or not allowed 2026-01-25 00:48:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock landed on page that is not a product page. 2026-01-25 00:48:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h-b-frio-temp-blood-bank-verification-thermometer-20-to-70f>: HTTP status code is not handled or not allowed 2026-01-25 00:48:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/bins_totes_containers/bins/cubby_bins_trays already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:48:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-25 00:48:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-rod-12-l-x-2-1-4-dia-black>: HTTP status code is not handled or not allowed 2026-01-25 00:48:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnetic-bead-separation-rack-for-standard-size-flat-microplates>: HTTP status code is not handled or not allowed 2026-01-25 00:48:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-1-5-8-channel-nut-p1008eg-electro-galvanized-3-8-16>: HTTP status code is not handled or not allowed 2026-01-25 00:48:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/bins_totes_containers/bins/cubby_bins_trays landed on page that is not a product page. 2026-01-25 00:48:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-rod-48-l-x-2-3-4-dia-black>: HTTP status code is not handled or not allowed 2026-01-25 00:48:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/threading/taps/spiral_point_taps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:48:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-aluminum-mailbox-without-pole-7w-x-19d-x-9h-mailbox-dim>: HTTP status code is not handled or not allowed 2026-01-25 00:48:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2x2-2-mil-white-block-reclosable-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 00:48:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:48:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/guard_rails_machine_guards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:48:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-x-6-2-mil-reclosable-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 00:48:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:48:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/threading/taps/spiral_point_taps landed on page that is not a product page. 2026-01-25 00:48:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-x-18-2-mil-reclosable-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 00:48:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers>: HTTP status code is not handled or not allowed 2026-01-25 00:48:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/guard_rails_machine_guards>: HTTP status code is not handled or not allowed 2026-01-25 00:49:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock landed on page that is not a product page. 2026-01-25 00:49:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plus-astm-s62c-03-individually-calibrated-liquid-in-glass-thermometer-38-2c>: HTTP status code is not handled or not allowed 2026-01-25 00:49:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:49:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:49:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:49:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-x-13-4-mil-reclosable-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 00:49:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:49:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 43682 pages (at 83 pages/min), scraped 21691 items (at 58 items/min) 2026-01-25 00:49:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plus-freezer-verification-thermometer-30-to-1c>: HTTP status code is not handled or not allowed 2026-01-25 00:49:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:49:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:49:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers>: HTTP status code is not handled or not allowed 2026-01-25 00:49:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers landed on page that is not a product page. 2026-01-25 00:49:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:49:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers landed on page that is not a product page. 2026-01-25 00:49:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kendall-howard-72-performance-work-bench-no-upper-shelving>: HTTP status code is not handled or not allowed 2026-01-25 00:49:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:49:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 00:49:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-75-basis-weight-kra-paper>: HTTP status code is not handled or not allowed 2026-01-25 00:49:21 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 00:49:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-165ml-two-piece-buchner-funnel>: HTTP status code is not handled or not allowed 2026-01-25 00:49:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:49:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-25 00:49:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 00:49:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:49:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-25 00:49:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/office_machines/time_clock_systems already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:49:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kendall-howard-1032-rack-screws-100>: HTTP status code is not handled or not allowed 2026-01-25 00:49:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers landed on page that is not a product page. 2026-01-25 00:49:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-sheet-24-l-x-24-w-x-2-thick-black>: HTTP status code is not handled or not allowed 2026-01-25 00:49:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:49:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:49:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-wall-mounted-bloodborne-pathogen-cleanup-kit>: HTTP status code is not handled or not allowed 2026-01-25 00:49:34 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers landed on page that is not a product page. 2026-01-25 00:49:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/office_machines/time_clock_systems>: HTTP status code is not handled or not allowed 2026-01-25 00:49:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precisionware-wide-mouth-250ml-8oz-autoclavable-polypropylene-bottles-pp-cap-12pk>: HTTP status code is not handled or not allowed 2026-01-25 00:49:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:49:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers landed on page that is not a product page. 2026-01-25 00:49:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/horizontal-or-vertical-discharge-fan-forced-suspended-unit-heater-3300w-277v>: HTTP status code is not handled or not allowed 2026-01-25 00:49:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 00:49:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-5-tier-24w-x-18d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 00:49:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:49:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:49:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-25 00:49:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:49:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/35-lb-table-umbrella-base-black>: HTTP status code is not handled or not allowed 2026-01-25 00:49:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/switches_receptacles_sensors/plugs_receptacles/surge_protection_plugs_receptacles already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:49:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/switches_receptacles_sensors/plugs_receptacles/surge_protection_plugs_receptacles already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:49:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spinbar-giant-polygon-teflon-magnetic-stirring-bar-108-x-27mm-white-without-pivot-ring>: HTTP status code is not handled or not allowed 2026-01-25 00:49:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 00:49:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-25 00:49:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:49:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 00:49:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_dishes_plates already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:50:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tray-and-shelf-fold-storage-with-25-clear-trays>: HTTP status code is not handled or not allowed 2026-01-25 00:50:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/switches_receptacles_sensors/plugs_receptacles/surge_protection_plugs_receptacles landed on page that is not a product page. 2026-01-25 00:50:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-72w-x-21d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 00:50:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 00:50:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/switches_receptacles_sensors/plugs_receptacles/surge_protection_plugs_receptacles landed on page that is not a product page. 2026-01-25 00:50:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/type-4000-medium-outdoor-waterproof-case-w-sponge-insert-foam-16-1-2-l-x-13-w-x-7h-black>: HTTP status code is not handled or not allowed 2026-01-25 00:50:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:50:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 43762 pages (at 80 pages/min), scraped 21737 items (at 46 items/min) 2026-01-25 00:50:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/fans_blowers/blowers/confined_space_blowers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:50:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:50:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cast-acrylic-bar-24-l-x-4-w-x-3-16-thick-clear-antistatic>: HTTP status code is not handled or not allowed 2026-01-25 00:50:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_dishes_plates landed on page that is not a product page. 2026-01-25 00:50:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-sterileware-powder-thief-5ml-100cml-individually-wrapped-20pk>: HTTP status code is not handled or not allowed 2026-01-25 00:50:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:50:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-5-tier-30w-x-14d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 00:50:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 00:50:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/fans_blowers/blowers/confined_space_blowers landed on page that is not a product page. 2026-01-25 00:50:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-face-out-w-disc-at-end-chrome>: HTTP status code is not handled or not allowed 2026-01-25 00:50:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-72w-x-21d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 00:50:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 00:50:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:50:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock landed on page that is not a product page. 2026-01-25 00:50:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-plastic-binding-combs-letter-size-40-sheet-capacity-5-16-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 00:50:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:50:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spinbar-teflon-micro-flea-magnetic-stirring-bar-15-x-15mm-white>: HTTP status code is not handled or not allowed 2026-01-25 00:50:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_storage_bench_protection/laboratory_bench_racks_stands already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:50:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-underbed-truck-box-24x24x30-black-1734303>: HTTP status code is not handled or not allowed 2026-01-25 00:50:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:50:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock>: HTTP status code is not handled or not allowed 2026-01-25 00:50:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:50:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-25 00:50:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:50:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spinstar-teflon-magnetic-stirring-bar-62mm-fits-250ml-beakers-white>: HTTP status code is not handled or not allowed 2026-01-25 00:50:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_storage_bench_protection/laboratory_bench_racks_stands landed on page that is not a product page. 2026-01-25 00:50:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/100-truckpak-6-watt-wintergard-plus-heating-cable-240v>: HTTP status code is not handled or not allowed 2026-01-25 00:50:42 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 00:50:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/competitor-ventilated-three-tier-locker-starter-1-wide-12w-x-12d-x-24h-unassembled-almond>: HTTP status code is not handled or not allowed 2026-01-25 00:50:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:50:45 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-25 00:50:45 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 00:50:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-ptfe-rod-72-l-x-1-1-2-dia-off-white-glass-filled>: HTTP status code is not handled or not allowed 2026-01-25 00:50:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-epoxy-wire-shelving-add-on-18365b-36w-x-18d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 00:50:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/diamond-plate-ergonomic-vinyl-mat-15-16-thick-36-x60-black>: HTTP status code is not handled or not allowed 2026-01-25 00:50:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 00:50:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:50:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:50:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-36-maple-square-edge-mobile-production-bench-blue>: HTTP status code is not handled or not allowed 2026-01-25 00:50:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:50:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-x-30-shop-top-safety-edge-mobile-production-bench-blue>: HTTP status code is not handled or not allowed 2026-01-25 00:51:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-five-tier-locker-2-wide-12w-x-15d-x-12h-assembled-blue-frost>: HTTP status code is not handled or not allowed 2026-01-25 00:51:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets>: HTTP status code is not handled or not allowed 2026-01-25 00:51:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-25 00:51:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eq-dual-speed-electric-chain-hoist-w-push-trolley-1-ton-10-lift-23-3-8-ft-min-460v>: HTTP status code is not handled or not allowed 2026-01-25 00:51:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:51:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 43844 pages (at 82 pages/min), scraped 21787 items (at 50 items/min) 2026-01-25 00:51:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-4-person-wall-locker-45w-x-18d-x-13h-unassembled-gray>: HTTP status code is not handled or not allowed 2026-01-25 00:51:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:51:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-25 00:51:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss-faucet-supply-line-3-8-od-x-3-8-od-x-12>: HTTP status code is not handled or not allowed 2026-01-25 00:51:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:51:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/core-dx-storage-cart-with-security-railed-top-and-hinged-door-white-with-watermelon-pink>: HTTP status code is not handled or not allowed 2026-01-25 00:51:18 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-25 00:51:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compact-bedside-cart-1-locking-drawer-white-with-silver-metallic>: HTTP status code is not handled or not allowed 2026-01-25 00:51:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:51:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-three-tier-locker-1-wide-12w-x-15d-x-24h-assembled-almond>: HTTP status code is not handled or not allowed 2026-01-25 00:51:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-25 00:51:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-head-thumb-screw-m4-x-0-7-12mm-thread-16mm-head-dia-3-5mm-head-h-steel-pkg-of-10>: HTTP status code is not handled or not allowed 2026-01-25 00:51:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-25 00:51:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/heaters/electric_heaters/portable_electric_heaters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:51:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-rack-netting-three-bay-7>: HTTP status code is not handled or not allowed 2026-01-25 00:51:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-25 00:51:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/174-b33-152-494-pk-b-494-color-polyester-labels-0-375-h-x-1-w-pink-white-1500-roll>: HTTP status code is not handled or not allowed 2026-01-25 00:51:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ecp4407tr-4-200-hp-1785-rpm>: HTTP status code is not handled or not allowed 2026-01-25 00:51:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:51:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/heaters/electric_heaters/portable_electric_heaters landed on page that is not a product page. 2026-01-25 00:51:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-hose-for-cat-21v-wet-dry-vacuum-641757>: HTTP status code is not handled or not allowed 2026-01-25 00:51:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/desk_accessories/desk_organizers_letter_trays already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:51:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-csswdm3541-75-hp-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-25 00:51:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:51:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beam-tongs-li-ing-attachment-2000-lb-capapcity>: HTTP status code is not handled or not allowed 2026-01-25 00:51:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/air_tools_accessories/pneumatic_tool_parts_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:51:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interlake-mecalux-pallet-rack-unslotted-beam-120-l-6678-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-25 00:51:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls>: HTTP status code is not handled or not allowed 2026-01-25 00:51:49 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/desk_accessories/desk_organizers_letter_trays landed on page that is not a product page. 2026-01-25 00:51:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72x24-maple-square-edge-production-bench-tan>: HTTP status code is not handled or not allowed 2026-01-25 00:51:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-single-phase-odp-motor-1-3-hp-3450-rpm-115-230v-odp-56c-frame>: HTTP status code is not handled or not allowed 2026-01-25 00:51:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-25 00:51:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/die_cut_labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:51:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/air_tools_accessories/pneumatic_tool_parts_accessories landed on page that is not a product page. 2026-01-25 00:51:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-hinged-top-basket-truck-14-bu-orange-vinyl-wood-base-all-swivel>: HTTP status code is not handled or not allowed 2026-01-25 00:51:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/work-benches/workbenches_worktables/open_base_workbenches/stationary_open_base_workbenches/standard_duty_stationary_workbenches already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:51:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-test-plugs-6>: HTTP status code is not handled or not allowed 2026-01-25 00:52:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stretch-fabric-table-cover-black-four-feet>: HTTP status code is not handled or not allowed 2026-01-25 00:52:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:52:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/l10-portable-outdoor-tankless-water-heater-eccoflo-diaphragm-12v-pump-strainer-shower-set>: HTTP status code is not handled or not allowed 2026-01-25 00:52:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/work-benches/workbenches_worktables/open_base_workbenches/stationary_open_base_workbenches/standard_duty_stationary_workbenches landed on page that is not a product page. 2026-01-25 00:52:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ecp2334t-4-20-hp-1765-rpm>: HTTP status code is not handled or not allowed 2026-01-25 00:52:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 43928 pages (at 84 pages/min), scraped 21840 items (at 53 items/min) 2026-01-25 00:52:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-three-phase-tenv-motor-3-4-hp-1725-rpm-200-230-460v-tenv>: HTTP status code is not handled or not allowed 2026-01-25 00:52:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels landed on page that is not a product page. 2026-01-25 00:52:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1010546-flip-out-bin-mobile-parts-cart-single-sided-with-40-bins>: HTTP status code is not handled or not allowed 2026-01-25 00:52:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/laptop-backpack-locker-54w-x-21d-x-78h-3-tier-3-wide-711-platinum-assembled>: HTTP status code is not handled or not allowed 2026-01-25 00:52:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/office_packing_tape/strapping_tape already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:52:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-log-book-for-electric-narrow-aisle-forklift-log-986886>: HTTP status code is not handled or not allowed 2026-01-25 00:52:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/digital-vellum-bristol-color-cover-8-1-2in-x-11in-67lb-goldenrod-250-sheets-pk>: HTTP status code is not handled or not allowed 2026-01-25 00:52:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-three-phase-tefc-motor-2-hp-1760-rpm-230-460v-tefc>: HTTP status code is not handled or not allowed 2026-01-25 00:52:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/office_packing_tape/strapping_tape>: HTTP status code is not handled or not allowed 2026-01-25 00:52:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-aluminum-aluminum-eggcrate-2>: HTTP status code is not handled or not allowed 2026-01-25 00:52:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/photo_presentation_paper already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:52:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-swivel-branch-tee-1-8-tube-x-1-4-swift-fit-universal-thread-1-2-7-l>: HTTP status code is not handled or not allowed 2026-01-25 00:52:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/men-indus-wrk-shirt-shrt-slv-whte-m-sp24>: HTTP status code is not handled or not allowed 2026-01-25 00:52:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cem3545-1-hp-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-25 00:52:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/paper_stationery/photo_presentation_paper landed on page that is not a product page. 2026-01-25 00:52:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/direct-drive-motor-1-4-hp-1625-rpm-115v-oao>: HTTP status code is not handled or not allowed 2026-01-25 00:52:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cewdm3710t-5-7-5-hp-1770-rpm>: HTTP status code is not handled or not allowed 2026-01-25 00:52:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cewdbm3611t-3-hp-1760-rpm>: HTTP status code is not handled or not allowed 2026-01-25 00:52:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/general_purpose_electric_motors/three_phase_motors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:52:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-swivel-elbow-3-8-tube-x-1-8-swift-fit-universal-thread-1-l>: HTTP status code is not handled or not allowed 2026-01-25 00:52:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-right-hand-return-table-gray>: HTTP status code is not handled or not allowed 2026-01-25 00:52:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-jml1409t-5-hp-3500-rpm>: HTTP status code is not handled or not allowed 2026-01-25 00:53:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-rivet-steel-extra-level-with-wood-deck-48-w-x-30-d-gray>: HTTP status code is not handled or not allowed 2026-01-25 00:53:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-series-clear-lid-for-qus236-polyethylene>: HTTP status code is not handled or not allowed 2026-01-25 00:53:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-bi-level-deck-scrub>: HTTP status code is not handled or not allowed 2026-01-25 00:53:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/filter-multi-pleat-xl10-24-w-x-24-h-x-2-d-12-ctn>: HTTP status code is not handled or not allowed 2026-01-25 00:53:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/general_purpose_electric_motors/three_phase_motors landed on page that is not a product page. 2026-01-25 00:53:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 44023 pages (at 95 pages/min), scraped 21911 items (at 71 items/min) 2026-01-25 00:53:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-brute-plastic-round-bucket-10-quart-red>: HTTP status code is not handled or not allowed 2026-01-25 00:53:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-rack-5-shelves-with-15-span-track-flow-units-48-w-x-48-d-x-84-h-textured-putty>: HTTP status code is not handled or not allowed 2026-01-25 00:53:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-piece-primo-cribbing-kit-with-locking-jack-plate-15266pl>: HTTP status code is not handled or not allowed 2026-01-25 00:53:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/appliances/cooking_equipment/ovens already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:53:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/28-piece-super-stacker-cribbing-set-15253>: HTTP status code is not handled or not allowed 2026-01-25 00:53:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/maxirough-janitor-cart-25-gallon-vinyl-replacement-bag-2-case-96985>: HTTP status code is not handled or not allowed 2026-01-25 00:53:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rivetwell-particle-board-decking-42-inch-w-x-15-inch-d-x-5-8-inch>: HTTP status code is not handled or not allowed 2026-01-25 00:53:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/appliances/cooking_equipment/ovens landed on page that is not a product page. 2026-01-25 00:53:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-x-48-reflective-roll-up-vinyl-sign-transition-symbol-right>: HTTP status code is not handled or not allowed 2026-01-25 00:53:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:53:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hol-dex-magnetic-shelf-bin-label-holders-1-2-inch-magnetic-label-holder-10-bx>: HTTP status code is not handled or not allowed 2026-01-25 00:53:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kent-foot-stool-30-w-x-8-h-x-12-d-w-handrail-mri-safe>: HTTP status code is not handled or not allowed 2026-01-25 00:53:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-m1508t-5-2-5-hp-1725-850-rpm>: HTTP status code is not handled or not allowed 2026-01-25 00:53:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits landed on page that is not a product page. 2026-01-25 00:53:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/55303-warning-stripe-checktape-vinyl-3-inches-wide-by-54-feet-long-black-yellow>: HTTP status code is not handled or not allowed 2026-01-25 00:53:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_seating/patio_dining_chairs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:53:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-flow-control-flow-knob-adjustment-8mm-tube-x-1-8-swift-fit-universal-thread-1-l>: HTTP status code is not handled or not allowed 2026-01-25 00:53:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-fdl3610m-3-hp-1750-rpm>: HTTP status code is not handled or not allowed 2026-01-25 00:53:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-em4115t-8-50-hp-1775-rpm>: HTTP status code is not handled or not allowed 2026-01-25 00:53:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_seating/patio_dining_chairs>: HTTP status code is not handled or not allowed 2026-01-25 00:53:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:53:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-swivel-run-tee-1-2-tube-x-1-2-ml-1-1-5-l>: HTTP status code is not handled or not allowed 2026-01-25 00:53:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/67387-120v-snap-on-circuit-breaker-lockout-each>: HTTP status code is not handled or not allowed 2026-01-25 00:53:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/wire_markers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:54:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/basket-5-w-x-13-3-8-d-x-7-h-steel-light-green-proform>: HTTP status code is not handled or not allowed 2026-01-25 00:54:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_labels landed on page that is not a product page. 2026-01-25 00:54:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-em4108t-8-30-hp-3520-rpm>: HTTP status code is not handled or not allowed 2026-01-25 00:54:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bleeding-control-kit-texas-mandate-fabric-red-13-pieces>: HTTP status code is not handled or not allowed 2026-01-25 00:54:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/wire_markers landed on page that is not a product page. 2026-01-25 00:54:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 44117 pages (at 94 pages/min), scraped 21981 items (at 70 items/min) 2026-01-25 00:54:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-el3612t-5-hp-1735-rpm>: HTTP status code is not handled or not allowed 2026-01-25 00:54:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-polyurethane-cleanroom-stool-with-casters-and-footring-high-bench-height-blue>: HTTP status code is not handled or not allowed 2026-01-25 00:54:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coverall-hood-and-socks-boots-white-serged-seam-xl>: HTTP status code is not handled or not allowed 2026-01-25 00:54:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/emergency_vehicle_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:54:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6in-amber-oval-mid-turn-signal-side-marker-light-kit-with-9-led-5626209>: HTTP status code is not handled or not allowed 2026-01-25 00:54:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/food_beverages/coffee_beverages/coffee already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:54:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metalout-bag-3-mm-18-x-24-100-pack>: HTTP status code is not handled or not allowed 2026-01-25 00:54:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oil-safe-mini-spout-lid-black>: HTTP status code is not handled or not allowed 2026-01-25 00:54:25 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-25 00:54:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-5in-amber-rectangular-dot-stick-on-reflectors-500-per-roll-5623521>: HTTP status code is not handled or not allowed 2026-01-25 00:54:26 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:22 2026-01-25 00:54:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/emergency_vehicle_lights landed on page that is not a product page. 2026-01-25 00:54:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/food_beverages/coffee_beverages/coffee>: HTTP status code is not handled or not allowed 2026-01-25 00:54:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/plumbing_parts_accessories/pipe_hose_tubing_accessories/pipe_markers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:54:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/bathroom_partition_replacement_hardware already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:54:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quicksnapuniversal-slot-adaptor-fits-29mm-to-33mm-slotted-safety-caps-black>: HTTP status code is not handled or not allowed 2026-01-25 00:54:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/go-specsfoam-lined-spectacle-gg-40g-af-gray-afpc-lens-black-frame>: HTTP status code is not handled or not allowed 2026-01-25 00:54:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-steel-platform-loading-dock-10-x-8-30-000-lb-cap-986961>: HTTP status code is not handled or not allowed 2026-01-25 00:54:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/plumbing_parts_accessories/pipe_hose_tubing_accessories/pipe_markers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:54:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/bathroom_partition_replacement_hardware>: HTTP status code is not handled or not allowed 2026-01-25 00:54:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-rhm3162ta-5-hp-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-25 00:54:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/plumbing/plumbing_parts_accessories/pipe_hose_tubing_accessories/pipe_markers landed on page that is not a product page. 2026-01-25 00:54:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interion-174-utility-table-60-x-30-rustic-gray>: HTTP status code is not handled or not allowed 2026-01-25 00:54:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/610bbh-jaws-hang-all-clip-wbk-hook-4-12l-x-34w-x4-12h-metal-bk-pkg-qty-100>: HTTP status code is not handled or not allowed 2026-01-25 00:54:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evx-s24-two-way-radio-2-watt-analog-3-watt-digital-256-channel-uhf-403-470-mhz-black>: HTTP status code is not handled or not allowed 2026-01-25 00:54:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-wide-welded-steel-lockers-double-tier-18x18x72-blue>: HTTP status code is not handled or not allowed 2026-01-25 00:54:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-16-mag-nutsetter-x-6-s2-modified-steel-gunmetal-grey>: HTTP status code is not handled or not allowed 2026-01-25 00:54:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wesco-aluminum-narrow-aisle-platform-truck-1322-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 00:54:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sld-16-shirtdress-hanger-16l-metal-chrome-pkg-qty-100>: HTTP status code is not handled or not allowed 2026-01-25 00:54:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2610pclh-skirtslack-hanger-infant-super-heavy-weight-wlong-hook-and-plastic-clipsplastic-bk>: HTTP status code is not handled or not allowed 2026-01-25 00:54:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/privacy-screen-vinyl-panel-white>: HTTP status code is not handled or not allowed 2026-01-25 00:54:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-vem3545-1-hp-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-25 00:54:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tray-fast-food-14x18-blue>: HTTP status code is not handled or not allowed 2026-01-25 00:54:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/first_aid_wound_treatment/first_aid_kits/first_aid_kits_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:55:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-window-premium-stacking>: HTTP status code is not handled or not allowed 2026-01-25 00:55:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electrical-rated-gloves-black-11-unlined-smooth-finish-beaded-class-0-8>: HTTP status code is not handled or not allowed 2026-01-25 00:55:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/height-adjustable-t-arms-for-comfortask-series-chairs-black-package-of-two>: HTTP status code is not handled or not allowed 2026-01-25 00:55:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bridges_returns already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:55:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bridges_returns already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:55:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-economy-lever-hoist-1-1-2-ton-capacity-15-lift>: HTTP status code is not handled or not allowed 2026-01-25 00:55:09 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/first_aid_wound_treatment/first_aid_kits/first_aid_kits_cabinets landed on page that is not a product page. 2026-01-25 00:55:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 44208 pages (at 91 pages/min), scraped 22041 items (at 60 items/min) 2026-01-25 00:55:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-economy-manual-chain-hoist-1-2-ton-capacity-10-lift>: HTTP status code is not handled or not allowed 2026-01-25 00:55:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/cubicle_partition_panels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:55:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/peapod-mini-8-15-locking-bike-shelter-21-10-l-x-7-5-w-15-bike-capacity-barrel-roof>: HTTP status code is not handled or not allowed 2026-01-25 00:55:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bridges_returns>: HTTP status code is not handled or not allowed 2026-01-25 00:55:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bridges_returns landed on page that is not a product page. 2026-01-25 00:55:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-partition-desktop-30-w-x-24-d-white-ash>: HTTP status code is not handled or not allowed 2026-01-25 00:55:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-vinyl-sign-danger-keep-off>: HTTP status code is not handled or not allowed 2026-01-25 00:55:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-m3463-3-4-hp-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-25 00:55:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/partitions_dividers/cubicle_partition_panels>: HTTP status code is not handled or not allowed 2026-01-25 00:55:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esp-kit-1-pair-black-esp-glove-1-pair-goat-class-00-size-12>: HTTP status code is not handled or not allowed 2026-01-25 00:55:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/furniture_parts_accessories/partition_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:55:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/111-t12-c8-eg-all-plastic-maintenance-free-type-ii-traffic-barricade-white-foldable>: HTTP status code is not handled or not allowed 2026-01-25 00:55:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torx-174-25-insert-bit-x-1-s2-modified-steel-gunmetal-grey>: HTTP status code is not handled or not allowed 2026-01-25 00:55:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shc-2-5-gal-electric-mini-tank-water-heater>: HTTP status code is not handled or not allowed 2026-01-25 00:55:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/top-grain-cowhide-drivers-wkevlar-grain-palm-keystone-regular-grade-l>: HTTP status code is not handled or not allowed 2026-01-25 00:55:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-zdvscp3770t-7-5-hp-6200-rpm>: HTTP status code is not handled or not allowed 2026-01-25 00:55:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sds-maxhammer-drilling-dust-extraction-attachment-8lx3h>: HTTP status code is not handled or not allowed 2026-01-25 00:55:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/csp-sw38-w-strongwall-ada-compliant-plastic-barricade-white-38-h-interlocking>: HTTP status code is not handled or not allowed 2026-01-25 00:55:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esp-kit-1-pair-esp-glove-1-pair-goat-protector-class-0-size-12>: HTTP status code is not handled or not allowed 2026-01-25 00:55:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/furniture_parts_accessories/partition_accessories landed on page that is not a product page. 2026-01-25 00:55:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-motor-for-48-blower-fan-for-model-600555>: HTTP status code is not handled or not allowed 2026-01-25 00:55:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torx-174-15-power-bit-x-2-gunmetal-bronze-s2-modified-steel>: HTTP status code is not handled or not allowed 2026-01-25 00:55:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zurn-9-x-9-square-floor-drain-wscrews-nickel>: HTTP status code is not handled or not allowed 2026-01-25 00:55:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/little-giant-wwc2-3672-heavy-duty-cabinet-workbench-steel-top-center-shelf>: HTTP status code is not handled or not allowed 2026-01-25 00:55:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mh-2d-2448-hdfl-48-w-mobile-bench-cabinets-heavy-duty-drawer-hardboard-top-2-doors>: HTTP status code is not handled or not allowed 2026-01-25 00:55:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cam-follower-sealed-hex-head-crowned-od-1-l-stud-0625-w-roller-1125-roller-dia>: HTTP status code is not handled or not allowed 2026-01-25 00:55:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cam-follower-sealed-hex-head-eccentric-stud-1-1-4-l-stud-075-w-roller-125-roller-dia>: HTTP status code is not handled or not allowed 2026-01-25 00:55:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-plastic-sign-men>: HTTP status code is not handled or not allowed 2026-01-25 00:55:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cam-follower-sealed-hex-head-crowned-od-1-3-4-l-stud-1-w-roller-1875-roller-dia>: HTTP status code is not handled or not allowed 2026-01-25 00:55:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-5-tier-wire-shelving-add-on-unit-60w-x-14d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 00:55:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:55:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-battery-cable-with-5-16-ring-terminals-11-0-awg-15-ft-red>: HTTP status code is not handled or not allowed 2026-01-25 00:55:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-vinyl-sign-caution-hazardous-material-storage-area>: HTTP status code is not handled or not allowed 2026-01-25 00:55:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-vinyl-sign-danger-area-front-of-this-electrical-panel-kept-clear-2>: HTTP status code is not handled or not allowed 2026-01-25 00:55:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/task_desk_chairs/task_office_chairs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:55:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelf-truck-esd-36w-x-36l-x-69h-polyurethane-4-swivel-2-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 00:55:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs landed on page that is not a product page. 2026-01-25 00:55:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flexbollard-52h-adhesive-anchoring-to-concrete-blackwhite-b52-d-blackwhite>: HTTP status code is not handled or not allowed 2026-01-25 00:55:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leviton-86602-2-gang-decora-gfci-device-decora-oversized-thermoset-ivory>: HTTP status code is not handled or not allowed 2026-01-25 00:55:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-4-tier-esd-24w-x-48l-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 00:55:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/task_desk_chairs/task_office_chairs>: HTTP status code is not handled or not allowed 2026-01-25 00:55:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:55:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-flow-heavy-duty-professional-100-ft-3-4>: HTTP status code is not handled or not allowed 2026-01-25 00:56:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:56:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-trash-container-garbage-can-lid-32-gallon-white>: HTTP status code is not handled or not allowed 2026-01-25 00:56:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/surface-mount-vertical-series-7-door-mailbox-anodized-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 00:56:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honeywell-super-tradeline-power-pro-model-2-pole-30a-24v>: HTTP status code is not handled or not allowed 2026-01-25 00:56:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/notice-sign-14x10-rigid-plastic-smoking-permitted-this-area>: HTTP status code is not handled or not allowed 2026-01-25 00:56:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-shelf-truck-esd-3-shelf-24w-x-30l-x-40h-polyurethane-4-swivel-2-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 00:56:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 44300 pages (at 92 pages/min), scraped 22089 items (at 48 items/min) 2026-01-25 00:56:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs>: HTTP status code is not handled or not allowed 2026-01-25 00:56:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/warning-sign-14x10-aluminum-keep-electrical-panel-clear>: HTTP status code is not handled or not allowed 2026-01-25 00:56:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs landed on page that is not a product page. 2026-01-25 00:56:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-inch-x-15-inch-x-5-inch-flat-corrugated-boxes>: HTTP status code is not handled or not allowed 2026-01-25 00:56:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-esd-21w-x-72l-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 00:56:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/freestanding-office-partition-panel-36-14-w-x-96-h-tan>: HTTP status code is not handled or not allowed 2026-01-25 00:56:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speakman-sc-5811-commander-service-sink-faucet-cross-handles>: HTTP status code is not handled or not allowed 2026-01-25 00:56:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/individual-character-stencil-12-letter-e>: HTTP status code is not handled or not allowed 2026-01-25 00:56:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mw-48426-hpfl-display-base-pallet-end-cap-spot-merchandiser-48w-x-42d-x-6h>: HTTP status code is not handled or not allowed 2026-01-25 00:56:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rust-oleum-high-performance-v2100-system-equip-aero-john-deere-yellow>: HTTP status code is not handled or not allowed 2026-01-25 00:56:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8x3-4-modified-truss-head-self-drilling-screws-0812KPM410>: HTTP status code is not handled or not allowed 2026-01-25 00:56:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fda-silicone-sanitary-gasket-for-34-tube>: HTTP status code is not handled or not allowed 2026-01-25 00:56:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/photo_presentation_paper already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:56:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-office-partition-panel-wpartial-window-raceway-60-14-w-x-47-12-h-tan>: HTTP status code is not handled or not allowed 2026-01-25 00:56:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rust-oleum-sierra-perform-metalmax-0-voc-dtm-acryl-ena-semi-gls-tnt-ba>: HTTP status code is not handled or not allowed 2026-01-25 00:56:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8x5-8-truss-head-self-tapping-screws-type-a-thread-0810APTB>: HTTP status code is not handled or not allowed 2026-01-25 00:56:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clearance-bar-kit-7-d-x-96-l-yellow-bar-no-tapes>: HTTP status code is not handled or not allowed 2026-01-25 00:56:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-safety-sign-fire-exit-with-le-arrow-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 00:56:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/paper_stationery/photo_presentation_paper>: HTTP status code is not handled or not allowed 2026-01-25 00:56:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rust-oleum-companion-marking-wand>: HTTP status code is not handled or not allowed 2026-01-25 00:56:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-24x1-1-2-round-head-machine-screws-1024MCR>: HTTP status code is not handled or not allowed 2026-01-25 00:56:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/push-spreader-50-lbs-20-inch-l-x-17-inch-w-x-34-inch-h-black>: HTTP status code is not handled or not allowed 2026-01-25 00:56:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/notebooks_pads already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:56:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss-m2-height-adjustable-petri-dish-holder-for-rk-10a-rack>: HTTP status code is not handled or not allowed 2026-01-25 00:56:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-panel-shipping-crate-with-lid-pallet-84-x-48-x-48-od>: HTTP status code is not handled or not allowed 2026-01-25 00:56:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:56:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brighton-queen-tufted-upholstered-platform-bed-dark-gray-with-pocket-spring-mattress>: HTTP status code is not handled or not allowed 2026-01-25 00:56:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=jtm-2-w-3x-203-knee-x-pwrfd already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:56:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8x3-4-phillips-bugle-head-exterior-wood-deck-screws-18-8-stainless-steel-0812DPG188>: HTTP status code is not handled or not allowed 2026-01-25 00:56:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/cash_register_pos_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:56:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-tool-1-2-inch-high-performance-industrial-impact-wrench>: HTTP status code is not handled or not allowed 2026-01-25 00:56:49 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/paper_stationery/notebooks_pads landed on page that is not a product page. 2026-01-25 00:56:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:56:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/display-unit-2-tier-black-epoxy-14-w-x-36-l-x-34-h>: HTTP status code is not handled or not allowed 2026-01-25 00:56:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:56:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards>: HTTP status code is not handled or not allowed 2026-01-25 00:56:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m4x8-metric-thread-rolling-screws-M48D7500T>: HTTP status code is not handled or not allowed 2026-01-25 00:56:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/paper_stationery/cash_register_pos_rolls>: HTTP status code is not handled or not allowed 2026-01-25 00:56:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rx-destroyer-64-oz-pro-series-lock-box-each>: HTTP status code is not handled or not allowed 2026-01-25 00:56:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs>: HTTP status code is not handled or not allowed 2026-01-25 00:56:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m8-1-25x45-din-6921-class-8-point-8-metric-flange-bolt-M845BF8>: HTTP status code is not handled or not allowed 2026-01-25 00:56:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/recycled-monthly-planner-black-6-7-8-x-8-3-4-2015-aag70120g05>: HTTP status code is not handled or not allowed 2026-01-25 00:56:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards>: HTTP status code is not handled or not allowed 2026-01-25 00:57:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-drilling-screw-10-x-1-1-2-hex-washer-head-teks-point-box-of-100>: HTTP status code is not handled or not allowed 2026-01-25 00:57:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/river-pro-portable-power-station-720wh-black>: HTTP status code is not handled or not allowed 2026-01-25 00:57:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tribeca-twin-tufted-upholstered-platform-bed-beige-with-memory-foam-mattress>: HTTP status code is not handled or not allowed 2026-01-25 00:57:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flagging-tape-orange-1>: HTTP status code is not handled or not allowed 2026-01-25 00:57:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:57:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/automotive_shop_equipment/vehicle_lifts_jacks/jack_stands already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:57:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-13-flex-type-hex-lock-nut-50NXL>: HTTP status code is not handled or not allowed 2026-01-25 00:57:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/beds_mattresses/mattresses already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:57:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=jtm-2-w-3x-203-knee-x-pwrfd landed on page that is not a product page. 2026-01-25 00:57:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ecotones-moonlight-cream-monthly-desk-pad-calendar-22-x-17-2015-hod12441>: HTTP status code is not handled or not allowed 2026-01-25 00:57:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 44389 pages (at 89 pages/min), scraped 22136 items (at 47 items/min) 2026-01-25 00:57:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-floor-standing-combination-belt-disc-sander>: HTTP status code is not handled or not allowed 2026-01-25 00:57:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/stamp_pads already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:57:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-16-20x2-hex-tap-bolts-grade-8-4432BHT8>: HTTP status code is not handled or not allowed 2026-01-25 00:57:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/heaters/stoves_fireplaces_log_sets_fire_pits/stove_heaters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:57:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards>: HTTP status code is not handled or not allowed 2026-01-25 00:57:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/vehicle-fleet-maintenance/automotive_shop_equipment/vehicle_lifts_jacks/jack_stands landed on page that is not a product page. 2026-01-25 00:57:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m10-1-5x12-metric-socket-set-screws-cup-point-M10012SSC>: HTTP status code is not handled or not allowed 2026-01-25 00:57:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/beds_mattresses/mattresses landed on page that is not a product page. 2026-01-25 00:57:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/heaters/stoves_fireplaces_log_sets_fire_pits/fireplaces_log_sets_fireboxes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:57:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dividers-for-4-h-drawer-of-modular-drawer-cabinet-36-wx24-d-black>: HTTP status code is not handled or not allowed 2026-01-25 00:57:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/stamp_pads landed on page that is not a product page. 2026-01-25 00:57:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/floor_machine_accessories_supplies/vacuum_bags already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:57:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/heaters/stoves_fireplaces_log_sets_fire_pits/stove_heaters>: HTTP status code is not handled or not allowed 2026-01-25 00:57:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:57:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/neoprene-foam-with-acrylic-adhesive-132-thick-x-316w-x-10l>: HTTP status code is not handled or not allowed 2026-01-25 00:57:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ice-bucket-liner-bags-12in-x-12in-96-oz-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 00:57:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3d-glow-sign-acrylic-emergency-exit-only>: HTTP status code is not handled or not allowed 2026-01-25 00:57:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/frixion-clicker-erasable-retractable-gel-pen-1-mm-black-ink-barrel-dozen>: HTTP status code is not handled or not allowed 2026-01-25 00:57:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/building_construction_tape/marking_warning_tape already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:57:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-25-11110-30inhg30psikpa-14-lm-plastic>: HTTP status code is not handled or not allowed 2026-01-25 00:57:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/floor_machine_accessories_supplies/vacuum_bags landed on page that is not a product page. 2026-01-25 00:57:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eva-foam-no-adhesive-1-thick-x-12w-x-12l>: HTTP status code is not handled or not allowed 2026-01-25 00:57:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges landed on page that is not a product page. 2026-01-25 00:57:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/heaters/stoves_fireplaces_log_sets_fire_pits/fireplaces_log_sets_fireboxes landed on page that is not a product page. 2026-01-25 00:57:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-15-11110-30-psi-18-lm-plastic>: HTTP status code is not handled or not allowed 2026-01-25 00:57:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glow-sign-rigid-plastic-emergency-shut-off-2>: HTTP status code is not handled or not allowed 2026-01-25 00:57:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/building_construction_tape/marking_warning_tape landed on page that is not a product page. 2026-01-25 00:57:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/illuminated-push-button-operator-22mm-white>: HTTP status code is not handled or not allowed 2026-01-25 00:57:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-labels-with-trueblock-technology-2-x-4-white-2500-box-5963>: HTTP status code is not handled or not allowed 2026-01-25 00:57:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:57:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4x2-1-2-split-drive-anchors-1440ADSPF>: HTTP status code is not handled or not allowed 2026-01-25 00:57:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/pens already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:57:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6009zznr-double-shielded-wsnap-ring-45mm-bore-75mm-od>: HTTP status code is not handled or not allowed 2026-01-25 00:57:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-8-x-9-working-length-j-hook-style-9>: HTTP status code is not handled or not allowed 2026-01-25 00:57:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rk-06-pl-extension-pole-for-rk-06a-stand>: HTTP status code is not handled or not allowed 2026-01-25 00:57:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents>: HTTP status code is not handled or not allowed 2026-01-25 00:57:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oscillating-tower-fan-37-inch-height-w-remote-fa1-0069-06>: HTTP status code is not handled or not allowed 2026-01-25 00:57:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/plumbing_parts_accessories/pipe_hose_tubing_accessories/pipe_markers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:57:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/pens>: HTTP status code is not handled or not allowed 2026-01-25 00:57:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/p/pressure-sensitive-pipe-marker-direction-arrow-red-1 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:57:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quick-ship-m-series-emergency-crash-cart-w-six-drawers-and-breakaway-lock>: HTTP status code is not handled or not allowed 2026-01-25 00:57:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rectangle-easy-peel-labels-2-x-3-clear-80-pack-22822>: HTTP status code is not handled or not allowed 2026-01-25 00:57:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/column-sentry-fit-column-protector-4x4-square-opening-43-inches-h-small-yellow>: HTTP status code is not handled or not allowed 2026-01-25 00:57:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:57:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6308nr-open-wsnap-ring-40mm-bore-90mm-od>: HTTP status code is not handled or not allowed 2026-01-25 00:57:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/plumbing/plumbing_parts_accessories/pipe_hose_tubing_accessories/pipe_markers>: HTTP status code is not handled or not allowed 2026-01-25 00:57:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/plumbing_parts_accessories/pipe_hose_tubing_accessories/pipe_markers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:57:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-8-x-14-working-length-j-hook-style-c>: HTTP status code is not handled or not allowed 2026-01-25 00:58:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-narrow-procedure-cart-6-drawers-key-lock-18-w-x-18-l-x-40-3-4-h-purple>: HTTP status code is not handled or not allowed 2026-01-25 00:58:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts landed on page that is not a product page. 2026-01-25 00:58:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/suredry-10-scope-cabinet-w-dri-scope-aid-e-lock-30-w-x-24-d-x-93-h-beige>: HTTP status code is not handled or not allowed 2026-01-25 00:58:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-34-inch-high-post-stainless-steel-post-4-pack>: HTTP status code is not handled or not allowed 2026-01-25 00:58:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-shopping-basket-28-liter-with-nylon-handle-206-28l-drk-green>: HTTP status code is not handled or not allowed 2026-01-25 00:58:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-series-narrow-medical-cart-6-drawers-e-lock-24-3-4-wx22-lx43-3-4-h-hammertone-red>: HTTP status code is not handled or not allowed 2026-01-25 00:58:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s373pt-ansi-class-e-lightweight-rain-pants-hi-vis-lime-xl-61512>: HTTP status code is not handled or not allowed 2026-01-25 00:58:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 44477 pages (at 88 pages/min), scraped 22180 items (at 44 items/min) 2026-01-25 00:58:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plate-clamp-30-45-hinged-yellow-11-000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 00:58:09 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/plumbing/plumbing_parts_accessories/pipe_hose_tubing_accessories/pipe_markers landed on page that is not a product page. 2026-01-25 00:58:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inspection-gloves-denier-tricot-nylon-rolled-hem-mens-97-740-l>: HTTP status code is not handled or not allowed 2026-01-25 00:58:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-dia-x-6-working-length-j-hook-style-a>: HTTP status code is not handled or not allowed 2026-01-25 00:58:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oty-11830-1-2-inch-1-inch-galvanized-no-calk-roof-flashing-solar-12-1-2-inchl-x-9-inchw-base>: HTTP status code is not handled or not allowed 2026-01-25 00:58:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/retract-a-badge-id-carabiner-5-pack-retractable-key-reel-with-36-cord>: HTTP status code is not handled or not allowed 2026-01-25 00:58:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-tall-isolation-cart-4-drawers-37-1-2-w-x-22-l-x-42-63-100-h-forest-green>: HTTP status code is not handled or not allowed 2026-01-25 00:58:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6015zznr-double-shielded-wsnap-ring-75mm-bore-115mm-od>: HTTP status code is not handled or not allowed 2026-01-25 00:58:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/right-container-lift-12-l-x-6-w-x-8-h-yellow-19000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 00:58:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-feet-x-20-feet-sf-9-93-oz-water-resist-canvas-tarp-brown>: HTTP status code is not handled or not allowed 2026-01-25 00:58:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-72-spread-adjustable-lift-yellow-10000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 00:58:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:58:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trex-6300tc-one-piece-ice-traction-device-w-tungsten-carbide-studs2xl>: HTTP status code is not handled or not allowed 2026-01-25 00:58:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6016nr-open-wsnap-ring-80mm-bore-125mm-od>: HTTP status code is not handled or not allowed 2026-01-25 00:58:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/limited-hose-for-jkl-115s-and-jkl-240s-models-96>: HTTP status code is not handled or not allowed 2026-01-25 00:58:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swf32guu-2-id-round-flange-type-linear-bearing-wresin-retainer-seals-steel>: HTTP status code is not handled or not allowed 2026-01-25 00:58:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-910-half-finger-impact-gloves-wrist-support-black-xl-17712>: HTTP status code is not handled or not allowed 2026-01-25 00:58:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/fans_blowers/ceiling_fans/residential_ceiling_fans already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:58:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-series-lightweight-aluminium-crash-cart-5-drawers-30-w-x-22-l-x-37-1-4-h-light-blue>: HTTP status code is not handled or not allowed 2026-01-25 00:58:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents>: HTTP status code is not handled or not allowed 2026-01-25 00:58:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anti-skid-additive-course-16-oz-jar-12-case-1-case-cp-sg180-220>: HTTP status code is not handled or not allowed 2026-01-25 00:58:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poly-z-brite-heavy-duty-wire-shelving-ds18246z-24w-x-18d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 00:58:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/fans_blowers/ceiling_fans/residential_ceiling_fans landed on page that is not a product page. 2026-01-25 00:58:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-3-shelf-container-rack-bk18306n-30w-x-18d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 00:58:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-8-id-double-flanged-sheet-metal-clip-bearing-m250-for-072-135-sheet-metal>: HTTP status code is not handled or not allowed 2026-01-25 00:58:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/horizontal-unit-heater-10kw-480v-3-phase>: HTTP status code is not handled or not allowed 2026-01-25 00:58:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vent-control-label-50ml-bio-reaction-tube-sheet-non-sterile-500-pk>: HTTP status code is not handled or not allowed 2026-01-25 00:58:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/oil_fluid_management/oil_fluid_transfer already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:58:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/boss-open-hutch-bookcase-mahogany>: HTTP status code is not handled or not allowed 2026-01-25 00:58:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bridges_returns already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:58:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-short-emergency-crash-cart-md30-emg1-std-width-4-drawers-breakaway-lock-teal>: HTTP status code is not handled or not allowed 2026-01-25 00:58:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-tall-emergency-crash-cart-6-drawers-36-3-4-w-x-22-l-x-43-3-4-h-light-blue>: HTTP status code is not handled or not allowed 2026-01-25 00:58:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/social_distancing_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:58:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-plastic-2-shelf-flat-black-service-utility-cart-38-x-17-1-2-5-rubber-casters>: HTTP status code is not handled or not allowed 2026-01-25 00:58:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/oil_fluid_management/oil_fluid_transfer>: HTTP status code is not handled or not allowed 2026-01-25 00:58:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tall-isolation-cart-3-drawers-standard-key-lock-37-1-2-wx22-lx42-63-100-h-white>: HTTP status code is not handled or not allowed 2026-01-25 00:59:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-circulation-fan-vk20-w-guard-cord-mount-1-3-hp-5470-cfm-115-230v>: HTTP status code is not handled or not allowed 2026-01-25 00:59:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stop-please-wait-here-8-round>: HTTP status code is not handled or not allowed 2026-01-25 00:59:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyjohn-double-restroom-trailer>: HTTP status code is not handled or not allowed 2026-01-25 00:59:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyjohn-well-care-ada-portable-restroom-evergreen>: HTTP status code is not handled or not allowed 2026-01-25 00:59:04 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/social_distancing_signs landed on page that is not a product page. 2026-01-25 00:59:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/social_distancing_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:59:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-back-end-stops-workbench-top-96-w-x-30-d-x-3-h-gray>: HTTP status code is not handled or not allowed 2026-01-25 00:59:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bridges_returns landed on page that is not a product page. 2026-01-25 00:59:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/social_distancing_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:59:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1000ml-filter-system-pes-filter-material-0-10-181-m-90mm-sterile-polystyrene-12-pk>: HTTP status code is not handled or not allowed 2026-01-25 00:59:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 44569 pages (at 92 pages/min), scraped 22234 items (at 54 items/min) 2026-01-25 00:59:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-shelf-for-boltless-heavy-duty-die-rack-96-w-x-36-d-medium-gray>: HTTP status code is not handled or not allowed 2026-01-25 00:59:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/suredry-10-scope-cabinet-w-dri-scope-aid-e-lock-30-w-x-24-d-x-93-h-pink>: HTTP status code is not handled or not allowed 2026-01-25 00:59:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-16-18-insert-for-soft-wood-flush-851618-25>: HTTP status code is not handled or not allowed 2026-01-25 00:59:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fb-rfb330-motors-and-blowers-1-20-hp-3000-rpm-115v-open>: HTTP status code is not handled or not allowed 2026-01-25 00:59:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-column-medical-storage-cabinet-key-lock-glass-doors-42-wx29-3-5-dx76-h-mauve>: HTTP status code is not handled or not allowed 2026-01-25 00:59:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/social_distancing_signs>: HTTP status code is not handled or not allowed 2026-01-25 00:59:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/d1048-oem-motor-for-iec-permanent-split-capacitor-1-20-hp-1075-rpm-277v-open-ventilated>: HTTP status code is not handled or not allowed 2026-01-25 00:59:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/275-omni-telescoping-case-trade-show-case-43l-x-28w-x-8h-black>: HTTP status code is not handled or not allowed 2026-01-25 00:59:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/social_distancing_signs landed on page that is not a product page. 2026-01-25 00:59:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wedgebit-drill-bit-sdsplus-size-12-usable-10-l-overall-12-l>: HTTP status code is not handled or not allowed 2026-01-25 00:59:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lightweight-rugged-travel-case-foam-filled-808-carry-case-27l-x-24w-x-15-3-4h-black>: HTTP status code is not handled or not allowed 2026-01-25 00:59:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/13-pin-male-plow-side-harness-repair-kit>: HTTP status code is not handled or not allowed 2026-01-25 00:59:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m3-05-tapered-ultrasonic-insert-double-vane-dv-m30-th>: HTTP status code is not handled or not allowed 2026-01-25 00:59:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a20h2408sslp-continuous-hinge-wclamps-type-4x-2000x2400x800in-ss-type-304>: HTTP status code is not handled or not allowed 2026-01-25 00:59:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-32-flanged-press-insert-stainless-260-006-cr>: HTTP status code is not handled or not allowed 2026-01-25 00:59:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eco-black-ns-precut-anti-spastic-splint-thumb-piece-1-8-non-perforated-large>: HTTP status code is not handled or not allowed 2026-01-25 00:59:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/notrax-174-germs-stop-here-safety-message-mat-3-8-thick-4-x-6-blue>: HTTP status code is not handled or not allowed 2026-01-25 00:59:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/graphic-machine-labels-danger-flammable>: HTTP status code is not handled or not allowed 2026-01-25 00:59:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-40-flush-press-insert-stainless-240-004-cr>: HTTP status code is not handled or not allowed 2026-01-25 00:59:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a644phcw-polyproencl-wwindow-612x412x438in>: HTTP status code is not handled or not allowed 2026-01-25 00:59:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polly-products-econo-mizer-8-ft-backed-bench-with-arms-green-bench-green-frame>: HTTP status code is not handled or not allowed 2026-01-25 00:59:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a12106phc-polyproencl-1208x1009x638in>: HTTP status code is not handled or not allowed 2026-01-25 00:59:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polly-products-econo-mizer-6-ft-backed-bench-with-arms-brown-bench-black-frame>: HTTP status code is not handled or not allowed 2026-01-25 00:59:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machine-labels-danger-moving-machinery>: HTTP status code is not handled or not allowed 2026-01-25 00:59:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-18-ga-ta50-locker-with-2-shelves-a-drawer-14-bins-6-moveable-hooks>: HTTP status code is not handled or not allowed 2026-01-25 00:59:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zinc-die-cast-adjustable-lever-steel-38-16-78-stud-248l>: HTTP status code is not handled or not allowed 2026-01-25 00:59:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corner-rack-protector-contour-fits-3-inch-wide>: HTTP status code is not handled or not allowed 2026-01-25 00:59:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tapered-roller-bearing-cone-395s>: HTTP status code is not handled or not allowed 2026-01-25 00:59:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polycarbonate-surface-mounted-90-corner-guard-2-1-2-inch-wing-4-foot-h-clear-drilled>: HTTP status code is not handled or not allowed 2026-01-25 00:59:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a16h1606sslp-continuous-hinge-wclamps-type-4x-1600x1200x600in-a16p16>: HTTP status code is not handled or not allowed 2026-01-25 00:59:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-cantilever-arm-no-lip>: HTTP status code is not handled or not allowed 2026-01-25 00:59:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polycarbonate-surface-mounted-90-corner-guard-3-4-inch-wing-4-foot-h-clear-un-drilled>: HTTP status code is not handled or not allowed 2026-01-25 00:59:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vrtcl-concrete-pressure-tong-w-urethane-pads-2200-lbs-cap-1-3-8-6-1-8-grip-range>: HTTP status code is not handled or not allowed 2026-01-25 00:59:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/think-osha-10x14-vinyl-others-depend-on-your-don-t-take-chances>: HTTP status code is not handled or not allowed 2026-01-25 00:59:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powerpoint-hoist-ring-360-pivot-11-000-lb-load-capacity>: HTTP status code is not handled or not allowed 2026-01-25 00:59:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 00:59:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-bar-18-thick-x-2-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 00:59:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:00:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vertical-lifting-locking-screw-clamp-orange-steel-6000-lbs-capacity-2-opening>: HTTP status code is not handled or not allowed 2026-01-25 01:00:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/beds_mattresses/mattresses already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:00:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/name_badges_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:00:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-232-dn10-ball-poppet-type-robust-screw-to-connect-nipple-1-27-50-l>: HTTP status code is not handled or not allowed 2026-01-25 01:00:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:00:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buna-n-rubber-sheet-no-adhesive-70a-116-thick-x-6-wide-x-6-long>: HTTP status code is not handled or not allowed 2026-01-25 01:00:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24in-fifth-wheel-lube-disks-w-steel-retention-clip-fwd24>: HTTP status code is not handled or not allowed 2026-01-25 01:00:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 01:00:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 44662 pages (at 93 pages/min), scraped 22285 items (at 51 items/min) 2026-01-25 01:00:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machine-labels-caution-do-not-operate-machinery-without-guards>: HTTP status code is not handled or not allowed 2026-01-25 01:00:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/name_badges_kits>: HTTP status code is not handled or not allowed 2026-01-25 01:00:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esdanti-static-operations-coverall-navy-polyesternylon-regular-s>: HTTP status code is not handled or not allowed 2026-01-25 01:00:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machine-labels-caution-lock-out-safety-before-you-start>: HTTP status code is not handled or not allowed 2026-01-25 01:00:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs>: HTTP status code is not handled or not allowed 2026-01-25 01:00:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/american-lock-solid-aluminum-rectangular-padlock-brown-no-a1167brn>: HTTP status code is not handled or not allowed 2026-01-25 01:00:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-310-compressed-air-hose-kit-78-37-50-l>: HTTP status code is not handled or not allowed 2026-01-25 01:00:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 01:00:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-lab-coat-white-polycombed-cotton-tall-52>: HTTP status code is not handled or not allowed 2026-01-25 01:00:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dripless-caulk-guns-si300-10-oz-industrial-caulker>: HTTP status code is not handled or not allowed 2026-01-25 01:00:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/odp-w-rigid-base-364t-frame-60-hp-1800-rpm>: HTTP status code is not handled or not allowed 2026-01-25 01:00:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/beds_mattresses/mattresses landed on page that is not a product page. 2026-01-25 01:00:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pintle-combination-hitch-mounting-kit-bh8-ph5-ph8-ph15-8520>: HTTP status code is not handled or not allowed 2026-01-25 01:00:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/handle-grip-for-8300-8400-8500-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 01:00:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:00:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-stac-a-shelf-wire-locker-shelf-pink-7-case>: HTTP status code is not handled or not allowed 2026-01-25 01:00:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:00:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lodestar-electric-chain-hoist-1-2-ton-6-ft-lift-16-fpm-230v-1-2-hp-3-step-controller>: HTTP status code is not handled or not allowed 2026-01-25 01:00:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:00:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-general-security-combo-padlock-lf-shackle-purple-dial>: HTTP status code is not handled or not allowed 2026-01-25 01:00:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-built-in-combo-lock-ada-compliant-lift-handle-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:00:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/composites/composite_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:00:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/highland8482-transparent-tape-3-4-inch-x-1000-inch-1-inch-core-clear-12-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:00:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-sheet-3-thick-x-16-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 01:00:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-25 01:00:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-built-in-combination-lock-with-metal-dial-right-hinged>: HTTP status code is not handled or not allowed 2026-01-25 01:00:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-25 01:00:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-11-cable-drive-manual-wallboard-lift-150-capacity-lbs>: HTTP status code is not handled or not allowed 2026-01-25 01:00:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits landed on page that is not a product page. 2026-01-25 01:00:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-built-in-combo-lock-ada-compliant-latch-lift-handle-red>: HTTP status code is not handled or not allowed 2026-01-25 01:00:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-built-in-combo-lock-ada-compliant-wrap-around-latch-red>: HTTP status code is not handled or not allowed 2026-01-25 01:00:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tumbler-clrw-5oz-72qty-clear>: HTTP status code is not handled or not allowed 2026-01-25 01:00:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:00:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/protective-desk-pads-24-inch-x-36-inch-black>: HTTP status code is not handled or not allowed 2026-01-25 01:00:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/composites/composite_sheet_stock landed on page that is not a product page. 2026-01-25 01:00:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-duty-mixer-200-quarts-cap-air-motor-33-hp-gear-drive-4-impeller-dia>: HTTP status code is not handled or not allowed 2026-01-25 01:00:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ground-static-strap-for-w-40z-45z-b218-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 01:00:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lorell-cubicle-nameplate-black>: HTTP status code is not handled or not allowed 2026-01-25 01:00:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-plastic-bar-w-lse-acrylic-adhesive-12-thick-x-1-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 01:00:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spacer-for-mpw060e-a897-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 01:00:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 01:00:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camcover-10-5-8-amber>: HTTP status code is not handled or not allowed 2026-01-25 01:00:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/koch-filter-merv-11-high-capacity-xl11-pleated-panel-extended-surface-18-inch-w-x-24-inch-h-x-2-inch-d>: HTTP status code is not handled or not allowed 2026-01-25 01:00:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:00:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tumbler-dm-16oz-ruby-red>: HTTP status code is not handled or not allowed 2026-01-25 01:00:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-ner-dual-speed-elec-hoist-push-trolley-1-12-15-183>: HTTP status code is not handled or not allowed 2026-01-25 01:01:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-ed-dual-speed-electric-chain-hoist-525-lbs-10-4410>: HTTP status code is not handled or not allowed 2026-01-25 01:01:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-temp-sensor-for-wp-3000-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 01:01:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/composites/composite_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:01:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/300-series-middle-tool-chest-w-2-drawers-26-w-x-16-d-black>: HTTP status code is not handled or not allowed 2026-01-25 01:01:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock>: HTTP status code is not handled or not allowed 2026-01-25 01:01:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caster-wheel-for-7hbw30-pallet-trucks-to-00590-04780-71-xl>: HTTP status code is not handled or not allowed 2026-01-25 01:01:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/tool_storage/chests_roller_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:01:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/koch-filter-merv-8-standard-capacity-xl8-pleated-panel-ext-surface-16-inch-w-x-20-inch-h-x-4-inch-d>: HTTP status code is not handled or not allowed 2026-01-25 01:01:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 44755 pages (at 93 pages/min), scraped 22333 items (at 48 items/min) 2026-01-25 01:01:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:01:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x36x74-chrome-wire-shelving-with-24-quickpick-double-open-bins-yellow-1>: HTTP status code is not handled or not allowed 2026-01-25 01:01:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/composites/composite_rod_stock>: HTTP status code is not handled or not allowed 2026-01-25 01:01:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/koch-filter-merv-8-high-capacity-xl8-pleated-extended-surface-18-inch-w-x-18-inch-h-x-1-inch-d>: HTTP status code is not handled or not allowed 2026-01-25 01:01:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/tool_storage/chests_roller_cabinets>: HTTP status code is not handled or not allowed 2026-01-25 01:01:18 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 01:01:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-bank-top-chest-w-12-drawers-42-w-x-22-3-8-d-x-27-2-3-h-tan>: HTTP status code is not handled or not allowed 2026-01-25 01:01:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20w-smart-solar-led-street-light-with-integrated-lithium-battery>: HTTP status code is not handled or not allowed 2026-01-25 01:01:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:01:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gripper-front-butcher-frock-wo-pockets-white-polyestercotton-twill-s>: HTTP status code is not handled or not allowed 2026-01-25 01:01:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/solenoid-for-smx-45-walkie-pallet-truck>: HTTP status code is not handled or not allowed 2026-01-25 01:01:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:01:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-ner-electric-chain-hoist-motor-trolley-2-20-14>: HTTP status code is not handled or not allowed 2026-01-25 01:01:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spring-for-w-40z-b218-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 01:01:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-5209-double-row-angular-contact-bearing-open-45mm-bore-x-85mm-od-x-30-2mm-w>: HTTP status code is not handled or not allowed 2026-01-25 01:01:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4800-series-dual-triple-truck-bus-rv-v-bar-tire-chains-pair-0484555>: HTTP status code is not handled or not allowed 2026-01-25 01:01:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cook-pants-unhemmed-black-white-check-polyestercotton-twill-50-x-36>: HTTP status code is not handled or not allowed 2026-01-25 01:01:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-25 01:01:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/switch-for-wp-2000-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 01:01:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/maxtrac-snow-blower-garden-tractor-tire-chains-2-link-spacing-pair-1063456>: HTTP status code is not handled or not allowed 2026-01-25 01:01:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 01:01:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crc-knocker-loose-penetrating-solvents-16-oz-aerosol-can>: HTTP status code is not handled or not allowed 2026-01-25 01:01:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ballymore-12-step-perforated-cantilever-ladder-14-overhang>: HTTP status code is not handled or not allowed 2026-01-25 01:01:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:01:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/biohazard-specimen-transport-bag-8-x-10-zipper-with-score-line-and-document-pouch-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:01:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/retail_fixtures_displays_equipment/garment_supplies/sewing_machines already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:01:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24x36x74-chrome-wire-shelving-with-44-4-h-shelf-bins-green>: HTTP status code is not handled or not allowed 2026-01-25 01:01:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gripper-front-butcher-coat-wknit-cuffs-pocket-less-spun-polyester-light-blue-m>: HTTP status code is not handled or not allowed 2026-01-25 01:01:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-x-20-12-oz-flame-retardant-canvas-floor-runner>: HTTP status code is not handled or not allowed 2026-01-25 01:01:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/royal-basket-trucks-174-linen-cart-19-5-cu-ft-29-l-x-29-w-x-47-h-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:01:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-sheet-2-thick-x-36-long-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 01:01:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:01:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/bin_shelving/wire_bin_shelving already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:01:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/retail_fixtures_displays_equipment/garment_supplies/sewing_machines>: HTTP status code is not handled or not allowed 2026-01-25 01:01:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/axle-shaft-for-wpt45-pallet-trucks-cl-1120-130010-10>: HTTP status code is not handled or not allowed 2026-01-25 01:01:56 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits landed on page that is not a product page. 2026-01-25 01:01:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alligator-clip-c-clamp-w-3-ft-7x19-stranded-flex-steel-cable>: HTTP status code is not handled or not allowed 2026-01-25 01:02:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xx-garolite-rod-1-18-diameter-x-4-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 01:02:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-8482-outdoor-bottle-filler-bi-level-fountain-w-pet-station-filter-ss>: HTTP status code is not handled or not allowed 2026-01-25 01:02:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/shelving/bin_shelving/wire_bin_shelving>: HTTP status code is not handled or not allowed 2026-01-25 01:02:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/office_furniture/literature_mail_sorters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:02:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-8482-outdoor-pedestal-drinking-w-water-filter-ss>: HTTP status code is not handled or not allowed 2026-01-25 01:02:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/button-front-short-sleeve-cook-shirt-white-polyestercotton-4xl>: HTTP status code is not handled or not allowed 2026-01-25 01:02:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:02:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x36x75-steel-shelving-with-72-6-h-shelf-bins-red>: HTTP status code is not handled or not allowed 2026-01-25 01:02:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/snow_ice_removal/snow_blowers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:02:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-34-thick-x-2-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 01:02:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/monitor_risers_stands already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:02:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 44844 pages (at 89 pages/min), scraped 22385 items (at 52 items/min) 2026-01-25 01:02:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xx-garolite-sheet-38-thick-x-6-wide-x-6-long>: HTTP status code is not handled or not allowed 2026-01-25 01:02:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/office_furniture/literature_mail_sorters>: HTTP status code is not handled or not allowed 2026-01-25 01:02:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poly-load-wheel-for-rd-series-reach-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 01:02:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits>: HTTP status code is not handled or not allowed 2026-01-25 01:02:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-15-yellow-steel-toe-boot-polyurethane-size-8>: HTTP status code is not handled or not allowed 2026-01-25 01:02:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-25 01:02:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/snow_ice_removal/snow_blowers>: HTTP status code is not handled or not allowed 2026-01-25 01:02:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xx-garolite-sheet-116-thick-x-12-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 01:02:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:02:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/monitor_risers_stands landed on page that is not a product page. 2026-01-25 01:02:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/siesta-snow-outdoor-dining-chair-red>: HTTP status code is not handled or not allowed 2026-01-25 01:02:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/rainwear already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:02:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ridgeline-cap-style-vented-hard-hat-matte-black-graphite-pattern-4-point-ratchet-suspension>: HTTP status code is not handled or not allowed 2026-01-25 01:02:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/semi-clear-fda-silicone-rubber-sheet-no-adhesive-40a-14-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 01:02:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-auto-darkening-filter-for-wham30>: HTTP status code is not handled or not allowed 2026-01-25 01:02:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits landed on page that is not a product page. 2026-01-25 01:02:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/rainwear already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:02:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24x72x74-chrome-wire-shelving-with-110-shelf-bins-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:02:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/rainwear landed on page that is not a product page. 2026-01-25 01:02:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/three-shelf-mesh-truck-24-x-48>: HTTP status code is not handled or not allowed 2026-01-25 01:02:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/power_tools/drills_drivers_impact_wrenches/drill_driver_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:02:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-bar-12-thick-x-1-14-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 01:02:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:02:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ridgeline-vented-cap-style-hard-hat-hi-vis-green-4-point-ratchet-suspension>: HTTP status code is not handled or not allowed 2026-01-25 01:02:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/import-diamond-5-12-needle-file-round-pattern-grit-220-fine>: HTTP status code is not handled or not allowed 2026-01-25 01:02:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/rainwear landed on page that is not a product page. 2026-01-25 01:02:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blue-hinged-lid-1-1-cu-yd-self-dumping-plastic-hopper>: HTTP status code is not handled or not allowed 2026-01-25 01:02:42 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/power_tools/drills_drivers_impact_wrenches/drill_driver_accessories landed on page that is not a product page. 2026-01-25 01:02:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tube-holder-use-with-gtr-hd-series-tube-rotators-16-vertical-places-for-15ml-microcentrifuge-tubes>: HTTP status code is not handled or not allowed 2026-01-25 01:02:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 01:02:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-washer-m6-steel-zinc-clear-class-8-8-din-125a-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 01:02:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/map-rail-accessory-2-hook-clip-with-flag-holder>: HTTP status code is not handled or not allowed 2026-01-25 01:02:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/boards_easels/project_display_boards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:02:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carabiner-steel-automatic-lock-1-gate>: HTTP status code is not handled or not allowed 2026-01-25 01:02:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:02:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-men-s-boot-16-goliath-black-plain-toe-w-power-lug-outsole-pvc-size-6>: HTTP status code is not handled or not allowed 2026-01-25 01:02:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/fans_blowers/exhaust_fans/duct_fans already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:02:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slide-box-for-12-slides-cork-lined-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:02:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-pallet-scale-truck-5000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 01:02:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/boards_easels/project_display_boards>: HTTP status code is not handled or not allowed 2026-01-25 01:02:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:02:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 01:03:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/fans_blowers/exhaust_fans/duct_fans>: HTTP status code is not handled or not allowed 2026-01-25 01:03:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-circuit-breaker-compact-padlock>: HTTP status code is not handled or not allowed 2026-01-25 01:03:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-point-chin-strap-with-chin-guard-black>: HTTP status code is not handled or not allowed 2026-01-25 01:03:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-vertical-hanger-bar-portable-bin-cart-putty>: HTTP status code is not handled or not allowed 2026-01-25 01:03:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-handle-641244-641265>: HTTP status code is not handled or not allowed 2026-01-25 01:03:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-25 01:03:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 44934 pages (at 90 pages/min), scraped 22435 items (at 50 items/min) 2026-01-25 01:03:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-flat-shelf-portable-bin-cart-black>: HTTP status code is not handled or not allowed 2026-01-25 01:03:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-mobile-computer-lanstation-sliding-keyboard-tray-5-casters-2-locking-75-hx24-wx72-l-black>: HTTP status code is not handled or not allowed 2026-01-25 01:03:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:03:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anvil-1-12x1-galv-ci-hex-bushing>: HTTP status code is not handled or not allowed 2026-01-25 01:03:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/composites/composite_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:03:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:03:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shop-crane-2000-lbs-capacity-powered-wire-rope-winch>: HTTP status code is not handled or not allowed 2026-01-25 01:03:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/composites/composite_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:03:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storage-bin-cabinet-138-yellow-hook-on-bins-36w-24d-84h>: HTTP status code is not handled or not allowed 2026-01-25 01:03:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/clear_view_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:03:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ibgb-436vs-combination-8-inch-variable-grinder-with-4-inch-x-36-inch-belt-sander>: HTTP status code is not handled or not allowed 2026-01-25 01:03:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/hvac_service_tools_parts/hvac_thermometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:03:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=jtm-949evs-dp700-3x-k-dro-x-y-pwrfd already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:03:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 01:03:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nps-wood-science-lab-table-24-x-54-x-36-whiteboard-top>: HTTP status code is not handled or not allowed 2026-01-25 01:03:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/composites/composite_rod_stock landed on page that is not a product page. 2026-01-25 01:03:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 01:03:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polycarbonate-chair-rail-3-h-x-8-l-clear>: HTTP status code is not handled or not allowed 2026-01-25 01:03:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/composites/composite_sheet_stock landed on page that is not a product page. 2026-01-25 01:03:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gh-1440-3-lathe-with-newall-dp500-dro-and-collet-closer>: HTTP status code is not handled or not allowed 2026-01-25 01:03:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-height-melamine-top-folding-table-30-x-96-walnut-277524>: HTTP status code is not handled or not allowed 2026-01-25 01:03:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/hvac_service_tools_parts/hvac_thermometers landed on page that is not a product page. 2026-01-25 01:03:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jtm-1254rvs-with-newall-dp7003x-k-dro-x-powerfeed>: HTTP status code is not handled or not allowed 2026-01-25 01:03:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-rated-insulated-prime-coat-dual-latch-16x16-frc16x16pc-dul>: HTTP status code is not handled or not allowed 2026-01-25 01:03:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/cabinets/clear_view_cabinets landed on page that is not a product page. 2026-01-25 01:03:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/freezing-box-cardboard-16-place-4x4-format-for-50ml-centrifuge-tubes-white-48-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:03:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=jtm-949evs-dp700-3x-k-dro-x-y-pwrfd landed on page that is not a product page. 2026-01-25 01:03:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/siesta-ocean-square-resin-side-table-white>: HTTP status code is not handled or not allowed 2026-01-25 01:03:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wireless-tube-rack-16mm-full-size-72-place-nylon-yellow>: HTTP status code is not handled or not allowed 2026-01-25 01:03:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-swivel-ring-dsr-u-025>: HTTP status code is not handled or not allowed 2026-01-25 01:03:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/freezer-merch-2-section-47-18w-29-78d-78-58h-gdm-43f>: HTTP status code is not handled or not allowed 2026-01-25 01:03:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/refrigerated-chef-base-72-38l-72-38w-32-18d-20-38h-trcb-72>: HTTP status code is not handled or not allowed 2026-01-25 01:03:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tall-corrugated-boxes-7-x-7-x-14>: HTTP status code is not handled or not allowed 2026-01-25 01:03:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2913-insulated-handle>: HTTP status code is not handled or not allowed 2026-01-25 01:03:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ice-merch-2-section-no-tank-54-18w-29-78d-78-58h-gdim-49nt>: HTTP status code is not handled or not allowed 2026-01-25 01:03:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3000-0000-right-hand-swivel-stem-repair-kit-polished-chrome>: HTTP status code is not handled or not allowed 2026-01-25 01:03:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/pneumatic_hoses_connectors/pneumatic_quick_connect_couplings already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:03:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=etm-949-mill-with-2-axis-newall-dp700-dro-and-servo-x-y-axis-powerfeeds already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:03:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blue-ridge-ergonomics-task-stool-with-chrome-foot-ring-carbon>: HTTP status code is not handled or not allowed 2026-01-25 01:03:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-607-6401-series-607-screw-to-connect-nipple-3-4-body-size-3-4-female-npt>: HTTP status code is not handled or not allowed 2026-01-25 01:03:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60x30-mobile-steel-square-edge-work-bench-fixed-height-1-3-4-top>: HTTP status code is not handled or not allowed 2026-01-25 01:03:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=etm-949-mill-with-3-axis-acu-rite-300-knee-dro-and-x-y-axis-jet-powerfeeds already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:03:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-x-30-shop-top-square-edge-work-bench-adjustable-height-1-1-2-top>: HTTP status code is not handled or not allowed 2026-01-25 01:03:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=evs-949-mill-2-axis-acu-rite-303-dro-x-axis-jet-powerfeed-usa-powered-draw-bar already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:03:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/pneumatic_hoses_connectors/pneumatic_quick_connect_couplings>: HTTP status code is not handled or not allowed 2026-01-25 01:03:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-packing-boxes-12-x-12-x-12>: HTTP status code is not handled or not allowed 2026-01-25 01:03:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foam-nitrile-coated-gloves-gray-black-medium-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 01:03:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=etm-949-mill-with-2-axis-newall-dp700-dro-and-servo-x-y-axis-powerfeeds landed on page that is not a product page. 2026-01-25 01:03:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-nesting-dunnage-rack-60x24x12>: HTTP status code is not handled or not allowed 2026-01-25 01:03:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=etm-949-mill-with-3-axis-acu-rite-300-knee-dro-and-x-y-axis-jet-powerfeeds landed on page that is not a product page. 2026-01-25 01:03:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wireless-microtube-rack-with-handles-for-15ml-and-20ml-microcentrifuge-tubes-96-place-red>: HTTP status code is not handled or not allowed 2026-01-25 01:03:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-corrugated-mailer-9-x-6-x-2->: HTTP status code is not handled or not allowed 2026-01-25 01:04:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/market-tray-12x30-green>: HTTP status code is not handled or not allowed 2026-01-25 01:04:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/melamine-dry-erase-whiteboard-36-x-24-double-sided-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-25 01:04:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=evs-949-mill-2-axis-acu-rite-303-dro-x-axis-jet-powerfeed-usa-powered-draw-bar landed on page that is not a product page. 2026-01-25 01:04:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m4848-dual-side-open-blast-system-w-dust-collector>: HTTP status code is not handled or not allowed 2026-01-25 01:04:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15es20c-3p-1-5-inch-cast-iron-straight-centrifugal-end-suct-chem-pump-epdm-epr-seal-2hp-3-phase>: HTTP status code is not handled or not allowed 2026-01-25 01:04:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/tables/laboratory_tables already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:04:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pipette-diamond-pro-fixed-volume-20ul-yellow>: HTTP status code is not handled or not allowed 2026-01-25 01:04:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rack-for-15ml-and-20ml-microcentrifuge-tubes-reinforced-polypropylene-20-place-white>: HTTP status code is not handled or not allowed 2026-01-25 01:04:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/368c-95-3-4-inch-x-1-2-inch-cast-iron-straight-centrifugal-pump-buna-n-seal-1-2hp-3-phase-motor>: HTTP status code is not handled or not allowed 2026-01-25 01:04:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 45024 pages (at 90 pages/min), scraped 22477 items (at 42 items/min) 2026-01-25 01:04:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pipette-diamond-pro-adjustable-volume-100-1000ul-orange>: HTTP status code is not handled or not allowed 2026-01-25 01:04:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/abrasives/points_burrs/burrs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:04:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1848-sf-41-1-8-x-1-8-standard-cut-round-nose-tree-bur>: HTTP status code is not handled or not allowed 2026-01-25 01:04:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=evs-949-mill-3-axis-newall-dp700-knee-dro-x-y-axis-jet-pwrfds-usa-pwred-drw-br already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:04:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camrack-base-rack-7-1-2-inch-inside-stack-height-brown-nsf>: HTTP status code is not handled or not allowed 2026-01-25 01:04:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/tables/laboratory_tables landed on page that is not a product page. 2026-01-25 01:04:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camrack-base-rack-7-1-2-inch-inside-stack-height-navy-blue-nsf>: HTTP status code is not handled or not allowed 2026-01-25 01:04:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-sided-cantilever-192-inch-upright-72-inch-brace-kit-adder>: HTTP status code is not handled or not allowed 2026-01-25 01:04:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-pump-parpmp01620b-for-hurricane-370>: HTTP status code is not handled or not allowed 2026-01-25 01:04:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1845-sa-2-5-16-x-1-4-standard-cut-cylindrical-bur-w-o-end-cut>: HTTP status code is not handled or not allowed 2026-01-25 01:04:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/abrasives/points_burrs/burrs landed on page that is not a product page. 2026-01-25 01:04:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=evs-949-mill-3-axis-newall-dp700-knee-dro-x-y-axis-jet-pwrfds-usa-pwred-drw-br>: HTTP status code is not handled or not allowed 2026-01-25 01:04:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/shelf_trucks/job_site_security_trucks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:04:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securall-self-latch-mobile-counter-high-industrial-cabinet-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:04:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/telescoping-outer-boxes-14-1-2-x-14-1-2-x-40>: HTTP status code is not handled or not allowed 2026-01-25 01:04:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bottle-narrow-mouth-ldpe-attached-polypropylene-screw-cap-500ml-12-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:04:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-step-aluminum-rolling-ladder-24w-grip-tread-28d-top-step>: HTTP status code is not handled or not allowed 2026-01-25 01:04:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1853-sd-2-5-16-x-1-4-standard-cut-ball-shaped-bur>: HTTP status code is not handled or not allowed 2026-01-25 01:04:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bottle-narrow-mouth-polypropylene-bottle-attached-polypropylene-screw-cap-4-liters-1-gallon>: HTTP status code is not handled or not allowed 2026-01-25 01:04:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/shelf_trucks/job_site_security_trucks landed on page that is not a product page. 2026-01-25 01:04:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manual-shutters-60-exhaust-fans>: HTTP status code is not handled or not allowed 2026-01-25 01:04:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poly-covered-clipboard-beige>: HTTP status code is not handled or not allowed 2026-01-25 01:04:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unsupported-9-step-30-cantilever-ladder-serrated>: HTTP status code is not handled or not allowed 2026-01-25 01:04:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/esd_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:04:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/preprinted-dividers-220910-top-open-16-tab-set>: HTTP status code is not handled or not allowed 2026-01-25 01:04:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machine-grade-polypropylene-1-2-6000-600lbs-8x8>: HTTP status code is not handled or not allowed 2026-01-25 01:04:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cimi-shield-knock-out-beg-bug-killer-gallon-bottle-pc-bb10>: HTTP status code is not handled or not allowed 2026-01-25 01:04:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/abrasives/points_burrs/burrs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:04:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1852-sl-3l6-3-8-x-1-4-double-cut-included-angle-bur>: HTTP status code is not handled or not allowed 2026-01-25 01:04:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/floor_mats/esd_mats>: HTTP status code is not handled or not allowed 2026-01-25 01:04:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hinged-design-tilter>: HTTP status code is not handled or not allowed 2026-01-25 01:04:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/graduated-cylinder-pmp-tpx-molded-graduations-100ml>: HTTP status code is not handled or not allowed 2026-01-25 01:04:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stacking-guest-chair-lava-shell-four-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:04:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-1-4-x-77-1-2-h-deluxe-office-partition-panel-with-pass-thru-cable-gray-277551gy>: HTTP status code is not handled or not allowed 2026-01-25 01:04:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-0kw-277v-flowco-electric-tankless-water-heater>: HTTP status code is not handled or not allowed 2026-01-25 01:04:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1785-wall-mount-storage-system-steel-16l-x-33w-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 01:04:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bottle-narrow-mouth-amber-polypropylene-bottle-attached-screw-cap-2-liters-05-gallons>: HTTP status code is not handled or not allowed 2026-01-25 01:04:56 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/abrasives/points_burrs/burrs landed on page that is not a product page. 2026-01-25 01:04:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wl069-ezkneelerkneeboard-24-x-14>: HTTP status code is not handled or not allowed 2026-01-25 01:04:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/impact-sport-electronic-earmuff-1030942>: HTTP status code is not handled or not allowed 2026-01-25 01:05:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gantry-jib-guard-column-protector>: HTTP status code is not handled or not allowed 2026-01-25 01:05:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-x-24-inch-heavy-duty-adjustable-height-steel-work-platform-orange-5-h-to-8-h>: HTTP status code is not handled or not allowed 2026-01-25 01:05:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/statfree-hj-dissipative-dual-layer-rubber-tray-liner16inx24in>: HTTP status code is not handled or not allowed 2026-01-25 01:05:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-auto-floor-scrubber-17-cleaning-path-two-80-amp-batteries>: HTTP status code is not handled or not allowed 2026-01-25 01:05:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 45120 pages (at 96 pages/min), scraped 22534 items (at 57 items/min) 2026-01-25 01:05:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/570h075-standard-timing-belt-h-3-4-x-57-t114-trapezoidal>: HTTP status code is not handled or not allowed 2026-01-25 01:05:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-frame-guard-le>: HTTP status code is not handled or not allowed 2026-01-25 01:05:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ex503nist-heavy-duty-industrial-multimeter-orangegreen-nist-certified>: HTTP status code is not handled or not allowed 2026-01-25 01:05:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2050h075-standard-timing-belt-h-3-4-x-205-t410-trapezoidal>: HTTP status code is not handled or not allowed 2026-01-25 01:05:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:05:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/58-pieces-5-8-pro-series-steel-clamping-kit-534174>: HTTP status code is not handled or not allowed 2026-01-25 01:05:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-40bs53x118-12-pitch-1-18-finished-bore-53-teeth>: HTTP status code is not handled or not allowed 2026-01-25 01:05:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vending-cart-cover-for-food-well-full-size-13-x-21-black>: HTTP status code is not handled or not allowed 2026-01-25 01:05:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gray-universal-drum-top-pads-25-bale-22-x-22>: HTTP status code is not handled or not allowed 2026-01-25 01:05:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hook-set-for-mobile-panel-carts-stainless-steel-pkg-qty-20>: HTTP status code is not handled or not allowed 2026-01-25 01:05:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/425-8-100-cr-gn-425-cabinet-u-handles-round-steel-with-tapped-holes>: HTTP status code is not handled or not allowed 2026-01-25 01:05:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-dimpled-oil-only-heavy-weight-pads-100-bale-18-x-15>: HTTP status code is not handled or not allowed 2026-01-25 01:05:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-boltless-shelving-96-w-x-48-d-x-84-h-3-level-starter-no-deck-tan>: HTTP status code is not handled or not allowed 2026-01-25 01:05:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves landed on page that is not a product page. 2026-01-25 01:05:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/graduated-cylinder-polypropylene-molded-graduations-2000ml>: HTTP status code is not handled or not allowed 2026-01-25 01:05:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tornado-cvd-30-deluxe-upright-vacuum-15-cleaning-width-black>: HTTP status code is not handled or not allowed 2026-01-25 01:05:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-capacity-laptop-charging-station-black>: HTTP status code is not handled or not allowed 2026-01-25 01:05:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50bs34hx1-58-pitch-1-finished-bore-34-teeth>: HTTP status code is not handled or not allowed 2026-01-25 01:05:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-motor-direct-drive-reversible-280-rpm-1-35-hp>: HTTP status code is not handled or not allowed 2026-01-25 01:05:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-motor-direct-drive-reversible-85-rpm-0-31-hp>: HTTP status code is not handled or not allowed 2026-01-25 01:05:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/growt-hgbb4-4-bamboo-stakes-natural-color-25-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:05:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/impact-wrench-accessory-tool-boot>: HTTP status code is not handled or not allowed 2026-01-25 01:05:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securall-1-24x60-panel-and-2-48x60-panels>: HTTP status code is not handled or not allowed 2026-01-25 01:05:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-o-smith-single-phase-jet-pump-motor-3450-rpm-230-volts-3hp-1>: HTTP status code is not handled or not allowed 2026-01-25 01:05:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securall-portable-eyewash-station-6-gallon>: HTTP status code is not handled or not allowed 2026-01-25 01:05:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kenwood-nx-p1200nvk-5-watt-two-way-vhf-analog-digital-analog-portable-radio-151-159-mhz>: HTTP status code is not handled or not allowed 2026-01-25 01:05:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-o-smith-5-5-8-split-capacitor-condenser-fan-motor-460-volts-1075-rpm-3>: HTTP status code is not handled or not allowed 2026-01-25 01:05:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/replacement_shelves_decks/plastic_shelving_additional_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:05:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-40btl54-taper-bushed-12-pitch-54-teeth>: HTTP status code is not handled or not allowed 2026-01-25 01:05:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/test-tube-12-x-86mm-5ml-polystyrene-with-rim-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:05:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/prolube-43681-flexible-grease-gun-hose-with-spring-guard-3500-psi-18-inch-1-8-npt>: HTTP status code is not handled or not allowed 2026-01-25 01:05:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leviton-16362-20-amp-125-volt-decora-plus-duplex-receptacle-self-grounding-brown>: HTTP status code is not handled or not allowed 2026-01-25 01:05:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shrink-bags-16-x-22-x-100-gauge-100-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:05:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/replacement_shelves_decks/plastic_shelving_additional_shelves landed on page that is not a product page. 2026-01-25 01:05:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shopping-paper-bags-13-x-7-x-13-kraft-250-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:05:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/red-current-calender-year-2-dia-fluorescent-red-black>: HTTP status code is not handled or not allowed 2026-01-25 01:05:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-8482-inground-mount-grass>: HTTP status code is not handled or not allowed 2026-01-25 01:06:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/hvac_electric_motors/unit_bearing_fan_motors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:06:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-3-black-rectangle>: HTTP status code is not handled or not allowed 2026-01-25 01:06:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stinger-1-1-2-ton-motorcyle-atv-lift>: HTTP status code is not handled or not allowed 2026-01-25 01:06:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-10bs25hx32-metric-58-pitch-32mm-finished-bore-25-teeth>: HTTP status code is not handled or not allowed 2026-01-25 01:06:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fragile-glass-2-3-fluorescent-orange>: HTTP status code is not handled or not allowed 2026-01-25 01:06:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 45217 pages (at 97 pages/min), scraped 22591 items (at 57 items/min) 2026-01-25 01:06:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/endurance-wall-mount-desk-2-0-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:06:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50bs30hx138-58-pitch-1-38-finished-bore-30-teeth>: HTTP status code is not handled or not allowed 2026-01-25 01:06:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-4-4-shaded-pole-motor-115-volts-1500-rpm-5>: HTTP status code is not handled or not allowed 2026-01-25 01:06:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/desktop-charging-station-system-light-use-bundle-kbep-3b3c3>: HTTP status code is not handled or not allowed 2026-01-25 01:06:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/hvac_electric_motors/unit_bearing_fan_motors landed on page that is not a product page. 2026-01-25 01:06:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ibg-12ssb-12-inch-single-speed-industrial-buffer>: HTTP status code is not handled or not allowed 2026-01-25 01:06:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hazard-class-2-non-flammable-gas-4-4-green-white>: HTTP status code is not handled or not allowed 2026-01-25 01:06:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-o-smith-5-split-capacitor-motor-230-volts-1550-rpm>: HTTP status code is not handled or not allowed 2026-01-25 01:06:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-40bs39x118-12-pitch-1-18-finished-bore-39-teeth>: HTTP status code is not handled or not allowed 2026-01-25 01:06:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/right-to-know-hazard-4-4-white-red-black-blue-yellow>: HTTP status code is not handled or not allowed 2026-01-25 01:06:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-16bs14hx42-metric-1-pitch-42mm-finished-bore-14-teeth>: HTTP status code is not handled or not allowed 2026-01-25 01:06:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-2-heavy-duty-master-link-traffic-blue-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 01:06:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:06:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50b24hx34mpb-58-pitch-34-plain-bore-24-teeth>: HTTP status code is not handled or not allowed 2026-01-25 01:06:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60bs19hx1316-34-pitch-1-316-finished-bore-19-teeth>: HTTP status code is not handled or not allowed 2026-01-25 01:06:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-3-new-designed-lightweight-shirt-rip-stop-bottom-down-shirt-w-spf-50-lime-xl>: HTTP status code is not handled or not allowed 2026-01-25 01:06:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-2-lady-short-sleeve-t-shirt-lime-with-pink-side-xl>: HTTP status code is not handled or not allowed 2026-01-25 01:06:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/line-voltage-thermostat-electric-heat-mechanical-4-wire-non-programmable-heat-only>: HTTP status code is not handled or not allowed 2026-01-25 01:06:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-25 01:06:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-multi-color-vest-black-l-xl>: HTTP status code is not handled or not allowed 2026-01-25 01:06:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/curbside-truck-silver-ep-24w-x-48l-x-72h-4-wire-shelves-pneumatic-2-rigid-casters>: HTTP status code is not handled or not allowed 2026-01-25 01:06:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-slotted-pallet-rack-add-on-no-deck-120-inches-w-x-36-inches-d-x-144-inches-h>: HTTP status code is not handled or not allowed 2026-01-25 01:06:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-piece-6-point-flex-metric-socket-set-with-38-drive-tang>: HTTP status code is not handled or not allowed 2026-01-25 01:06:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/pest_control/bug_zappers_traps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:06:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qa-approved-4-4-blue-white>: HTTP status code is not handled or not allowed 2026-01-25 01:06:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-piece-pozidriv-dual-material-screwdriver-set>: HTTP status code is not handled or not allowed 2026-01-25 01:06:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1496-1-4-fnpt-kwik-grip-safety-air-chuck>: HTTP status code is not handled or not allowed 2026-01-25 01:06:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/bathroom_partition_replacement_hardware already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:06:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exq-pro-extreme-power-workstation-hutch-72-w-x-30-d-x-h-26-3-8-black-w-red-handle>: HTTP status code is not handled or not allowed 2026-01-25 01:06:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/pest_control/bug_zappers_traps landed on page that is not a product page. 2026-01-25 01:06:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rx-3-drawer-3-shelf-side-locker-19-w-x-25-d-x-h-61-black-w-red-trim>: HTTP status code is not handled or not allowed 2026-01-25 01:06:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-wd12aa2pz7-1-2-hp-1-phase-1700-rpm>: HTTP status code is not handled or not allowed 2026-01-25 01:06:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/tables/training_tables already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:06:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-dj2c2p18m-2-hp-1-phase-1730-rpm>: HTTP status code is not handled or not allowed 2026-01-25 01:07:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-5463-3-4-hp-1-phase-1075-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:07:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-6078-1-2-hp-1-phase-1425-rpm>: HTTP status code is not handled or not allowed 2026-01-25 01:07:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/antimicrobial-leather-conference-chair-black>: HTTP status code is not handled or not allowed 2026-01-25 01:07:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/privacy_screens already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:07:04 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/bathroom_partition_replacement_hardware landed on page that is not a product page. 2026-01-25 01:07:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nature-s-garden-healthy-heart-mix-1-2-oz-7-count-6-pack-gr>: HTTP status code is not handled or not allowed 2026-01-25 01:07:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medium-duty-hamper-with-foot-pedal-steel-white>: HTTP status code is not handled or not allowed 2026-01-25 01:07:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/tables/training_tables landed on page that is not a product page. 2026-01-25 01:07:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-1789-1-3-hp-1-phase-1725-1140-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:07:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=us-motor-5463-3-4-hp-1-phase-1075-rpm>: HTTP status code is not handled or not allowed 2026-01-25 01:07:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 45312 pages (at 95 pages/min), scraped 22650 items (at 59 items/min) 2026-01-25 01:07:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/three-panel-stationary-medical-privacy-screen-81-wx69-h-white-aml-vinyl-panels>: HTTP status code is not handled or not allowed 2026-01-25 01:07:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/partitions_dividers/privacy_screens landed on page that is not a product page. 2026-01-25 01:07:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-t1c1j-1-hp-1-phase-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-25 01:07:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-t2c2p18-2-hp-1-phase-1740-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:07:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/york-dark-chocolate-peppermint-patties-candy-35-2-oz-gr>: HTTP status code is not handled or not allowed 2026-01-25 01:07:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-t2c2p21z-2-hp-1-phase-1740-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:07:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lateral-44-3-drawer-fire-and-water-resistant-file-cabinet-light-gray-lat3w44lg>: HTTP status code is not handled or not allowed 2026-01-25 01:07:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-t12cm1j-1-2-hp-1-phase-3450-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:07:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=us-motor-1789-1-3-hp-1-phase-1725-1140-rpm>: HTTP status code is not handled or not allowed 2026-01-25 01:07:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-step-16-w-stainless-steel-fold-store-ladder-serrated-grating>: HTTP status code is not handled or not allowed 2026-01-25 01:07:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2gtl2-led-recessed-troffer-24-inch-l-x-24-inch-w-lp840-4000k-3300-lumens-silver>: HTTP status code is not handled or not allowed 2026-01-25 01:07:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=us-motor-t2c2p18-2-hp-1-phase-1740-rpm landed on page that is not a product page. 2026-01-25 01:07:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=us-motor-t2c2p21z-2-hp-1-phase-1740-rpm>: HTTP status code is not handled or not allowed 2026-01-25 01:07:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=us-motor-t12cm1j-1-2-hp-1-phase-3450-rpm landed on page that is not a product page. 2026-01-25 01:07:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/closed-shelving-drawer-unit-36x12x75-36-drawers-yellow>: HTTP status code is not handled or not allowed 2026-01-25 01:07:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wwe-khn67-40-1-h-56c-cast-iron-helical-bevel-speed-reducer-56c-input-flange-40-1-foot-mt>: HTTP status code is not handled or not allowed 2026-01-25 01:07:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-9-drawer-shallow-depth-cabinet-blue-2>: HTTP status code is not handled or not allowed 2026-01-25 01:07:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beaker-globe-glass-low-form-griffin-style-dual-graduations-astm-e960-1000ml-6-box>: HTTP status code is not handled or not allowed 2026-01-25 01:07:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flask-volumetric-globe-glass-class-b-to-contain-tc-astme288-250ml-6-box>: HTTP status code is not handled or not allowed 2026-01-25 01:07:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-agl10fl2s-1-1-8-hp-1-phase-3450-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 01:07:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-assembled-electronic-access-locker-double-tier-3-wide-12x18x78>: HTTP status code is not handled or not allowed 2026-01-25 01:07:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-assembled-electronic-access-locker-triple-tier-3-wide-12x12x78>: HTTP status code is not handled or not allowed 2026-01-25 01:07:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-u32p1g-1-5-hp-3-phase-3500-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:07:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-yard-front-loading-dumpster-with-bumpers-dark-cool-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:07:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:07:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:07:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microstat-2-high-120vac-amber-green-msl2-120ag>: HTTP status code is not handled or not allowed 2026-01-25 01:07:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-yc2p3b-2-hp-3-phase-1175-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:07:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/standard_duty_boltless_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:07:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:07:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-galvanized-steel-blower-wheel-9-31-32-diameter-1-2-bore-3>: HTTP status code is not handled or not allowed 2026-01-25 01:07:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lau-4-blade-condenser-propeller-18-diameter-ccw-rotation-3>: HTTP status code is not handled or not allowed 2026-01-25 01:07:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=us-motor-u32p1g-1-5-hp-3-phase-3500-rpm landed on page that is not a product page. 2026-01-25 01:07:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-inch-heavy-duty-plastic-chain-100-feet-safety-orange>: HTTP status code is not handled or not allowed 2026-01-25 01:07:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cork-strips-bulletin-bars-4-ft-6-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:07:51 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves landed on page that is not a product page. 2026-01-25 01:07:51 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=us-motor-yc2p3b-2-hp-3-phase-1175-rpm landed on page that is not a product page. 2026-01-25 01:07:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/standard_duty_boltless_shelves>: HTTP status code is not handled or not allowed 2026-01-25 01:07:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves landed on page that is not a product page. 2026-01-25 01:07:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wesco-galvanized-steel-narrow-aisle-platform-truck-60-x-24-1750-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 01:07:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-1696-3-4-hp-1-phase-1625-rpm>: HTTP status code is not handled or not allowed 2026-01-25 01:07:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-t-2000nl-no-lead-forged-brass-full-port-ball-valve-101-418nl>: HTTP status code is not handled or not allowed 2026-01-25 01:07:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves landed on page that is not a product page. 2026-01-25 01:07:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-x1p3b-1-hp-3-phase-1160-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:07:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/252952-clear-euro-style-print-holder-8-5-x-11>: HTTP status code is not handled or not allowed 2026-01-25 01:08:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-3846-1-4-hp-1-phase-1075-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:08:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-br34s2acr3-0-75-hp-3-phase-1720-rpm>: HTTP status code is not handled or not allowed 2026-01-25 01:08:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-4960-1-3-hp-1-phase-1075-rpm>: HTTP status code is not handled or not allowed 2026-01-25 01:08:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camwear-food-storage-container-18x26x6-875-gallon-capacity-safety-red>: HTTP status code is not handled or not allowed 2026-01-25 01:08:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/door-panel-cradle-dolly-4-polyurethane-wheels-350-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 01:08:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=us-motor-x1p3b-1-hp-3-phase-1160-rpm landed on page that is not a product page. 2026-01-25 01:08:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 45400 pages (at 88 pages/min), scraped 22698 items (at 48 items/min) 2026-01-25 01:08:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=us-motor-3846-1-4-hp-1-phase-1075-rpm>: HTTP status code is not handled or not allowed 2026-01-25 01:08:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interion-deluxe-mobile-office-partition-panel-48-1-4-w-x-96-h-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:08:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-machine-stand-mahogany>: HTTP status code is not handled or not allowed 2026-01-25 01:08:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/square-food-container-w-handles-18-quart-11-1-4x12-1-4x12-5-8-translucent-blue-graduation>: HTTP status code is not handled or not allowed 2026-01-25 01:08:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crock-cover-snap-on-for-cp27-white>: HTTP status code is not handled or not allowed 2026-01-25 01:08:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camwear-food-pan-1-2-size-long-2-1-2-deep-white-nsf>: HTTP status code is not handled or not allowed 2026-01-25 01:08:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hot-cold-cart-tall-profile-granite-gray-celsius-10-rear-casters>: HTTP status code is not handled or not allowed 2026-01-25 01:08:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-camtainer-beverage-carrier-insulated-plastic-2-3-4-gallon-capacity-dark-brown>: HTTP status code is not handled or not allowed 2026-01-25 01:08:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camcarrier-ultra-pancarrier-top-loading-approximately-cap-20-qt-stackable-slate-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:08:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/folding-shower-seat-with-sponge-cushion-le-hand-seat>: HTTP status code is not handled or not allowed 2026-01-25 01:08:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camlink-for-dunnage-racks>: HTTP status code is not handled or not allowed 2026-01-25 01:08:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/belt-replacement-part-push-sweeper>: HTTP status code is not handled or not allowed 2026-01-25 01:08:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storage-container-round-1-qt-6-1-16-diameter-x-5-high-natural-white-polyethylene-nsf>: HTTP status code is not handled or not allowed 2026-01-25 01:08:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anemostat-diffuser-5-20kw-downflow-unit-heaters>: HTTP status code is not handled or not allowed 2026-01-25 01:08:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plug-8mm-metal>: HTTP status code is not handled or not allowed 2026-01-25 01:08:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camshelving-elements-add-on-unit-18x48x72-brushed-graphite>: HTTP status code is not handled or not allowed 2026-01-25 01:08:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cast-iron-jaw-coupling-hub-style-l225-2-1-2-inch-bore-diameter-5-8-x-5-16-keyway>: HTTP status code is not handled or not allowed 2026-01-25 01:08:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cast-iron-jaw-coupling-hub-style-l095-3-4-inch-bore-diameter-3-16-x-3-32-keyway>: HTTP status code is not handled or not allowed 2026-01-25 01:08:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plug-in-y-conn-8mm-stud-x-8mm-tube>: HTTP status code is not handled or not allowed 2026-01-25 01:08:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-8npt-breather-vent-brass-body-100-micron-14-to-176-f>: HTTP status code is not handled or not allowed 2026-01-25 01:08:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-bspp-breather-vent-brass-body-304-stainless-steel-mesh-100-micron-14-to-176-f>: HTTP status code is not handled or not allowed 2026-01-25 01:08:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carbon-steel-inside-spring-caliper-0-10-5-8-range>: HTTP status code is not handled or not allowed 2026-01-25 01:08:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electronic-screw-thread-micrometer-0-1-inch-0-25mm-range>: HTTP status code is not handled or not allowed 2026-01-25 01:08:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-vernier-caliper-0-6-inch-0-150mm-range-1-128-inch-graduation>: HTTP status code is not handled or not allowed 2026-01-25 01:08:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/surface-roughness-specimen-set>: HTTP status code is not handled or not allowed 2026-01-25 01:08:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/expandable-conveyor-w-steel-rollers-720l-x-34-58w>: HTTP status code is not handled or not allowed 2026-01-25 01:08:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-sidedshield-36-inch-reach-w-o-interlock-3-arms-16-inch-l-x-14-inch-w-x-12-inch-h>: HTTP status code is not handled or not allowed 2026-01-25 01:08:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cafeteria-tray-17-78l-x-14w-antimicrobial-black>: HTTP status code is not handled or not allowed 2026-01-25 01:08:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/packaging/protective_packaging/roll_dispensers_cutters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:08:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/birdslide-small-6-seam-connectors-gray-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-25 01:08:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eled-rgb-led-ufo-round-post-light-20w>: HTTP status code is not handled or not allowed 2026-01-25 01:09:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/muskoka-adirondack-deck-dining-chair-46-12h-coastal-teak>: HTTP status code is not handled or not allowed 2026-01-25 01:09:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_seating/adirondack_lounger_chairs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:09:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/muskoka-adirondack-deck-dining-chair-40-12h-whitewash>: HTTP status code is not handled or not allowed 2026-01-25 01:09:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/outdoor_structures_storage/gazebos_pavilions_pergolas already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:09:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/muskoka-adirondack-3-piece-bistro-dining-set-30h-table-white>: HTTP status code is not handled or not allowed 2026-01-25 01:09:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labella-slab-door-wood-36w-x-80h-whiskey-oak>: HTTP status code is not handled or not allowed 2026-01-25 01:09:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_seating/patio_dining_chairs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:09:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/packaging/protective_packaging/roll_dispensers_cutters landed on page that is not a product page. 2026-01-25 01:09:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gainesville-gold-lines-slab-door-wood-28w-x-80h-whiskey-oak>: HTTP status code is not handled or not allowed 2026-01-25 01:09:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:09:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:09:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hog-heavenanti-fatigue-mat-58-thick-25l-x-3w-black-striped-border>: HTTP status code is not handled or not allowed 2026-01-25 01:09:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 45492 pages (at 92 pages/min), scraped 22753 items (at 55 items/min) 2026-01-25 01:09:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:09:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_seating/adirondack_lounger_chairs>: HTTP status code is not handled or not allowed 2026-01-25 01:09:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/outdoor_structures_storage/gazebos_pavilions_pergolas landed on page that is not a product page. 2026-01-25 01:09:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hog-heavenanti-fatigue-mat-78-thick-20l-x-5w-black-striped-border>: HTTP status code is not handled or not allowed 2026-01-25 01:09:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_seating/patio_dining_chairs landed on page that is not a product page. 2026-01-25 01:09:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-enhanced-visibility-softshell-insulated-jacket-2xl-black-charcoal>: HTTP status code is not handled or not allowed 2026-01-25 01:09:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:09:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:09:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergoforce-mens-waterproof-insulated-jacket-s-black>: HTTP status code is not handled or not allowed 2026-01-25 01:09:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:09:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extreme-hiker-boots-size-7-black>: HTTP status code is not handled or not allowed 2026-01-25 01:09:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mediumweight-wrapped-soup-spoon-polypropylene-white-pack-of-1000>: HTTP status code is not handled or not allowed 2026-01-25 01:09:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/renewable-compostable-clamshells-container-8-l-x-8-w-x-3-h-white-pack-of-200>: HTTP status code is not handled or not allowed 2026-01-25 01:09:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polystyrene-portion-cups-1-oz-translucent-pack-of-2500>: HTTP status code is not handled or not allowed 2026-01-25 01:09:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-cold-drink-cups-polypropylene-3-oz-translucent-pack-of-120>: HTTP status code is not handled or not allowed 2026-01-25 01:09:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/doors_windows/doors>: HTTP status code is not handled or not allowed 2026-01-25 01:09:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/doors_windows/doors>: HTTP status code is not handled or not allowed 2026-01-25 01:09:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/doors_windows/doors>: HTTP status code is not handled or not allowed 2026-01-25 01:09:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smartlock-container-w-3-compartment-8-l-x-7-1-2-w-x-2-5-8-h-pack-of-150>: HTTP status code is not handled or not allowed 2026-01-25 01:09:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caterline-pack-n-serve-plastic-bowl-80-oz-10-dia-x-4-h-black-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-25 01:09:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/non-lifting-synthetic-shackle-1-4-dia-x-4-l-2-3-4-ton-wll>: HTTP status code is not handled or not allowed 2026-01-25 01:09:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/impact-driver-kit-brushless-24v>: HTTP status code is not handled or not allowed 2026-01-25 01:09:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/colormaxx-paint-primer-12-oz-gloss-smoke-gray-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 01:09:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blood-bank-refrigerator-w-chart-recorder-23-cuft-cap-solid-door>: HTTP status code is not handled or not allowed 2026-01-25 01:09:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/k1-basic-wheelchair-w-full-length-arms-elevating-leg-rests-nylon-18w-seat>: HTTP status code is not handled or not allowed 2026-01-25 01:09:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elite-adult-stethoscope-22l-tubing-black>: HTTP status code is not handled or not allowed 2026-01-25 01:09:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/curad-compression-hosiery-w-15-20-mmhg-knee-high-x-large-size-d-regular-black>: HTTP status code is not handled or not allowed 2026-01-25 01:09:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/curad-cushioned-compression-socks-w-15-20-mmhg-knee-high-xx-large-size-e-regular-white>: HTTP status code is not handled or not allowed 2026-01-25 01:09:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tool-trolley-w-round-ring-control-vinyl-black>: HTTP status code is not handled or not allowed 2026-01-25 01:09:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flexible-stainless-steel-faucet-connector-36l-38-x-12-od-x-fip-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-25 01:09:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbk1827-rebuild-kit-for-gerber-shower-valves>: HTTP status code is not handled or not allowed 2026-01-25 01:09:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deck-mount-add-on-faucet-w-10-spout>: HTTP status code is not handled or not allowed 2026-01-25 01:09:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-faucet-w-8-spout--soap-dish>: HTTP status code is not handled or not allowed 2026-01-25 01:09:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/water-dumbbells-medium>: HTTP status code is not handled or not allowed 2026-01-25 01:09:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mounted-rack-for-belts-tubing--jump-ropes-26l-black>: HTTP status code is not handled or not allowed 2026-01-25 01:09:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/covered-tube-heavy-resistance-48l-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:09:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/physical_therapy/exercise_fitness_equipment/exercise_weights_weight_training already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:09:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beverage-center-1-glass-door-45-cuft-capacity>: HTTP status code is not handled or not allowed 2026-01-25 01:10:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/apartment-sized-refrigeratorfreezer-combo-unit-101-cuft-capacity-white>: HTTP status code is not handled or not allowed 2026-01-25 01:10:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/power_tools/drills_drivers_impact_wrenches/power_drills already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:10:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-tier-4-door-digital-wood-locker-12-w-x-15-d-x-72-h-cherry-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 01:10:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cabinet-w-fixed-shelves-assembled-36w-x-24d-x-72h-leaf-green>: HTTP status code is not handled or not allowed 2026-01-25 01:10:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sliding-door-cabinet-assembled-36w-x-21d-x-72h-dove-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:10:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bookcase-assembled-36w-x-13d-x-60h-hunter-green>: HTTP status code is not handled or not allowed 2026-01-25 01:10:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bookcase-assembled-36w-x-18d-x-60h-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:10:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/physical_therapy/exercise_fitness_equipment/exercise_weights_weight_training>: HTTP status code is not handled or not allowed 2026-01-25 01:10:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-hype-lite-heavy-duty-vest-lgxl-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:10:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onyx-enhanced-visibility-job-shirt-w-14-zipper-polyester-fleece-black-2xl>: HTTP status code is not handled or not allowed 2026-01-25 01:10:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/full_height_cabinet_w_drawers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:10:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 45585 pages (at 93 pages/min), scraped 22803 items (at 50 items/min) 2026-01-25 01:10:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/power_tools/drills_drivers_impact_wrenches/power_drills>: HTTP status code is not handled or not allowed 2026-01-25 01:10:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-sided-towable-multi-shelf-truck-2000-lb-cap-65-12l-x-30w-x-58-12h-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:10:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8386-hi-vis-shell-jacket-medium-orange>: HTTP status code is not handled or not allowed 2026-01-25 01:10:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8256z-self-extinguishing-vest-zipper-class-2-sm-lime>: HTTP status code is not handled or not allowed 2026-01-25 01:10:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/cabinets/full_height_cabinet_w_drawers>: HTTP status code is not handled or not allowed 2026-01-25 01:10:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triple-bulk-dispenser-17-1316w-x-5-78d-x-11-12h-transparent>: HTTP status code is not handled or not allowed 2026-01-25 01:10:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_storage_bench_protection/isolation_station_dispensers_organizers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:10:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-air-impact-wrench-quiet-2-extended-anvil-titanium-hammer-case-pistol-grip>: HTTP status code is not handled or not allowed 2026-01-25 01:10:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-tier-2-door-digital-locker-12in-w-x-18in-d-x-78in-h-blue-assembled>: HTTP status code is not handled or not allowed 2026-01-25 01:10:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bd-7075-w-bp-classic-battery-powered-walk-behind-floor-scrubber-28-cleaning-width>: HTTP status code is not handled or not allowed 2026-01-25 01:10:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cutting-bracket-w-blade-for-w-series-long-hand--w-series-i-bar-sealers>: HTTP status code is not handled or not allowed 2026-01-25 01:10:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/medical_laboratory_refrigeration/medical_refrigerators already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:10:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silicone-rubber-pad-for-kf-300h>: HTTP status code is not handled or not allowed 2026-01-25 01:10:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/lab_storage_bench_protection/isolation_station_dispensers_organizers>: HTTP status code is not handled or not allowed 2026-01-25 01:10:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tough-coat-adv-spray-paint-w-rust-barrier-technology-20-oz-cap-gloss-safety-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:10:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ball-shaped-bur-double-cut-2-18l-x-14-shank-dia>: HTTP status code is not handled or not allowed 2026-01-25 01:10:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cylindrical-w-endcut-bur-single-cut-2-12l-x-14-shank-dia>: HTTP status code is not handled or not allowed 2026-01-25 01:10:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cut-off-wheel-1-14-chuck-size-bulk-pack-of-20>: HTTP status code is not handled or not allowed 2026-01-25 01:10:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/medical_laboratory_refrigeration/medical_refrigerators>: HTTP status code is not handled or not allowed 2026-01-25 01:10:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/radius-cylindrical-bur-double-cut-6-34l-x-14-shank-dia-58-len-of-cut>: HTTP status code is not handled or not allowed 2026-01-25 01:10:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/led_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:10:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-strip-light-fixture-20w-2300-lumens-48-size>: HTTP status code is not handled or not allowed 2026-01-25 01:10:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/led_wraparound_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:10:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-cylindrical-wall-sconce-12w-800-lumens-80-cri-10-size-white>: HTTP status code is not handled or not allowed 2026-01-25 01:10:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mc-12-high-speed-microcentrifuge-w-12-place-rotor-15000-rpm-115v>: HTTP status code is not handled or not allowed 2026-01-25 01:10:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/outdoor_lighting_fixtures/outdoor_sconces already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:10:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tamper-proof-magnetic-pull-cord-alarm-4l-x-3w-x-1h>: HTTP status code is not handled or not allowed 2026-01-25 01:10:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/360-aisle-signal-1-box-4-sensors-3-lights-15l-cord>: HTTP status code is not handled or not allowed 2026-01-25 01:10:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/led_bulbs>: HTTP status code is not handled or not allowed 2026-01-25 01:10:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/led_wraparound_lights>: HTTP status code is not handled or not allowed 2026-01-25 01:10:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ball-end-mill-power-a-2-flute-2-12l-x-14-dia-34-cut-length>: HTTP status code is not handled or not allowed 2026-01-25 01:10:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-add-on-48l-x-36w-x-72h-black>: HTTP status code is not handled or not allowed 2026-01-25 01:10:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/outdoor_lighting_fixtures/outdoor_sconces>: HTTP status code is not handled or not allowed 2026-01-25 01:10:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-sided-starter-36l-x-20w-x-84h-antique-white>: HTTP status code is not handled or not allowed 2026-01-25 01:10:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-starter-48l-x-36w-x-54h-antique-white>: HTTP status code is not handled or not allowed 2026-01-25 01:10:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/end-unit-w-t-joiner-support-clip-48l-x-16w-x-60h-antique-white>: HTTP status code is not handled or not allowed 2026-01-25 01:11:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dawn-professional-manual-pot--pan-dish-detergent-original-scent-1-gal-bottle-pk-of-2>: HTTP status code is not handled or not allowed 2026-01-25 01:11:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v-flex-full-body-safety-harness-tongue-buckle-leg-strap-sxl>: HTTP status code is not handled or not allowed 2026-01-25 01:11:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cylinder-graduated-globe-glass-class-b-to-deliver-td-dual-grads-astm-e1272-2000ml>: HTTP status code is not handled or not allowed 2026-01-25 01:11:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/air_conditioners_chillers/wall_air_conditioners already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:11:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/skullerz-aegir-mi-half-frame-safety-glasses-blue-mirror-lens-orange-frame>: HTTP status code is not handled or not allowed 2026-01-25 01:11:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/identification-tag-4-inch-x-2-inch-tan-37082>: HTTP status code is not handled or not allowed 2026-01-25 01:11:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cottony-soft-sanitizing-hand-wipes-individually-wrapped-5l-x-7w-white-pk-of-1000>: HTTP status code is not handled or not allowed 2026-01-25 01:11:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 45677 pages (at 92 pages/min), scraped 22859 items (at 56 items/min) 2026-01-25 01:11:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/healthy-soap-05-pcmx-foam-refill-for-cs8-dispensers-1200-ml-capacity-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-25 01:11:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exhalation-valve-filter-604-for-6000-series-respirators>: HTTP status code is not handled or not allowed 2026-01-25 01:11:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/protect-liquid-cooled-standby-generator-100kw-120-208v-3-phase-lp>: HTTP status code is not handled or not allowed 2026-01-25 01:11:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=protect-liquid-cooled-standby-generator-150kw-277-480v-3-phase-ng already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:11:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-battery-agm-12v-120ah-cat-174-c26r-26-auto-ride-on-floor-scrubber-641811>: HTTP status code is not handled or not allowed 2026-01-25 01:11:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ambu-adult-bag-valve-mask-w-hepa-filter>: HTTP status code is not handled or not allowed 2026-01-25 01:11:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trailgrip-work-athletic-hiker-boots-alloy-toe-size-12w-4-inch-h-coyote-black>: HTTP status code is not handled or not allowed 2026-01-25 01:11:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-safety-crafted-waterproof-logger-work-boots-steel-toe-size-10-5w-8-inch-h-black>: HTTP status code is not handled or not allowed 2026-01-25 01:11:21 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/air_conditioners_chillers/wall_air_conditioners landed on page that is not a product page. 2026-01-25 01:11:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waterless-surgical-scrub-gel-hand-sanitizer-1200-ml-refill-for-cs-8-dispenser-2-ctn>: HTTP status code is not handled or not allowed 2026-01-25 01:11:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tuff-job-s500-high-performance-wipers-9-1-4-inch-x-12-1-2-inch-white-176-box-10-box-carton>: HTTP status code is not handled or not allowed 2026-01-25 01:11:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=protect-liquid-cooled-standby-generator-150kw-277-480v-3-phase-ng landed on page that is not a product page. 2026-01-25 01:11:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-safety-crafted-waterproof-work-boots-steel-toe-size-9-5m-6-inch-h-dark-brown>: HTTP status code is not handled or not allowed 2026-01-25 01:11:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-kitchen-roll-towels-2-ply-6-x-11-white-110-roll-12-rolls-carton>: HTTP status code is not handled or not allowed 2026-01-25 01:11:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/select-kitchen-roll-towels-2-ply-8-x-11-250-roll-12-carton>: HTTP status code is not handled or not allowed 2026-01-25 01:11:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35sh40-38-pitch-qd-bushed-40-teeth>: HTTP status code is not handled or not allowed 2026-01-25 01:11:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lk4405bf-outdoor-wall-mount-bottle-filling-station-black>: HTTP status code is not handled or not allowed 2026-01-25 01:11:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-100bs18hx11516-1-14-pitch-1-1516-finished-bore-18-teeth>: HTTP status code is not handled or not allowed 2026-01-25 01:11:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-40bs17hx78-12-pitch-78-finished-bore-17-teeth>: HTTP status code is not handled or not allowed 2026-01-25 01:11:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zig-elusion-heritage-work-high-top-sneaker-composite-toe-size-16m-black-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:11:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-24bs13hx42-metric-1-12-pitch-42mm-finished-bore-13-teeth>: HTTP status code is not handled or not allowed 2026-01-25 01:11:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/groundbreaker-work-boots-w-external-met-guard-steel-toe-size-14m-8-inch-h-brown>: HTTP status code is not handled or not allowed 2026-01-25 01:11:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/attest-rapid-bi-steam-test-pack-41382-16-bx-4-bx-cs>: HTTP status code is not handled or not allowed 2026-01-25 01:11:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/surgical_supplies/sterilization already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:11:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-boltless-shelving-60-in-w-x-24-in-d-x-72-in-h-5-shelves-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:11:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:11:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-racks-for-30-x-15-ml-20-x-50-ml-tubes-orange-5-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:11:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/truflex-fly-skylar-sneaker-composite-toe-size-11-5m-black>: HTTP status code is not handled or not allowed 2026-01-25 01:11:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-propette-le-single-channel-pipette-10-100ul>: HTTP status code is not handled or not allowed 2026-01-25 01:11:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/truflex-fly-blucher-sneaker-composite-toe-size-8-5w-black-gold>: HTTP status code is not handled or not allowed 2026-01-25 01:11:50 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/surgical_supplies/sterilization landed on page that is not a product page. 2026-01-25 01:11:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:11:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50bs17hx118-58-pitch-1-18-finished-bore-17-teeth>: HTTP status code is not handled or not allowed 2026-01-25 01:11:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-spreader-triangle-30-x-207-mm-10-sterile-bag-10-bags-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:11:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:11:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/true-skate-inspired-work-shoes-composite-toe-size-11m-navy>: HTTP status code is not handled or not allowed 2026-01-25 01:11:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:11:56 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-25 01:11:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35bs30hx78-38-pitch-78-finished-bore-30-teeth>: HTTP status code is not handled or not allowed 2026-01-25 01:11:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guldmann-b7-medical-replacement-battery-9ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 01:12:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-25 01:12:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:12:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50sds26h-qd-bushed-58-pitch-26-teeth>: HTTP status code is not handled or not allowed 2026-01-25 01:12:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:12:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sealed-lead-acid---agm---vrla-battery-1-3ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 01:12:04 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-25 01:12:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:12:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-25 01:12:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alaris-medical-1310-medical-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 01:12:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 45767 pages (at 90 pages/min), scraped 22909 items (at 50 items/min) 2026-01-25 01:12:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots>: HTTP status code is not handled or not allowed 2026-01-25 01:12:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-25 01:12:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-25 01:12:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caterpillar-572-industrial-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 01:12:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-seal-bubble-bags-9-x-13-1-2-350-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:12:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50bs23hx114-58-pitch-1-14-finished-bore-23-teeth>: HTTP status code is not handled or not allowed 2026-01-25 01:12:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nano-x1-adventure-athletic-work-shoes-composite-toe-size-7-5w-slate-blue-salmon>: HTTP status code is not handled or not allowed 2026-01-25 01:12:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-41bs30hx1-12-pitch-1-finished-bore-30-teeth>: HTTP status code is not handled or not allowed 2026-01-25 01:12:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adi-pwps12600-alarm-replacement-battery-75ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 01:12:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adi-vista-250bpt-alarm-replacement-battery-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 01:12:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/tool_storage/jobsite_boxes_storage already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:12:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/altronix-al400ulm-alarm-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 01:12:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ademco-vista-20p-alarm-replacement-battery-4-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 01:12:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merits-mp1in-wheelchair-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 01:12:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-40bs09hx12-12-pitch-12-finished-bore-9-teeth>: HTTP status code is not handled or not allowed 2026-01-25 01:12:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lavante-trail-2-running-work-shoes-composite-toe-size-7-5m-gray-red-black>: HTTP status code is not handled or not allowed 2026-01-25 01:12:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nano-x1-adventure-athletic-work-shoes-composite-toe-size-9-5m-gray-black>: HTTP status code is not handled or not allowed 2026-01-25 01:12:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/tool_storage/jobsite_boxes_storage landed on page that is not a product page. 2026-01-25 01:12:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nano-x1-adventure-athletic-work-shoes-composite-toe-size-12m-gray-black>: HTTP status code is not handled or not allowed 2026-01-25 01:12:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ag10-series-4-x-14-solid-alum-fixed-bar-supply-return-air-vent-grille-white-matte>: HTTP status code is not handled or not allowed 2026-01-25 01:12:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:12:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nano-x1-adventure-athletic-work-shoes-composite-toe-size-8-5m-silver-gray-clay-black>: HTTP status code is not handled or not allowed 2026-01-25 01:12:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-loop-sew-on-2-inch-x-75-feet>: HTTP status code is not handled or not allowed 2026-01-25 01:12:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-20bs13hx40-metric-1-14-pitch-40mm-finished-bore-13-teeth>: HTTP status code is not handled or not allowed 2026-01-25 01:12:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wet-umbrella-bag-holdersign-panel>: HTTP status code is not handled or not allowed 2026-01-25 01:12:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35b12hx12mpb-38-pitch-12-plain-bore-12-teeth>: HTTP status code is not handled or not allowed 2026-01-25 01:12:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-25 01:12:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-25bs13hx14-14-pitch-14-finished-bore-13-teeth>: HTTP status code is not handled or not allowed 2026-01-25 01:12:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/food_storage/food_storage_containers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:12:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/09797-food-pan-polycarbonate-15-qt-stackable-clear>: HTTP status code is not handled or not allowed 2026-01-25 01:12:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:12:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/toro-724-lawn-and-garden-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 01:12:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/best-r-2-emergency-light-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 01:12:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shoprider-777e-wheelchair-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 01:12:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-40b10hx12mpb-12-pitch-12-plain-bore-10-teeth>: HTTP status code is not handled or not allowed 2026-01-25 01:12:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-25a48x12-a-plate-14-pitch-12-bore-48-teeth>: HTTP status code is not handled or not allowed 2026-01-25 01:12:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/food_storage/food_storage_containers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:12:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/column-protector-round-for-24-dia-round-columns-black-3carton>: HTTP status code is not handled or not allowed 2026-01-25 01:12:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/food_storage/food_storage_containers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:13:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classic-leather-boots-brown-15f-comfort-rating-size-95>: HTTP status code is not handled or not allowed 2026-01-25 01:13:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/food_storage/food_storage_containers>: HTTP status code is not handled or not allowed 2026-01-25 01:13:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:13:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-25 01:13:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:13:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classic-leather-boots-brown-15f-comfort-rating-size-14>: HTTP status code is not handled or not allowed 2026-01-25 01:13:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:13:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-e-a-standard-wood-beam-socket-1845>: HTTP status code is not handled or not allowed 2026-01-25 01:13:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:13:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/food_storage/food_storage_containers>: HTTP status code is not handled or not allowed 2026-01-25 01:13:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 45853 pages (at 86 pages/min), scraped 22956 items (at 47 items/min) 2026-01-25 01:13:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evolve-skate-inspired-high-top-work-shoes-composite-toe-size-13m-black-yellow>: HTTP status code is not handled or not allowed 2026-01-25 01:13:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-25 01:13:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-u-handle-803l-110-dia-black-m8-516-socket-cap-screw>: HTTP status code is not handled or not allowed 2026-01-25 01:13:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-25 01:13:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pressboard-report-cover-prong-clip-letter-2-capacity-red-2-piece-prong>: HTTP status code is not handled or not allowed 2026-01-25 01:13:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/food_storage/food_storage_containers landed on page that is not a product page. 2026-01-25 01:13:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats landed on page that is not a product page. 2026-01-25 01:13:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pressguard-coated-report-cover-prong-clip-legal-2-capacity-black>: HTTP status code is not handled or not allowed 2026-01-25 01:13:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-25 01:13:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honeywell-remote-bulb-refrigeration-temp-controller-t6031a1029-30-f-to-90f-8-ft-capillary>: HTTP status code is not handled or not allowed 2026-01-25 01:13:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/peg-perego-270-ride-on-toys-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 01:13:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/can-am-outlander-400-400cc-atv-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 01:13:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commercial-bottleless-hot-cold-cooler-w-filtration-black-w-grey-trim>: HTTP status code is not handled or not allowed 2026-01-25 01:13:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kymco-250cc-motorcycle-replacement-battery-2009-2010-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 01:13:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-frame-motor-wwht3-12-213t-gp-tefc-rigid-3-ph-213t-43-fla>: HTTP status code is not handled or not allowed 2026-01-25 01:13:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buell-rr1000-1000cc-motorcycle-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 01:13:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mbk-evolis-50cc-motorcycle-replacement-battery-1998-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 01:13:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cc-pump-motor-wwe15-36-215jp-tefc-rigid-c-3-ph-215jp-15-hp-3600-rpm>: HTTP status code is not handled or not allowed 2026-01-25 01:13:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/healthflex-privacy-screen-5-panel-primary-yellow>: HTTP status code is not handled or not allowed 2026-01-25 01:13:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/healthflex-privacy-screen-5-panel-mallard>: HTTP status code is not handled or not allowed 2026-01-25 01:13:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pm-dc-motor-wpmdc13-18-90v-56cb-tenv-tefc-56c-90v-1-3-hp-1800-rpm>: HTTP status code is not handled or not allowed 2026-01-25 01:13:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chs-2-womens-restroom-sign-bl>: HTTP status code is not handled or not allowed 2026-01-25 01:13:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pm-dc-motor-wpmdc14-18-90v-56cb-tenv-tefc-56c-90v-1-4-hp-1800-rpm>: HTTP status code is not handled or not allowed 2026-01-25 01:13:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yamaha-vmax-700-xtc-700cc-snowmobile-replacement-battery-12v-i>: HTTP status code is not handled or not allowed 2026-01-25 01:13:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-frame-motor-wwht15-12-284t-gp-tefc-rigid-3-ph-284t-193-fla>: HTTP status code is not handled or not allowed 2026-01-25 01:13:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chs-5-girls-restroom-sign-bl>: HTTP status code is not handled or not allowed 2026-01-25 01:13:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-16-ga-steel-foot-guards-steel-toe-clip-leather-strap-5w-standard-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 01:13:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-mxz-tnt-1200-1170cc-snowmobile-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 01:13:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15ft-usb-30-superspeed-device-cable-5gbps-a-male-b-male-black>: HTTP status code is not handled or not allowed 2026-01-25 01:13:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carton-sealing-tape-mixed-merchandise-2-x-110-yds-redwhite-pack-of-36>: HTTP status code is not handled or not allowed 2026-01-25 01:13:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbi-sala174-nano-lok-personal-self-retracting-lifeline-w-steel-snap-anchor-hook-6l>: HTTP status code is not handled or not allowed 2026-01-25 01:13:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-purpose-metal-access-panel-cam-lock-24l-x-30w>: HTTP status code is not handled or not allowed 2026-01-25 01:13:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-visibility-class-3-waterproof-quilt-lined-bomber-jacket-lime-black-xl>: HTTP status code is not handled or not allowed 2026-01-25 01:13:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ranger-agm1234t-sealed-lead-acid-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 01:13:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-in-heavy-duty-plastic-chain-50-feet-yellow>: HTTP status code is not handled or not allowed 2026-01-25 01:13:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/epdm-commercial-grade-60a-rubber-sheet-1-16-thick-x-3-x-18-length-black>: HTTP status code is not handled or not allowed 2026-01-25 01:13:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/painting_supplies/specialty_coatings_paint already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:13:58 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-25 01:13:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-blade-for-katanaboy-500mm>: HTTP status code is not handled or not allowed 2026-01-25 01:13:59 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:23 2026-01-25 01:14:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extreme-cold-weather-engine-block-heater-for-automatic-standby-80w-46l-generators>: HTTP status code is not handled or not allowed 2026-01-25 01:14:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/solar-booster-pac-jump-starter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 01:14:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:14:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yellow-post-safety-barrier-11-ft-yellow-black-belt-caution-do-not-enter-w-roller-base>: HTTP status code is not handled or not allowed 2026-01-25 01:14:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/painting_supplies/specialty_coatings_paint>: HTTP status code is not handled or not allowed 2026-01-25 01:14:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caterpillar-e300-industrial-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 01:14:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/boards_easels/dry_erase_whiteboards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:14:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/calm90-151-56c-aluminum-worm-gear-reducer-90mm-151-56c-nema-frame>: HTTP status code is not handled or not allowed 2026-01-25 01:14:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 45950 pages (at 97 pages/min), scraped 23010 items (at 54 items/min) 2026-01-25 01:14:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/close-style-deep-bin-shelving-8-shelves-starter-unit-36w-x-72d-x-87h-bluegray>: HTTP status code is not handled or not allowed 2026-01-25 01:14:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions landed on page that is not a product page. 2026-01-25 01:14:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-tech-closed-shelving-6-shelves-starter-unit-500-lb-cap-36w-x-24d-x-87h-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:14:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/champion-16l-b-sealed-lead-acid-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 01:14:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/upsonic-pcm-35-sealed-lead-acid-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 01:14:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/boards_easels/dry_erase_whiteboards landed on page that is not a product page. 2026-01-25 01:14:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sealer-sales-w-450a-18-2-7mm-automatic-sealer>: HTTP status code is not handled or not allowed 2026-01-25 01:14:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/northflex-cold-grip-plus-5-cut-resistant-gloves-hi-vis-orange-black-size-xxl-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 01:14:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-in-aluminum-wall-exhaust-shutter-front-flange>: HTTP status code is not handled or not allowed 2026-01-25 01:14:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crown-12ce7-5-sealed-lead-acid-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 01:14:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-tech-open-style-metal-shelving-6-shelves-375-lb-cap-48w-x-12d-x-87h-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:14:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ritar-ra12-38-sealed-lead-acid-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 01:14:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-in-x-12-in-steel-door-louver>: HTTP status code is not handled or not allowed 2026-01-25 01:14:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-tech-open-style-shelving-sway-braces-6-shelves-36w-x-18d-x-87h-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:14:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mighty-products-mn-dn-0818-debris-safety-netting-8-x-18>: HTTP status code is not handled or not allowed 2026-01-25 01:14:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-tech-open-stand-alone-shelving-5-shelves-400-lb-cap-48w-x-12d-x-87h-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:14:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stair-tread-round-nose-36l-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:14:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cove-base-pinnacle-rubber-no-toe-4x18x120-coil-blackbrown>: HTTP status code is not handled or not allowed 2026-01-25 01:14:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/fuel_safety_cans/gas_fuel_cans_caddies already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:14:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-tech-closed-stand-alone-shelving-5-shelves-900-lb-cap-48w-x-24d-x-87h-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:14:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/falcon-electric-sg15k-1t-sg2k-1t-sg2k-1t-hw-12v-7ah-ups-battery>: HTTP status code is not handled or not allowed 2026-01-25 01:14:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e-track-weld-on-contoured-unfinished-single-strap-anchor>: HTTP status code is not handled or not allowed 2026-01-25 01:14:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pivot-hitch-pin-nickelchrome-plated-58>: HTTP status code is not handled or not allowed 2026-01-25 01:14:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exide-ep26-12w-sealed-lead-acid-replacement-battery-26ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 01:14:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/fuel_safety_cans/gas_fuel_cans_caddies landed on page that is not a product page. 2026-01-25 01:14:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-rackall-model-ra18-high-temp-polysulfone-nonesd-25-slots>: HTTP status code is not handled or not allowed 2026-01-25 01:14:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-manual-chain-hoist-14-ton-capacity-30-lift>: HTTP status code is not handled or not allowed 2026-01-25 01:14:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cryo-coder-for-skirted-base-vials-white-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 01:14:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cryo-coder-for-skirted-base-vials-yellow-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 01:14:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-manual-chain-hoist-5-ton-capacity-10-lift>: HTTP status code is not handled or not allowed 2026-01-25 01:14:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extinguisher-cabinet-full-tempered-safety-glass-stl-surface-mt-14-5-8-inch-l-x-29-3-8-inch-h-x-8-inch-d>: HTTP status code is not handled or not allowed 2026-01-25 01:14:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eveleen-indoor-outdoor-rocking-chair-mochasilver>: HTTP status code is not handled or not allowed 2026-01-25 01:14:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dsc-alarm-systems-ps1240-alarm-replacement-battery-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 01:14:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/plumbing_fittings/pipe_clamps_fittings/black_malleable_pipe_fittings already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:14:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/d18326-hand-flat-dolly-bundough-pan-w-handle-supports-pans-18w-x-32h-x-28d>: HTTP status code is not handled or not allowed 2026-01-25 01:14:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dynamometer-top-hook-assembly-for-9-ton-capacity-lever-hoists>: HTTP status code is not handled or not allowed 2026-01-25 01:14:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dyno-lever-hoist-34-ton-capacity-15-lift>: HTTP status code is not handled or not allowed 2026-01-25 01:14:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-rubber-window-squeegee-w-plastic-handle-pr35>: HTTP status code is not handled or not allowed 2026-01-25 01:14:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/altronix-al1024ulxpd4-alarm-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 01:14:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50-replacement-supply-cord-gray-plastic-36>: HTTP status code is not handled or not allowed 2026-01-25 01:14:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microfiber-cloths-16x16-red-200-pack-csa006e>: HTTP status code is not handled or not allowed 2026-01-25 01:14:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dcc-shoprider-777e-wheelchair-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 01:14:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ev-rider-snr2-mini-wheelchair-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 01:14:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/submersible-high-head-effluent-pump-ws50hm-115v-130-gpm-at-5>: HTTP status code is not handled or not allowed 2026-01-25 01:14:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/laboratory-scoops-pp-10ml-capacity-white-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-25 01:14:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mars-acoustical-panels-shadowline-tapered-edge-24l-x-48w-white>: HTTP status code is not handled or not allowed 2026-01-25 01:14:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/plumbing/plumbing_fittings/pipe_clamps_fittings/black_malleable_pipe_fittings landed on page that is not a product page. 2026-01-25 01:14:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/type-ii-accuflow-steel-safety-can-5-gal-5-8-metal-hose-yellow-7250220>: HTTP status code is not handled or not allowed 2026-01-25 01:15:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reagent-bottle-narrow-mouth-hdpe-4ml-capacity-amber-pack-of-72>: HTTP status code is not handled or not allowed 2026-01-25 01:15:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6201zz-p53-deep-groove-ball-bearing-double-shielded-abec-5-12mm-bore-32mm-od>: HTTP status code is not handled or not allowed 2026-01-25 01:15:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merits-s341-deluxe-wheelchair-replacement-battery-75ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 01:15:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/62072rsnr-deep-groove-ball-bearing-double-sealed-snap-ring-35mm-bore-72mm-od>: HTTP status code is not handled or not allowed 2026-01-25 01:15:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 46048 pages (at 98 pages/min), scraped 23058 items (at 48 items/min) 2026-01-25 01:15:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merits-p302-2hdsp-s-wheelchair-replacement-battery-55ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 01:15:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-tier-9-door-digital-wood-locker-36-w-x-15-d-x-72-h-gray-assembled>: HTTP status code is not handled or not allowed 2026-01-25 01:15:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interstate-dcm0022-wheelchair-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 01:15:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:15:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/graphgear-500-automatic-pencil-refillable-07mm-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:15:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:15:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelf-truck-30l-x-18w-x-60h-polyurethane-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 01:15:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gilson-22e-lawn-and-garden-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 01:15:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelf-truck-30l-x-18w-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 01:15:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers>: HTTP status code is not handled or not allowed 2026-01-25 01:15:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:15:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers>: HTTP status code is not handled or not allowed 2026-01-25 01:15:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:15:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:15:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sears-256619-lawn-and-garden-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 01:15:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelf-truck-30l-x-14w-x-69h-polyurethane-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 01:15:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hinged-lid-for-5-cu-yd-capacity-global-industrial-self-dumping-hopper-yellow>: HTTP status code is not handled or not allowed 2026-01-25 01:15:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers>: HTTP status code is not handled or not allowed 2026-01-25 01:15:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers>: HTTP status code is not handled or not allowed 2026-01-25 01:15:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-25 01:15:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/energel-liquid-gel-ink-pen-refillable-07mm-black-barrelink>: HTTP status code is not handled or not allowed 2026-01-25 01:15:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/petropad153-smart-polymer-spill-pad-35-1316l-x-53-78w-large-white-pk-of-2>: HTTP status code is not handled or not allowed 2026-01-25 01:15:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dyna-ray-dr709-30s-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 01:15:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/drill_presses_tapping_machinery already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:15:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/805-0064-external-spindle-series-57-05-wheel-bore-1500-max-speed>: HTTP status code is not handled or not allowed 2026-01-25 01:15:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precise-p700-gel-rollerball-pen-fine-07mm-purple-barrelink-dozen>: HTTP status code is not handled or not allowed 2026-01-25 01:15:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precise-v7-rolling-ball-pen-nonrefillable-fine-07mm-red-ink-dozen>: HTTP status code is not handled or not allowed 2026-01-25 01:15:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/128-in3-hydraulic-hand-pump-p2301m-2speed-4ways-control-wcarry-handle>: HTTP status code is not handled or not allowed 2026-01-25 01:15:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/machining/machinery/drill_presses_tapping_machinery>: HTTP status code is not handled or not allowed 2026-01-25 01:15:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dyna-ray-737-5sg-emergency-light-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 01:15:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wg4rn-svmc-outdoor-speaker-strobe-red-no-fire>: HTTP status code is not handled or not allowed 2026-01-25 01:15:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mpsr1-d45wx-ge-single-action-cat-45-key-reset-explosionproof>: HTTP status code is not handled or not allowed 2026-01-25 01:16:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/johnlite-2950-emergency-light-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 01:16:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-trouble-light-sl426-50l-cord-183-ga-yellow-6pk>: HTTP status code is not handled or not allowed 2026-01-25 01:16:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-patrol-sla1075-sealed-lead-acid-agm-vrla-battery>: HTTP status code is not handled or not allowed 2026-01-25 01:16:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-esc22-emergency-light-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 01:16:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 46139 pages (at 91 pages/min), scraped 23118 items (at 60 items/min) 2026-01-25 01:16:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sentry-lite-3904-emergency-light-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 01:16:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_tools/mops/mop_buckets_wringers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:16:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dyna-ray-be10-emergency-light-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 01:16:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crown-6ce12-sealed-lead-acid-agm-vrla-battery>: HTTP status code is not handled or not allowed 2026-01-25 01:16:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/spill_control_supplies/chemical_sorbents_spill_kits/chemical_sorbents already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:16:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lithonia-ind636-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 01:16:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-panel-5w-x-8h>: HTTP status code is not handled or not allowed 2026-01-25 01:16:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_tools/mops/mop_buckets_wringers>: HTTP status code is not handled or not allowed 2026-01-25 01:16:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iron-two-stage-air-compressor-10-hp-120-gal-capacity-3-phase-230v-830-lb-wt>: HTTP status code is not handled or not allowed 2026-01-25 01:16:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/spill_control_supplies/chemical_sorbents_spill_kits/chemical_sorbents landed on page that is not a product page. 2026-01-25 01:16:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-stage-electric-air-compressor-5-hp-80-gal-cap-3-phase-460v-481-lb-wt>: HTTP status code is not handled or not allowed 2026-01-25 01:16:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/parasystems-ps12180-12v-18ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-25 01:16:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/umbrellas_awnings_shades/outdoor_umbrellas_bases/umbrellas already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:16:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-cvt3rb3-emergency-light-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 01:16:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/framed-view-binder-with-gap-free-slant-rings-1-2inch-capacity-navy-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:16:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3794-double-end-fastener-double-wing-clip>: HTTP status code is not handled or not allowed 2026-01-25 01:16:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dyna-ray-s18197-emergency-light-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 01:16:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2026-end-cap-3l>: HTTP status code is not handled or not allowed 2026-01-25 01:16:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triple-tooth-glove-box-dispenser-10-34w-x-3-34d-x-14-78h>: HTTP status code is not handled or not allowed 2026-01-25 01:16:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-profile-steel-self-dumping-forklift-hopper-14-cu-yd-2000-lb-capacity-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:16:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/big-beam-2se6s20-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 01:16:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/tool_storage/chests_roller_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:16:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5930-double-horizontal-base-15>: HTTP status code is not handled or not allowed 2026-01-25 01:16:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esd-jacket-collared-knit-cuff-light-blue-3xl>: HTTP status code is not handled or not allowed 2026-01-25 01:16:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/outdoor_furniture/umbrellas_awnings_shades/outdoor_umbrellas_bases/umbrellas landed on page that is not a product page. 2026-01-25 01:16:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-cell-pc6120-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 01:16:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4304-8-hole-inside-corner-bracket>: HTTP status code is not handled or not allowed 2026-01-25 01:16:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esd-jacket-v-neck-knit-cuff-light-blue-2xl>: HTTP status code is not handled or not allowed 2026-01-25 01:16:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hydraulic-high-performance-puller-with-3-jaws-165-ton-cap-14-34-opening-7-78-reach>: HTTP status code is not handled or not allowed 2026-01-25 01:16:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/tool_storage/chests_roller_cabinets>: HTTP status code is not handled or not allowed 2026-01-25 01:16:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/umbrellas_awnings_shades/outdoor_umbrellas_bases/umbrellas already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:16:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2281-roller-wheel>: HTTP status code is not handled or not allowed 2026-01-25 01:16:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-w-x-30-d-production-workbench-with-drawer-upright-shelf-plastic-laminate-safety-edge-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:16:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portalac-pe22b1-emergency-light-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 01:16:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3088-double-tnut-3l>: HTTP status code is not handled or not allowed 2026-01-25 01:16:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/outdoor_furniture/umbrellas_awnings_shades/outdoor_umbrellas_bases/umbrellas>: HTTP status code is not handled or not allowed 2026-01-25 01:16:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-1799118st-emergency-light-replacement-battery-7ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 01:16:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lightalarms-605p1-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 01:16:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/parking_traffic_signs/parking_traffic_signs_1 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:16:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-lites-39-40-emergency-light-replacement-battery-1-3ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 01:16:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/loading-dock-w-left-arrow-traffic-sign-egp-refl-aluminum-18-x-24-blackwhite>: HTTP status code is not handled or not allowed 2026-01-25 01:16:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/parking_traffic_signs/parking_traffic_signs_1 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:16:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nerp003sd-15-ner-electric-hoist-push-trolley-14-ton-15-lift-366-ftmin-208v>: HTTP status code is not handled or not allowed 2026-01-25 01:17:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ion-audio-total-pa-plus-speaker-replacement-battery-9ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 01:17:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/parking_traffic_signs/parking_traffic_signs_1>: HTTP status code is not handled or not allowed 2026-01-25 01:17:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mbk-forte-50cc-motorcycle-replacement-battery-1984-2008-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 01:17:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nerm030c-l-20-ner-electric-hoist-motor-trolley-3-ton-20-lift-17-ftmin-230v>: HTTP status code is not handled or not allowed 2026-01-25 01:17:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/parking_traffic_signs/parking_traffic_signs_1 landed on page that is not a product page. 2026-01-25 01:17:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ner010ld-15-ner-dual-speed-electric-chain-hoist-1-ton-15-lift-1425-ftmin-208v>: HTTP status code is not handled or not allowed 2026-01-25 01:17:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 46233 pages (at 94 pages/min), scraped 23171 items (at 53 items/min) 2026-01-25 01:17:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/al100-hand-chain-hoist-aluminum-1-ton-capacity-30-ft-lift>: HTTP status code is not handled or not allowed 2026-01-25 01:17:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tner020l-10-tner-theatrical-chain-hoist-2-ton-capacity-10-lift-208v>: HTTP status code is not handled or not allowed 2026-01-25 01:17:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/snerp030c-15-sner-electric-hoist-push-trolley-3-ton-15-lift-35-ftmin-230v>: HTTP status code is not handled or not allowed 2026-01-25 01:17:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pgo-pa50da-50cc-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 01:17:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nerp003hd-20-ner-electric-hoist-push-trolley-14-ton-20-lift-539-ftmin-230v>: HTTP status code is not handled or not allowed 2026-01-25 01:17:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tner010l-10-tner-theatrical-chain-hoist-1-ton-capacity-10-lift-230v>: HTTP status code is not handled or not allowed 2026-01-25 01:17:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60090002001-roll-211k-1-1-2w-x-50-yd-100>: HTTP status code is not handled or not allowed 2026-01-25 01:17:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heartway-s17-mobility-scooter-replacement-battery-55ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 01:17:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ceramic-magnetic-hook-mhhh9-9-lbs-pull-white-enamel-paint>: HTTP status code is not handled or not allowed 2026-01-25 01:17:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pgo-g-max-125-125cc-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 01:17:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:17:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ceramic-round-base-magnets-hmkr-50-with-knob-25-lbs-pull-nickel-chrome-plating>: HTTP status code is not handled or not allowed 2026-01-25 01:17:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/heaters/electric_heaters/overhead_infrared_heaters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:17:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/import-hss-str-shank-3-flute-core-drill-1-inch-dia-x-6-3-8-inch-flute-x-11-inch-oal>: HTTP status code is not handled or not allowed 2026-01-25 01:17:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/fall_protection/anchorage_connectors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:17:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-subject-notebook-8-inch-x-10-1-2-inch-college-ruled-bright-white-70-sheets-pad>: HTTP status code is not handled or not allowed 2026-01-25 01:17:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/airborne-life-support-systems-185a-infant-isolette-medical-battery-26ah-12v>: HTTP status code is not handled or not allowed 2026-01-25 01:17:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots>: HTTP status code is not handled or not allowed 2026-01-25 01:17:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-side-printable-inkjet-business-card-2-inch-x-3-1-2-inch-matte-white-200-cards-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:17:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/heaters/electric_heaters/overhead_infrared_heaters landed on page that is not a product page. 2026-01-25 01:17:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flex-cover-comp-book-8-inch-x-10-inch-wide-ruled-black-mable-60-sheets-pad>: HTTP status code is not handled or not allowed 2026-01-25 01:17:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/fall_protection/anchorage_connectors>: HTTP status code is not handled or not allowed 2026-01-25 01:17:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/varidyne-500-22e-vacuum-controller-medical-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 01:17:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tidal-mp-paper-8-1-2-inch-x-11-inch-20-lb-white-2500-sheets-carton>: HTTP status code is not handled or not allowed 2026-01-25 01:17:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hill-rom-totalcare-bed-intellidrive-medical-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 01:17:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onestep-printable-toc-divider-printed-jan-to-dec-9x11-12-tabs-white-multicolor>: HTTP status code is not handled or not allowed 2026-01-25 01:17:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/masterly-solar-generator-msl04-10a-solar-replacement-battery-100ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 01:17:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/open-deck-pallet-galvanized-steel-40l-x-48w-8000-lb-floor-capacity-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:17:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/big-tab-plastic-insertable-divider-print-on-8-tabs-multicolor-multicolor>: HTTP status code is not handled or not allowed 2026-01-25 01:17:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stackable-shelf-bin-6w-x-18d-x-7h-ivory>: HTTP status code is not handled or not allowed 2026-01-25 01:17:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-source-wp5-12-91-300-sealed-lead-acid-replacement-battery-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 01:17:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eaglepicher-carefree-cf17-sealed-lead-acid-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 01:17:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hook-loop-tape-individual-dots-with-adhesive-34l-x-34w-white-pack-of-200>: HTTP status code is not handled or not allowed 2026-01-25 01:18:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/computers_electronics/audio_video_equipment/headphones already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:18:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/big-16-compartment-drawer-organizer-plastic-tray-19l-x-2-1-4w-x-17-1-4h>: HTTP status code is not handled or not allowed 2026-01-25 01:18:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=jtm-4vs-mill3-axis-acu-rite-vue-dro-powerfeeds already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:18:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nilfisk-advance-micromatic-group-27-floor-scrubber-industrial-battery-35ah-12v>: HTTP status code is not handled or not allowed 2026-01-25 01:18:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/estes-38-thick-entrance-floor-mat-3x6-navy-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:18:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=jtm-1050-mill3-axis-newall-dp700-dro-powerfeeds-power-draw-bar already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:18:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-rectangular-perforated-picnic-table-green-694553GN>: HTTP status code is not handled or not allowed 2026-01-25 01:18:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tsh-series-fan-forced-kickspace-specialty-heater-brown-tsh20j-1500-1000w-240-208v>: HTTP status code is not handled or not allowed 2026-01-25 01:18:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 46322 pages (at 89 pages/min), scraped 23225 items (at 54 items/min) 2026-01-25 01:18:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/computers_electronics/audio_video_equipment/headphones>: HTTP status code is not handled or not allowed 2026-01-25 01:18:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=jtm-4vs-mill3-axis-acu-rite-vue-dro-powerfeeds landed on page that is not a product page. 2026-01-25 01:18:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-super-size-12-3-8w-x-20d-x-6h>: HTTP status code is not handled or not allowed 2026-01-25 01:18:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/no-thread-coupling-3-12-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 01:18:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=jtm-1050-mill3-axis-newall-dp700-dro-powerfeeds-power-draw-bar landed on page that is not a product page. 2026-01-25 01:18:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ef-flame-resistant-team-jacket-jet2-navy-size-s-regular>: HTTP status code is not handled or not allowed 2026-01-25 01:18:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ef-ct-flame-resistant-universal-fit-snap-on-hood-hlh2-navy-size-m>: HTTP status code is not handled or not allowed 2026-01-25 01:18:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/no-thread-connector-1-14-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 01:18:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ef-ct-fr-dress-uniform-shirt-slu2-khaki-size-m-regular>: HTTP status code is not handled or not allowed 2026-01-25 01:18:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s30240s-infrared-heater-30kw-208240v-silvergrey>: HTTP status code is not handled or not allowed 2026-01-25 01:18:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vest-style-harness-1110600-w-back-d-ring-quick-connect-buckles-universal>: HTTP status code is not handled or not allowed 2026-01-25 01:18:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pride-mobility-go-go-elite-traveller-plus-hd-4-wheel-wheelchair-battery-22ah-12v>: HTTP status code is not handled or not allowed 2026-01-25 01:18:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/garden-way-garden-way-55295-lawn-and-garden-replacement-battery-7ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 01:18:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylon-button-head-drive-anchor-kit-w-pin-tuff-pack-316-x-1-size-100-pieces>: HTTP status code is not handled or not allowed 2026-01-25 01:18:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5502-moisture-wicking-short-sleeve-safety-tshirt-with-chest-pocket-orange-4xl>: HTTP status code is not handled or not allowed 2026-01-25 01:18:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/phillips-and-square-flat-head-machine-screw-14-20-x-2-12-size-pack-od-100>: HTTP status code is not handled or not allowed 2026-01-25 01:18:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1003-standard-class-2-mesh-hook-loop-safety-vest-lime-2xl>: HTTP status code is not handled or not allowed 2026-01-25 01:18:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2002-standard-class-3-mesh-zipper-safety-vest-orange-large>: HTTP status code is not handled or not allowed 2026-01-25 01:18:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibration-mount-2-threaded-studs-56-inch-dia-50-inch-h-10-32-thread>: HTTP status code is not handled or not allowed 2026-01-25 01:18:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sonnenschein-3000-infusion-pump-emergency-light-replacement-battery-1-3ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 01:18:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/69000-series-hutch-with-doors-66-inchw-x-15-inchd-x-36-inchh-cherry>: HTTP status code is not handled or not allowed 2026-01-25 01:18:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perf-24-w-5-step-steel-roll-ladder-21-d-top-hndrl-orange-fsh52621po>: HTTP status code is not handled or not allowed 2026-01-25 01:18:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grip-24-w-7-step-steel-roll-ladder-14-d-top-stp-hndrl-orange-fsh726go>: HTTP status code is not handled or not allowed 2026-01-25 01:18:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/69000-series-box-file-hanging-pedestal-16-inchw-x-22-inchd-x-19-inchh-mahogany>: HTTP status code is not handled or not allowed 2026-01-25 01:18:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manual-l-o-x-pneumatic-lockout-valve-y1523c400212-npt>: HTTP status code is not handled or not allowed 2026-01-25 01:18:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/pneumatic_valves/directional_control_valves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:18:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pneumatic-mushroom-green-push-button-valve-d1223b1mpg-18-bspp>: HTTP status code is not handled or not allowed 2026-01-25 01:18:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/concrete-screw-anchor-slotted-hex-washer-head-14-x-2-34-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 01:18:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/concrete-screw-anchor-slotted-hex-washer-head-316-x-1-14-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 01:18:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zamac-alloy-anchor-w-carbon-steel-pin-flat-head-14-x-34-size-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 01:19:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/short-wall-grip-anchor-carbon-steel-14-size-50-pk>: HTTP status code is not handled or not allowed 2026-01-25 01:19:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/pneumatic_valves/directional_control_valves landed on page that is not a product page. 2026-01-25 01:19:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-hole-pipe-strap-carbon-steel-22-gauge-12-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 01:19:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polaris-sportsman-850-efi-touring-eps-850cc-atv-replacement-battery-2010-2013-12v>: HTTP status code is not handled or not allowed 2026-01-25 01:19:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/punches_1 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:19:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/can-am-outlander-max-1000-efi-xt-p-1000cc-atv-replacement-battery-2013-2016-12v>: HTTP status code is not handled or not allowed 2026-01-25 01:19:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrf133401h56c-cast-iron-right-angle-worm-gear-reducer-401-ratio-56c-frame>: HTTP status code is not handled or not allowed 2026-01-25 01:19:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrf206501de56c-cast-iron-right-angle-worm-gear-reducer-501-ratio-56c-frame>: HTTP status code is not handled or not allowed 2026-01-25 01:19:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hello-my-name-is-name-badge-3-1-2-x-2-1-4-red-border-100-box>: HTTP status code is not handled or not allowed 2026-01-25 01:19:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 46417 pages (at 95 pages/min), scraped 23283 items (at 58 items/min) 2026-01-25 01:19:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aprilia-tuono-v4-r-aprc-abs-1000cc-motorcycle-replacement-battery-2014-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 01:19:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-staple-cartridge-1-4-leg-length-5000-per-cartridge-1-cartridge-box>: HTTP status code is not handled or not allowed 2026-01-25 01:19:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/clips_fasteners/rubber_bands already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:19:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/basic_supplies/punches_1>: HTTP status code is not handled or not allowed 2026-01-25 01:19:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrf262201h145tc-cast-iron-right-angle-worm-gear-reducer-201-ratio-145t-frame>: HTTP status code is not handled or not allowed 2026-01-25 01:19:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honda-gl1200sei-gold-wing-aspencade-1200cc-motorcycle-replacement-battery-12v-i>: HTTP status code is not handled or not allowed 2026-01-25 01:19:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-head-tap-bolt-low-carbon-steel-34-10-x-2-34-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 01:19:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honda-cbx1000-super-sport-optional-1000cc-motorcycle-replacement-battery-12v-i>: HTTP status code is not handled or not allowed 2026-01-25 01:19:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harley-davidson-softail-heritage-flst-1340cc-motorcycle-battery-1986-1990-12v>: HTTP status code is not handled or not allowed 2026-01-25 01:19:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/clips_fasteners/rubber_bands landed on page that is not a product page. 2026-01-25 01:19:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ar49-reach-in-refrigerator-2-door-white-49-cu-ft-54w-x-32-34d-x-82-34h>: HTTP status code is not handled or not allowed 2026-01-25 01:19:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/american-ironhorse-texas-chopper-motorcycle-replacement-battery-1995-2009-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 01:19:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/closed-starter-csb97-3048-8x-48-inch-w-x-30-inch-d-x-73-inch-h-8-shelves-18-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:19:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harley-davidson-cvo-fxd-fat-bob-dyna-1803cc-motorcycle-battery-2007-2015-12v>: HTTP status code is not handled or not allowed 2026-01-25 01:19:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honda-nrx1800-valkyrie-rune-1800cc-motorcycle-replacement-battery-2004-2005-12v>: HTTP status code is not handled or not allowed 2026-01-25 01:19:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f70eb12040000001-mike-pendant-6-button-black-2speed-buttons>: HTTP status code is not handled or not allowed 2026-01-25 01:19:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/prga0012pe-hook-use-wmike-victor-pendants>: HTTP status code is not handled or not allowed 2026-01-25 01:19:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-head-lag-screw-carbon-steel-516-x-5-12-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 01:19:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_batteries/motorcycle_batteries already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:19:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/closed-starter-csb85-1842-5x-42-inch-w-x-18-inch-d-x-85-inch-h-5-shelves-18-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:19:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aprilia-rsv4-factory-aprc-abs-1000cc-motorcycle-replacement-battery-2013-2015-12v>: HTTP status code is not handled or not allowed 2026-01-25 01:19:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/pneumatic_valves/directional_control_valves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:19:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/misty-si-dry-silicone-spray-lubricant-11oz-aerosol-12-case-amra32916>: HTTP status code is not handled or not allowed 2026-01-25 01:19:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-vis-hooded-sweatshirt-type-o-ansi-class-2-m-blackfluorescent-yellowgreen>: HTTP status code is not handled or not allowed 2026-01-25 01:19:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-ucf215-75mm-flange-mount-bearing-4-bolt-flange-set-screw-locking-bore-75-mm>: HTTP status code is not handled or not allowed 2026-01-25 01:19:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-hcpk208-24-pillow-block-bearing-low-base-ecc-lock-collar-bore-38-1-mm>: HTTP status code is not handled or not allowed 2026-01-25 01:19:45 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_batteries/motorcycle_batteries landed on page that is not a product page. 2026-01-25 01:19:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zipr-mobility-3traveler-mobility-scooter-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 01:19:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/pneumatic_valves/directional_control_valves landed on page that is not a product page. 2026-01-25 01:19:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-high-visibility-waterproof-bomber-jacket-type-r-ansi-class-3-smmd-limepink>: HTTP status code is not handled or not allowed 2026-01-25 01:19:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-ucf205-16-flange-mount-bearing-4-bolt-flange-set-screw-locking-bore-25-4-mm>: HTTP status code is not handled or not allowed 2026-01-25 01:19:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-ucf211-55mm-flange-mount-bearing-4-bolt-flange-set-screw-locking-bore-55-mm>: HTTP status code is not handled or not allowed 2026-01-25 01:19:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zapworld-transportbike-electric-scooter-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 01:19:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sure-grip-ex-safety-cabinet-self-close-4-gal-cap-17w-x-17d-x-22h-royal-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:19:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-row-spherical-roller-bearing-23148ew33-240mm-bore-400mm-od>: HTTP status code is not handled or not allowed 2026-01-25 01:19:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-row-cylindrical-roller-bearing-nu314eg-70mm-bore-150mm-od-high-capacity>: HTTP status code is not handled or not allowed 2026-01-25 01:19:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flammable-cabinet-self-close-double-door-22-gallon-capacity-35w-x-22d-x-35h>: HTTP status code is not handled or not allowed 2026-01-25 01:20:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flammable-cabinet-manual-close-double-door-90-gal-capacity-43w-x-34d-x-65h>: HTTP status code is not handled or not allowed 2026-01-25 01:20:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8089-noncertified-tshirt-orange-xl>: HTTP status code is not handled or not allowed 2026-01-25 01:20:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-solid-shelf-18-x-48-186CP201>: HTTP status code is not handled or not allowed 2026-01-25 01:20:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/cart_truck_accessories/vehicle_tugger_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:20:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-in-stainless-steel-gate-valve-200-psi>: HTTP status code is not handled or not allowed 2026-01-25 01:20:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 46510 pages (at 93 pages/min), scraped 23338 items (at 55 items/min) 2026-01-25 01:20:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-chrome-esd-wire-shelving-unit-starter-72w-x-12d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 01:20:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/everest-jennings-metro-poweriii-mobility-scooter-replacement-battery-35ah-12v>: HTTP status code is not handled or not allowed 2026-01-25 01:20:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dalton-4-wheel-scooter-sc-s145hd-mobility-scooter-replacement-battery-35ah-12v>: HTTP status code is not handled or not allowed 2026-01-25 01:20:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-chrome-esd-wire-shelving-unit-starter-48w-x-14d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 01:20:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mototec-solar-electric-go-kart-electric-scooter-replacement-battery-14ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 01:20:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-4-in-304-stainless-steel-cap-fnpt-125-psi-import>: HTTP status code is not handled or not allowed 2026-01-25 01:20:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-in-x-1-1-8-in-stainless-steel-pipe-nipple-16168-psi-sch-40-domestic>: HTTP status code is not handled or not allowed 2026-01-25 01:20:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/cart_truck_accessories/vehicle_tugger_accessories landed on page that is not a product page. 2026-01-25 01:20:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-shelf-stainless-steel-wire-shelving-unit-starter-48w-x-36d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 01:20:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-bolt-open-add-on-oat75-1836-7x-36-inch-w-x-18-inch-d-x-75-inch-h-7-shelves-18-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:20:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5205zz-double-row-angular-contact-ball-bearing-double-shielded-od-52mm-bore-25mm-metric>: HTTP status code is not handled or not allowed 2026-01-25 01:20:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-berm-builder-kit-8640>: HTTP status code is not handled or not allowed 2026-01-25 01:20:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ink-refill-for-pre-inked-stamps-024-fl-oz-bottle-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:20:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wtta-30-15-gallon-expansion-tank-asme-1-in-fnpt>: HTTP status code is not handled or not allowed 2026-01-25 01:20:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blank-award-certificate-golden-seal-25-x-8-1-2-blue-25-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:20:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yamaha-fx-nytro-rtx-1049cc-snowmobile-replacement-battery-2008-2009-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 01:20:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-expedition-tuv-600-h-o-sdi-594cc-snowmobile-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 01:20:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/long-corrugated-boxes-12-x-6-x-3-200lb-test-ect-32-25-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:20:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-inking-date-stamp-10-year-date-band-black>: HTTP status code is not handled or not allowed 2026-01-25 01:20:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/money-rent-receipt-book-2-part-carbonless-5-1-4-x-11-200-sets-pad>: HTTP status code is not handled or not allowed 2026-01-25 01:20:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/at-a-glance-visitor-registration-book-8-7-8-x-11-7-16-black-cover-60-sheets-pad>: HTTP status code is not handled or not allowed 2026-01-25 01:20:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-literature-mailers-22-x-18-x-2-3-4-200lb-test-ect-32-white-25-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:20:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-3-x-5-8-mil-clear-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:20:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-circle-removable-labels-fluorescent-yellow-500-per-roll>: HTTP status code is not handled or not allowed 2026-01-25 01:20:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-poppet-type-regulator-rv48-3-4-up-to-250000-btu>: HTTP status code is not handled or not allowed 2026-01-25 01:20:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-poly-green-wire-shelving-unit-add-on-42w-x-18d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 01:20:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-poly-green-wire-shelving-unit-add-on-60w-x-12d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 01:20:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50lb-cable-ties-natural-7-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:20:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-1-2-straight-flow-thru-regulator-rv91-2-1-2-up-to-7500000-btu>: HTTP status code is not handled or not allowed 2026-01-25 01:20:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-ski-doo-mx-z-renegade-x-800-ho-ptek-800cc-snowmobile-battery-2004-2013-12v>: HTTP status code is not handled or not allowed 2026-01-25 01:20:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bollardgard-anywhere-yellow-4-x-52-with-25-lb-octagon-rubber-base-bgaw452yw>: HTTP status code is not handled or not allowed 2026-01-25 01:20:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bombardier-renegade-backcountry-800r-800cc-snowmobile-battery-2014-2016-12v>: HTTP status code is not handled or not allowed 2026-01-25 01:20:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/k-50-sectional-machine-115v-1-6hp-400rpm>: HTTP status code is not handled or not allowed 2026-01-25 01:20:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-poly-green-wire-shelving-unit-starter-60w-x-21d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 01:20:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-8664-containment-berm-rapid-rise-15-x-50-22-oz-pvc>: HTTP status code is not handled or not allowed 2026-01-25 01:20:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-8587-containment-berm-foam-wall-15-x-20-copolymer-2000>: HTTP status code is not handled or not allowed 2026-01-25 01:20:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chemical-resistant-gloves-smooth-grip-butyl-13-mil-thick-size-8-black>: HTTP status code is not handled or not allowed 2026-01-25 01:20:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-plastic-locker-11-9z1215720-z-style-12-x-15-x-72-1-wide-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:20:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/-ligature-resistant-adjustable-clothestowel-hook-5-18w-x-78d-x-5-18h-white>: HTTP status code is not handled or not allowed 2026-01-25 01:21:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/-dock-seal-draft-blocker-24l-x-12w-x-9h-yellow>: HTTP status code is not handled or not allowed 2026-01-25 01:21:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-plus-phenolic-locker-11-8z1518601-z-style-15-x-18-x-60-silver-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:21:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/-600-pro-universal-turbo-diamond-blade-9-dia-x-316t-x-78-center-hole-dia>: HTTP status code is not handled or not allowed 2026-01-25 01:21:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-ski-doo-mx-z-tnt-600-ho-600cc-snowmobile-replacement-battery-2014-2016-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 01:21:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cooler-thermometer-535-0-8-with-magnetic-back>: HTTP status code is not handled or not allowed 2026-01-25 01:21:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-grand-touring-se-800-sdi-800cc-snowmobile-replacement-battery-2003-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 01:21:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 46611 pages (at 101 pages/min), scraped 23394 items (at 56 items/min) 2026-01-25 01:21:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5th-generation-smart-padlock-long-shackle-satin-silver>: HTTP status code is not handled or not allowed 2026-01-25 01:21:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/parking_traffic_signs/parking_traffic_signs_1 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:21:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ch395-e16-gas-engine-horizontal-shaft-9-12-hp-74957-lb-wt>: HTTP status code is not handled or not allowed 2026-01-25 01:21:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sky-ares-3-piece-outdoor-bar-set-23-5-8-w-x-42-1-2-h-table-taupe>: HTTP status code is not handled or not allowed 2026-01-25 01:21:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/severe-duty-motor-e619-215ttfs6526-10hp-460v-1800rpm-3ph-215t-fr-tefc>: HTTP status code is not handled or not allowed 2026-01-25 01:21:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storage-argos-6w-ammo-bin-ws-6006-ws23-battleship-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:21:21 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/parking_traffic_signs/parking_traffic_signs_1 landed on page that is not a product page. 2026-01-25 01:21:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/argosrack-gun-open-rack-awr83-24r-1-holds-24-rifles-with-2-half-shelves-42w-x-15d-x-83h-desert-sand>: HTTP status code is not handled or not allowed 2026-01-25 01:21:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-mailbox-2210-23-1-2w-x-15-1-2d-x-30h-10-doors-standard-system>: HTTP status code is not handled or not allowed 2026-01-25 01:21:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inverter-duty-motor-y595-213thfs8028-75hp-230-460v-1800rpm-3ph-213tc-tefc>: HTTP status code is not handled or not allowed 2026-01-25 01:21:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/severe-duty-motor-w587-364thfs9036-60hp-460v-1800rpm-3ph-364t-fr-tefc>: HTTP status code is not handled or not allowed 2026-01-25 01:21:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/severe-duty-motor-e621-256ttfna6526-20hp-460v-1800rpm-3ph-256t-fr-tefc>: HTTP status code is not handled or not allowed 2026-01-25 01:21:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inverter-duty-motor-y531-449thfs8046-250hp-460v-1800rpm-3ph-449t-tebc>: HTTP status code is not handled or not allowed 2026-01-25 01:21:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_alarms_signals/forklift_alert_sensors_systems already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:21:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/113789a-round-dome-utility-sewer-marker-white-pole-78h-54-above-ground-green>: HTTP status code is not handled or not allowed 2026-01-25 01:21:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/indicator-bin-36462blkred-6-3-4x-11-5-8x-4-double-hopper-shelf-bin-pkg-qty-12>: HTTP status code is not handled or not allowed 2026-01-25 01:21:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g951-56c17d5308-1-1-2hp-1800rpm-115-208-230v-1ph-56h-fr-dp>: HTTP status code is not handled or not allowed 2026-01-25 01:21:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/r-22gtf-commercial-microwave-oven-heavy-duty-1200w-ss-20-18w-x-18-12h-x-13-14d>: HTTP status code is not handled or not allowed 2026-01-25 01:21:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_alarms_signals/forklift_alert_sensors_systems>: HTTP status code is not handled or not allowed 2026-01-25 01:21:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hvac-motor-h290-5kh36mn137-1-2hp-1725rpm-115v-1-ph-56cz-fr>: HTTP status code is not handled or not allowed 2026-01-25 01:21:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-3-classic-wicking-birdseye-t-shirt-with-pocket-orange-m>: HTTP status code is not handled or not allowed 2026-01-25 01:21:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cooling-tower-duty-motor-y389-50-12-1-2-hp-1800-900-rpm-460v-3ph-364t-fr-tefc>: HTTP status code is not handled or not allowed 2026-01-25 01:21:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-image-glass-surface-cleaner-rain-fresh-scent-1-gal-capacity-bottle-4carton>: HTTP status code is not handled or not allowed 2026-01-25 01:21:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cortina-plastx-type-i-fold-flat-barricade-with-direction-indicator-24l-x-12w-panel-orangewhite>: HTTP status code is not handled or not allowed 2026-01-25 01:21:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fan-blower-motor-x015-48a17o151-1-3hp-1625rpm-115v-1ph-48y-fr-opao>: HTTP status code is not handled or not allowed 2026-01-25 01:21:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-pipe-fitting-single-socket-tee-1-1-2-dia>: HTTP status code is not handled or not allowed 2026-01-25 01:21:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oil-burner-motor-4731-5kh39qn9696x-1-3hp-1725rpm-115v-1ph-48y-dp>: HTTP status code is not handled or not allowed 2026-01-25 01:21:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/orange-traffic-cone-mount-retracting-belt-barrier-20-yellowblack-diagonal-stripe-belt>: HTTP status code is not handled or not allowed 2026-01-25 01:21:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v2015-tee-fitting-125v-15a-6l>: HTTP status code is not handled or not allowed 2026-01-25 01:21:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vestil-floor-lock-with-polyurethane-foot-pad-fl-lkh-5-5-casters>: HTTP status code is not handled or not allowed 2026-01-25 01:21:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wa-2848-lp-48w-x-28d-welded-workbench-with-backstops-and-adj-height>: HTTP status code is not handled or not allowed 2026-01-25 01:22:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wd-40-174-specialist-174-electrical-contact-cleaner-spray-11-oz-aerosol-can-300080>: HTTP status code is not handled or not allowed 2026-01-25 01:22:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-tee-22200-use-for-1w-grid-white>: HTTP status code is not handled or not allowed 2026-01-25 01:22:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/duracell-durg12-31j-a-sealed-lead-acid-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 01:22:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-tee-32007-satin-black>: HTTP status code is not handled or not allowed 2026-01-25 01:22:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-nexelon-blue-wire-shelving-unit-starter-54w-x-18d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 01:22:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/creamer-in-a-canister-non-dairy-12-oz>: HTTP status code is not handled or not allowed 2026-01-25 01:22:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-black-epoxy-wire-shelving-unit-add-on-54w-x-14d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 01:22:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fullriver-hghl12285w-sealed-lead-acid-replacement-battery-75ah-12v-it>: HTTP status code is not handled or not allowed 2026-01-25 01:22:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 46710 pages (at 99 pages/min), scraped 23455 items (at 61 items/min) 2026-01-25 01:22:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gogreen-power-gg-16000tw-6-outlet-wall-tap-white>: HTTP status code is not handled or not allowed 2026-01-25 01:22:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-patrol-sla0915-sealed-lead-acid-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 01:22:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-nexelate-silver-epoxy-wire-shelving-unit-add-on-36w-x-14d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 01:22:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electric-mobility-wispa-wheelchair-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 01:22:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gallery-wall-clock-16-black>: HTTP status code is not handled or not allowed 2026-01-25 01:22:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-nexelate-silver-epoxy-wire-shelving-unit-starter-36w-x-30d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 01:22:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/potter-electric-pfc-7500series-alarm-replacement-battery-8ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 01:22:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fallte-7017-contractor-3-d-full-body-harness-3-d-rings-back-and-side-size-unifit>: HTTP status code is not handled or not allowed 2026-01-25 01:22:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quick-coupler-39040915-38-body-coupler-set-poppet-38fnpt>: HTTP status code is not handled or not allowed 2026-01-25 01:22:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fallte-82403-elastech-4-1-2-to-6-shock-absorbing-lanyard-adjustable-single-leg>: HTTP status code is not handled or not allowed 2026-01-25 01:22:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fallte-82563-viewpack-6-shock-absorbing-lanyard-with-1-snap-hook-and-1-rebar-hook>: HTTP status code is not handled or not allowed 2026-01-25 01:22:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quantum-rehab-freestyle1-wheelchair-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 01:22:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/apc-smart-ups-su1000-rack-wheelchair-replacement-battery-8ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 01:22:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-inlet-blower-wheel-6-1-4-inch-dia-cw-2000-rpm-1-2-inch-bore-4-1-4-inch-w-galvanized>: HTTP status code is not handled or not allowed 2026-01-25 01:22:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-inlet-blower-wheel-3-27-32-inch-dia-cw-4500-rpm-1-4-inch-bore-1-1-4-inch-w-galvanized>: HTTP status code is not handled or not allowed 2026-01-25 01:22:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-poly-z-brite-wire-shelving-unit-starter-60w-x-18d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 01:22:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:22:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/hvac_system_parts_components/replacement_hvac_blades_wheels/single_double_inlet_blower_wheels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:22:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-poly-z-brite-wire-shelving-unit-starter-36w-x-21d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 01:22:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hv-14-high-velocity-fan-14-230460v-3-ph>: HTTP status code is not handled or not allowed 2026-01-25 01:22:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-poly-z-brite-wire-shelving-unit-starter-72w-x-14d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 01:22:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:22:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-base-cabinet-bc-4300-1-hinged-door-shelf-sink-bowl-18x21x36>: HTTP status code is not handled or not allowed 2026-01-25 01:22:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-inlet-blower-wheel-8-1-2-inch-dia-cw-1650-rpm-1-2-inch-bore-3-3-16-inch-w-galvanized>: HTTP status code is not handled or not allowed 2026-01-25 01:22:50 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-25 01:22:50 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/hvac_system_parts_components/replacement_hvac_blades_wheels/single_double_inlet_blower_wheels landed on page that is not a product page. 2026-01-25 01:22:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eagle-column-protector-10-column-opening-24-high-red-1724-8-red>: HTTP status code is not handled or not allowed 2026-01-25 01:22:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/technacell-ep1640-emergency-light-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 01:22:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-25 01:22:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-inlet-blower-wheel-5-13-32-inch-dia-ccw-3600-rpm-3-8-inch-bore-6-55-64-inch-w-galvanized>: HTTP status code is not handled or not allowed 2026-01-25 01:22:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergi-lite-002027-emergency-light-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 01:23:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-workbench-vise-with-swivel-base-8-jaw-width-4-3164-throat-depth>: HTTP status code is not handled or not allowed 2026-01-25 01:23:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/p-touch-tz-industrial-series-fabric-iron-on-tape-navy-on-white-1-2-x-98ft>: HTTP status code is not handled or not allowed 2026-01-25 01:23:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-adhesive-folder-labels-9-16-x-3-7-16-white-260-box>: HTTP status code is not handled or not allowed 2026-01-25 01:23:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-wide-boltless-shelving-add-on-72-in-w-x-36-in-d-x-96-in-h-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 01:23:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 46808 pages (at 98 pages/min), scraped 23521 items (at 66 items/min) 2026-01-25 01:23:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/workfit-c-dual-sit-stand-workstation-cart>: HTTP status code is not handled or not allowed 2026-01-25 01:23:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/ink_toner/laser_toner_cartridges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:23:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/palmgren-9611351-cradle-style-angel-vise-4>: HTTP status code is not handled or not allowed 2026-01-25 01:23:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:23:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hl10000-series-corner-l-steel-desk-left-hand-return-66-x-72-putty-oak>: HTTP status code is not handled or not allowed 2026-01-25 01:23:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/medical_chart_file_holders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:23:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/wall_corner_bumper_guards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:23:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traffic-spiked-speed-bump-end-caps-15-14l-x-10-58w-x-2h-yellow>: HTTP status code is not handled or not allowed 2026-01-25 01:23:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ps-70-f-rye-privacy-screen>: HTTP status code is not handled or not allowed 2026-01-25 01:23:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/ink_toner/laser_toner_cartridges landed on page that is not a product page. 2026-01-25 01:23:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/room_dividers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:23:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/female-ballistic-bullet-resistant-vest-level-iii-a-foam-s-black>: HTTP status code is not handled or not allowed 2026-01-25 01:23:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks>: HTTP status code is not handled or not allowed 2026-01-25 01:23:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/can-am-outlander-1000-1000cc-atv-replacement-battery-2012-2016-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 01:23:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/medical_chart_file_holders landed on page that is not a product page. 2026-01-25 01:23:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/five-shelf-extra-hd-boltless-shelving-add-on-48inw-x-12ind-x-96inh-laminate-deck-usa>: HTTP status code is not handled or not allowed 2026-01-25 01:23:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/wall_corner_bumper_guards landed on page that is not a product page. 2026-01-25 01:23:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regenerative-blower-ab-850-3-phase-1-stage-10-hp>: HTTP status code is not handled or not allowed 2026-01-25 01:23:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/acoustical_partition_panels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:23:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/partitions_dividers/room_dividers>: HTTP status code is not handled or not allowed 2026-01-25 01:23:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/three-shelf-wide-boltless-shelving-add-on-2400-lb-96inwx24indx60inh-laminate-deck-usa>: HTTP status code is not handled or not allowed 2026-01-25 01:23:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/prospec-hd100-e12-full-zipper-replacement-cloth-vacuum-bag-gk-sanicloth>: HTTP status code is not handled or not allowed 2026-01-25 01:23:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smoking-shelter-s64fdkb-3sided-wleft-open-front-15l-x-10w-flat-roof-dk-bronze>: HTTP status code is not handled or not allowed 2026-01-25 01:23:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/national-power-gs012ps6-emergency-light-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 01:23:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smoking-shelter-52wsfca-4sided-wl-r-open-front-12l-x-5w-flat-roof-clear>: HTTP status code is not handled or not allowed 2026-01-25 01:23:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/five-shelf-heavy-duty-boltless-shelving-add-on-36in-w-x-24in-d-x-60in-h-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 01:23:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/partitions_dividers/acoustical_partition_panels>: HTTP status code is not handled or not allowed 2026-01-25 01:23:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/ink_toner/laser_toner_cartridges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:23:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poly-acid-corrosive-cabinet-cra4510-with-self-close-45-gallon-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:23:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/loading-dock-safety-net-4-feet-x-20-feet-OHPW420>: HTTP status code is not handled or not allowed 2026-01-25 01:23:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/three-shelf-wide-boltless-shelving-add-on-60-in-w-x-48-in-d-x-60-in-h-laminate-deck-usa>: HTTP status code is not handled or not allowed 2026-01-25 01:23:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/heaters/stoves_fireplaces_log_sets_fire_pits/fireplace_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:23:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f30-face-shield-window-acetate-8l-x-15-12w-116-thick-clear>: HTTP status code is not handled or not allowed 2026-01-25 01:23:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/ink_toner/laser_toner_cartridges>: HTTP status code is not handled or not allowed 2026-01-25 01:23:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/import-carbide-tipped-tool-bit-d8-1-2-inch-x-1-2-inch-x-3-1-2-inch-general-purpose>: HTTP status code is not handled or not allowed 2026-01-25 01:23:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbi-sala-nano-lok-twin-leg-hot-work-self-retracting-lifeline-web--steel-comfort-grip-hook-6l>: HTTP status code is not handled or not allowed 2026-01-25 01:23:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/925-case-w-padded-divider-18-11-16l-x-14-13-16w-x-7h-silver>: HTTP status code is not handled or not allowed 2026-01-25 01:23:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/heaters/stoves_fireplaces_log_sets_fire_pits/fireplace_accessories landed on page that is not a product page. 2026-01-25 01:23:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motobecane-active-80cc-motorcycle-replacement-battery-1942-1984-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 01:23:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/padded-divider-for-940-case>: HTTP status code is not handled or not allowed 2026-01-25 01:23:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/footwear_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:23:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbi-sala-nano-lok-twin-leg-arc-flash-self-retracting-lifeline-quick-connect-web--carabiner-8l>: HTTP status code is not handled or not allowed 2026-01-25 01:23:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/heaters/electric_heaters/electric_unit_heaters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:24:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd-welded-workbench-60-x-24-adj-height-louvered-panel-steel-square-edge>: HTTP status code is not handled or not allowed 2026-01-25 01:24:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-welded-workbench-48-x-30-louvered-panel-butcher-block-square-edge>: HTTP status code is not handled or not allowed 2026-01-25 01:24:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dmm-s13n-7ll1-dm-dual-mag-dual-led-stereomicroscope-wlight-stand-10x-30x>: HTTP status code is not handled or not allowed 2026-01-25 01:24:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/footwear_accessories>: HTTP status code is not handled or not allowed 2026-01-25 01:24:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/286930000-graduated-cylinder-tpx-polymethylpentene-100ml>: HTTP status code is not handled or not allowed 2026-01-25 01:24:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 46895 pages (at 87 pages/min), scraped 23571 items (at 50 items/min) 2026-01-25 01:24:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powercat-scooter-electric-scooter-replacement-battery-14ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 01:24:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/bin_cabinets/stationary_bin_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:24:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-welded-workbench-72-x-30-pegboard-panel-steel-square-edge>: HTTP status code is not handled or not allowed 2026-01-25 01:24:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bsm-inc-ada-ss-sink-5-station-with-electronic-faucet-wall-mount-120l-x-20w-x-6d-sink>: HTTP status code is not handled or not allowed 2026-01-25 01:24:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merits-pioneer-1-mobility-scooter-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 01:24:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/si-0236-vortex-genie-2-vortex-mixer-120v>: HTTP status code is not handled or not allowed 2026-01-25 01:24:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gssafe-5113-class-3-microfiber-birdseye-short-sleeve-t-shrit-w-black-bottom-lime-xl>: HTTP status code is not handled or not allowed 2026-01-25 01:24:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gssafe-5113-class-3-microfiber-birdseye-short-sleeve-t-shrit-w-black-bottom-lime-2xl>: HTTP status code is not handled or not allowed 2026-01-25 01:24:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/si-1612-250ml-flask-clamp>: HTTP status code is not handled or not allowed 2026-01-25 01:24:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hoveround-tekniquefwd-mobility-scooter-replacement-battery-55ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 01:24:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0k-0500-902-3-inch-platform-with-rubber-cover>: HTTP status code is not handled or not allowed 2026-01-25 01:24:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gssafe-2702-class-3-heavy-duty-safety-vest-orange-l>: HTTP status code is not handled or not allowed 2026-01-25 01:24:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/cabinets/bin_cabinets/stationary_bin_cabinets>: HTTP status code is not handled or not allowed 2026-01-25 01:24:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/meyra-2482adult-mobility-scooter-replacement-battery-55ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 01:24:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/heaters/electric_heaters/electric_unit_heaters landed on page that is not a product page. 2026-01-25 01:24:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/amigo-mobility-rt-mobility-scooter-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 01:24:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/appliances/refrigerators_freezers/refrigerator_freezer_combos/undercounter_refrigerator_freezer_combos already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:24:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/voyager-gt-overboots-cleated-outsole-m-11h-black>: HTTP status code is not handled or not allowed 2026-01-25 01:24:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/appliances/refrigerators_freezers/refrigerator_freezer_combos/undercounter_refrigerator_freezer_combos already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:24:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/summit-ct663bsstb-freestanding-counter-height-refrigerator-freezer-51-cu-ft-24-wide>: HTTP status code is not handled or not allowed 2026-01-25 01:24:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/workbench-48-x-24-pegboard-panel--3-shelves-steel-square-edge>: HTTP status code is not handled or not allowed 2026-01-25 01:24:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-nexelate-48w-x-14d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 01:24:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/appliances/refrigerators_freezers/refrigerator_freezer_combos/undercounter_refrigerator_freezer_combos>: HTTP status code is not handled or not allowed 2026-01-25 01:24:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/appliances/refrigerators_freezers/refrigerator_freezer_combos/undercounter_refrigerator_freezer_combos>: HTTP status code is not handled or not allowed 2026-01-25 01:24:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/freerider-fr168-3a3-electric-scooter-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 01:24:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/teftec-omega-trac-mobility-scooter-replacement-battery-75ah-12v-it>: HTTP status code is not handled or not allowed 2026-01-25 01:24:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2568-1-2-drive-43-piece-master-sae-standard-deep-impact-socket-set>: HTTP status code is not handled or not allowed 2026-01-25 01:24:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/computers_electronics/server_racks/network_server_cabinets_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:24:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dp20-large-capacity-package-passer-drawer-ul-approved-bullet-resistant>: HTTP status code is not handled or not allowed 2026-01-25 01:24:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelate-36w-x-30d-x-60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 01:24:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/salad-table-72-3-door-megatop>: HTTP status code is not handled or not allowed 2026-01-25 01:24:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/razor-pocket-modbetty-electric-scooter-replacement-battery-8ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 01:24:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2du20107-cantilever-rack-double-sided-upright-107d-x-20h>: HTTP status code is not handled or not allowed 2026-01-25 01:24:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/computers_electronics/server_racks/network_server_cabinets_racks>: HTTP status code is not handled or not allowed 2026-01-25 01:24:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sym-hd200-200cc-scooter-and-moped-replacement-battery-2010-2013-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 01:24:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/plumbing_pumps/sewage_pumps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:24:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-shelf-support-slips-2-14l-x-2-38h>: HTTP status code is not handled or not allowed 2026-01-25 01:24:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelate-54w-x-18d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 01:24:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/peel-stick-literature-pockets-4-116w-x-5h-x-1-14d>: HTTP status code is not handled or not allowed 2026-01-25 01:24:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelate-72w-x-21d-x-80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 01:25:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-lab-pack-drum-1601mbbg-with-metal-lever-lock-bung-lid-30-gallon>: HTTP status code is not handled or not allowed 2026-01-25 01:25:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/plumbing/plumbing_pumps/sewage_pumps landed on page that is not a product page. 2026-01-25 01:25:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bombardier-summit-highmark-x-800-800cc-snowmobile-battery-2001-2004-12v>: HTTP status code is not handled or not allowed 2026-01-25 01:25:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/angled-countertop-signcard-holder-2-38l-x-1-916w>: HTTP status code is not handled or not allowed 2026-01-25 01:25:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-stick-cardboard-easel-single-wing-5>: HTTP status code is not handled or not allowed 2026-01-25 01:25:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arctic-cat-bearcat-570-xt-565cc-snowmobile-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 01:25:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 46988 pages (at 93 pages/min), scraped 23623 items (at 52 items/min) 2026-01-25 01:25:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-wheels-for-global-36-blower-fan-model-258320-600595>: HTTP status code is not handled or not allowed 2026-01-25 01:25:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ventilated-locker-vsl-121872-a-214-single-tier-no-legs-1-wide-12x18x72-welded-sand>: HTTP status code is not handled or not allowed 2026-01-25 01:25:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-gsx-600-limited-snowmobile-replacement-battery-2004-2007-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 01:25:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-black-epoxy-54w-x-18d-x-80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 01:25:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/evaporative_coolers_swamp_coolers/evaporative_cooler_accessories/media_pads already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:25:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-shelf-high-end-platform-trucks-48-x-16-2000-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-25 01:25:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ul372-stainless-steel-work-bench-top-30-x-72>: HTTP status code is not handled or not allowed 2026-01-25 01:25:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vestil-heavy-duty-panel-cart-w-4x-2-casters-4000-lb-capacity-black>: HTTP status code is not handled or not allowed 2026-01-25 01:25:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vestil-steel-magnetic-spreader-beam-144l-1000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 01:25:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-a105-v-belt-1-2-x-107>: HTTP status code is not handled or not allowed 2026-01-25 01:25:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/evaporative_coolers_swamp_coolers/evaporative_cooler_accessories/media_pads landed on page that is not a product page. 2026-01-25 01:25:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-a97-4l990-v-belt-1-2-x-99>: HTTP status code is not handled or not allowed 2026-01-25 01:25:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8210hl-s-mesh-hi-vis-safety-vest-class-2-economy-single-size-5xl-orange>: HTTP status code is not handled or not allowed 2026-01-25 01:25:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/key-blank-uncut>: HTTP status code is not handled or not allowed 2026-01-25 01:25:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8283bk-lightweight-performance-hi-vis-t-shirt-class-2-black-bottom-l-orange>: HTTP status code is not handled or not allowed 2026-01-25 01:25:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-b77-5l800-v-belt-5-8-x-80>: HTTP status code is not handled or not allowed 2026-01-25 01:25:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-b218-v-belt-5-8-x-221>: HTTP status code is not handled or not allowed 2026-01-25 01:25:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-b212-v-belt-5-8-x-215>: HTTP status code is not handled or not allowed 2026-01-25 01:25:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hvac-motor-em3711t-g-3-ph-10-hp-208-230-460-v-3600-rpm-tefc-215t-frame>: HTTP status code is not handled or not allowed 2026-01-25 01:25:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/locks_lockout_devices/lockout_tagout_devices/lockout_padlocks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:25:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-phase-motor-vem3713t-5-15-hp-3500-rpm-215tc-frame-c-face-mount-tefc-575-volts>: HTTP status code is not handled or not allowed 2026-01-25 01:25:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/severe-duty-motor-ecp82394t-5-3-ph-15-hp-575-v-3525-rpm-tefc-254t-frame>: HTTP status code is not handled or not allowed 2026-01-25 01:25:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-motor-230-460-v-60-hp-1780-rpm-3-ph-364ts-tefc>: HTTP status code is not handled or not allowed 2026-01-25 01:25:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-linea-u-desk-bark-grey>: HTTP status code is not handled or not allowed 2026-01-25 01:25:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelite-48w-x-24d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 01:25:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/whistler-cap-style-hard-hat-hdpe-shell-vented-4-pt-textile-suspension-ratchet-adj-hi-vis-orange>: HTTP status code is not handled or not allowed 2026-01-25 01:25:56 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/locks_lockout_devices/lockout_tagout_devices/lockout_padlocks landed on page that is not a product page. 2026-01-25 01:25:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-motor-230-460-v-50-hp-1775-rpm-3-ph-326t-opsb>: HTTP status code is not handled or not allowed 2026-01-25 01:25:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-poly-green-24wx14dx80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 01:26:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-poly-green-24w-x-21d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 01:26:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washdown-motor-cesswdm3554t-3-ph-15-hp-1760-rpm-208-230-460-v-tefc-145tc-fr>: HTTP status code is not handled or not allowed 2026-01-25 01:26:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-flat-free-wheelbarrow-cart-wheel-w-universal-hub-408rbpu32-480-400-8-5-8bb>: HTTP status code is not handled or not allowed 2026-01-25 01:26:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/severe-duty-motor-ecp84111t-5-3-ph-25-hp-575-v-1180-rpm-tefc-324t-frame>: HTTP status code is not handled or not allowed 2026-01-25 01:26:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 47083 pages (at 95 pages/min), scraped 23686 items (at 63 items/min) 2026-01-25 01:26:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-poly-green-60wx21dx80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 01:26:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-poly-green-24w-x-24d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 01:26:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hvac-motor-ehm2334t-5-3-ph-20-hp-575-v-1765-rpm-tefc-256t-frame>: HTTP status code is not handled or not allowed 2026-01-25 01:26:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-round-chrome-cafeteria-table-with-8-stools-gray-mdf-core-top-blue-stools>: HTTP status code is not handled or not allowed 2026-01-25 01:26:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hvac-motor-em4109t-g-3-ph-40-hp-230-460-v-3600-rpm-tefc-324ts-frame>: HTTP status code is not handled or not allowed 2026-01-25 01:26:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/material-handling/mezzanines_modular_structures/catwalks_mezzanines/mezzanines already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:26:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machined-chrome-steel-needle-roller-bearing-w-inner-ring-90-x-120-x-36mm>: HTTP status code is not handled or not allowed 2026-01-25 01:26:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/severe-duty-motor-ecp84413t-5-3-ph-150-hp-575-v-3600-rpm-tefc-445ts-frame>: HTTP status code is not handled or not allowed 2026-01-25 01:26:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/permanent-magnet-motor-cspm3611t-3-ph-230-460-v-3-hp-1800-rpm-tefc-182tc-frame>: HTTP status code is not handled or not allowed 2026-01-25 01:26:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machined-chrome-steel-needle-roller-bearing-w-inner-ring-70-x-95-x-25mm>: HTTP status code is not handled or not allowed 2026-01-25 01:26:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machined-chrome-steel-needle-roller-bearing-w-inner-ring-60-x-85-x-34mm>: HTTP status code is not handled or not allowed 2026-01-25 01:26:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-nexelon-36w-x-18d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 01:26:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evap-cooling-bandana-w-cooling-towel-h-l-solid-blue-12577>: HTTP status code is not handled or not allowed 2026-01-25 01:26:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/material-handling/mezzanines_modular_structures/catwalks_mezzanines/mezzanines landed on page that is not a product page. 2026-01-25 01:26:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-poly-green-54wx18dx80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 01:26:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultratech-gloves-nitrile-cut-a4-impact-1-abrasion-4-puncture-4-gray-black-xl>: HTTP status code is not handled or not allowed 2026-01-25 01:26:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/gloves_sleeves/cut_resistant_gloves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:26:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plug-in-horsepwer-resistor-and-fuse-kit-br0180-018-ohms-085-amps>: HTTP status code is not handled or not allowed 2026-01-25 01:26:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/feedback-cable-w-assembly-ms-connector-cbl061zd-2-20-ft-extension-length>: HTTP status code is not handled or not allowed 2026-01-25 01:26:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electric-pump-0-5-hp-1-gallon-3-way-3-position-manual-valve>: HTTP status code is not handled or not allowed 2026-01-25 01:26:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/awn-en1030-8bb-window-entry-awning-8-4-1-2-w-x-2-6-d-x-1-4-h-bright-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:26:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelon-24w-x-18d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 01:26:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/awn-h23-5cop-seam-slope-awning-5-8-w-x-3d-x-2h-metal>: HTTP status code is not handled or not allowed 2026-01-25 01:26:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/gloves_sleeves/cut_resistant_gloves landed on page that is not a product page. 2026-01-25 01:26:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-nexelon-30w-x-24d-x-60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 01:26:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-acting-hydraulic-cylinder-30-ton-14-stroke>: HTTP status code is not handled or not allowed 2026-01-25 01:26:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-w-louver-panel-rack-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:26:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1008-x-3-4-3-4-x-133-1008-series-tapered-locking-steel-bushing-3-4-bore>: HTTP status code is not handled or not allowed 2026-01-25 01:26:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1610-x-1-7-16-1-7-16-x-23-1610-series-tapered-locking-steel-bushing-1-7-16-bore>: HTTP status code is not handled or not allowed 2026-01-25 01:26:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-ac-motor-c-face-kit-field-conversion-40-1400-odp-40-284-6uc-tc-frame>: HTTP status code is not handled or not allowed 2026-01-25 01:26:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/activity-table-height-insert-set-of-4>: HTTP status code is not handled or not allowed 2026-01-25 01:27:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelon-42w-x-18d-x-80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 01:27:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-snap-on-open-seal-for-poly-1000case>: HTTP status code is not handled or not allowed 2026-01-25 01:27:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-first-aid-refill-kit-25-50-person-ansi-compliant-class-a>: HTTP status code is not handled or not allowed 2026-01-25 01:27:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waring-wct704-toaster-4-slice-2-slot-120v>: HTTP status code is not handled or not allowed 2026-01-25 01:27:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelon-36w-x-21d-x-69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 01:27:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 47178 pages (at 95 pages/min), scraped 23748 items (at 62 items/min) 2026-01-25 01:27:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-head-mechanism-w-snap-switch-150s-hd-use-with-150-s-series>: HTTP status code is not handled or not allowed 2026-01-25 01:27:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bolted-teardrop-pallet-rack-starter-144inw-x-42ind-x-192inh-5702-lbs-shelf-cap>: HTTP status code is not handled or not allowed 2026-01-25 01:27:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-lids-perfect-touch-dome-fits-8-oz-1000-carton-white>: HTTP status code is not handled or not allowed 2026-01-25 01:27:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-house-blend-coffee-regular-25-oz-18-carton>: HTTP status code is not handled or not allowed 2026-01-25 01:27:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/outdoor-curved-bench-steel-slat-45-degree-unit-black>: HTTP status code is not handled or not allowed 2026-01-25 01:27:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/foodservice_disposables/disposable_plates_bowls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:27:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/omnicart-cab-36-cap-big-beam-w-locking-panel-beige-49-3-4w-x-17d-x-55-1-8h>: HTTP status code is not handled or not allowed 2026-01-25 01:27:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/construction-harness-back-front-shoulder-side-d-ring-m-1112576>: HTTP status code is not handled or not allowed 2026-01-25 01:27:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-1675-back-support-brace-with-coolingwarming-pack-l-black>: HTTP status code is not handled or not allowed 2026-01-25 01:27:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-electric-infrared-heater-fsp-4348-3-heavy-duty-430kw-480v>: HTTP status code is not handled or not allowed 2026-01-25 01:27:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/foodservice_disposables/disposable_plates_bowls landed on page that is not a product page. 2026-01-25 01:27:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/infrared-electric-heater-fss-9524-3-heavy-duty-950-kw-240v>: HTTP status code is not handled or not allowed 2026-01-25 01:27:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cordless-impact-wrench-kit-34-chuck-size-standard-friction-ring-anvil-20v>: HTTP status code is not handled or not allowed 2026-01-25 01:27:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/19n132-0051-f1-model-19-troybilt-tiller-horizontal-shaft>: HTTP status code is not handled or not allowed 2026-01-25 01:27:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/gas_engines already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:27:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:27:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jurni-multi-purpose-table-with-t-legs--glides-60l-x-24w-x-29h-fusion-maple>: HTTP status code is not handled or not allowed 2026-01-25 01:27:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/twin-series-clamp-for-hose-tsch062-5-8-inch-id-1034-od>: HTTP status code is not handled or not allowed 2026-01-25 01:27:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/western-global-fuelcube153-type-s-diesel-fuel-storage-tank-119-gallon-capacity>: HTTP status code is not handled or not allowed 2026-01-25 01:27:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/floor_scrubbers_floor_machines already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:27:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90-straight-thread-hose-connector-h890x20x16-1-inch-male-sae-thread-1-1-4-inch-hose-id>: HTTP status code is not handled or not allowed 2026-01-25 01:27:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/heaters/heater_accessories_components/heater_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:27:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/motors_power_transmission/gas_engines>: HTTP status code is not handled or not allowed 2026-01-25 01:27:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats landed on page that is not a product page. 2026-01-25 01:27:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simpson-professional-pressure-washer-spray-gun-hotcold-water-powered-machines-to-5000-psi>: HTTP status code is not handled or not allowed 2026-01-25 01:27:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-pressure-hose-assembly-hp16108-1-inch-npt-x-1-inch-npt-x-9-foot-long-hose>: HTTP status code is not handled or not allowed 2026-01-25 01:27:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/floor_scrubbers_floor_machines>: HTTP status code is not handled or not allowed 2026-01-25 01:27:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/heaters/heater_accessories_components/heater_accessories>: HTTP status code is not handled or not allowed 2026-01-25 01:27:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wsmr2-151-shaft-mount-reducer-size-2-151-ratio-1-1516-tapered-bore>: HTTP status code is not handled or not allowed 2026-01-25 01:27:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/gearboxes_speed_reducers/shaft_mount_reducers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:27:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wing-type-quick-detach-hydraulic-coupler-qdwc122-3-4-inch-npt-coupler-18-flow-gpm>: HTTP status code is not handled or not allowed 2026-01-25 01:27:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wing-type-quick-detach-hydraulic-coupler-qdwc241-1-1-2-inch-npt-coupler-100-flow-gpm>: HTTP status code is not handled or not allowed 2026-01-25 01:28:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-replacement-handle-for-push-brooms-wood-threaded-handle>: HTTP status code is not handled or not allowed 2026-01-25 01:28:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reducer-bushing-h3109x16x6-1-inch-x-1-2-inch-npt-male-to-female>: HTTP status code is not handled or not allowed 2026-01-25 01:28:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sa-alphabetter-adjustable-height-desk-swinging-footrest-bar-28-x-20-dryerase>: HTTP status code is not handled or not allowed 2026-01-25 01:28:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metro-extra-shelf-for-open-wire-shelving-24x14>: HTTP status code is not handled or not allowed 2026-01-25 01:28:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/gearboxes_speed_reducers/shaft_mount_reducers landed on page that is not a product page. 2026-01-25 01:28:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medical-transporter-infinite-recline-5-casters-push-bar-side-table-300-lbs-max-colonial-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:28:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/modular_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:28:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sbl4-lp835-4-led-square-basket-wraparound-mvolt-30l-3500-cct>: HTTP status code is not handled or not allowed 2026-01-25 01:28:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:28:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4000-2-1-2-ball-valve-flanged-2-1-2-cast-iron>: HTTP status code is not handled or not allowed 2026-01-25 01:28:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 47271 pages (at 93 pages/min), scraped 23807 items (at 59 items/min) 2026-01-25 01:28:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-long-sleeve-button-down-work-shirt-m-tall-navy>: HTTP status code is not handled or not allowed 2026-01-25 01:28:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:28:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/worldwide-electric-winl147-120-1-254-6tc-helical-inline-reducer-254-6tc-input-flange-120-1-ratio>: HTTP status code is not handled or not allowed 2026-01-25 01:28:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:28:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wind-pro-full-zip-hooded-sweater-reflective-flame-resistant-s-orange>: HTTP status code is not handled or not allowed 2026-01-25 01:28:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-racksolid-steel-deck-58-w-x-24-d-797215>: HTTP status code is not handled or not allowed 2026-01-25 01:28:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/floor_mats/modular_mats landed on page that is not a product page. 2026-01-25 01:28:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hickory-half-zip-pull-over-long-sleeve-shirt-4xl-tall-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:28:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts>: HTTP status code is not handled or not allowed 2026-01-25 01:28:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:28:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wpk-a-corner-wall-protection-kit-1-64-black-yellow-60-6866>: HTTP status code is not handled or not allowed 2026-01-25 01:28:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:28:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-long-sleeve-button-down-work-shirt-3xl-navy>: HTTP status code is not handled or not allowed 2026-01-25 01:28:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/traffic_parking_lot_safety/delineator_posts_channelizer_cones already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:28:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-vis-long-sleeve-shirt-tear-and-rip-resistant-m-orange>: HTTP status code is not handled or not allowed 2026-01-25 01:28:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:28:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-vis-long-sleeve-shirt-tear-and-rip-resistant-l-yellow>: HTTP status code is not handled or not allowed 2026-01-25 01:28:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:28:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4979300-str-wrap-20x1000-80ga>: HTTP status code is not handled or not allowed 2026-01-25 01:28:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts landed on page that is not a product page. 2026-01-25 01:28:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/laptop-tablet-riser-black-red>: HTTP status code is not handled or not allowed 2026-01-25 01:28:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts landed on page that is not a product page. 2026-01-25 01:28:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:28:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats landed on page that is not a product page. 2026-01-25 01:28:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/under-desk-mouse-platform-black>: HTTP status code is not handled or not allowed 2026-01-25 01:28:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/desk_accessories/document_holders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:28:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/traffic_parking_lot_safety/delineator_posts_channelizer_cones>: HTTP status code is not handled or not allowed 2026-01-25 01:28:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/underbed-truck-bx-black-steel-flatbedplatform-truck-11-2-cuft-560-5-02>: HTTP status code is not handled or not allowed 2026-01-25 01:28:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats landed on page that is not a product page. 2026-01-25 01:28:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts>: HTTP status code is not handled or not allowed 2026-01-25 01:28:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/truck_boxes_utility_boxes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:28:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-33w-x-36l-navy>: HTTP status code is not handled or not allowed 2026-01-25 01:28:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/truck_boxes_utility_boxes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:28:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/overhead-tool-trolley-pbtt-72-pb-series-light-duty-for-72-benches-black>: HTTP status code is not handled or not allowed 2026-01-25 01:28:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:28:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:28:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats landed on page that is not a product page. 2026-01-25 01:28:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts>: HTTP status code is not handled or not allowed 2026-01-25 01:28:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traffic-cone-chain-kit-blue-mrc-93206-6>: HTTP status code is not handled or not allowed 2026-01-25 01:28:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:28:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galvanized-steel-solid-shelving-add-on-189909>: HTTP status code is not handled or not allowed 2026-01-25 01:28:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/truck_boxes_utility_boxes>: HTTP status code is not handled or not allowed 2026-01-25 01:28:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/truck_boxes_utility_boxes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:28:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/truck_boxes_utility_boxes>: HTTP status code is not handled or not allowed 2026-01-25 01:28:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/desk_accessories/document_holders landed on page that is not a product page. 2026-01-25 01:28:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 01:28:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/activity-tables-48-l-x-30-w-standard-height-rectangular-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:28:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-42w-x-33l-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:28:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 01:28:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-38w-x-36l-green>: HTTP status code is not handled or not allowed 2026-01-25 01:28:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 01:28:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:28:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/truck_boxes_utility_boxes>: HTTP status code is not handled or not allowed 2026-01-25 01:28:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5697400-36x18x86-add-on-4-shelf-unit>: HTTP status code is not handled or not allowed 2026-01-25 01:28:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-60-x-14-3-shelf-media-stand-chrome-798802>: HTTP status code is not handled or not allowed 2026-01-25 01:28:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:28:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-double-wall-cardboard-corrugated-box-12-x-10-x-10-275-lb-ect-48-15-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:28:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:28:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:28:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-33w-x-28l-navy>: HTTP status code is not handled or not allowed 2026-01-25 01:28:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:28:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 01:28:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lewiston-mailbox-post-ornate-base-pineapple-finial-w-3-address-plates-support-brace-bronze>: HTTP status code is not handled or not allowed 2026-01-25 01:28:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:29:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/friedrich-packaged-terminal-air-conditioner9000-btu-cool-292279>: HTTP status code is not handled or not allowed 2026-01-25 01:29:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:29:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-inch-drive-socket-10-mm-6-point-1-3-32-inch-long>: HTTP status code is not handled or not allowed 2026-01-25 01:29:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 01:29:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 01:29:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 01:29:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 01:29:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-inch-drive-socket-9-16-inch-12-point-1-1-8-inch-l>: HTTP status code is not handled or not allowed 2026-01-25 01:29:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 01:29:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 47348 pages (at 77 pages/min), scraped 23833 items (at 26 items/min) 2026-01-25 01:29:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-54w-x-31l-black>: HTTP status code is not handled or not allowed 2026-01-25 01:29:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:29:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-50w-x-33l-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:29:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/boost-plus-1000-amp-lithium-jump-starter-gb40>: HTTP status code is not handled or not allowed 2026-01-25 01:29:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-set-7>: HTTP status code is not handled or not allowed 2026-01-25 01:29:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-42w-x-36l-black>: HTTP status code is not handled or not allowed 2026-01-25 01:29:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manchester-stucco-locking-column-mailbox-in-sandstone-w-decorative-scroll-door-in-antique-copper>: HTTP status code is not handled or not allowed 2026-01-25 01:29:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-32w-x-31l-brown>: HTTP status code is not handled or not allowed 2026-01-25 01:29:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 01:29:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-48w-x-30l-black>: HTTP status code is not handled or not allowed 2026-01-25 01:29:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:29:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-brushes-canister-6-color-set-144-pcs-set>: HTTP status code is not handled or not allowed 2026-01-25 01:29:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 01:29:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/horizontal-mobile-paper-rack-holds-8-rolls-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:29:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-46w-x-36l-black>: HTTP status code is not handled or not allowed 2026-01-25 01:29:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-50w-x-34l-navy>: HTTP status code is not handled or not allowed 2026-01-25 01:29:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:29:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-34w-x-34l-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:29:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:29:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 01:29:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gkt-2006-gasket-for-motor-terminal-conduit-box>: HTTP status code is not handled or not allowed 2026-01-25 01:29:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:29:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-42w-x-32l-tan>: HTTP status code is not handled or not allowed 2026-01-25 01:29:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:29:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:29:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-34w-x-unhemmed-black>: HTTP status code is not handled or not allowed 2026-01-25 01:29:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:29:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 01:29:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-48w-x-32l-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:29:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 01:29:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esafe-series-310-safety-coupling-38-male-npt-industrial-intch-14>: HTTP status code is not handled or not allowed 2026-01-25 01:29:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:29:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 01:29:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ctl-bandsaw-blade-11-foot-10-inch-long-x-3-4-inch-wide-4-6-tpi-x-0035-thick>: HTTP status code is not handled or not allowed 2026-01-25 01:29:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 01:29:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ctl-bandsaw-blade-6-foot-10-inch-long-x-3-4-inch-wide-8-12-tpi-x-0035-thick>: HTTP status code is not handled or not allowed 2026-01-25 01:29:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 01:29:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:29:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-36w-x-32l-navy>: HTTP status code is not handled or not allowed 2026-01-25 01:29:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wedge-locking-washer-carbon-steel-zinc-flake-coated-m6-pkg-of-20>: HTTP status code is not handled or not allowed 2026-01-25 01:29:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 01:29:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:29:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 01:29:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-38w-x-32l-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:29:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inf-tek-mate-refrigerant-leak-detector-705-202-g1-detects-r22-r410a-r134a>: HTTP status code is not handled or not allowed 2026-01-25 01:29:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 01:29:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:29:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/earplugs-corded-poly-bag-100-pair>: HTTP status code is not handled or not allowed 2026-01-25 01:29:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:29:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/07060-butcher-steel-high-carbon-steel-14-l>: HTTP status code is not handled or not allowed 2026-01-25 01:29:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:29:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 01:29:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-28w-x-34l-dark-navy>: HTTP status code is not handled or not allowed 2026-01-25 01:30:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:30:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-jeans-indura-denim-14-oz-31w-x-30l-flame-resistant-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:30:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:30:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 01:30:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:30:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 01:30:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rcmp-181801-rubber-cork-pads-18-l-x-18-w-x-7-8-h>: HTTP status code is not handled or not allowed 2026-01-25 01:30:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a18k-v-belt-kevlar-1-2-x-20>: HTTP status code is not handled or not allowed 2026-01-25 01:30:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 47426 pages (at 78 pages/min), scraped 23867 items (at 34 items/min) 2026-01-25 01:30:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:30:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 01:30:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-visibility-ripstop-cargo-pants-36w-x-34l-navy>: HTTP status code is not handled or not allowed 2026-01-25 01:30:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 01:30:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:30:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-work-pants-reflective-material-flame-resistant-42w-x-unhemmed-red>: HTTP status code is not handled or not allowed 2026-01-25 01:30:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 01:30:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/en-7-316ss-drop-forged-eye-nut-5-8-11-style-a-316-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 01:30:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:30:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-sheet-diamond-tread-56022-24-inchl-x-12-inchw-x-073-inchh-silver>: HTTP status code is not handled or not allowed 2026-01-25 01:30:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-westex-33w-x-unhemmed-black>: HTTP status code is not handled or not allowed 2026-01-25 01:30:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 01:30:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 01:30:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:30:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:30:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spkhz-55-hazmat-55-gallon-spill-kit>: HTTP status code is not handled or not allowed 2026-01-25 01:30:21 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 01:30:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-wire-shelving-add-on-54-inch-w-x-24-inch-d-x-54-inch-h>: HTTP status code is not handled or not allowed 2026-01-25 01:30:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-wire-shelving-72-inch-w-x-21-inch-d-x-63-inch-h>: HTTP status code is not handled or not allowed 2026-01-25 01:30:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-wire-shelving-add-on-42-inch-w-x-21-inch-d-x-86-inch-h>: HTTP status code is not handled or not allowed 2026-01-25 01:30:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-wire-shelving-24-inch-w-x-24-inch-d-x-86-inch-h>: HTTP status code is not handled or not allowed 2026-01-25 01:30:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-shelf-blue-poly-cabinet-24180>: HTTP status code is not handled or not allowed 2026-01-25 01:30:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-14-piece-grilling-tool-set>: HTTP status code is not handled or not allowed 2026-01-25 01:30:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-work-pants-reflective-material-flame-resistant-44w-x-unhemmed-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:30:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 01:30:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 01:30:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:30:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 01:30:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simply-grilling-non-stick-12x16-grilling-platter>: HTTP status code is not handled or not allowed 2026-01-25 01:30:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-4040-v-cross-cut-shredder-press-combination-120-sheet-capacity>: HTTP status code is not handled or not allowed 2026-01-25 01:30:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-jeans-indura-denim-14-oz-58w-x-30l-flame-resistant-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:30:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:30:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/optional-duct-kit-ck-24s-for-kpac2421-2>: HTTP status code is not handled or not allowed 2026-01-25 01:30:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/x-o-rust-12-oz-aerosol-rust-preventative-paint-primer-in-one-almond-gloss-125739>: HTTP status code is not handled or not allowed 2026-01-25 01:30:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 01:30:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:30:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-30w-x-21d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 01:30:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/start-right-interior-latex-vapor-barrier-primer-5-gallon-775863>: HTTP status code is not handled or not allowed 2026-01-25 01:30:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 01:30:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-9a-3ph-remote-start-reset-button-250-500v-1-7-2-3a>: HTTP status code is not handled or not allowed 2026-01-25 01:30:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-9a-3ph-remote-start-terminals-reset-button-250-500v-1-7-2-3a>: HTTP status code is not handled or not allowed 2026-01-25 01:30:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-epoxy-wire-shelving-add-on-72w-x-18d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 01:30:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-epoxy-wire-shelving-42w-x-24d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 01:30:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rtl100017-walker-glide-ski-gray-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 01:30:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-14w-x-61-12h-deluxe-freestanding-office-partition-panel-gray-with-cherry-277529FGY>: HTTP status code is not handled or not allowed 2026-01-25 01:30:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-9a-1ph-remote-start-reset-button-100-250v-1-7-2-3a>: HTTP status code is not handled or not allowed 2026-01-25 01:30:49 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 01:30:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-12a-3ph-remote-start-reset-button-250-500v-3-1-4-2a>: HTTP status code is not handled or not allowed 2026-01-25 01:30:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/itemizer-drawer-unit-lateral-49-in-314-3>: HTTP status code is not handled or not allowed 2026-01-25 01:30:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_storage_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:30:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_storage_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:30:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/comm-shelv-van-pack-high-rf-ford-transit-600-8410x>: HTTP status code is not handled or not allowed 2026-01-25 01:30:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1500054-trigger2trigger-lanyard-05-x-12-10-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:30:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-16a-3ph-separate-coil-voltage-start-stop-100-250v-7-6-10a>: HTTP status code is not handled or not allowed 2026-01-25 01:30:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-16a-3ph-reset-button-24-60v-10-13a>: HTTP status code is not handled or not allowed 2026-01-25 01:30:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_shelving already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:31:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-16a-3ph-remote-start-reset-button-100-250v-16-20a>: HTTP status code is not handled or not allowed 2026-01-25 01:31:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:31:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:31:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deep-groove-ball-bearings-metric-6005-2rs-2-rubber-seals-light-duty-25mm-bore-47mm-od>: HTTP status code is not handled or not allowed 2026-01-25 01:31:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-8482-thermoplastic-coated-32-gallon-perforated-receptacle-w-dome-lid-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:31:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/attached-lid-shipping-container-27-3-16-x-16-5-8-x-12-1-2-red-with-dolly-combo>: HTTP status code is not handled or not allowed 2026-01-25 01:31:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deep-groove-ball-bearings-metric-6204-2rs-2-rubber-seals-medium-duty-20mm-bore-47mm-od>: HTTP status code is not handled or not allowed 2026-01-25 01:31:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 47513 pages (at 87 pages/min), scraped 23904 items (at 37 items/min) 2026-01-25 01:31:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment>: HTTP status code is not handled or not allowed 2026-01-25 01:31:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_cabinets_drawers?token=null already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:31:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:31:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-26a-1ph-remote-start-reset-button-24-60v-24-26a>: HTTP status code is not handled or not allowed 2026-01-25 01:31:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/nuts/lock_nuts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:31:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/390140-flat-washer-3-8-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 01:31:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tm30g-traffic-sign-no-parking-between-signs-wright-arrow-18-x-12-whitered>: HTTP status code is not handled or not allowed 2026-01-25 01:31:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precision-compression-spring-018-od-x-0022-x-075-85-lbsin-rate-music-wire-pkg-of-5>: HTTP status code is not handled or not allowed 2026-01-25 01:31:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tm7g-traffic-sign-visitor-parking-only-18-x-12-whitered>: HTTP status code is not handled or not allowed 2026-01-25 01:31:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment>: HTTP status code is not handled or not allowed 2026-01-25 01:31:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-pit-cover-hvd-1010cvr-m-rectangle-heavy-duty-waterproof-50l-x-38w-x-23h-mocha>: HTTP status code is not handled or not allowed 2026-01-25 01:31:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/fasteners/nuts/lock_nuts>: HTTP status code is not handled or not allowed 2026-01-25 01:31:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/travira-stacking-side-chair-tekwood-vintage-2pack>: HTTP status code is not handled or not allowed 2026-01-25 01:31:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_cabinets_drawers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:31:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n43pb-osha-sign-notice-wash-Hands-before-returning-to-work-10-x-14-whiteblueblack>: HTTP status code is not handled or not allowed 2026-01-25 01:31:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n6rb-osha-sign-notice-safety-glasses-required-in-this-area-10-x-14-whiteblueblack>: HTTP status code is not handled or not allowed 2026-01-25 01:31:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bluff-174-b6024-aluminum-dock-plate-11-850-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 01:31:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-65a-1ph-direct-online-start-stop-100-250v-36-47a>: HTTP status code is not handled or not allowed 2026-01-25 01:31:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vestil-air-bag-scissor-li-table-ablt-2000-48-x-32-2000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 01:31:34 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_cabinets_drawers landed on page that is not a product page. 2026-01-25 01:31:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vsm-abrasive-belt-104833-zirconia-alumina-2-x-60-24-grit>: HTTP status code is not handled or not allowed 2026-01-25 01:31:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-80a-3ph-remote-start-terminals-start-stop-100-250v-75-87a>: HTTP status code is not handled or not allowed 2026-01-25 01:31:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vsm-abrasive-belt-1966-aluminum-oxide-25-x-48-120-grit>: HTTP status code is not handled or not allowed 2026-01-25 01:31:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelate-wire-shelving-60-w-x-24-d-x-86-h>: HTTP status code is not handled or not allowed 2026-01-25 01:31:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/af-f-jack-stand-10-ton-ratchet-type-medium-height-red-pair>: HTTP status code is not handled or not allowed 2026-01-25 01:31:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vsm-abrasive-belt-8181-silicon-carbide-4-x-106-120-grit>: HTTP status code is not handled or not allowed 2026-01-25 01:31:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vsm-abrasive-belt-83344-zirconia-alumina-4-x-54-60-grit>: HTTP status code is not handled or not allowed 2026-01-25 01:31:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-step-800-lb-capacity-work-platform-with-24-w-x-48-l-platform>: HTTP status code is not handled or not allowed 2026-01-25 01:31:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/builder-plus-52-indoor-ceiling-fan-53236-snow-white>: HTTP status code is not handled or not allowed 2026-01-25 01:31:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5555461780bx1-14-10x-1-x-0035-bimetal-independence-ii-46-bandsaw-blade>: HTTP status code is not handled or not allowed 2026-01-25 01:31:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-step-configurable-forward-descent-roll-ladder-perforated-ukdec108246>: HTTP status code is not handled or not allowed 2026-01-25 01:31:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-frame-slab-rack-single-sided-wall-display-aframewallrack-60w-x-36d-x-60h>: HTTP status code is not handled or not allowed 2026-01-25 01:31:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/62-ibc-stairway-stair36i-9-62>: HTTP status code is not handled or not allowed 2026-01-25 01:31:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/29-in-portable-atomizing-evaporative-cooler-26-gallon-5885-cfm>: HTTP status code is not handled or not allowed 2026-01-25 01:31:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/under-sink-recirculation-system-for-hot-water-tank-without-timer>: HTTP status code is not handled or not allowed 2026-01-25 01:31:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/34-793-fiberglass-open-reel-long-tape-1-2-inch-x-200>: HTTP status code is not handled or not allowed 2026-01-25 01:32:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/131-osha-stairway-stair36o-15-131>: HTTP status code is not handled or not allowed 2026-01-25 01:32:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keyless-rigid-coupling-c600e350-steel-35d-x-7283d-312l-shaft>: HTTP status code is not handled or not allowed 2026-01-25 01:32:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-54-inch-w-x-18-inch-d-x-62-inch-h-2000-lbs-cap-5-open-grid-shelves-dunnage-stand>: HTTP status code is not handled or not allowed 2026-01-25 01:32:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-shaft-arbor-al4-lefthand-type-b-11316l-thread-fits-12-shaft>: HTTP status code is not handled or not allowed 2026-01-25 01:32:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-48-inch-w-x-18-inch-d-x-74-inch-h-2000-lbs-cap-3-open-and-1-solid-grid-shelves>: HTTP status code is not handled or not allowed 2026-01-25 01:32:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 47606 pages (at 93 pages/min), scraped 23960 items (at 56 items/min) 2026-01-25 01:32:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pillow-block-ball-bearing-pbpsbl050-selfaligning-pressed-steel-housing-12-bore>: HTTP status code is not handled or not allowed 2026-01-25 01:32:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-frame-pvc-decontamination-pool-stfdp-33-36l-x-12w-x-39h-60-gallon-cap-yellow>: HTTP status code is not handled or not allowed 2026-01-25 01:32:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-frame-pvc-decontamination-pool-stfdp-33-36l-x-12w-x-39h-60-gallon-cap-black>: HTTP status code is not handled or not allowed 2026-01-25 01:32:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drink-water-bladder-tank-bt-500pw-108l-x-84w-x-16h-500-gallon-cap-powder-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:32:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pws-plus-003-micro-precision-wire-strippers-030-1mm-solid-stranded-wires>: HTTP status code is not handled or not allowed 2026-01-25 01:32:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pc-strip-cable-stripper-5-13mm-pvc-insulated-data-cables>: HTTP status code is not handled or not allowed 2026-01-25 01:32:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dpg54-9d-ansi-z87-1-protector-i-o>: HTTP status code is not handled or not allowed 2026-01-25 01:32:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/wire_shelving/poly_zbrite_wire_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:32:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-24-inch-w-x-21-inch-d-x-74-inch-h-2000-lbs-cap-4-open-grid-shelves>: HTTP status code is not handled or not allowed 2026-01-25 01:32:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-panel-drug-cup-test-25-tests-per-box>: HTTP status code is not handled or not allowed 2026-01-25 01:32:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-2-series-air-curtain-72-wide-unheated-208-230v-1-ph-obsidian-black>: HTTP status code is not handled or not allowed 2026-01-25 01:32:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-basket-truck-with-vinyl-liner-6-bushel-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:32:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jettrac-satellite-post-39h-65l-florescent-yellow-retractable-belt-s-hook>: HTTP status code is not handled or not allowed 2026-01-25 01:32:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/shelving/wire_shelving/poly_zbrite_wire_shelves>: HTTP status code is not handled or not allowed 2026-01-25 01:32:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medina-right-curved-desk-extension-47w-x-28d-x-29-1-2h-gray-steel>: HTTP status code is not handled or not allowed 2026-01-25 01:32:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/exam_patient_room_supplies/exam_room_consumables/cotton_balls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:32:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sponge-bowl-1-qt>: HTTP status code is not handled or not allowed 2026-01-25 01:32:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/versatile-4c-mailbox-4c06d-04-23-14h-4-mailboxes-1-parcel-front-loading-silver-usps>: HTTP status code is not handled or not allowed 2026-01-25 01:32:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sensi-wrap-self-adherent-bandage-rolls-2w-x-5-yards-dark-blue-36-pcs>: HTTP status code is not handled or not allowed 2026-01-25 01:32:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/siligentle-silicone-bordered-foam-dressing-2l-x-2w-120-pcs>: HTTP status code is not handled or not allowed 2026-01-25 01:32:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clean-room-24-x36-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:32:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/exam_patient_room_supplies/exam_room_consumables/cotton_balls>: HTTP status code is not handled or not allowed 2026-01-25 01:32:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sensi-wrap-self-adherent-bandage-rolls-4w-x-5-yards-red-18-pcs>: HTTP status code is not handled or not allowed 2026-01-25 01:32:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sensi-wrap-self-adherent-bandage-rolls-2w-x-5-yards-assorted-36-pcs>: HTTP status code is not handled or not allowed 2026-01-25 01:32:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/two_way_radios_accessories/two_way_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:32:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-torx-power-bit-16625x-1-4-hexagon-t25-drive-6-long>: HTTP status code is not handled or not allowed 2026-01-25 01:32:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-quiet-oil-free-air-compressor-with-sound-proof-cabinet-10-gal-vertical>: HTTP status code is not handled or not allowed 2026-01-25 01:32:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aberdeen-series-pedestal-mobile-kit-for-apbf20-suspended-credenzas-mocha>: HTTP status code is not handled or not allowed 2026-01-25 01:32:51 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-25 01:32:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medina-series-low-wall-cabinet-with-wood-glass-door-textured-sea-salt>: HTTP status code is not handled or not allowed 2026-01-25 01:32:52 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:24 2026-01-25 01:32:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/tool_storage/tool_boxes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:32:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aneroid-sphygmomanometer-toddler-1-to-4-years-old-black-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 01:32:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/two_way_radios_accessories/two_way_accessories landed on page that is not a product page. 2026-01-25 01:32:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kid-seat-coat-locker-2-wide-20w-x-17-1-2d-x-50-1-2h-birch-plywood>: HTTP status code is not handled or not allowed 2026-01-25 01:32:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-tub-mobile-cubbie-w-o-tubs-60w-x-15d-x-29-1-2h-gray-laminate-black-edge>: HTTP status code is not handled or not allowed 2026-01-25 01:32:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tm016j-traffic-sign-no-parking-any-time-with-double-arrow-18-x-12-white>: HTTP status code is not handled or not allowed 2026-01-25 01:32:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mounted-storage-diaper-organizer-48w-x-12d-x-25h-birch-plywood>: HTTP status code is not handled or not allowed 2026-01-25 01:33:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waterproof-bordered-foam-dressing-4l-x-4w-120-pcs>: HTTP status code is not handled or not allowed 2026-01-25 01:33:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/tool_storage/tool_boxes landed on page that is not a product page. 2026-01-25 01:33:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paper-tray-tub-lid13-1-2l-x-11w-x-1-4h-almond>: HTTP status code is not handled or not allowed 2026-01-25 01:33:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tm98h-traffic-sign-no-parking-bi-lingual-18-x-12-white>: HTTP status code is not handled or not allowed 2026-01-25 01:33:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-sheet-panel-truck-36-x-72>: HTTP status code is not handled or not allowed 2026-01-25 01:33:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18l-synthetic-absorbable-sutures-c3-needle-violet-size-4-to-0-12-pcs>: HTTP status code is not handled or not allowed 2026-01-25 01:33:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dynarest-airfloat-deluxe-air-mattress-w-digital-pump-80l-x-36w-x-8h>: HTTP status code is not handled or not allowed 2026-01-25 01:33:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premier-free-standing-undercounter-refrigerator-abt-ucfs-0204g-2-3-cu-ft>: HTTP status code is not handled or not allowed 2026-01-25 01:33:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12oz-hs-shredder-oil-6-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:33:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-large-capacity-refrigerator-for-pharmacies-abt-47b-47-cu-ft>: HTTP status code is not handled or not allowed 2026-01-25 01:33:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 47699 pages (at 93 pages/min), scraped 24011 items (at 51 items/min) 2026-01-25 01:33:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/back-pain-relief-red-regular>: HTTP status code is not handled or not allowed 2026-01-25 01:33:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-low-profile-beveled-slimline-vision-lite-vsl2434b-24-x-34-bronze>: HTTP status code is not handled or not allowed 2026-01-25 01:33:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/firerated-door-louver-1900a2424b-adjustable-zblade-selfattach-24-x-24-bronze>: HTTP status code is not handled or not allowed 2026-01-25 01:33:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anatomical-chart-parkinsons-disease-paper>: HTTP status code is not handled or not allowed 2026-01-25 01:33:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kd-adjustable-height-h-stand-24in-to-36in-3000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 01:33:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anatomical-model-leo-the-ligament-skeleton-on-roller-stand>: HTTP status code is not handled or not allowed 2026-01-25 01:33:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anatomical-chart-spinal-nerves-laminated>: HTTP status code is not handled or not allowed 2026-01-25 01:33:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/desk-mount-cable-anchor-kmw64613-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:33:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/cable_management/desk_grommets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:33:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/monitor_risers_stands already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:33:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cd-recordable-media-ccs72102-cd-rw-media-12x-speed-700-mb-capcity>: HTTP status code is not handled or not allowed 2026-01-25 01:33:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rp-works-two-eye-tie-work-sneaker-breathable-knit-textile-leather-blue-tan-8w>: HTTP status code is not handled or not allowed 2026-01-25 01:33:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inner-ring-for-shell-type-needle-roller-bearing-inch-212-bore-234-od-3213mm-width>: HTTP status code is not handled or not allowed 2026-01-25 01:33:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/abrasives/general_purpose_sanding/cloth_sanding_bands already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:33:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rp-works-two-eye-tie-work-sneaker-breathable-knit-textile-leather-brown-tan-8m>: HTTP status code is not handled or not allowed 2026-01-25 01:33:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/cable_management/desk_grommets landed on page that is not a product page. 2026-01-25 01:33:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/monitor_risers_stands>: HTTP status code is not handled or not allowed 2026-01-25 01:33:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24021-black-oxide-drill-bit-21-piece-set-135-1-16-to-1-2-by-32nds>: HTTP status code is not handled or not allowed 2026-01-25 01:33:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/77203-end-brush-nylon-abrasive-shank-1-4-fine-grit-150-1>: HTTP status code is not handled or not allowed 2026-01-25 01:33:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/abrasives/general_purpose_sanding/cloth_sanding_bands>: HTTP status code is not handled or not allowed 2026-01-25 01:33:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbk-athletic-work-shoe-ultraknit-flexweave-grey-white-14m>: HTTP status code is not handled or not allowed 2026-01-25 01:33:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fl-slip-on-boat-shoe-smooth-leather-cognac-8-5d>: HTTP status code is not handled or not allowed 2026-01-25 01:33:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ia-6-inch-waterproof-work-boot-waterproof-leather-brown-10w>: HTTP status code is not handled or not allowed 2026-01-25 01:33:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:33:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ia-6-inch-wedge-work-boot-leather-brown-7-5w>: HTTP status code is not handled or not allowed 2026-01-25 01:33:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/coat_racks_umbrella_stands/shoe_racks_trees already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:33:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f24653-0001-broken-glass-disposal-box-floor-model-12w-x-12d-x-27h-6pk>: HTTP status code is not handled or not allowed 2026-01-25 01:33:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pilot-kit-for-bakers-pride-m1062x>: HTTP status code is not handled or not allowed 2026-01-25 01:33:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u3848-1-premium-locker-single-tier-18x24x72-3-doors-ready-to-assemble-parchment>: HTTP status code is not handled or not allowed 2026-01-25 01:33:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/general_purpose_cleaners already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:33:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/specialty_cleaners/deck_house_wash already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:33:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u1518-2a-premium-locker-double-tier-15x21x36-2-doors-assembled-parchment>: HTTP status code is not handled or not allowed 2026-01-25 01:33:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/400-series-solid-door-storage-cabinet-48wx18dx72h-gray-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 01:33:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/coat_racks_umbrella_stands/shoe_racks_trees landed on page that is not a product page. 2026-01-25 01:33:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u3282-6a-mr-black-tie-locker-six-tier-12x18x12-18-doors-assembled-black-red>: HTTP status code is not handled or not allowed 2026-01-25 01:33:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u3258-1hv-a-heavy-duty-ventilated-locker-single-tier-12x15x72-3-doors-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:33:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-25 01:34:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u3282-1mp-black-tie-locker-single-tier-12x18x72-3-doors-unassembled-black-light-grey>: HTTP status code is not handled or not allowed 2026-01-25 01:34:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/katahdin-pickeroon-with-brush-cutter-t-000-036-b660-hardwood-handle-36>: HTTP status code is not handled or not allowed 2026-01-25 01:34:04 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/specialty_cleaners/deck_house_wash landed on page that is not a product page. 2026-01-25 01:34:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-caster-creeper-powder-coated-black-frame-red-vinyl-cover>: HTTP status code is not handled or not allowed 2026-01-25 01:34:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enclosed-split-core-ac-sensor-ribxgtf-fixed-35-150a-30vac-dc-terminals>: HTTP status code is not handled or not allowed 2026-01-25 01:34:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/general_purpose_cleaners landed on page that is not a product page. 2026-01-25 01:34:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/electric_motor_parts_accessories/electric_motor_power_supplies already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:34:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stop-slow-paddle-18-inch-hip-abs-plastic-w-10-inch-fiberglass-handle>: HTTP status code is not handled or not allowed 2026-01-25 01:34:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 47790 pages (at 91 pages/min), scraped 24065 items (at 54 items/min) 2026-01-25 01:34:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/titanium-speed-tip-drill-bit-set-dw1341-14-piece-set>: HTTP status code is not handled or not allowed 2026-01-25 01:34:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbwp-10-bushel-antimicrobial-vinyl-basket-truck-all-swivel-casters-navy>: HTTP status code is not handled or not allowed 2026-01-25 01:34:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/building_construction_tape/marking_warning_tape already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:34:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/horizon-mfg-double-top-loading-plastic-bo-glove-dispenser-5104-w-4d>: HTTP status code is not handled or not allowed 2026-01-25 01:34:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/electric_motor_parts_accessories/electric_motor_power_supplies landed on page that is not a product page. 2026-01-25 01:34:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elite-watertight-case-with-pre-cubed-foam-el1105-13-1-4x11-13-16x5-13-16>: HTTP status code is not handled or not allowed 2026-01-25 01:34:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/horizon-mfg-wire-basket-for-safety-glasses-4004-13l>: HTTP status code is not handled or not allowed 2026-01-25 01:34:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/office-partition-panel-with-window-48-1-4w-x-60h-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:34:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roof-mount-damper-5acc19rd-19-x-19>: HTTP status code is not handled or not allowed 2026-01-25 01:34:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/td-1-6-gpf-elongated-bowl-toilet-17-3-16-inch-w-x-28-3-8-inch-d-x-30-1-8-inch-h-colonial-white>: HTTP status code is not handled or not allowed 2026-01-25 01:34:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ajc-leoch-djw12-7-12v-7ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-25 01:34:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/building_construction_tape/marking_warning_tape landed on page that is not a product page. 2026-01-25 01:34:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flash-chef-high-speed-oven-066-cuft-capacity-black>: HTTP status code is not handled or not allowed 2026-01-25 01:34:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ajc-amstron-ap12-35-12v-35ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-25 01:34:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-temperature-strip-door-sd11-12-10x12-10w-x-12h-12-smooth-clear-pvc>: HTTP status code is not handled or not allowed 2026-01-25 01:34:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-charleston-recycled-plastic-plank-bench-brown>: HTTP status code is not handled or not allowed 2026-01-25 01:34:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-6-step-tilt-roll-ladder-serrated-grating-tr6g>: HTTP status code is not handled or not allowed 2026-01-25 01:34:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/800078-wht-8-opaque-plastic-hook-50-pack-black>: HTTP status code is not handled or not allowed 2026-01-25 01:34:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/532fa-floor-mount-scrub-sink-with-ac-powered-sensor>: HTTP status code is not handled or not allowed 2026-01-25 01:34:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/162723-horizontal-wall-mount-acrylic-sign-holder-7-x-5-acrylic>: HTTP status code is not handled or not allowed 2026-01-25 01:34:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wern-proform-f3-climbing-harness-w-quick-connect-legs-xxl>: HTTP status code is not handled or not allowed 2026-01-25 01:34:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/252302-single-pocket-letter-size-countertop-brochure-holder-9-x-11-10-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:34:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700818-8-metal-scan-hook-galvanized-metal-finish-50-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:34:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mail-boss-epoch-wall-mount-locking-drop-box-7413-11-14w-x-4-34d-x-16-14h-galaxy>: HTTP status code is not handled or not allowed 2026-01-25 01:34:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-147-medium-white-plastic-kitchen-floor-drain-strainer>: HTTP status code is not handled or not allowed 2026-01-25 01:34:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/252012-50pk-countertop-tri-fold-brochure-holder-50-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:34:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-502l-royal-series-8-center-deck-mount-faucet-8-1-2-gooseneck-spout-low-lead>: HTTP status code is not handled or not allowed 2026-01-25 01:34:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vs-fc6s-flocculator-100-240v-50-60hz>: HTTP status code is not handled or not allowed 2026-01-25 01:35:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lyn-slope-top-hood-for-locker-18-inch-w-x-18-inch-d-gray-pack-of-1>: HTTP status code is not handled or not allowed 2026-01-25 01:35:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lyn-closed-style-front-base-for-locker-18-inch-w-x-6-inch-h-gray-pack-of-1>: HTTP status code is not handled or not allowed 2026-01-25 01:35:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/771360-pnk-pegboard-wall-panel-13-5-x-60-pink-opaque>: HTTP status code is not handled or not allowed 2026-01-25 01:35:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/igg-digital-speed-indicating-micrometer-w-usb-data-output-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:35:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 47890 pages (at 100 pages/min), scraped 24134 items (at 69 items/min) 2026-01-25 01:35:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hygrade-heavy-duty-super-cover-poly-tarp-mtgb1520-10-mil-greenblack-15l-x-20w>: HTTP status code is not handled or not allowed 2026-01-25 01:35:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/igg-12-inch-digital-electronic-height-gauge-w-absolute-technology>: HTTP status code is not handled or not allowed 2026-01-25 01:35:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20v-max-7-tool-combo-kit>: HTTP status code is not handled or not allowed 2026-01-25 01:35:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inplant-office-vinyl-cov-hdbd-16x16-3-wall-5-lgts-4-dplx-1-door-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:35:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baf-aireye-36-inch-directional-fan-w-wall-mount>: HTTP status code is not handled or not allowed 2026-01-25 01:35:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90-series-swivel-threaded-stem-caster-900685-polyurethane-4-dia-300-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 01:35:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-plastic-barrier-chain-1-1-2-inch-x-25-feet-l-army-tan>: HTTP status code is not handled or not allowed 2026-01-25 01:35:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esd-storage-container-19-3-8l-x-15-1-2w-x-8-1-4h>: HTTP status code is not handled or not allowed 2026-01-25 01:35:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wash-bottle-117320001-ldpe-1000ml-capacity-acetone-4-pk-red-cap-wide-mouth>: HTTP status code is not handled or not allowed 2026-01-25 01:35:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rust-bullet-standard-formula-rust-inhibitive-coating-gallon-can-1-rb14>: HTTP status code is not handled or not allowed 2026-01-25 01:35:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/antimicrobial-exercise-putty-black-1-pound-x-firm>: HTTP status code is not handled or not allowed 2026-01-25 01:35:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-putty-plum-x-firm-2-ounce>: HTTP status code is not handled or not allowed 2026-01-25 01:35:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microwaveable-exercise-putty-soft-orange-6-ounce>: HTTP status code is not handled or not allowed 2026-01-25 01:35:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-finial-fin03x04tr-338od-x-434h>: HTTP status code is not handled or not allowed 2026-01-25 01:35:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/balance-system-black-ball-only-level-5-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 01:35:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-wide-designer-wood-locker-22361-double-tier-3-wide-15x21x36-maple-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 01:35:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3c12x16212x-nsf-sink-3compartment-w-12l-x-16w-bowl-12-deep-2-12-drainboards>: HTTP status code is not handled or not allowed 2026-01-25 01:35:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-rigid-stem-casters-polyurethane-set-of-4-w-bumper-1200-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 01:35:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hand-exerciser-small-blue-heavy>: HTTP status code is not handled or not allowed 2026-01-25 01:35:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hand-exerciser-purple-soft>: HTTP status code is not handled or not allowed 2026-01-25 01:35:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/storage_panels/louvered_panels_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:35:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exerciser-light-red-large>: HTTP status code is not handled or not allowed 2026-01-25 01:35:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rigid-media-110132-1-8-1516w-x-2d-x-14-78h>: HTTP status code is not handled or not allowed 2026-01-25 01:35:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/physical_therapy/exercise_fitness_equipment/resistance_devices already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:35:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-mat-24-x-75-x-0-6-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:35:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-8482-72-16-outlet-aluminum-power-strip-with-6-cord>: HTTP status code is not handled or not allowed 2026-01-25 01:35:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-stp-steel-safe-roll-lad-wgt-act-lock-24-w-serr-stp-org-wa083214g-o>: HTTP status code is not handled or not allowed 2026-01-25 01:35:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-stp-steel-safe-roll-lad-wgt-act-lock-24-w-perf-stp-org-wa063214p-o>: HTTP status code is not handled or not allowed 2026-01-25 01:35:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/plumbing_parts_accessories/toilet_urinal_carriers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:35:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30003-5a-z-3-no-hub-floor-drain-body-5-round-nikaloy-strainer>: HTTP status code is not handled or not allowed 2026-01-25 01:35:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/storage_panels/louvered_panels_accessories landed on page that is not a product page. 2026-01-25 01:35:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/physical_therapy/exercise_fitness_equipment/resistance_devices landed on page that is not a product page. 2026-01-25 01:35:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nsf-aluminum-lug-cart-23-l-x-15-1-2-w-x-19-h-1-tote-capacity-all-welded>: HTTP status code is not handled or not allowed 2026-01-25 01:35:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibration-shock-absorption-mounts-conical-type-1-50-146-max-load-B1121248>: HTTP status code is not handled or not allowed 2026-01-25 01:35:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50071-3-slot-slanted-auto-pipettor-holder-5-w-x-8-h-x-7-d-white-acrylic>: HTTP status code is not handled or not allowed 2026-01-25 01:35:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/plumbing_drains_traps/drains_traps/floor_drains_cleanouts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:35:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medstoremax-single-column-shelf-w-2-adjustable-dividers>: HTTP status code is not handled or not allowed 2026-01-25 01:36:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/starbrite-to-uni-snap-adapter-aluminum-metallic-sw66620>: HTTP status code is not handled or not allowed 2026-01-25 01:36:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/plumbing_parts_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:36:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-174-contractor-black-trash-bags-42-gallon-3-0-mil-50-case>: HTTP status code is not handled or not allowed 2026-01-25 01:36:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-32-x-2-security-cap-screw-button-socket-head-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 01:36:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/liquid-bag-filter-polyester-felt-7-3-50-dia-x-32-l-50-micron-plastic-sure-seal-ring-pkg-qty-50>: HTTP status code is not handled or not allowed 2026-01-25 01:36:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/plumbing/plumbing_drains_traps/drains_traps/floor_drains_cleanouts landed on page that is not a product page. 2026-01-25 01:36:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25mm-hex-t-handle-9-length-15354>: HTTP status code is not handled or not allowed 2026-01-25 01:36:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 47983 pages (at 93 pages/min), scraped 24188 items (at 54 items/min) 2026-01-25 01:36:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-13-x-1-security-cap-screw-button-socket-head-18-8-stainless-steel-pkg-of-10>: HTTP status code is not handled or not allowed 2026-01-25 01:36:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rectangle-activity-table-30w-x-60l-x-15-to-24h-classic-maple>: HTTP status code is not handled or not allowed 2026-01-25 01:36:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-20-x-3-4-spanner-machine-screw-truss-head-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 01:36:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/office-partition-panel-with-partial-window-and-pass-thru-cable-48-1-4w-x-76h-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:36:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-16-18-x-3-4-spanner-machine-screw-flat-head-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 01:36:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-sealing-pre-slit-tubular-poly-foam-pipe-ins-38-thick-for-1-copper>: HTTP status code is not handled or not allowed 2026-01-25 01:36:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aberdeen-series-36-inch-freestanding-lateral-file-mocha>: HTTP status code is not handled or not allowed 2026-01-25 01:36:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelving-72w-x-24d-x-14h-1-shelf-starter-chrome>: HTTP status code is not handled or not allowed 2026-01-25 01:36:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-elbow-8mm-tube-x-1-4-female>: HTTP status code is not handled or not allowed 2026-01-25 01:36:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/plumbing/plumbing_parts_accessories landed on page that is not a product page. 2026-01-25 01:36:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-bspp-poly-muffler-145-psi-75-micron-5-to-212-f>: HTTP status code is not handled or not allowed 2026-01-25 01:36:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelving-42w-x-18d-x-14h-1-shelf-add-on-chrome>: HTTP status code is not handled or not allowed 2026-01-25 01:36:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nilfisk-alto-attix-30-50-complete-hose>: HTTP status code is not handled or not allowed 2026-01-25 01:36:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-way-block-valve-1-4-male-sf-1-4-female-npt>: HTTP status code is not handled or not allowed 2026-01-25 01:36:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/hvac_controls/thermostats_temperature_controls/temperature_controllers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:36:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelving-36w-x-24d-x-54h-3-shelf-add-on-chrome>: HTTP status code is not handled or not allowed 2026-01-25 01:36:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0555100-1-4-ball-valve>: HTTP status code is not handled or not allowed 2026-01-25 01:36:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelf-level-48w-x-14d-chrome>: HTTP status code is not handled or not allowed 2026-01-25 01:36:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-male-elbow-metal-release-collet-10mm-tube-x-3-8-swift-fit>: HTTP status code is not handled or not allowed 2026-01-25 01:36:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelf-level-60w-x-14d-chrome>: HTTP status code is not handled or not allowed 2026-01-25 01:36:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nilfisk-eliminator-gwd-replacement-hepa-filter>: HTTP status code is not handled or not allowed 2026-01-25 01:36:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nilfisk-eliminator-series-straight-aluminum-wand>: HTTP status code is not handled or not allowed 2026-01-25 01:36:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelving-60w-x-14d-x-54h-3-shelf-starter-chrome>: HTTP status code is not handled or not allowed 2026-01-25 01:36:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4n10p46an-set-screw-w-full-ball-m4-x-07-thread-10mm-thread-length>: HTTP status code is not handled or not allowed 2026-01-25 01:36:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultradrain-seal-wall-mount-large>: HTTP status code is not handled or not allowed 2026-01-25 01:36:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/hvac_controls/thermostats_temperature_controls/temperature_controllers landed on page that is not a product page. 2026-01-25 01:36:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultradrain-circular-seal-plus-12inches-diameter>: HTTP status code is not handled or not allowed 2026-01-25 01:36:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelf-level-60w-x-24d-chrome>: HTTP status code is not handled or not allowed 2026-01-25 01:36:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/men-dry-protection-anti-perspirant-cool-rush-1-2-oz-36-tubes-case>: HTTP status code is not handled or not allowed 2026-01-25 01:36:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-paper-wiper-4-ply-12-4-5-x-16-1-2-blue-180-wipes-box>: HTTP status code is not handled or not allowed 2026-01-25 01:36:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o-ring-for-15ml-tubes-for-digital-bottle-tube-roller-88881003-20-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:36:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:36:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/half-size-test-tube-rack-clamp-for-17-20mm-tubes-4-x-5-array-white>: HTTP status code is not handled or not allowed 2026-01-25 01:36:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-800-series-wardrobe-cabinet-36w-x-24d-x-78h-midnight-ebony-single-tier-knockdown>: HTTP status code is not handled or not allowed 2026-01-25 01:36:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-value-max-locker-36-w-x-18-d-x-78-h-725-hallowell-gray-double-tier-3-wide-assembled>: HTTP status code is not handled or not allowed 2026-01-25 01:36:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-48-w-x-48-d-x-96-h-starter-3-shelf-levels-wood-deck-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:36:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50ml-erlenmeyer-flask-clamp-236011-for-use-with-maxq-8000-shaker>: HTTP status code is not handled or not allowed 2026-01-25 01:36:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-value-max-locker-12-w-x-12-d-x-78-h-729-parchment-double-tier-1-wide-knock-down>: HTTP status code is not handled or not allowed 2026-01-25 01:36:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-48-w-x-24-d-x-96-h-starter-3-shelf-levels-wood-deck-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:36:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/touch-screen-dry-bath-block-heater-4-block-capacity-100-120v-50-60hz>: HTTP status code is not handled or not allowed 2026-01-25 01:36:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-48-w-x-48-d-x-96-h-add-on-3-shelf-levels-wood-deck-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:36:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:36:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-coil-thread-repair-kit-for-metal-m18-20-x-15d>: HTTP status code is not handled or not allowed 2026-01-25 01:36:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/staplers_staples/staplers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:36:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clipboard-21565>: HTTP status code is not handled or not allowed 2026-01-25 01:36:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:36:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/legacy-reception-desk-shell-ash-grey>: HTTP status code is not handled or not allowed 2026-01-25 01:36:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-48-w-x-48-d-x-120-h-starter-3-shelf-levels-wood-deck-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:36:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks landed on page that is not a product page. 2026-01-25 01:36:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:36:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-series-wheel-500005929-ws-5-x-2-polyurethane-on-polyolefin-1-2-straight-roller-bearing>: HTTP status code is not handled or not allowed 2026-01-25 01:37:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks landed on page that is not a product page. 2026-01-25 01:37:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8015339-fan-assembly-115v-60hz-1550rpm-for-star-manufacturing>: HTTP status code is not handled or not allowed 2026-01-25 01:37:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/staplers_staples/staplers landed on page that is not a product page. 2026-01-25 01:37:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks>: HTTP status code is not handled or not allowed 2026-01-25 01:37:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-48-w-x-48-d-x-96-h-starter-3-shelf-levels-wire-deck-medium-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:37:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/locator-jr-cryogenic-rack-and-box-system-60-liters>: HTTP status code is not handled or not allowed 2026-01-25 01:37:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/scales/compact_digital_scales already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:37:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks>: HTTP status code is not handled or not allowed 2026-01-25 01:37:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/primo-digital-scalewhite>: HTTP status code is not handled or not allowed 2026-01-25 01:37:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 48075 pages (at 92 pages/min), scraped 24226 items (at 38 items/min) 2026-01-25 01:37:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crayola-528019-classpack-regular-crayons-assorted-13-caddies-832box>: HTTP status code is not handled or not allowed 2026-01-25 01:37:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-shelf-level-for-bulk-storage-rack-48-w-x-48-d-wire-deck-medium-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:37:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/exit_devices already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:37:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trilogy-pc-software-kit-usb-trilogy-locks>: HTTP status code is not handled or not allowed 2026-01-25 01:37:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kimberly-clark-83630-wypall-cloths-wmicrobanmicrofiber-15-34-x-15-34green6pack>: HTTP status code is not handled or not allowed 2026-01-25 01:37:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:37:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/scales/compact_digital_scales landed on page that is not a product page. 2026-01-25 01:37:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scraper-outdoor-mat-72l-x-48w-black-gjo70467>: HTTP status code is not handled or not allowed 2026-01-25 01:37:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-shelf-level-for-bulk-storage-rack-48-w-x-48-d-steel-deck-medium-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:37:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alera-aleva312414my-valencia-series-center-drawer-24-12w-x-15d-x-2h-mahogany>: HTTP status code is not handled or not allowed 2026-01-25 01:37:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-72-w-x-36-d-x-72-h-add-on-3-shelf-levels-wire-deck-medium-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:37:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/door_window_hardware/exit_devices landed on page that is not a product page. 2026-01-25 01:37:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-48-w-x-24-d-x-120-h-starter-3-shelf-levels-wood-deck-medium-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:37:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bridges_returns already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:37:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010442-grommet-temp-sensor-for-bunn-o-matic>: HTTP status code is not handled or not allowed 2026-01-25 01:37:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks landed on page that is not a product page. 2026-01-25 01:37:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1301035-knob-utility-1-4-20thd-1-inch-rd>: HTTP status code is not handled or not allowed 2026-01-25 01:37:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/electronic_queuing_systems already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:37:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premier-pharmacy-vaccine-undercounter-refrigerator-freestanding-solid-door-2-5-cu-ft-ph-abt-hc-ucfs-0204>: HTTP status code is not handled or not allowed 2026-01-25 01:37:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:37:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-2-inch-x-50-foot-pvc-lay-flat-discharge-hose-coupled-with-cxe-aluminum-cam-and-groove-fittings>: HTTP status code is not handled or not allowed 2026-01-25 01:37:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-72-w-x-24-d-x-72-h-add-on-3-shelf-levels-wood-deck-medium-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:37:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bridges_returns landed on page that is not a product page. 2026-01-25 01:37:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chateau-series-mahogany-laminate-desking-414x236x30-bridge>: HTTP status code is not handled or not allowed 2026-01-25 01:37:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:37:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/electronic_queuing_systems>: HTTP status code is not handled or not allowed 2026-01-25 01:37:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-quiet-ceiling-exhaust-fan-110-cfm-0-3-sones-120v>: HTTP status code is not handled or not allowed 2026-01-25 01:37:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks landed on page that is not a product page. 2026-01-25 01:37:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-ton-engine-stand-hw93776>: HTTP status code is not handled or not allowed 2026-01-25 01:37:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/100-ton-shop-press-with-electric-motor-pump-hw93610>: HTTP status code is not handled or not allowed 2026-01-25 01:37:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/55-ton-shop-press-accessories-kit-hw93409>: HTTP status code is not handled or not allowed 2026-01-25 01:37:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-40-14w-mobile-cabinet-6-drawers-84-compart-bright-blue-keyed-alike>: HTTP status code is not handled or not allowed 2026-01-25 01:37:44 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks landed on page that is not a product page. 2026-01-25 01:37:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1711196-brush-replacement-coarse-for-prince-castle>: HTTP status code is not handled or not allowed 2026-01-25 01:37:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2631037-fuse-15-amp-for-hobart>: HTTP status code is not handled or not allowed 2026-01-25 01:37:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/memphis-96731s-memphis-flex-seamless-nylon-knit-gloves-small-bluegray-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 01:37:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-ae21z-poly-z-brite-safety-push-handle-21w>: HTTP status code is not handled or not allowed 2026-01-25 01:37:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tubing-exerciser-yellow-22>: HTTP status code is not handled or not allowed 2026-01-25 01:37:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-tubing-with-handles-black-48>: HTTP status code is not handled or not allowed 2026-01-25 01:37:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unisex-navy-cotton-polyester-fashion-blend-reversible-set-in-sleeve-scrub-shirt-size-xl>: HTTP status code is not handled or not allowed 2026-01-25 01:37:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/wiring_devices/wire_connectors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:37:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fm-pro-series-steel-industrial-mobile-sanitzation-cart-41w-x-20d-x-43h-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:37:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/starter-ground-magnalugs-6404-050e-5-16-50-pcs>: HTTP status code is not handled or not allowed 2026-01-25 01:38:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/cable_management/corrugated_tubing_fittings already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:38:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/cable_management/cable_ties_wraps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:38:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ironwood-2-compartment-wadrobe-storage-open-storage-locker-cactus-star-color-finish>: HTTP status code is not handled or not allowed 2026-01-25 01:38:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-free-pre-cut-exercise-band-yellow-48-l-strips-1-pk>: HTTP status code is not handled or not allowed 2026-01-25 01:38:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/381448-fuse-for-cleveland-range>: HTTP status code is not handled or not allowed 2026-01-25 01:38:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/wiring_devices/wire_connectors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:38:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/wiring_devices/wire_connectors>: HTTP status code is not handled or not allowed 2026-01-25 01:38:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-band-yellow-6-yard-roll>: HTTP status code is not handled or not allowed 2026-01-25 01:38:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 48162 pages (at 87 pages/min), scraped 24273 items (at 47 items/min) 2026-01-25 01:38:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leviton-13357-660w-600v-medium-base-bi-pin-linear-fluorescent-lampholder>: HTTP status code is not handled or not allowed 2026-01-25 01:38:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/cable_management/corrugated_tubing_fittings landed on page that is not a product page. 2026-01-25 01:38:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/cable_management/cable_ties_wraps>: HTTP status code is not handled or not allowed 2026-01-25 01:38:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-3284-3-inch-swivel-threaded-stem-caster-with-brake-1-2-inch-13-x-1-1-2-inch-stem>: HTTP status code is not handled or not allowed 2026-01-25 01:38:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/wiring_devices/wire_connectors>: HTTP status code is not handled or not allowed 2026-01-25 01:38:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/716066-24-single-blade-ultra-hygiene-squeegee-brown>: HTTP status code is not handled or not allowed 2026-01-25 01:38:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/38-1105-1-4-inch-push-turn-bayonet-cap-fuse-holder-20-amp-32v>: HTTP status code is not handled or not allowed 2026-01-25 01:38:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010452-burner-assembly-rear-for-jade-range>: HTTP status code is not handled or not allowed 2026-01-25 01:38:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/561547-wall-mount-faucet-economy-w-16-inch-spout>: HTTP status code is not handled or not allowed 2026-01-25 01:38:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/85-1067-blue-female-vinyl-insulated-quick-disconnect-1-4-inch-tab-wire-gauge-16-14-10pk>: HTTP status code is not handled or not allowed 2026-01-25 01:38:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rigpro174-purofort-full-safety-boots-steel-toe-12h-size-14-tan>: HTTP status code is not handled or not allowed 2026-01-25 01:38:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/barnstead-oxygen-and-demineralization-cartridge-d50216-half-size-1-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:38:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-cylinder-bracket-4>: HTTP status code is not handled or not allowed 2026-01-25 01:38:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/lockers/clear_view_lockers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:38:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-classicseries-recessed-sanitary-disposal>: HTTP status code is not handled or not allowed 2026-01-25 01:38:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/27u-server-cabinet-vertical-rail-kit-10-32-tapped>: HTTP status code is not handled or not allowed 2026-01-25 01:38:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-folding-bulk-shipping-container-32x30x34-1800lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 01:38:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/77545-24-foam-blade-squeegee-white>: HTTP status code is not handled or not allowed 2026-01-25 01:38:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-1-1-2-dia-straight-satin-grab-bar-42-w>: HTTP status code is not handled or not allowed 2026-01-25 01:38:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/lockers/clear_view_lockers landed on page that is not a product page. 2026-01-25 01:38:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/bollards/safety_bollards_posts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:38:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-frameless-mirror-23-1-2-w-x-35-1-2-h>: HTTP status code is not handled or not allowed 2026-01-25 01:38:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/hand_tools_1/clamps/toggle_clamps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:38:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-16-18-x-1-2-machine-screw-pan-head-phillips-steel-zinc-ft-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 01:38:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/bollards/safety_bollards_posts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:38:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/hand_tools_1/clamps/toggle_clamps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:38:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-16-18-x-5-8-machine-screw-pan-head-phillips-steel-zinc-ft-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 01:38:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/85-1055-8-stainless-steel-hose-clamp-1-2-inch-to-7-8-inch>: HTTP status code is not handled or not allowed 2026-01-25 01:38:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/52-bollard-flex-with-sign-post-natural-ground-white-cover-blue-tapes>: HTTP status code is not handled or not allowed 2026-01-25 01:38:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/bollards/safety_bollards_posts>: HTTP status code is not handled or not allowed 2026-01-25 01:38:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/hand_tools_1/clamps/toggle_clamps>: HTTP status code is not handled or not allowed 2026-01-25 01:38:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/overhead_clearance_bars_door_guards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:38:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/52-bollard-flex-with-sign-post-natural-ground-white-cover-black-tapes>: HTTP status code is not handled or not allowed 2026-01-25 01:38:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/bollards/safety_bollards_posts>: HTTP status code is not handled or not allowed 2026-01-25 01:38:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/hand_tools_1/clamps/toggle_clamps>: HTTP status code is not handled or not allowed 2026-01-25 01:38:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-x-18-x-72-phenolic-locker-single-tier-dove-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:38:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-shelf-liner-60w-x-24d-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:38:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-x-12-x-60-phenolic-locker-single-tier-graphite-grafix>: HTTP status code is not handled or not allowed 2026-01-25 01:38:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5380-90-9-216-3-5-pipe-brush-medium-black>: HTTP status code is not handled or not allowed 2026-01-25 01:38:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/overhead_clearance_bars_door_guards>: HTTP status code is not handled or not allowed 2026-01-25 01:39:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/bins_totes_containers/storage_totes_tubs/storage_totes_with_lids already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:39:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-polyester-filter-12-w-x-12-h-x-1-d>: HTTP status code is not handled or not allowed 2026-01-25 01:39:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-wire-shelf-truck-36x24x80-1200-pound-capacity-1>: HTTP status code is not handled or not allowed 2026-01-25 01:39:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lexmark-50f1h00-high-yield-toner-5000-page-yield-black>: HTTP status code is not handled or not allowed 2026-01-25 01:39:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6961md3-3-metal-detectable-scraper-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:39:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 48250 pages (at 88 pages/min), scraped 24323 items (at 50 items/min) 2026-01-25 01:39:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/bins_totes_containers/storage_totes_tubs/storage_totes_with_lids>: HTTP status code is not handled or not allowed 2026-01-25 01:39:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/ventilated_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:39:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8619-containment-berms-repair-kit-11-by-36-pvc>: HTTP status code is not handled or not allowed 2026-01-25 01:39:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dinex-dinet-entree-plate-9-12-cs-ivory-dx9cp02>: HTTP status code is not handled or not allowed 2026-01-25 01:39:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/full_height_storage_cabinets/stationary_storage_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:39:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dinex-side-dish-one-compartment-6-oz-2000-cs-white-dxtt1>: HTTP status code is not handled or not allowed 2026-01-25 01:39:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/-replacement-fill-door-for-apex153-500--700-portable-evaporative-coolers>: HTTP status code is not handled or not allowed 2026-01-25 01:39:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metal-access-door-8-x-8>: HTTP status code is not handled or not allowed 2026-01-25 01:39:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stronghold-flammable-drum-storage-cabinet-32-x-30-x-72>: HTTP status code is not handled or not allowed 2026-01-25 01:39:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/breakfast-blend-dark-coffee-24-pods>: HTTP status code is not handled or not allowed 2026-01-25 01:39:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/cabinets/ventilated_cabinets landed on page that is not a product page. 2026-01-25 01:39:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-rack-108x42x96-starter-unit-blue-orange-5480-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-25 01:39:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96-x-36-maple-square-edge-work-bench-adjustable-height-1-3-4-top>: HTTP status code is not handled or not allowed 2026-01-25 01:39:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:39:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/cabinets/full_height_storage_cabinets/stationary_storage_cabinets landed on page that is not a product page. 2026-01-25 01:39:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-sign-please-wait-here-the-next-available-cashier-11-x14-w-adapter-2>: HTTP status code is not handled or not allowed 2026-01-25 01:39:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-x-30-shop-top-work-bench-fixed-height-1-3-4-top>: HTTP status code is not handled or not allowed 2026-01-25 01:39:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-rack-108x36x120-add-on-unit-green-orange-5480-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-25 01:39:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/profrost-ci-55-gallons>: HTTP status code is not handled or not allowed 2026-01-25 01:39:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/small-cube-ice-machine-313lb>: HTTP status code is not handled or not allowed 2026-01-25 01:39:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/appliances/ice_machines_equipment/ice_machines/undercounter_ice_machines already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:39:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wireway-husky-tear-drop-pallet-rack-add-on-120x42x96>: HTTP status code is not handled or not allowed 2026-01-25 01:39:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leak-seek-hvac-blue-1-gallon>: HTTP status code is not handled or not allowed 2026-01-25 01:39:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-glad-hand-lock-keyed-different>: HTTP status code is not handled or not allowed 2026-01-25 01:39:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hand-held-weighted-ball-black-3-kg-6-6-lb>: HTTP status code is not handled or not allowed 2026-01-25 01:39:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits landed on page that is not a product page. 2026-01-25 01:39:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-handle-chest-weight-pulley-system-with-dual-weight-stack-10-x-2-2-lb-weights>: HTTP status code is not handled or not allowed 2026-01-25 01:39:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/azp-waterproof-heavy-duty-glass-tube-portable-heater-cover-hvd-gttcv-s-20-diameter-38h-silver>: HTTP status code is not handled or not allowed 2026-01-25 01:39:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/appliances/ice_machines_equipment/ice_machines/undercounter_ice_machines>: HTTP status code is not handled or not allowed 2026-01-25 01:39:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-wrist-and-ankle-weight-7-piece-set>: HTTP status code is not handled or not allowed 2026-01-25 01:39:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cdc03008-socket-cap-screw>: HTTP status code is not handled or not allowed 2026-01-25 01:39:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2089-bb-60-w-x-30-d-x-49-h-jobsite-storage-chest>: HTTP status code is not handled or not allowed 2026-01-25 01:39:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/cookware_bakeware/pots_pans/saute_pots_pans already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:39:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-wrist-and-ankle-weight-1-lb-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:39:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/74-h-nexel-chrome-wire-shelving-72-w-x-14-d>: HTTP status code is not handled or not allowed 2026-01-25 01:39:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a214-benchtop-ph-ise-meter-kit-stara2145>: HTTP status code is not handled or not allowed 2026-01-25 01:39:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chc05010-machine-screw>: HTTP status code is not handled or not allowed 2026-01-25 01:39:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-inch-1-piece-utility-tong-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:39:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/cookware_bakeware/pots_pans/saute_pots_pans landed on page that is not a product page. 2026-01-25 01:39:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8013963-screw-mach-10-24-x-1-2-inch-stainless-steel-truss-for-blodgett-oven>: HTTP status code is not handled or not allowed 2026-01-25 01:40:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cnc06045-machine-screw>: HTTP status code is not handled or not allowed 2026-01-25 01:40:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ripstop-insulated-safety-bib-overall-m-fluorescent-green>: HTTP status code is not handled or not allowed 2026-01-25 01:40:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/original-wrist-and-ankle-weight-15-lb-tan>: HTTP status code is not handled or not allowed 2026-01-25 01:40:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insulated-safety-coverall-2xl-black>: HTTP status code is not handled or not allowed 2026-01-25 01:40:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-in-pedestal-fan-8723-cfm-explosion-proof-motor>: HTTP status code is not handled or not allowed 2026-01-25 01:40:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 48344 pages (at 94 pages/min), scraped 24377 items (at 54 items/min) 2026-01-25 01:40:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/34-x-6-14-stud-wedge-anchor-steel-zinc-pkg-of-20>: HTTP status code is not handled or not allowed 2026-01-25 01:40:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/anchors_hangers/wedge_anchors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:40:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-w-machinery-wire-fence-partition-panel>: HTTP status code is not handled or not allowed 2026-01-25 01:40:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/anchors_hangers/wedge_anchors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:40:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/servewell-economy-breath-guard-4>: HTTP status code is not handled or not allowed 2026-01-25 01:40:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/lockers/locker_accessories/locker_locks_keys already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:40:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insulated-flex-safety-pants-s-black>: HTTP status code is not handled or not allowed 2026-01-25 01:40:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insulated-flex-safety-pants-2xl-fluorescent-green>: HTTP status code is not handled or not allowed 2026-01-25 01:40:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/fasteners/anchors_hangers/wedge_anchors>: HTTP status code is not handled or not allowed 2026-01-25 01:40:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/54-h-nexel-chrome-wire-shelving-add-on-72-w-x-14-d>: HTTP status code is not handled or not allowed 2026-01-25 01:40:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/file_folders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:40:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/anchors_hangers/wedge_anchors landed on page that is not a product page. 2026-01-25 01:40:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rivet-nut-1-4-20-x-200-grip-flat-head-aluminum-usa-pkg-of-50>: HTTP status code is not handled or not allowed 2026-01-25 01:40:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lathes-jet321579>: HTTP status code is not handled or not allowed 2026-01-25 01:40:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ball-handle-zinc-plated-cast-iron-3-4-10-thread-6-inch-wide>: HTTP status code is not handled or not allowed 2026-01-25 01:40:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/small-kickboard-yellow-14-011-yel>: HTTP status code is not handled or not allowed 2026-01-25 01:40:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/lockers/locker_accessories/locker_locks_keys landed on page that is not a product page. 2026-01-25 01:40:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hubbel-industrial-bare-end-14-3-35>: HTTP status code is not handled or not allowed 2026-01-25 01:40:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transparent-vinyl-welding-curtain--steel-frame-combo-kit-6w-x-6h-gold>: HTTP status code is not handled or not allowed 2026-01-25 01:40:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/file_folders landed on page that is not a product page. 2026-01-25 01:40:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nfpa-rod-alignment-coupler-1-1-4>: HTTP status code is not handled or not allowed 2026-01-25 01:40:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/barr8-br-bk-crowdmaster-1000-barricade-100-by-43-bridge-feet-black>: HTTP status code is not handled or not allowed 2026-01-25 01:40:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/relaxed-fit-flex-twill-cargo-pants-34l-inseam-x-42-waist-navy>: HTTP status code is not handled or not allowed 2026-01-25 01:40:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fanny-pack-with-screenprint-guard-red-no-logo-10-103-red-nl>: HTTP status code is not handled or not allowed 2026-01-25 01:40:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-1730-threaded-insert-10-24-50-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:40:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/file_folders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:40:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-80mm-bore-x-70mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 01:40:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aquarad-rain-coat-hood-only-25-mil-tpu-200d-nylon-yellow-rh33-nsyy-univ>: HTTP status code is not handled or not allowed 2026-01-25 01:40:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aquarad-rb33-nsyy-xl-25-mil-tpu-200d-nylon-rain-bibs-yellow-xl>: HTTP status code is not handled or not allowed 2026-01-25 01:40:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8ft-long-shirred-pleat-skirting-8h-stage-green>: HTTP status code is not handled or not allowed 2026-01-25 01:40:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cold-pain-relief-gel-3-oz-roll-on-bottle-box-of-12>: HTTP status code is not handled or not allowed 2026-01-25 01:40:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/file_folders landed on page that is not a product page. 2026-01-25 01:40:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chicago-pneumatic-heavy-duty-air-hammer-cp7111-6-7l-3000-bpm-7-2-joule-B1222331>: HTTP status code is not handled or not allowed 2026-01-25 01:40:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012085-hose-kit-for-blodgett-oven>: HTTP status code is not handled or not allowed 2026-01-25 01:40:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-poly-oxford-5-in-1-safety-jacket-2xlt-black>: HTTP status code is not handled or not allowed 2026-01-25 01:40:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mf1-mf2-fl-mount-kit-3-1-4-bore-nfpa-cylinder>: HTTP status code is not handled or not allowed 2026-01-25 01:41:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rg59-u-18-2-siamese-camera-cable-cmr-500-ft-spool-black>: HTTP status code is not handled or not allowed 2026-01-25 01:41:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/262628-conveyor-roller>: HTTP status code is not handled or not allowed 2026-01-25 01:41:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/type-r-class-3-heavy-duty-engineer-vest-green-m-sv55-3zgd-m>: HTTP status code is not handled or not allowed 2026-01-25 01:41:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-wire-shelf-truck-36x24x69-1200-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 01:41:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kit-male-clevis-bracket-al-mount-125-for-iso-15552-cylinders>: HTTP status code is not handled or not allowed 2026-01-25 01:41:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/needle-valve-89978-06-04-knob-adj-38-tube-x-14-swift-fit-metallic-collet>: HTTP status code is not handled or not allowed 2026-01-25 01:41:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 48435 pages (at 91 pages/min), scraped 24431 items (at 54 items/min) 2026-01-25 01:41:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paraffin-bath-refill-6-lb-beads-in-bag-rose-blossom-fragrance>: HTTP status code is not handled or not allowed 2026-01-25 01:41:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2971015-dial-control-120-570f-for-anvil-america-fryers-warmers-etc>: HTTP status code is not handled or not allowed 2026-01-25 01:41:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galvanized-garbage-can-lid-5>: HTTP status code is not handled or not allowed 2026-01-25 01:41:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-quilted-safety-freezer-jacket-xlt-fluorescent-green>: HTTP status code is not handled or not allowed 2026-01-25 01:41:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60x30x64-75-cabinet-leg-workstation-w-4-drawers-stat-riser-shelf-2-ovhd-cabs-butcher-block-top>: HTTP status code is not handled or not allowed 2026-01-25 01:41:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inline-flow-control-82815-04-14-tube-nickel-plated-brass>: HTTP status code is not handled or not allowed 2026-01-25 01:41:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1591181-goodyear-ax35-belt>: HTTP status code is not handled or not allowed 2026-01-25 01:41:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/moist-heat-pack-contour-wrap-wrist-5-5-x-12-with-3-x-27-elastic-belt>: HTTP status code is not handled or not allowed 2026-01-25 01:41:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/needle-valve-85978-53-02-knob-adj-532-tube-x-18-swift-fit-composite-body>: HTTP status code is not handled or not allowed 2026-01-25 01:41:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/female-coupling-82300n-04-14-nptf-nickel-plated-brass>: HTTP status code is not handled or not allowed 2026-01-25 01:41:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/moto-vest-4-pockets-cottonpolyester-2xl-dark-brown>: HTTP status code is not handled or not allowed 2026-01-25 01:41:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-expandable-starter-rack-72x36x84-gray-with-3-level-wood-deck-750lb-cap-per-level>: HTTP status code is not handled or not allowed 2026-01-25 01:41:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pain-relief-spray-4-oz-bottle>: HTTP status code is not handled or not allowed 2026-01-25 01:41:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48x24x63-galvanized-steel-solid-shelving-add-on>: HTTP status code is not handled or not allowed 2026-01-25 01:41:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/moist-heat-pack-spine-large-10-x-24>: HTTP status code is not handled or not allowed 2026-01-25 01:41:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-control-55905-8-38-knob-adj-8mm-tube-x-38-swift-fit-composite-body>: HTTP status code is not handled or not allowed 2026-01-25 01:41:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-pigeon-hole-organizer-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:41:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-50mm-bore-x-250mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 01:41:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insulated-cold-gear-jacket-xl-black>: HTTP status code is not handled or not allowed 2026-01-25 01:41:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exl-unitized-wheel-tr-2-inch-x-nh-fin-grit-silicon-carbide>: HTTP status code is not handled or not allowed 2026-01-25 01:41:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/union-60040-05-516-tube-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 01:41:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-keyed-padlock-shackle-keyed-alike-1>: HTTP status code is not handled or not allowed 2026-01-25 01:41:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/vests already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:41:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-guard-rail-3-rib-4-ft-l-yellow>: HTTP status code is not handled or not allowed 2026-01-25 01:41:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stretch-wrap-machine-112-in-wrap-height-10-free-rolls-of-stretch-wrap>: HTTP status code is not handled or not allowed 2026-01-25 01:41:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14mm-x-25mm-metric-keystock-gold-dichromate-finish-12-inch-length-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-25 01:41:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5mm-x-8mm-metric-keystock-plain-finish-1-meter-length>: HTTP status code is not handled or not allowed 2026-01-25 01:41:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-x-2-mold-on-rubber-caster-kit-2-swivel-2-rigid>: HTTP status code is not handled or not allowed 2026-01-25 01:41:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sl-surface-conditioning-disc-7-inch-x-nh-med-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-25 01:41:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/vests landed on page that is not a product page. 2026-01-25 01:41:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1201061-wheel-4-inch-5-16-inch-id-blk>: HTTP status code is not handled or not allowed 2026-01-25 01:42:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulkhead-union-89050-02-18-tube-metallic-collet>: HTTP status code is not handled or not allowed 2026-01-25 01:42:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ps30-rectangular-boom-for-maxima-ultra-36--giotto-ultra-36-barrier-gate-openers-20l>: HTTP status code is not handled or not allowed 2026-01-25 01:42:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/405-nrs-rw-cast-iron-gate-valve-4-in-flg-0700105>: HTTP status code is not handled or not allowed 2026-01-25 01:42:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thru-wall-package-drop-vault-plus-dvwm0062sa-w-bottom-hold-rear-access-black>: HTTP status code is not handled or not allowed 2026-01-25 01:42:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011354-gasket-for-turbo-air-refrigeration>: HTTP status code is not handled or not allowed 2026-01-25 01:42:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 48534 pages (at 99 pages/min), scraped 24495 items (at 64 items/min) 2026-01-25 01:42:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1421280-kit-allen-head-set-200-piece>: HTTP status code is not handled or not allowed 2026-01-25 01:42:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0004-inch-stainless-steel-shim-stock-6-inch-x-60-inch-roll>: HTTP status code is not handled or not allowed 2026-01-25 01:42:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30008-500-plastic-barrier-chain-hdpe-1-5-x500-6-38mm-silver>: HTTP status code is not handled or not allowed 2026-01-25 01:42:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-tefc-1ph-3500rpm-p184k34fk2dd>: HTTP status code is not handled or not allowed 2026-01-25 01:42:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tote-box-dividable-4>: HTTP status code is not handled or not allowed 2026-01-25 01:42:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m3-x-0-5-x-10mm-machine-screw-phillips-pan-head-304-stainless-steel-din-7985-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 01:42:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b10hspx-316-stainless-worm-gear-hose-clamp-916-inch-1-116-inch-clamping-diam-10-pcs>: HTTP status code is not handled or not allowed 2026-01-25 01:42:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m4-x-0-7-x-10mm-flat-head-socket-cap-screw-304-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 01:42:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-tecgen-flame-resistant-work-shirt-2xl-tall-tan>: HTTP status code is not handled or not allowed 2026-01-25 01:42:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/union-89040-06-38-tube-metallic-collet>: HTTP status code is not handled or not allowed 2026-01-25 01:42:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50723-10-master-link-acetal-copolymer-2-purple-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 01:42:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-shelving-48x18x72-1>: HTTP status code is not handled or not allowed 2026-01-25 01:42:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30713-10-master-link-acetal-copolymer-1-5-traffic-orange-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 01:42:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-arm-for-orbit-with-vesa-mount-black>: HTTP status code is not handled or not allowed 2026-01-25 01:42:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-cu-yd-yellow-self-dumping-forkli-hopper-with-heavy-gauge-base>: HTTP status code is not handled or not allowed 2026-01-25 01:42:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010229-heating-element-120v-1kw>: HTTP status code is not handled or not allowed 2026-01-25 01:42:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-poly-scrub-brush-20-electric-floor-scrubber>: HTTP status code is not handled or not allowed 2026-01-25 01:42:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dixie-pathways-paper-hot-cups-10-oz-1000carton>: HTTP status code is not handled or not allowed 2026-01-25 01:42:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-partition-flexible-nine-panel-10>: HTTP status code is not handled or not allowed 2026-01-25 01:42:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trimline-kickspace-heater-with-built-in-thermostat-112-1500750w-240120v-white>: HTTP status code is not handled or not allowed 2026-01-25 01:42:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/38-inch-id-x-58-inch-od-x-0003-inch-steel-arbor-shim-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 01:42:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/room_dividers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:42:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s8950-28w-torpedo-led-candelabra-base-3000k-1card>: HTTP status code is not handled or not allowed 2026-01-25 01:42:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/room_dividers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:42:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gray-plastic-box-truck-8-bushel-medium-duty>: HTTP status code is not handled or not allowed 2026-01-25 01:42:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microfiber-cleaning-cloths-12-x-12-green-50-ragsbox>: HTTP status code is not handled or not allowed 2026-01-25 01:42:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-flame-resistant-dh-air153-work-shirt-2xl-tall-navy>: HTTP status code is not handled or not allowed 2026-01-25 01:42:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bookcases_hutches already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:42:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/partitions_dividers/room_dividers>: HTTP status code is not handled or not allowed 2026-01-25 01:42:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/partitions_dividers/room_dividers landed on page that is not a product page. 2026-01-25 01:42:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/263596-kason-10930a0004-latch-strike-for-kason>: HTTP status code is not handled or not allowed 2026-01-25 01:43:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thread-repair-kit-for-metal-thin-wall-3-8-16-x-1-2-13-ez-310-6>: HTTP status code is not handled or not allowed 2026-01-25 01:43:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s9282-3w-led-mr11-gu4-base-4000k-12v>: HTTP status code is not handled or not allowed 2026-01-25 01:43:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-magnetic-sweeper-with-release-lever-30-w>: HTTP status code is not handled or not allowed 2026-01-25 01:43:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-hang-type-magnetic-sweeper-with-forkli-hanger-60-w>: HTTP status code is not handled or not allowed 2026-01-25 01:43:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-clip-white-100-pkg-qty>: HTTP status code is not handled or not allowed 2026-01-25 01:43:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carpenter-saws-irwin-marathon-2011202>: HTTP status code is not handled or not allowed 2026-01-25 01:43:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bookcases_hutches>: HTTP status code is not handled or not allowed 2026-01-25 01:43:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-14-inch-id-x-1-34-inch-od-steel-arbor-shim-assortment>: HTTP status code is not handled or not allowed 2026-01-25 01:43:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-10000mah-dual-port-mobile-power-bank-usb-battery-charger-with-led-flashlight>: HTTP status code is not handled or not allowed 2026-01-25 01:43:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 48628 pages (at 94 pages/min), scraped 24551 items (at 56 items/min) 2026-01-25 01:43:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010689-caster-w-brake-for-henny-penny>: HTTP status code is not handled or not allowed 2026-01-25 01:43:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-gantry-crane-adjustable-height-2000-lb-capacity-2>: HTTP status code is not handled or not allowed 2026-01-25 01:43:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bin-nestable-clear-12>: HTTP status code is not handled or not allowed 2026-01-25 01:43:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-piece-stainless-steel-shim-stock-assortment-6-inch-x-12-inch-sheets>: HTTP status code is not handled or not allowed 2026-01-25 01:43:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exo2-mig-05-xlmotor-impact-glove>: HTTP status code is not handled or not allowed 2026-01-25 01:43:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-round-rack>: HTTP status code is not handled or not allowed 2026-01-25 01:43:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/900va-standby-compact-low-profile-12-outlet-usb>: HTTP status code is not handled or not allowed 2026-01-25 01:43:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carriage-bolt-12-13-2-round-head-18-8-stainless-steel-unc-pkg-of-50-781408>: HTTP status code is not handled or not allowed 2026-01-25 01:43:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hydraulic-hand-pump-for-4-ton-body--frame-repair-kits-8000-psi>: HTTP status code is not handled or not allowed 2026-01-25 01:43:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-18-gallon-wet-dry-squeegee-vacuum>: HTTP status code is not handled or not allowed 2026-01-25 01:43:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/save-a-stud-rethread-die-12-20>: HTTP status code is not handled or not allowed 2026-01-25 01:43:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-1055-stainless-steel-round-head-10-32-x-1-1-2-inch-phillips-machine-screw-100-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:43:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/cabinet_accessories/cabinet_locks_keys already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:43:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flammable-cabinet-6-gallon-manual-close-single-door-23-w-x-18-d-x-22-h>: HTTP status code is not handled or not allowed 2026-01-25 01:43:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beige-outdoor-ashtray>: HTTP status code is not handled or not allowed 2026-01-25 01:43:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cors-outdoor-singlee-head-remote-led-fixture-120-277v>: HTTP status code is not handled or not allowed 2026-01-25 01:43:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/furniture_parts_accessories/partition_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:43:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traex-squeeze-dispensers-2808-02-single-tip-8-oz>: HTTP status code is not handled or not allowed 2026-01-25 01:43:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/furniture_parts_accessories/partition_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:43:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plate-covers-1100-13-fits-plate-size-11-plastic>: HTTP status code is not handled or not allowed 2026-01-25 01:43:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/cabinets/cabinet_accessories/cabinet_locks_keys landed on page that is not a product page. 2026-01-25 01:43:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/emergency_lighting_signage/emergency_lighting already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:43:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-portable-picnic-table-36-square-x-30h-black>: HTTP status code is not handled or not allowed 2026-01-25 01:43:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oxford-4-outdoor-bench-with-back-surface-mount-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:43:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/augusta-steel-outdoor-trash-can-with-ash-urn-lid-36-gallon-capacity-black>: HTTP status code is not handled or not allowed 2026-01-25 01:43:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/room_dividers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:43:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/push-to-connect-tube-fitting-nylon-90-degree-elbow-adapter-18-tube-od-x-14-npt-female>: HTTP status code is not handled or not allowed 2026-01-25 01:43:45 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/furniture_parts_accessories/partition_accessories landed on page that is not a product page. 2026-01-25 01:43:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1421645-yellow-20-inch-pot-brush>: HTTP status code is not handled or not allowed 2026-01-25 01:43:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/furniture_parts_accessories/partition_accessories landed on page that is not a product page. 2026-01-25 01:43:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/emergency_lighting_signage/emergency_lighting>: HTTP status code is not handled or not allowed 2026-01-25 01:43:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spiral-wound-flange-gasket-with-ptfe-filler-for-1-pipe-18t-class-150>: HTTP status code is not handled or not allowed 2026-01-25 01:43:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-flat-washer-sae-9-32-i-d-steel-zinc-grade-2-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 01:43:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-way-metal-push-switch-5-8-in-bushing-brass-finish>: HTTP status code is not handled or not allowed 2026-01-25 01:43:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/madison-6-ipe-wood-flat-bench-without-back-with-armrests-brown>: HTTP status code is not handled or not allowed 2026-01-25 01:43:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/partitions_dividers/room_dividers landed on page that is not a product page. 2026-01-25 01:43:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washdown-motor-56h-tenv-1ph-1725rpm-nz6c17vk1bb>: HTTP status code is not handled or not allowed 2026-01-25 01:44:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-square-plate-washer-9-16-i-d-1-8-thick-steel-galvanized-grade-2-pkg-of-25>: HTTP status code is not handled or not allowed 2026-01-25 01:44:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5ml-serological-pipet-individually-wrapped-in-bags-sterile-200-case>: HTTP status code is not handled or not allowed 2026-01-25 01:44:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-flat-washer-uss-13-16-i-d-122-146-thick-steel-yellow-zinc-grade-8-pkg-of-50>: HTTP status code is not handled or not allowed 2026-01-25 01:44:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/infinity-locker-double-tier-12x12x36-6-door-tan>: HTTP status code is not handled or not allowed 2026-01-25 01:44:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/infinity-locker-double-tier-12x18x36-2-door-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:44:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/miramar-french-oven-w-cover-49431-7-quart-capacity-satin-finish>: HTTP status code is not handled or not allowed 2026-01-25 01:44:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 48719 pages (at 91 pages/min), scraped 24605 items (at 54 items/min) 2026-01-25 01:44:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fowler-combination-square-protractor-center-head-12-inch-blade>: HTTP status code is not handled or not allowed 2026-01-25 01:44:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-flanged-shaft-15-hp-2-speed-230v>: HTTP status code is not handled or not allowed 2026-01-25 01:44:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/304-ss-high-pressure-clamp-with-bolt-for-quick-clamp-fittings-for-1-and-1-12-tube>: HTTP status code is not handled or not allowed 2026-01-25 01:44:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-square-table-with-2-imme-armless-barstools-sky-blue-seatdesigner-white-top>: HTTP status code is not handled or not allowed 2026-01-25 01:44:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1901180-dispenser-iced-tea-w-brw-lid-for-bunn-o-matic>: HTTP status code is not handled or not allowed 2026-01-25 01:44:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-straight-arm-only-for-k10>: HTTP status code is not handled or not allowed 2026-01-25 01:44:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8000ab-adjustable-case-sealer-with-accuglide-nph-taping-head-side-belt-drive-2w-tape>: HTTP status code is not handled or not allowed 2026-01-25 01:44:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/air_purifiers/hepa_media_air_purifiers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:44:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fiberglass-aerial-safety-cage-ladder-5-9-type-1aa-18509-243>: HTTP status code is not handled or not allowed 2026-01-25 01:44:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pack-of-25-aflas-o-ring-dash-108>: HTTP status code is not handled or not allowed 2026-01-25 01:44:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-round-table-with-4-imme-armless-chairs-white-seatdesigner-white-top>: HTTP status code is not handled or not allowed 2026-01-25 01:44:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-48x24x60-with-5-shelves-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 01:44:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waste-wtcher-single-mixed-recyclables-23-gallon-gray-blue-8105014-2>: HTTP status code is not handled or not allowed 2026-01-25 01:44:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/air_purifiers/hepa_media_air_purifiers landed on page that is not a product page. 2026-01-25 01:44:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/battery-bin-2-1-4-gallon-brown-batbin-24bo>: HTTP status code is not handled or not allowed 2026-01-25 01:44:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/front-mount-direct-surface-hooks-polished-chrome>: HTTP status code is not handled or not allowed 2026-01-25 01:44:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traex-squeeze-dispensers-4924-02-wide-mouth-24-oz>: HTTP status code is not handled or not allowed 2026-01-25 01:44:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-round-table-with-4-imme-armless-caster-chairs-white-seatriver-cherry-top>: HTTP status code is not handled or not allowed 2026-01-25 01:44:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5t50a02o-zinc-die-cast-adjustable-lever-w-steel-components-5-16-18-x-1-97-stud>: HTTP status code is not handled or not allowed 2026-01-25 01:44:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/lathes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:44:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-starter-rack-72x36x84-with-3-levels-wire-deck-1000lb-cap-per-shelf-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:44:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-square-table-with-4-imme-armless-barstools-coral-seatstudio-teak-top>: HTTP status code is not handled or not allowed 2026-01-25 01:44:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-round-table-with-4-imme-armless-caster-chairs-light-gray-seatmaple-top>: HTTP status code is not handled or not allowed 2026-01-25 01:44:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1571108-connector-gas-3-4-inch-x48-inch-for-dormont-mfg>: HTTP status code is not handled or not allowed 2026-01-25 01:44:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/machining/machinery/lathes>: HTTP status code is not handled or not allowed 2026-01-25 01:44:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hvac-odor-clean-odor-eliminator-cleaner-32-oz-1-qty>: HTTP status code is not handled or not allowed 2026-01-25 01:44:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-round-table-with-4-imme-armless-caster-chairs-white-seatstudio-teak-top>: HTTP status code is not handled or not allowed 2026-01-25 01:44:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mtm-hydro-4200-psi-38-brass-quick-coupler-plug-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:44:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/691175-jvm-836-1-milling-machine-w-newall-dp700-dro-x-y-axis-1-1-2-hp>: HTTP status code is not handled or not allowed 2026-01-25 01:44:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-ultra-parking-block-with-hardware-3-1-4-H-yellow-ULTRA3672PGN>: HTTP status code is not handled or not allowed 2026-01-25 01:44:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-handle-magliner-hand-truck>: HTTP status code is not handled or not allowed 2026-01-25 01:44:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-heat-fuel-oil-diesel-treatment-8-in-1-1-pt-1-qty>: HTTP status code is not handled or not allowed 2026-01-25 01:44:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mirage-induction-rethermalizer-74110140-11-quart-red>: HTTP status code is not handled or not allowed 2026-01-25 01:44:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jtm-4vs-mill-w-200s-dro-3axis-q-andx-tpfa>: HTTP status code is not handled or not allowed 2026-01-25 01:44:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cm-valuestar-electric-chain-hoist-with-chain-container-500-lb-capacity-3>: HTTP status code is not handled or not allowed 2026-01-25 01:44:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/321233-kason-401sa6063884-strip-curtain-for-kason>: HTTP status code is not handled or not allowed 2026-01-25 01:44:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-stacking-student-chair-black>: HTTP status code is not handled or not allowed 2026-01-25 01:45:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/orbit-power-kit-for-68-and-81-posts-black>: HTTP status code is not handled or not allowed 2026-01-25 01:45:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjust-stepmate-stand-stainless-steel-ahwh1948ss>: HTTP status code is not handled or not allowed 2026-01-25 01:45:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spring-loaded-folding-steel-step-sfs149sl>: HTTP status code is not handled or not allowed 2026-01-25 01:45:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_supplies_dispensers/feminine_hygiene_dispensers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:45:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/690098-jtm-4vs-milling-machine-w-acu-rite-200s-dro-x-y-axis-powerfeeds-3-hp>: HTTP status code is not handled or not allowed 2026-01-25 01:45:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=691201-jtm-4vs-milling-machine-w-newall-dp700-dro-x-y-axis-powerfeeds-3-hp already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:45:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=690202-jtm-4vs-milling-machine-w-acu-rite-200s-dro-3-hp already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:45:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/easy-adjust-boltless-shelf-truck-with-laminate-shelves-17>: HTTP status code is not handled or not allowed 2026-01-25 01:45:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/norton-66261138453-neon-fiber-disc-4-1-2-inch-x-7-8-inch-zirconia-alumina-60-grit>: HTTP status code is not handled or not allowed 2026-01-25 01:45:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 48814 pages (at 95 pages/min), scraped 24659 items (at 54 items/min) 2026-01-25 01:45:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galvanized-structural-guard-rail-drop-in-style-4>: HTTP status code is not handled or not allowed 2026-01-25 01:45:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-rack-dolly-w-2-swivel-2-fixed-casters-52292-20-x-20-200-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 01:45:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_supplies_dispensers/feminine_hygiene_dispensers landed on page that is not a product page. 2026-01-25 01:45:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=691201-jtm-4vs-milling-machine-w-newall-dp700-dro-x-y-axis-powerfeeds-3-hp landed on page that is not a product page. 2026-01-25 01:45:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=690202-jtm-4vs-milling-machine-w-acu-rite-200s-dro-3-hp>: HTTP status code is not handled or not allowed 2026-01-25 01:45:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/redco-lettuce-king-i-485-28-blades-1-4-cut>: HTTP status code is not handled or not allowed 2026-01-25 01:45:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/skullerz-saga-frameless-safety-glasses-matte-black-smoke-lens>: HTTP status code is not handled or not allowed 2026-01-25 01:45:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gearmotor-ac-parallel-41rpm-c4c17fz35c>: HTTP status code is not handled or not allowed 2026-01-25 01:45:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grid-wall-bin-accessory-bin-divider-sold-pack-of-6-8>: HTTP status code is not handled or not allowed 2026-01-25 01:45:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/skullerz-8992-safety-helmet-replacement-chin-strap-black>: HTTP status code is not handled or not allowed 2026-01-25 01:45:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1971102-shaft-discharge-for-duke-manufacturing>: HTTP status code is not handled or not allowed 2026-01-25 01:45:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/reel_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:45:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/skullerz-vali-frameless-safety-glasses-matte-black-anti-fog-smoke-lens>: HTTP status code is not handled or not allowed 2026-01-25 01:45:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kyorene-pro-cut-resistant-gloves-hct-micro-foam-nitrile-coated-ansi-a3-s-gray-12-pair>: HTTP status code is not handled or not allowed 2026-01-25 01:45:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/generic-yellow-wall-case-w-label-kit-1-shelf-small>: HTTP status code is not handled or not allowed 2026-01-25 01:45:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/cookware_bakeware/chafing_dishes_fuel already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:45:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1571103-connector-gas-1-inch-x36-inch-for-dormont-mfg>: HTTP status code is not handled or not allowed 2026-01-25 01:45:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/reel_racks landed on page that is not a product page. 2026-01-25 01:45:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/graphic-facility-signs-fire-extinguisher-vinyl-7x7>: HTTP status code is not handled or not allowed 2026-01-25 01:45:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bb1279-full-mortise-five-knuckle-ball-bearing-standard-weight-hinge-45-x-45-usp-nrp>: HTTP status code is not handled or not allowed 2026-01-25 01:45:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/guest_chairs/conference_training_chairs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:45:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-d-double-flange-cross-bar-interlake-pallet-rack>: HTTP status code is not handled or not allowed 2026-01-25 01:45:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ptp186-floor-stripe>: HTTP status code is not handled or not allowed 2026-01-25 01:45:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009561-castor-set-4-for-true-manufacturing>: HTTP status code is not handled or not allowed 2026-01-25 01:45:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/cookware_bakeware/chafing_dishes_fuel landed on page that is not a product page. 2026-01-25 01:45:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-syle-2-48-3-8-x-24-3-8-pvc-glue-up-tile-in-smoked-pewter-pg5127>: HTTP status code is not handled or not allowed 2026-01-25 01:45:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/locksets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:45:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-filling-can-type-2-with-flexible-nozzle-two-gallon-galvanized-steel>: HTTP status code is not handled or not allowed 2026-01-25 01:45:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/Spiral-Bowl-Brush-96301>: HTTP status code is not handled or not allowed 2026-01-25 01:45:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/framing-nails-fr131-314grs-3-1-4-x-131-plastic-collated-galv-ring-shank-2000-bx>: HTTP status code is not handled or not allowed 2026-01-25 01:45:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/revel-fidget-stool-with-adjustable-height-backless-fabric-seat-charcoal>: HTTP status code is not handled or not allowed 2026-01-25 01:45:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/guest_chairs/conference_training_chairs landed on page that is not a product page. 2026-01-25 01:45:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-syle-1-48-3-8-x-24-3-8-pvc-glue-up-tile-in-brushed-nickel-pg5029>: HTTP status code is not handled or not allowed 2026-01-25 01:45:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/fasteners/door_window_hardware/locksets>: HTTP status code is not handled or not allowed 2026-01-25 01:46:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-1-2-w-x-8-d-x-2-1-2-h-steel-assembly-bin-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:46:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kee-klamp-medium-flange-6>: HTTP status code is not handled or not allowed 2026-01-25 01:46:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eye-face-wash-wall-mounted-stainless-steel-bowl-and-cover-g1750bc>: HTTP status code is not handled or not allowed 2026-01-25 01:46:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/241217-burner-ring-12>: HTTP status code is not handled or not allowed 2026-01-25 01:46:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-x725-6-mil-poly-tubing>: HTTP status code is not handled or not allowed 2026-01-25 01:46:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 48908 pages (at 94 pages/min), scraped 24718 items (at 59 items/min) 2026-01-25 01:46:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnetic-multi-function-tool-holder-kti-72465-20l-x-6w-x-4h-black>: HTTP status code is not handled or not allowed 2026-01-25 01:46:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wood-deck-36-wx18-dx5-8-thick>: HTTP status code is not handled or not allowed 2026-01-25 01:46:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10194910-harness-back-hip-d-rings-tongue-buckle-leg-straps-extra-large>: HTTP status code is not handled or not allowed 2026-01-25 01:46:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16x6x4-corrugated-mailers>: HTTP status code is not handled or not allowed 2026-01-25 01:46:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-marker-booklet-a-z-1-90-symbols>: HTTP status code is not handled or not allowed 2026-01-25 01:46:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:46:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7x4x4-kraft-corrugated-mailers>: HTTP status code is not handled or not allowed 2026-01-25 01:46:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-extra-heavy-duty-boltless-shelving-starter-36in-w-x-12in-d-x-60in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-25 01:46:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1381248-thermometer-digital-candy-fry-for-taylor-precision-products-lp>: HTTP status code is not handled or not allowed 2026-01-25 01:46:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/881036-solenoid-coilevr-3-3-8-inch-odf>: HTTP status code is not handled or not allowed 2026-01-25 01:46:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10194864-harness-back-chest-hip-d-rings-quick-connect-leg-straps-standard>: HTTP status code is not handled or not allowed 2026-01-25 01:46:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats landed on page that is not a product page. 2026-01-25 01:46:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/plastic_laminate_bathroom_partitions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:46:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10194657-harness-back-chest-d-rings-quick-connect-leg-straps-extra-large>: HTTP status code is not handled or not allowed 2026-01-25 01:46:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15x12x5-flat-corrugated-boxes>: HTTP status code is not handled or not allowed 2026-01-25 01:46:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-24-x-54-blue-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:46:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/80-gauge-stretch-wrap-18-x-1500-hand-dispenser-use-black>: HTTP status code is not handled or not allowed 2026-01-25 01:46:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bassick-prism-stainless-steel-rigid-caster-thermal-plastic-rubber-flat-tread-4-dia>: HTTP status code is not handled or not allowed 2026-01-25 01:46:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/start-panel-to-wall-panel-to-pilaster-bracket-kit>: HTTP status code is not handled or not allowed 2026-01-25 01:46:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dynamic-2-1-2-fluid-glycerine-filled-pressure-gage-panel-flange-3000-psi>: HTTP status code is not handled or not allowed 2026-01-25 01:46:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-24-x-24-light-grey-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:46:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dynamic-hydraulic-gear-pump-0-61-cu-in-rev-5-8-dia-straight-shaft-9-51-gpm-max-3600-rpm>: HTTP status code is not handled or not allowed 2026-01-25 01:46:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/plastic_laminate_bathroom_partitions landed on page that is not a product page. 2026-01-25 01:46:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ratchet-lever-hoist-3-4-ton-capacity-10-lift>: HTTP status code is not handled or not allowed 2026-01-25 01:46:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tr1-9501fs-a-triton-carpet-wand-2540ac-hp-25-ft-vacuum-hose-solution-line-assembly-500-psi-units>: HTTP status code is not handled or not allowed 2026-01-25 01:46:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-plastic-platform-truck-60-x-30-1000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 01:46:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30x18x18-double-wall-boxes>: HTTP status code is not handled or not allowed 2026-01-25 01:46:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-add-on-96in-w-x-48in-d-x-72in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-25 01:46:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-heavy-duty-boltless-shelving-add-on-96in-w-x-48in-d-x-72in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-25 01:46:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012354-impeller-for-hoshizaki-of-america>: HTTP status code is not handled or not allowed 2026-01-25 01:46:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8013203-rinse-arm-assembly-for-hobart>: HTTP status code is not handled or not allowed 2026-01-25 01:46:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-x14-x36-3-mil-gusseted-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 01:46:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lxt-174-3-speed-impact-driver-tool-only-li-ion-18v-brushless-cordless>: HTTP status code is not handled or not allowed 2026-01-25 01:46:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/anti_fatigue_mats/wet_area_anti_fatigue_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:46:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smooth-military-switchboard-matting-black-1-8in-x-3ft-x-5ft>: HTTP status code is not handled or not allowed 2026-01-25 01:47:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-24-x-30-grey-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:47:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-heavy-duty-boltless-shelving-add-on-48in-w-x-18in-d-x-60in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-25 01:47:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-2-rubber-voltage-glove-premium-kit-black-size-9-kitgc209ag>: HTTP status code is not handled or not allowed 2026-01-25 01:47:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-hi-vis-rain-jacket-ansi-class-3-type-r-yellow-m>: HTTP status code is not handled or not allowed 2026-01-25 01:47:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/floor_mats/anti_fatigue_mats/wet_area_anti_fatigue_mats landed on page that is not a product page. 2026-01-25 01:47:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-add-on-72in-w-x-48in-d-x-72in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-25 01:47:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fillet-7-stiff-blk-fibrox>: HTTP status code is not handled or not allowed 2026-01-25 01:47:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 49003 pages (at 95 pages/min), scraped 24774 items (at 56 items/min) 2026-01-25 01:47:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durastripe-20-round-sign-eye-protection-required>: HTTP status code is not handled or not allowed 2026-01-25 01:47:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-closed-single-solenoid-valve-pilot-spr-return-g-1-4-24v-dc-3w-coil-blk-connection>: HTTP status code is not handled or not allowed 2026-01-25 01:47:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-00-arcguard-rubber-voltage-glove-kit-red-size-11-kitgc0011r>: HTTP status code is not handled or not allowed 2026-01-25 01:47:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/push-cart-dolly-sl1500p4b-4-casters-1500-lb-cap-black>: HTTP status code is not handled or not allowed 2026-01-25 01:47:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-work-shirt-4x-ln-royal-blue-tcg01130232>: HTTP status code is not handled or not allowed 2026-01-25 01:47:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2041-12k-bitrex-test-solution-6box>: HTTP status code is not handled or not allowed 2026-01-25 01:47:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-0-arcguard-rubber-voltage-glove-kit-black-size-8-kitgc008>: HTTP status code is not handled or not allowed 2026-01-25 01:47:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-12-x-24-light-blue-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:47:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1381254-thermometer-stand-hang-20-60f-for-taylor-precision-products-lp>: HTTP status code is not handled or not allowed 2026-01-25 01:47:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kit2cv08b3x09-8-cal-cm2-arc-flash-kit-w-fr-coverall-w-balaclava-3xl-glove-size-09>: HTTP status code is not handled or not allowed 2026-01-25 01:47:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-dbl-solenoid-valve-pilot-spring-closed-cntr-1-4-nptf-220v-ac-5va-coil-black-conn>: HTTP status code is not handled or not allowed 2026-01-25 01:47:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-2-double-solenoid-valve-pilot-g-1-4-110v-ac-5va-coil-led-connection>: HTTP status code is not handled or not allowed 2026-01-25 01:47:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-12-x-36-white-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:47:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/51-1114-coaxial-thermocouple-36-inch-11-32-inch-32-thread>: HTTP status code is not handled or not allowed 2026-01-25 01:47:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mh7100-plus-replacement-suction-cup-7-wall-mount-barrier>: HTTP status code is not handled or not allowed 2026-01-25 01:47:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-high-capacity-bulk-rack-without-decking-72x24x72-starter>: HTTP status code is not handled or not allowed 2026-01-25 01:47:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-12-x-60-black-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:47:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-high-capacity-bulk-rack-without-decking-60x24x96-add-on>: HTTP status code is not handled or not allowed 2026-01-25 01:47:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-2-single-solenoid-valve-pilot-spring-return-g-1-2-220v-ac-5va-coil-led-connection>: HTTP status code is not handled or not allowed 2026-01-25 01:47:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-2-double-solenoid-valve-pilot-1-8-nptf-220v-ac-5va-coil-black-connection>: HTTP status code is not handled or not allowed 2026-01-25 01:47:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thhn-10-gauge-building-wire-stranded-type-white>: HTTP status code is not handled or not allowed 2026-01-25 01:47:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-vis-work-shirt-type-r-class-3-xl-fluorescent-yellow-df2-ax3-324ls-hy-xl>: HTTP status code is not handled or not allowed 2026-01-25 01:47:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-16-24-x-1-hex-head-cap-screw-steel-zinc-yellow-unf-grade-8-usa-100-pack-454108>: HTTP status code is not handled or not allowed 2026-01-25 01:47:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-add-on-96inw-x-24ind-x-72inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 01:47:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-shelving-open-shelving-20>: HTTP status code is not handled or not allowed 2026-01-25 01:47:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-2-single-solenoid-valve-pilot-spr-return-1-2-nptf-12v-dc-3w-coil-blk-connection>: HTTP status code is not handled or not allowed 2026-01-25 01:47:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/621121-thermometer-candy-fryer-for-comark-instruments>: HTTP status code is not handled or not allowed 2026-01-25 01:47:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-add-on-72inw-x-24ind-x-96inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 01:47:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-2-rubber-voltage-gloves-red-size-10-dwh14210r>: HTTP status code is not handled or not allowed 2026-01-25 01:47:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-1-2-x-2-1-2-x-40-225-edge-protectors-cased>: HTTP status code is not handled or not allowed 2026-01-25 01:48:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2241222-cup-suction-for-nemco>: HTTP status code is not handled or not allowed 2026-01-25 01:48:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/511400-pilot-burner-lp-nat-for-jade-range>: HTTP status code is not handled or not allowed 2026-01-25 01:48:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/541001-gas-valve3-4-inch>: HTTP status code is not handled or not allowed 2026-01-25 01:48:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-x-24-flush-cut-foam-pouches>: HTTP status code is not handled or not allowed 2026-01-25 01:48:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stackable-platform-starter-24x96>: HTTP status code is not handled or not allowed 2026-01-25 01:48:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 49099 pages (at 96 pages/min), scraped 24835 items (at 61 items/min) 2026-01-25 01:48:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylon-nail-anchor-1-4-x-1-1-2-mushroom-head-plain-finish-pkg-of-100-r14003>: HTTP status code is not handled or not allowed 2026-01-25 01:48:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-dbl-solenoid-vlv-ext-pilot-open-cntr-spr-cntred-1-4-nptf-24v-dc-2w-coil-black>: HTTP status code is not handled or not allowed 2026-01-25 01:48:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-extra-heavy-duty-boltless-shelving-add-on-96inw-x-36ind-x-72inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 01:48:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-hose-double-male-adapter-378-1561524-1-1-2-nh-x-1-1-2-npt-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 01:48:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-stacking-bin-11-x-16-x-8-green>: HTTP status code is not handled or not allowed 2026-01-25 01:48:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/outdoor-enclosed-satin-letterboard-36-in-w-x-36-in-h>: HTTP status code is not handled or not allowed 2026-01-25 01:48:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-sided-floor-rack-with-bins-10>: HTTP status code is not handled or not allowed 2026-01-25 01:48:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/knit-sleeve-with-thumbhole-black-s00ca0318>: HTTP status code is not handled or not allowed 2026-01-25 01:48:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/snow-cone-syrups-lemon>: HTTP status code is not handled or not allowed 2026-01-25 01:48:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-dbl-solenoid-vlv-ext-pilot-closed-cntr-spr-cntred-g1-4-220v-ac-5va-coil-black>: HTTP status code is not handled or not allowed 2026-01-25 01:48:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beam-seating-guest-chairs-2-seater-black-white>: HTTP status code is not handled or not allowed 2026-01-25 01:48:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-unlined-bib-overall-32-x-30-brown-bib6dcm32xl30>: HTTP status code is not handled or not allowed 2026-01-25 01:48:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40-cal-compliance-bib-overalls-3xl-navy-c45uquq403xl32>: HTTP status code is not handled or not allowed 2026-01-25 01:48:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/321737-bowl-cutter-plastic-for-robot-coupe>: HTTP status code is not handled or not allowed 2026-01-25 01:48:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/321207-gauge-glass>: HTTP status code is not handled or not allowed 2026-01-25 01:48:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-heavy-duty-boltless-shelving-add-on-48inw-x-36ind-x-72inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 01:48:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-58-22-amp208-230v-1075rpm-reversible>: HTTP status code is not handled or not allowed 2026-01-25 01:48:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-s-brass-bubbler-flexible-mouth-guard-push-button-metering-handle>: HTTP status code is not handled or not allowed 2026-01-25 01:48:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-s-brass-installation-kit-for-b-0230-style-faucets>: HTTP status code is not handled or not allowed 2026-01-25 01:48:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/33-double-shaft-115v-1550rpm-516-4>: HTTP status code is not handled or not allowed 2026-01-25 01:48:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-closed-single-solenoid-valve-pilot-spr-return-g-1-8-220v-ac-5va-coil-blk-conn>: HTTP status code is not handled or not allowed 2026-01-25 01:48:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nesbitt-replacement-motors-850rpm-115v-ball>: HTTP status code is not handled or not allowed 2026-01-25 01:48:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-cal-ultrasoft-flame-resistant-coverall-md-x-32-navy-c88ujmd32>: HTTP status code is not handled or not allowed 2026-01-25 01:48:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-30-gallon-dual-action-hand-pump>: HTTP status code is not handled or not allowed 2026-01-25 01:48:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sodium-chloride-ice-and-snow-melt-30-lb-pail-12987>: HTTP status code is not handled or not allowed 2026-01-25 01:48:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/69632-container-assembly-plastic-44oz-for-hamilton-beach>: HTTP status code is not handled or not allowed 2026-01-25 01:48:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnalugs-butt-splice-102457-050-1-0-gauge-50-pcs>: HTTP status code is not handled or not allowed 2026-01-25 01:48:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-powder-exercise-band-pack-4-band-easy-yellow-red-green>: HTTP status code is not handled or not allowed 2026-01-25 01:48:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silt-fence-3-x-100-black>: HTTP status code is not handled or not allowed 2026-01-25 01:48:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-piece-heavy-duty-rainsuit-yellow-polyester-5xl>: HTTP status code is not handled or not allowed 2026-01-25 01:48:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/71252-10-single-blade-ultra-hygiene-bench-squeegee-green>: HTTP status code is not handled or not allowed 2026-01-25 01:48:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_hinges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:48:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/53707-216-2-4-tube-brush-medium-orange>: HTTP status code is not handled or not allowed 2026-01-25 01:48:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyester-strapping-58w-x-3600l-x-003-thick-16-x-6-core-green>: HTTP status code is not handled or not allowed 2026-01-25 01:48:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyester-strapping-58w-x-2200l-x-0025-thick-16-x-3-core-black>: HTTP status code is not handled or not allowed 2026-01-25 01:48:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-tier-5-door-clear-view-locker-w-digital-lock-12-w-x-15-d-x-66-h-tan-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 01:48:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1831128-link-belt-1-2-inch-pitch-for-roundup-food-equipment>: HTTP status code is not handled or not allowed 2026-01-25 01:48:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cold-weather-survival-kit-70410-2-person>: HTTP status code is not handled or not allowed 2026-01-25 01:49:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_hinges landed on page that is not a product page. 2026-01-25 01:49:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fieldpro-purofort-full-safety-boots-steel-toe-14h-size-14-black>: HTTP status code is not handled or not allowed 2026-01-25 01:49:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/263806-stuf-bx-3-8-fpt-x-1-4mpt-2-pk>: HTTP status code is not handled or not allowed 2026-01-25 01:49:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/seapro-purofort-full-safety-boots-steel-toe-15h-size-5-blueyellow>: HTTP status code is not handled or not allowed 2026-01-25 01:49:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/barnstead-cation-removal-cartridge-d0815-full-size-1-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:49:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/56877-3-gallon-bucket-lid-orange>: HTTP status code is not handled or not allowed 2026-01-25 01:49:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 49194 pages (at 95 pages/min), scraped 24887 items (at 52 items/min) 2026-01-25 01:49:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-shelf-liner-42w-x-12d-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:49:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/record-storage-open-44>: HTTP status code is not handled or not allowed 2026-01-25 01:49:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/53743-overhead-brush-medium-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:49:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-black-cable-tie>: HTTP status code is not handled or not allowed 2026-01-25 01:49:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-20-x-1-1-4-machine-screw-pan-head-phillips-18-8-stainless-steel-ft-unc-pkg-of-500>: HTTP status code is not handled or not allowed 2026-01-25 01:49:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-surface-mounted-toiletry-shelf-24-w>: HTTP status code is not handled or not allowed 2026-01-25 01:49:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/53726-hook-brush-medium-yellow>: HTTP status code is not handled or not allowed 2026-01-25 01:49:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portability-package-bishamon-li-2k-li-25k-scissor-li-table>: HTTP status code is not handled or not allowed 2026-01-25 01:49:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/52-bollard-flex-with-sign-post-asphalt-installation-red-cover-blue-tapes>: HTTP status code is not handled or not allowed 2026-01-25 01:49:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/458766-bench-brush-soft-brown>: HTTP status code is not handled or not allowed 2026-01-25 01:49:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/56889-1-5-gallon-bucket-black>: HTTP status code is not handled or not allowed 2026-01-25 01:49:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60w-left-hand-3-position-sit-stand-l-desk-mocha-cherry-series-c>: HTTP status code is not handled or not allowed 2026-01-25 01:49:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/bollards/safety_bollards_posts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:49:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/150-series-acetylene-regulator-1-12-side-mount>: HTTP status code is not handled or not allowed 2026-01-25 01:49:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/stands/computer_stands_carts/computer_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:49:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5380-90-5-216-3-5-pipe-brush-medium-white>: HTTP status code is not handled or not allowed 2026-01-25 01:49:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leviton-41089-2wp-quickport-surface-mount-housing-2-port-white>: HTTP status code is not handled or not allowed 2026-01-25 01:49:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-fiberglass-filter-20-w-x-24-h-x-2-d>: HTTP status code is not handled or not allowed 2026-01-25 01:49:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/bollards/safety_bollards_posts landed on page that is not a product page. 2026-01-25 01:49:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/29342-6-mini-handle-green>: HTTP status code is not handled or not allowed 2026-01-25 01:49:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/stands/computer_stands_carts/computer_cabinets>: HTTP status code is not handled or not allowed 2026-01-25 01:49:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/29596-59-basic-aluminum-handle-yellow>: HTTP status code is not handled or not allowed 2026-01-25 01:49:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dinex-classic-translucent-lidfits-dx1186-12-oz-bowl-500-cs-translucent-dx11868714>: HTTP status code is not handled or not allowed 2026-01-25 01:49:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/29112-nylon-floor-scraper-green>: HTTP status code is not handled or not allowed 2026-01-25 01:49:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40112-4-stainless-steel-handle-mounted-scraper-green>: HTTP status code is not handled or not allowed 2026-01-25 01:49:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8751-replacement-liner-for-13-by-13-modular-model-berm-textured-tan>: HTTP status code is not handled or not allowed 2026-01-25 01:49:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/hand_tools_1/hand_saws_cutting_tools_scrapers/scrapers_putty_knives already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:49:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010564-knob-open-burner-for-garland-range>: HTTP status code is not handled or not allowed 2026-01-25 01:49:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/hand_tools_1/hand_saws_cutting_tools_scrapers/scrapers_putty_knives already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:49:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fir-plywood-premium-grade-crate-premium76-57-23-collapsible-inside-76l-x-57w-x-23>: HTTP status code is not handled or not allowed 2026-01-25 01:49:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/29754-62-113-telescopic-aluminum-handle-red>: HTTP status code is not handled or not allowed 2026-01-25 01:49:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/-protective-cover-for-apex153-700--jetstream-230-portable-evaporative-coolers>: HTTP status code is not handled or not allowed 2026-01-25 01:50:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/outdoor-snow-ice-melting-heated-walkway-mat-3-x-20-120-volts>: HTTP status code is not handled or not allowed 2026-01-25 01:50:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1211162-glide-table-shox-1-4-20>: HTTP status code is not handled or not allowed 2026-01-25 01:50:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/hand_tools_1/hand_saws_cutting_tools_scrapers/scrapers_putty_knives>: HTTP status code is not handled or not allowed 2026-01-25 01:50:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/511039-pilot-tube-for-randell>: HTTP status code is not handled or not allowed 2026-01-25 01:50:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/hand_tools_1/hand_saws_cutting_tools_scrapers/scrapers_putty_knives landed on page that is not a product page. 2026-01-25 01:50:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-rack-108x42x144-starter-unit-green-orange-5480-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-25 01:50:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-36-shop-top-work-bench-fixed-height-1-3-4-top>: HTTP status code is not handled or not allowed 2026-01-25 01:50:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-height-steel-floor-lock-8-10-casters>: HTTP status code is not handled or not allowed 2026-01-25 01:50:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 49291 pages (at 97 pages/min), scraped 24948 items (at 61 items/min) 2026-01-25 01:50:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bolted-teardrop-pallet-rack-starter-96inw-x-48ind-x-120inh-5030-lbs-shelf-cap>: HTTP status code is not handled or not allowed 2026-01-25 01:50:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-mesh-security-cage-48x24x60>: HTTP status code is not handled or not allowed 2026-01-25 01:50:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bolted-teardrop-pallet-rack-add-on-96inw-x-42ind-x-192inh-5030-lbs-shelf-cap>: HTTP status code is not handled or not allowed 2026-01-25 01:50:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-unit-blk-yellow-15-caution-do-not-enter-yellow-w-clamp-belt-end>: HTTP status code is not handled or not allowed 2026-01-25 01:50:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/31748-16-combo-push-broom-soft-stiff-purple>: HTTP status code is not handled or not allowed 2026-01-25 01:50:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manual-powerstop-vehicle-truck-restraint-kit-with-automatic-lights-signs>: HTTP status code is not handled or not allowed 2026-01-25 01:50:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/toolarama-shop-desk-combo-36-x-28-x-54>: HTTP status code is not handled or not allowed 2026-01-25 01:50:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1400202-ps80l210-80v-pro-series-10-inch-pole-saw-kit-w-20ah-battery-and-charger>: HTTP status code is not handled or not allowed 2026-01-25 01:50:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triple-drawer-bank-mobile-12-drawer-cabinet-steel-top-finish-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:50:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forkli-able-shipping-container-with-lid-42-x-42-3>: HTTP status code is not handled or not allowed 2026-01-25 01:50:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lockouttagout-toolbox-kit-with-32-components-13w-x-5d-x-6-12h-redyellowgray>: HTTP status code is not handled or not allowed 2026-01-25 01:50:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-dolly-with-tilt-rollers-36-l-x-36-w-4000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 01:50:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/350-10-3ble-dual-ramp-wheelchair-scale-with-handrail-bluetooth-ble-4-0-1000-lb-x-0-2-lb>: HTTP status code is not handled or not allowed 2026-01-25 01:50:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ccc06006-socket-set-screw>: HTTP status code is not handled or not allowed 2026-01-25 01:50:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-caster-dolly-42-l-x-24-w-2000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 01:50:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ten-quarts-stock-pot>: HTTP status code is not handled or not allowed 2026-01-25 01:50:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cdc05030-socket-cap-screw>: HTTP status code is not handled or not allowed 2026-01-25 01:50:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cube-ice-maker-undercounter-air-cooled-approx-309-lb-production-full-size-cubes-full-size-cube>: HTTP status code is not handled or not allowed 2026-01-25 01:50:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/263810-bushing-flange-bj>: HTTP status code is not handled or not allowed 2026-01-25 01:50:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/63-h-nexel-chrome-wire-shelving-add-on-60-w-x-14-d>: HTTP status code is not handled or not allowed 2026-01-25 01:50:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/original-wrist-and-ankle-weight-with-mobile-rack-24-piece-set>: HTTP status code is not handled or not allowed 2026-01-25 01:50:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-level-incline-board-plastic-14-x-14>: HTTP status code is not handled or not allowed 2026-01-25 01:50:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/porta-shield-vinyl-divider-6-foot-w-x-6-half-foot-h-clear>: HTTP status code is not handled or not allowed 2026-01-25 01:50:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stacking-bin-16-1-2-x-14-3-4-x-7-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:50:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-inch-french-whip-with-hi-temp-handle>: HTTP status code is not handled or not allowed 2026-01-25 01:50:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industries-adjustable-saddle-ball-for-15-tubing-polished-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 01:50:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mixing-bowl-5-quart-1>: HTTP status code is not handled or not allowed 2026-01-25 01:50:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1591155-belt-bx48>: HTTP status code is not handled or not allowed 2026-01-25 01:50:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wooden-fixed-level-incline-board-16-1-4-x-15-surface-set-of-4-boards>: HTTP status code is not handled or not allowed 2026-01-25 01:50:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/282121-brush-attach-f>: HTTP status code is not handled or not allowed 2026-01-25 01:50:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/anchors_hangers/wedge_anchors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:50:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelite-vented-plastic-mat-shelf-truck-72l-x-21w-x-69h-polyurethane-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 01:50:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/welded-steel-bookcase-52-h-putty>: HTTP status code is not handled or not allowed 2026-01-25 01:50:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/servewell-2-well-hot-food-table-3>: HTTP status code is not handled or not allowed 2026-01-25 01:50:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-residential-safes-expandable-depth-wall-safe-15-w-x-3-1-4-6-d-x-22-1-8-h-new>: HTTP status code is not handled or not allowed 2026-01-25 01:51:00 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-25 01:51:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/morado-super-cleaner-55-gallon-drum>: HTTP status code is not handled or not allowed 2026-01-25 01:51:01 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:25 2026-01-25 01:51:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/tool_storage/chests_roller_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:51:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smead-manila-self-adhesive-endtop-tab-folder-dividers-two-fasteners-letter-100box>: HTTP status code is not handled or not allowed 2026-01-25 01:51:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/hanging_file_folders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:51:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22-1047-slicer-handle>: HTTP status code is not handled or not allowed 2026-01-25 01:51:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/anchors_hangers/wedge_anchors landed on page that is not a product page. 2026-01-25 01:51:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/performance-gravity-hvlp-atomizing-head-refill-kit-26718-1-8mm-clear-pack-of-5>: HTTP status code is not handled or not allowed 2026-01-25 01:51:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 49388 pages (at 97 pages/min), scraped 25006 items (at 58 items/min) 2026-01-25 01:51:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-hand-knob-w-ss-screw-5-16-18-thread-11-16-inch-thread-length-1-1-2-inch-knob-dia>: HTTP status code is not handled or not allowed 2026-01-25 01:51:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clamps-wilton14302>: HTTP status code is not handled or not allowed 2026-01-25 01:51:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hubbel-industrial-bare-end-12-3-45>: HTTP status code is not handled or not allowed 2026-01-25 01:51:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/tool_storage/chests_roller_cabinets landed on page that is not a product page. 2026-01-25 01:51:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/hanging_file_folders>: HTTP status code is not handled or not allowed 2026-01-25 01:51:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/crowd_control/crowd_control_barricades already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:51:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/child-universal-life-vest-red-black-20-002-child-red>: HTTP status code is not handled or not allowed 2026-01-25 01:51:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/classification_folders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:51:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/63-h-nexel-chrome-wire-shelving-add-on-42-w-x-24-d>: HTTP status code is not handled or not allowed 2026-01-25 01:51:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-insulated-hydro-parka-jacket-m-black>: HTTP status code is not handled or not allowed 2026-01-25 01:51:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/86-h-nexel-chrome-wire-shelving-add-on-30-w-x-14-d>: HTTP status code is not handled or not allowed 2026-01-25 01:51:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1421647-brush-fryer-angled-nylon>: HTTP status code is not handled or not allowed 2026-01-25 01:51:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rivetinghammers-jet-jsg-0101r>: HTTP status code is not handled or not allowed 2026-01-25 01:51:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/crowd_control/crowd_control_barricades landed on page that is not a product page. 2026-01-25 01:51:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009335-ignition-module-for-henny-penny>: HTTP status code is not handled or not allowed 2026-01-25 01:51:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/classification_folders landed on page that is not a product page. 2026-01-25 01:51:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4ft-long-box-pleat-skirting-8h-stage-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:51:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fanny-pack-stuffed-with-cpr-maskbandagesgauze-red-10-103-red-s1>: HTTP status code is not handled or not allowed 2026-01-25 01:51:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-63mm-bore-x-30mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 01:51:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/hanging_file_folders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:51:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-wire-shelving-add-on-60-w-x-24-d-x-63-h>: HTTP status code is not handled or not allowed 2026-01-25 01:51:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stage-pie-set-carpet-36w-stage-units-8h-16h-24h-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:51:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silicone-coated-fiberglass-welding-blanket-6w-x-8h-red>: HTTP status code is not handled or not allowed 2026-01-25 01:51:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/86-h-nexel-chrome-wire-shelving-60-w-x-18-d>: HTTP status code is not handled or not allowed 2026-01-25 01:51:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-wire-shelving-add-on-60-w-x-18-d-x-86-h>: HTTP status code is not handled or not allowed 2026-01-25 01:51:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/hanging_file_folders landed on page that is not a product page. 2026-01-25 01:51:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-63mm-bore-x-270mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 01:51:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stage-set-hardboard-96l-x-36w-8h-16h-24h-two-guard-rails>: HTTP status code is not handled or not allowed 2026-01-25 01:51:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-desiccant-dehumidifier-dd900-460v-10-amps-7-6-kw-364-pints>: HTTP status code is not handled or not allowed 2026-01-25 01:51:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/23awg-4-pr-utp-cat6-ez-250-mhz-blue-cmp-500-ft-pullbox-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:51:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/physical_therapy/pain_relief_gel already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:51:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-poly-oxford-5-in-1-safety-jacket-3xlt-black>: HTTP status code is not handled or not allowed 2026-01-25 01:51:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-insulated-quilted-jacket-l-black>: HTTP status code is not handled or not allowed 2026-01-25 01:51:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-poly-oxford-5-in-1-safety-jacket-3xlt-fluorescent-green>: HTTP status code is not handled or not allowed 2026-01-25 01:51:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-wire-shelf-instrument-cart-36x24-3-shelves-1200-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 01:51:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kit-male-clevis-bracket-al-mount-32-for-iso-15552-cylinders>: HTTP status code is not handled or not allowed 2026-01-25 01:51:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-wire-shelf-truck-48x24x92-1200-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 01:51:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/physical_therapy/pain_relief_gel landed on page that is not a product page. 2026-01-25 01:51:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-solid-galvanized-shelf-cart-60x24-2-shelves-800-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 01:52:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-quilted-safety-freezer-jacket-lt-black>: HTTP status code is not handled or not allowed 2026-01-25 01:52:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-32mm-bore-x-590mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 01:52:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-wide-span-rack-48x48x96-with-3-shelves-wood-deck-1200-lb-capacity-per-level>: HTTP status code is not handled or not allowed 2026-01-25 01:52:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reusable-blue-vinyl-cold-pack-oversize-11-x-21-12-pk>: HTTP status code is not handled or not allowed 2026-01-25 01:52:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bib-apron-25x28-naugahyde-leather-look-brown>: HTTP status code is not handled or not allowed 2026-01-25 01:52:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-run-tee-85222-53-02-532-tube-x-18-swift-fit-univ-thread-composite-body>: HTTP status code is not handled or not allowed 2026-01-25 01:52:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 49482 pages (at 94 pages/min), scraped 25059 items (at 53 items/min) 2026-01-25 01:52:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-z-brite-wire-shelving-36-w-x-24-d-x-63-h-1>: HTTP status code is not handled or not allowed 2026-01-25 01:52:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-quilted-safety-freezer-jacket-m-fluorescent-green>: HTTP status code is not handled or not allowed 2026-01-25 01:52:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-14>: HTTP status code is not handled or not allowed 2026-01-25 01:52:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-wide-span-rack-48x36x60-with-3-shelves-wood-deck-1200-lb-capacity-per-level>: HTTP status code is not handled or not allowed 2026-01-25 01:52:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beanie-watch-cap-xl-olive>: HTTP status code is not handled or not allowed 2026-01-25 01:52:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/moist-heat-pack-and-cover-set-circular-pack-with-foam-filled-cover>: HTTP status code is not handled or not allowed 2026-01-25 01:52:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-polypropylene-seal-top-bag-with-hang-hole-5-x-3-2-mil>: HTTP status code is not handled or not allowed 2026-01-25 01:52:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/saddle-pack-printed-portion-control-bags-tuesday-print-in-yellow-ink-7-x-65-05-mil>: HTTP status code is not handled or not allowed 2026-01-25 01:52:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/moist-heat-pack-contour-wrap-cervical-6-x-24-with-3-x-27-elastic-belt>: HTTP status code is not handled or not allowed 2026-01-25 01:52:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-run-tee-55222-8-14-8mm-tube-x-14-swift-fit-univ-thread-composite-body>: HTTP status code is not handled or not allowed 2026-01-25 01:52:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-extra-shelving-48x18x72-with-4-shelves-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 01:52:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/264061-grinder-plate-3-16-inch>: HTTP status code is not handled or not allowed 2026-01-25 01:52:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eagle-poly-waste-can-yellow-foot-lever-14-gallon>: HTTP status code is not handled or not allowed 2026-01-25 01:52:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-run-tee-89222-53-02-532-tube-x-18-swift-fit-metallic-collet>: HTTP status code is not handled or not allowed 2026-01-25 01:52:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-control-55905-5-18-knob-adj-5mm-tube-x-18-swift-fit-composite-body>: HTTP status code is not handled or not allowed 2026-01-25 01:52:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-heavy-duty-shelving-48x18x72-with-4-shelves-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 01:52:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eagle-type-i-safety-can-5-gallon-funnel-yellow>: HTTP status code is not handled or not allowed 2026-01-25 01:52:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roof-edge-delineator-cone-without-base-39h-yellow>: HTTP status code is not handled or not allowed 2026-01-25 01:52:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elbow-82520n-08-12-male-nptf-x-12-female-nptf-nickel-plated-brass>: HTTP status code is not handled or not allowed 2026-01-25 01:52:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-inch-x-14-inch-x-716-inch-x-532-inch-step-keystock-type-1-zinc-plated-12-inch-length>: HTTP status code is not handled or not allowed 2026-01-25 01:52:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-utility-cloth-roll-kit-314d-1-in-p60x-p80j-p120j-p180j-p240j>: HTTP status code is not handled or not allowed 2026-01-25 01:52:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/150-piece-hitch-pin-clip-assortment>: HTTP status code is not handled or not allowed 2026-01-25 01:52:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-elbow-85110-53-32-532-tube-x-10-32-male-unf-thread-composite-body>: HTTP status code is not handled or not allowed 2026-01-25 01:52:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-add-on-36-in-w-x-18-in-d-x-72-in-h-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 01:52:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/file-belt-sander-attachment-arm-for-corners-28370>: HTTP status code is not handled or not allowed 2026-01-25 01:52:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelf-36x24-with-4-clips-1>: HTTP status code is not handled or not allowed 2026-01-25 01:52:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/power_tools/power_sanders/abrasive_tool_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:52:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-base-for-globalindustrial-2-3-cu-yd-dumping-hopper-w-bump-release>: HTTP status code is not handled or not allowed 2026-01-25 01:52:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/plumbing_pumps/utility_pumps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:52:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/atrium153-forklift-enclosure-vinyl-large-60l-x-52w-super-clear>: HTTP status code is not handled or not allowed 2026-01-25 01:52:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2231027-block-support-w-feet>: HTTP status code is not handled or not allowed 2026-01-25 01:52:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/51025-500-heavy-duty-plastic-barrier-chain-hdpe-2-x500-54mm-safety-pink>: HTTP status code is not handled or not allowed 2026-01-25 01:52:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/power_tools/power_sanders/abrasive_tool_accessories>: HTTP status code is not handled or not allowed 2026-01-25 01:53:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ls12-sliding-arm-for-model-e5-bt-a12-ul>: HTTP status code is not handled or not allowed 2026-01-25 01:53:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/plumbing/plumbing_pumps/utility_pumps landed on page that is not a product page. 2026-01-25 01:53:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-215-tefc-1ph-1740rpm-p215k17fb2ll>: HTTP status code is not handled or not allowed 2026-01-25 01:53:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m10-x-1-5-x-80mm-flat-head-socket-cap-screw-304-stainless-steel-pkg-of-25>: HTTP status code is not handled or not allowed 2026-01-25 01:53:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/best-value-manual-li-stacker-880-lb-cap-59-li>: HTTP status code is not handled or not allowed 2026-01-25 01:53:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72in-rectangular-metal-mesh-picnic-table-red>: HTTP status code is not handled or not allowed 2026-01-25 01:53:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/union-tee-89230-05-516-tube-metallic-collet>: HTTP status code is not handled or not allowed 2026-01-25 01:53:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-182-dp-1ph-3450rpm-c182c34db2aa>: HTTP status code is not handled or not allowed 2026-01-25 01:53:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 49582 pages (at 100 pages/min), scraped 25120 items (at 61 items/min) 2026-01-25 01:53:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-kickboard-with-2-hand-holes-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:53:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m4-x-0-7-x-30mm-machine-screw-phillips-pan-head-304-stainless-steel-din-7985-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 01:53:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8004109-probe-w-ties-kit-for-frymaster>: HTTP status code is not handled or not allowed 2026-01-25 01:53:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smartcompliance-plastic-bandage-38-x-1-12-80box>: HTTP status code is not handled or not allowed 2026-01-25 01:53:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1271141-gasket-wendy-s-only-18-inch-x-18-3-4-inch-d2d>: HTTP status code is not handled or not allowed 2026-01-25 01:53:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m10-x-1-5-hex-nut-304-stainless-steel-din-934-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 01:53:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011343-drawer-rail-for-turbo-air-refrigeration>: HTTP status code is not handled or not allowed 2026-01-25 01:53:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-branch-tee-55210-8-18-8mm-tube-x-18-swift-fit-composite-body>: HTTP status code is not handled or not allowed 2026-01-25 01:53:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-elbow-85110-08-04-12-tube-x-14-swift-fit-univ-thread-composite-body>: HTTP status code is not handled or not allowed 2026-01-25 01:53:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hog-heaven-fashion-mat-five-eighths-3x5-cocoa-brown>: HTTP status code is not handled or not allowed 2026-01-25 01:53:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cs52h-collared-screw-worm-gear-hose-clamp-1-78-inch-3-34-inch-clamping-diam-10-pcs>: HTTP status code is not handled or not allowed 2026-01-25 01:53:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-tecgen-flame-resistant-work-shirt-l-tall-tan>: HTTP status code is not handled or not allowed 2026-01-25 01:53:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b40h-part-ss-worm-gear-hose-clamp-2-116-inch-3-inch-clamping-diameter-box-of-10>: HTTP status code is not handled or not allowed 2026-01-25 01:53:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1221221-kason-10533d00004-latch-w-strike>: HTTP status code is not handled or not allowed 2026-01-25 01:53:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-quart-pail-galvanized-steel-for-freestanding-ashtray>: HTTP status code is not handled or not allowed 2026-01-25 01:53:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-143-tefc-3ph-1760-1450rpm-c143t17fk9ee>: HTTP status code is not handled or not allowed 2026-01-25 01:53:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-184-dp-1ph-1740rpm-p184c17db21aa>: HTTP status code is not handled or not allowed 2026-01-25 01:53:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/comfy-hand-thumb-orthosis-adult-large-with-one-cover>: HTTP status code is not handled or not allowed 2026-01-25 01:53:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1221079-latch-w-strike>: HTTP status code is not handled or not allowed 2026-01-25 01:53:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-182-dp-1ph-3500rpm-c182k34db3bb>: HTTP status code is not handled or not allowed 2026-01-25 01:53:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lite-insulated-gas-cylinder-heater-pbl100-100-lb-capacity-90f-fixed>: HTTP status code is not handled or not allowed 2026-01-25 01:53:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1201010-caster-stem-3-1-2-inch-w-brk-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:53:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smoking-shelter-dome-roof-four-sided-with-le-front-opening-10-x-5>: HTTP status code is not handled or not allowed 2026-01-25 01:53:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-flame-resistant-dh-air153-work-shirt-xl-tall-navy>: HTTP status code is not handled or not allowed 2026-01-25 01:53:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-channel-ph-ise-benchtop-meter-and-stand-vstar40b>: HTTP status code is not handled or not allowed 2026-01-25 01:53:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s9842-95w-a19-led-220-beam-spread-gu24-base-4000k>: HTTP status code is not handled or not allowed 2026-01-25 01:53:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/red-plastic-box-truck-6-bushel-medium-duty>: HTTP status code is not handled or not allowed 2026-01-25 01:53:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bookcases_hutches already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:53:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10500-series-bookcase-hutch-36-w-x-14-63-d-x-37-13-h-harvest>: HTTP status code is not handled or not allowed 2026-01-25 01:53:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/foodservice_disposables/disposable_cups already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:53:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72w-x-30d-modular-workbench-with-3-drawers-esd-laminate-safety-edge-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:53:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/biodegradable-grill-cleaning-cloth-wipes-80-wipes-per-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:53:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/red-plastic-box-truck-10-bushel>: HTTP status code is not handled or not allowed 2026-01-25 01:53:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bookcases_hutches landed on page that is not a product page. 2026-01-25 01:53:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-s-hook-with-2-clamp-black>: HTTP status code is not handled or not allowed 2026-01-25 01:53:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/foodservice_disposables/disposable_cups landed on page that is not a product page. 2026-01-25 01:54:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-sided-tray-truck-38x24x66-1200-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 01:54:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0010-inch-steel-shim-stock-6-inch-x-100-inch-roll>: HTTP status code is not handled or not allowed 2026-01-25 01:54:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-12-inch-id-x-2-18-inch-od-x-0025-inch-plastic-color-coded-arbor-shim-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 01:54:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/heaters/electric_heaters/electric_unit_heaters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:54:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1421406-brush-grill-steel-bristle>: HTTP status code is not handled or not allowed 2026-01-25 01:54:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 49678 pages (at 96 pages/min), scraped 25178 items (at 58 items/min) 2026-01-25 01:54:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/500va-omnitower-full-isolation-avr-6-outlet>: HTTP status code is not handled or not allowed 2026-01-25 01:54:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/half-round-rack>: HTTP status code is not handled or not allowed 2026-01-25 01:54:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acrylic-heavy-duty-shelves-for-slat-wall-24w-x-12d-x-3-16h>: HTTP status code is not handled or not allowed 2026-01-25 01:54:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multiple-15-amp-6-outlet-15ft>: HTTP status code is not handled or not allowed 2026-01-25 01:54:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-inch-plastic-classroom-chair-set-of-five-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:54:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-stud-wedge-anchor-3-8-x-3-3-4-304-stainless-steel-unc-pkg-of-50>: HTTP status code is not handled or not allowed 2026-01-25 01:54:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smartrack-45u-black-2-post-frame-threaded>: HTTP status code is not handled or not allowed 2026-01-25 01:54:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-double-air-actuated-valve-pilot-spring-cent-closed-center-1-2-nptf-ports>: HTTP status code is not handled or not allowed 2026-01-25 01:54:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/heaters/electric_heaters/electric_unit_heaters landed on page that is not a product page. 2026-01-25 01:54:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-dock-plate-4383-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 01:54:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/06135-1-4-x-2-x-1-plastic-horseshoe-shim-solid-74s-1020pc>: HTTP status code is not handled or not allowed 2026-01-25 01:54:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-panel-display-tower-6-ft-5-inch-high-fabric-greysmoke>: HTTP status code is not handled or not allowed 2026-01-25 01:54:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keyless-unglazed-porcelain-socket-w-snap-in-clip>: HTTP status code is not handled or not allowed 2026-01-25 01:54:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/furniture_parts_accessories/partition_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:54:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keyless-glazed-porcelain-socket-w-paper-liner-mounting-screws-held-captive-2-wireways-and-18-in-leads>: HTTP status code is not handled or not allowed 2026-01-25 01:54:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-boxes-14-x-14-x-14-200lb-test-ect-32-25-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:54:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/push-to-connect-tube-fitting-nylon-plastic-straight-adapter-38-tube-od-x-38-npt-male>: HTTP status code is not handled or not allowed 2026-01-25 01:54:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/seating/stools/bar_counter_height_stools/bar_stools already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:54:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tabletop-personal-safety-partition-18w-x-24h-clear>: HTTP status code is not handled or not allowed 2026-01-25 01:54:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/furniture_parts_accessories/partition_accessories>: HTTP status code is not handled or not allowed 2026-01-25 01:54:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-plastic-shelf-truck-24-w-x-18-d-x-33-1-2-h-2-shelves>: HTTP status code is not handled or not allowed 2026-01-25 01:54:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washdown-motor-213-tefc-3ph-1760-1450rpm-c213t17wk2dd>: HTTP status code is not handled or not allowed 2026-01-25 01:54:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/seating/stools/bar_counter_height_stools/bar_stools>: HTTP status code is not handled or not allowed 2026-01-25 01:54:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keyless-threaded-glazed-porcelain-socket-w-cap-ring-1-8-ips>: HTTP status code is not handled or not allowed 2026-01-25 01:54:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-countersunk-finishing-washer-307-267-i-d-steel-plain-grade-2-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 01:54:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/measurement_layout_tools/test_indicators_holders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:54:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/signature-heavy-duty-single-compartment-bus-box-52661-20-x-15-x-7-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:54:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010425-blower-assembly-for-turbochef>: HTTP status code is not handled or not allowed 2026-01-25 01:54:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-hp-emod-motor56j-c-face15-sf>: HTTP status code is not handled or not allowed 2026-01-25 01:54:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-48x18x96-with-5-shelves-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 01:54:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-hook-for-gridwall>: HTTP status code is not handled or not allowed 2026-01-25 01:54:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/measurement_layout_tools/test_indicators_holders>: HTTP status code is not handled or not allowed 2026-01-25 01:55:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iec-metric-motor-180m-ip55-3ph-c180t34fz1cc>: HTTP status code is not handled or not allowed 2026-01-25 01:55:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-2-carbon-filter-2pk>: HTTP status code is not handled or not allowed 2026-01-25 01:55:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-round-table-with-4-imme-armless-caster-chairs-coral-seatmaple-top>: HTTP status code is not handled or not allowed 2026-01-25 01:55:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blue-thank-you-for-social-distancing-sign>: HTTP status code is not handled or not allowed 2026-01-25 01:55:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/304-ss-straight-reducers-tubeto-tubefor-quick-clamp-fittings-for-2-x-1-12-tubeod>: HTTP status code is not handled or not allowed 2026-01-25 01:55:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 49770 pages (at 92 pages/min), scraped 25236 items (at 58 items/min) 2026-01-25 01:55:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/optional-two-pedal-foot-control-for-portable-electric-hydraulic-lift-tables>: HTTP status code is not handled or not allowed 2026-01-25 01:55:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/321121-bdb-1340a-belt-drive-bench-lathe-w-acu-rite-200s-dro-2-hp>: HTTP status code is not handled or not allowed 2026-01-25 01:55:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-width-freestanding-coat-rack-sandstone>: HTTP status code is not handled or not allowed 2026-01-25 01:55:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010654-caster-stem-w-o-brake-for-alto-shaam>: HTTP status code is not handled or not allowed 2026-01-25 01:55:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/321567-gh-2280zx-large-spindle-bore-lathe-w-acu-rite-200s-dro-taper-attachment-10-hp>: HTTP status code is not handled or not allowed 2026-01-25 01:55:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/27008-20-x-16-ss-hood-filter-w-hooks-for-captive-air>: HTTP status code is not handled or not allowed 2026-01-25 01:55:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-container-without-lid-13>: HTTP status code is not handled or not allowed 2026-01-25 01:55:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arkadia-16-quart-stock-pot-7304-6-gauge-12-depth>: HTTP status code is not handled or not allowed 2026-01-25 01:55:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-starter-rack-72x24x96-with-3-levels-wood-deck-1000lb-cap-per-shelf-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:55:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/lathes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:55:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-round-table-with-4-imme-armless-caster-chairs-black-seatriver-cherry-top>: HTTP status code is not handled or not allowed 2026-01-25 01:55:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/equipto-wide-drawer-bolted-shelving-36-w-x-24-d-x-4-half-h-400lb-capacity-textured-evergreen>: HTTP status code is not handled or not allowed 2026-01-25 01:55:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-starter-rack-72x36x96-with-3-level-steel-deck-1000lb-cap-per-shelf-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:55:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-round-table-with-4-imme-armless-caster-chairs-light-gray-seatdesigner-white-top>: HTTP status code is not handled or not allowed 2026-01-25 01:55:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-add-on-rack-60x24x84-with-3-levels-wood-deck-1300-lb-cap-per-level-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:55:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-add-on-rack-48x36x84-with-3-levels-steel-deck-1500lb-cap-per-level-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:55:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=jvm-836-3-mill-c80-3-axis-q-dro-and-x-pwf already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:55:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jvm-836-3-mill-acu-rite-200s-dro-x-powerfeed-installed>: HTTP status code is not handled or not allowed 2026-01-25 01:55:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012582-grate-reversible-6-ribscb>: HTTP status code is not handled or not allowed 2026-01-25 01:55:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/lathes landed on page that is not a product page. 2026-01-25 01:55:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sled-base-leather-guest-chair-black>: HTTP status code is not handled or not allowed 2026-01-25 01:55:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=690008-jtm-4vs-milling-machine-w-x-y-axis-powerfeeds-3-hp already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:55:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-level-48x48-high-capacity-rack-steel-deck-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:55:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nickel-pull-switch-spst-6a-125-vac-o-f>: HTTP status code is not handled or not allowed 2026-01-25 01:55:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=jvm-836-3-mill-c80-3-axis-q-dro-and-x-pwf>: HTTP status code is not handled or not allowed 2026-01-25 01:55:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-2656-zinc-wire-shelf-24-1-2-inch-x-25-inch>: HTTP status code is not handled or not allowed 2026-01-25 01:55:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/releasable-cable-tie-8-25-pk>: HTTP status code is not handled or not allowed 2026-01-25 01:55:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8013952-fan-motor-condensor-115v-16w-cw-for-delfield>: HTTP status code is not handled or not allowed 2026-01-25 01:55:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/851231-filter-hot-oil-disc-100-for-keating>: HTTP status code is not handled or not allowed 2026-01-25 01:55:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gearmotor-dc-parallel-51rpm-tenv>: HTTP status code is not handled or not allowed 2026-01-25 01:55:51 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=690008-jtm-4vs-milling-machine-w-x-y-axis-powerfeeds-3-hp landed on page that is not a product page. 2026-01-25 01:55:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cayenne-36-heavy-duty-gas-griddle-936ggm-3-burners-90000-btu>: HTTP status code is not handled or not allowed 2026-01-25 01:55:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48w-30d-mobile-production-workbench-with-power-apron-shop-top-square-edge-tan>: HTTP status code is not handled or not allowed 2026-01-25 01:55:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/221154-kason-10055000028-latch-for-kason>: HTTP status code is not handled or not allowed 2026-01-25 01:55:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-lok-174-red-g2-500-pk>: HTTP status code is not handled or not allowed 2026-01-25 01:55:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/task_desk_chairs/executive_chairs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:56:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/skullerz-8974-safety-helmet-is-a-class-e-safety-helmet-orange>: HTTP status code is not handled or not allowed 2026-01-25 01:56:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-7551-coated-waterproof-winter-work-gloves-medium-orange-a5>: HTTP status code is not handled or not allowed 2026-01-25 01:56:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nitrile-coated-general-purpose-work-gloves-xs-gray-12-pair>: HTTP status code is not handled or not allowed 2026-01-25 01:56:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/34-20-belt-sander>: HTTP status code is not handled or not allowed 2026-01-25 01:56:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10ft-fiberglass-leansafe-ladder-w-plastic-tool-tray-375-lb-capacity-l7310>: HTTP status code is not handled or not allowed 2026-01-25 01:56:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/task_desk_chairs/executive_chairs landed on page that is not a product page. 2026-01-25 01:56:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rw10-3s1y-lightweight-rain-jacket-hi-viz-lime-2xl>: HTTP status code is not handled or not allowed 2026-01-25 01:56:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 49865 pages (at 95 pages/min), scraped 25292 items (at 56 items/min) 2026-01-25 01:56:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/appliances/refrigerators_freezers/refrigerators/refrigerated_merchandisers_display_cases already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:56:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manhole-shield-fits-28-30-32-diameter-manholes-high-impact-polymer>: HTTP status code is not handled or not allowed 2026-01-25 01:56:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/faultless-swivel-plate-caster-6-steel-wheel>: HTTP status code is not handled or not allowed 2026-01-25 01:56:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magliner-aluminum-hand-truck-loop-handle-balloon-wheels>: HTTP status code is not handled or not allowed 2026-01-25 01:56:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pistol-recoilless-air-hammer>: HTTP status code is not handled or not allowed 2026-01-25 01:56:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traffic-cone-non-reflective-4>: HTTP status code is not handled or not allowed 2026-01-25 01:56:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-pcs-30-inch-auger-anchor>: HTTP status code is not handled or not allowed 2026-01-25 01:56:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/office_machines/label_makers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:56:21 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/appliances/refrigerators_freezers/refrigerators/refrigerated_merchandisers_display_cases landed on page that is not a product page. 2026-01-25 01:56:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pc-connectable-twin-turbo-label-printer-71-labels-per-minute>: HTTP status code is not handled or not allowed 2026-01-25 01:56:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/159-lever-style-blow-gun-and-4-extension-1-4-npt-pack-of-5>: HTTP status code is not handled or not allowed 2026-01-25 01:56:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_hinges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:56:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1303-full-surface-spring-double-acting-hinge-6-usp>: HTTP status code is not handled or not allowed 2026-01-25 01:56:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_hinges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:56:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mcsloxyvsp-gas-cylinder-shoulder-label>: HTTP status code is not handled or not allowed 2026-01-25 01:56:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_closers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:56:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/office_machines/label_makers landed on page that is not a product page. 2026-01-25 01:56:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/packaging/bags_covers/poly_bags_tubing_sheeting/reclosable_bags already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:56:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/740x-portable-therapeutic-ultrasound-unit-with-3-applicators>: HTTP status code is not handled or not allowed 2026-01-25 01:56:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/thresholds_bottom_sweeps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:56:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coffer-23-3-4-x-23-3-4-pvc-lay-in-tile-in-matte-white-pl6101>: HTTP status code is not handled or not allowed 2026-01-25 01:56:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_hinges landed on page that is not a product page. 2026-01-25 01:56:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-finish-nails-sf16-25-16-gauge-2-1-2-1000-bx>: HTTP status code is not handled or not allowed 2026-01-25 01:56:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_hinges landed on page that is not a product page. 2026-01-25 01:56:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/border-fill-23-3-4-x-23-3-4-pvc-lay-in-tile-in-polished-copper-pl5925>: HTTP status code is not handled or not allowed 2026-01-25 01:56:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_closers landed on page that is not a product page. 2026-01-25 01:56:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-platter-cart-12-trays-10-inch>: HTTP status code is not handled or not allowed 2026-01-25 01:56:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/packaging/bags_covers/poly_bags_tubing_sheeting/reclosable_bags landed on page that is not a product page. 2026-01-25 01:56:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012704-blower-assembly-115v-for-cres-cor>: HTTP status code is not handled or not allowed 2026-01-25 01:56:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/door_window_hardware/thresholds_bottom_sweeps landed on page that is not a product page. 2026-01-25 01:56:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-platter-cart-10-trays-12-inch>: HTTP status code is not handled or not allowed 2026-01-25 01:56:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26x26-1-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 01:56:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kee-klamp-side-outlet-tee-4>: HTTP status code is not handled or not allowed 2026-01-25 01:56:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kee-klamp-double-swivel-socket-2>: HTTP status code is not handled or not allowed 2026-01-25 01:56:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milwaukee-174-48-11-1812-m18-174-redlithium-174-high-output-12-0ah-battery-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:56:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kee-klamp-tee-crossover-2>: HTTP status code is not handled or not allowed 2026-01-25 01:56:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kee-klamp-corner-swivel-socket-2>: HTTP status code is not handled or not allowed 2026-01-25 01:56:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-x725-6-mil-poly-tubing>: HTTP status code is not handled or not allowed 2026-01-25 01:56:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-socket-cross-1-5-pipe-fitting>: HTTP status code is not handled or not allowed 2026-01-25 01:56:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kee-klamp-swivel-fitting-1>: HTTP status code is not handled or not allowed 2026-01-25 01:56:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kee-klamp-plug-2>: HTTP status code is not handled or not allowed 2026-01-25 01:56:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-glasses-mirror-lens-clear-mirror-frame>: HTTP status code is not handled or not allowed 2026-01-25 01:56:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010783-motor-1-4-hp-for-stero-dishwashers>: HTTP status code is not handled or not allowed 2026-01-25 01:56:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-marker-booklet-a-z-0-15-symbols>: HTTP status code is not handled or not allowed 2026-01-25 01:57:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergo-mat-grande-2-ft-x-custom-lengths-black>: HTTP status code is not handled or not allowed 2026-01-25 01:57:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poly-truck-bumper-for-8-bu-poly-truck-r08-kkx-pbn>: HTTP status code is not handled or not allowed 2026-01-25 01:57:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010739-hi-limit-for-star-manufacturing>: HTTP status code is not handled or not allowed 2026-01-25 01:57:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/marble-sof-tyle-grande-mat-3-ft-x-custom-length-grey>: HTTP status code is not handled or not allowed 2026-01-25 01:57:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010748-timer-120-minute-for-star-manufacturing>: HTTP status code is not handled or not allowed 2026-01-25 01:57:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36x18x12-double-wall-boxes>: HTTP status code is not handled or not allowed 2026-01-25 01:57:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:57:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tear-down-bench-30x48x29-1-2-red>: HTTP status code is not handled or not allowed 2026-01-25 01:57:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16x14x10-double-wall-boxes>: HTTP status code is not handled or not allowed 2026-01-25 01:57:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 49957 pages (at 92 pages/min), scraped 25339 items (at 47 items/min) 2026-01-25 01:57:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-8-inch-carbon-steel-machined-drop-in-ball-transfer>: HTTP status code is not handled or not allowed 2026-01-25 01:57:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tutco-crankcase-54-120-39-58>: HTTP status code is not handled or not allowed 2026-01-25 01:57:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/packard-foot-mount-120208-240va-output-20va>: HTTP status code is not handled or not allowed 2026-01-25 01:57:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents landed on page that is not a product page. 2026-01-25 01:57:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-plastic-chain-safety-pink>: HTTP status code is not handled or not allowed 2026-01-25 01:57:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acrylic-mirror-160-degree-outdoor-36-dia>: HTTP status code is not handled or not allowed 2026-01-25 01:57:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/hvac_system_parts_components/replacement_fan_blades_blower_wheels/repair_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:57:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-14-x-54-white-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:57:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inox-flex-grinding-wheel-36-grit-type-27-4-1-2-dia-x-5-32-x-7-8-cntr-holedia-25-pk>: HTTP status code is not handled or not allowed 2026-01-25 01:57:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/561267-3-4-inch-backflow-prevent>: HTTP status code is not handled or not allowed 2026-01-25 01:57:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/circulation_reception_desks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:57:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergency-shower-mat-kit-red-w-yellow-edging-5-8in-x-2ft-3in-x-3ft-6in>: HTTP status code is not handled or not allowed 2026-01-25 01:57:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/hvac_system_parts_components/replacement_fan_blades_blower_wheels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:57:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/144-x-36-extra-long-bench-plastic-top>: HTTP status code is not handled or not allowed 2026-01-25 01:57:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-heavy-duty-boltless-shelving-add-on-48in-w-x-12in-d-x-60in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-25 01:57:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enclosure-kit-slide-door-14-x-36-x-16-white>: HTTP status code is not handled or not allowed 2026-01-25 01:57:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/desks_workstations/circulation_reception_desks>: HTTP status code is not handled or not allowed 2026-01-25 01:57:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-14-x-72-black-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 01:57:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/hvac_system_parts_components/replacement_fan_blades_blower_wheels landed on page that is not a product page. 2026-01-25 01:57:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16x24-2-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 01:57:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-open-single-solenoid-valve-ext-pilot-g-1-4-110v-ac-5va-coil-led-connection>: HTTP status code is not handled or not allowed 2026-01-25 01:57:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enclosure-kit-slide-door-14-x-36-x-16-light-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:57:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-heavy-duty-boltless-shelving-starter-36inw-x-18ind-x-96inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 01:57:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a41a-repair-kit-16-closet>: HTTP status code is not handled or not allowed 2026-01-25 01:57:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-work-shirt-4x-ln-light-blue-tcg01190232>: HTTP status code is not handled or not allowed 2026-01-25 01:57:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v551a-vac-brkr-repair-kit-v500>: HTTP status code is not handled or not allowed 2026-01-25 01:57:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-dbl-solenoid-valve-pilot-spring-closed-cntr-1-8-nptf-220v-ac-5va-coil-black-conn>: HTTP status code is not handled or not allowed 2026-01-25 01:57:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9248-01-deluxe-supplied-air-shieldhelmet-system-1-worker-50-hose-adf>: HTTP status code is not handled or not allowed 2026-01-25 01:57:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/342290-evaporator-15-qt-240v-1500w>: HTTP status code is not handled or not allowed 2026-01-25 01:57:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vacuumblowers-computersappliances-metrodatavacelectricduster-261430>: HTTP status code is not handled or not allowed 2026-01-25 01:57:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/461544-control-temperature>: HTTP status code is not handled or not allowed 2026-01-25 01:57:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/translucent-liner-14-x-60>: HTTP status code is not handled or not allowed 2026-01-25 01:57:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-heavy-duty-boltless-shelving-starter-42inw-x-15ind-x-96inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 01:57:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sc-drawer-layout-12-compartments-5-5-w-x-3-h>: HTTP status code is not handled or not allowed 2026-01-25 01:58:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conductive-anti-static-hand-scoop-black-135-x-185-x-310mm-750g>: HTTP status code is not handled or not allowed 2026-01-25 01:58:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coveralls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:58:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/511131-thermocouple-baso-slmjim-48-for-johnson-controls>: HTTP status code is not handled or not allowed 2026-01-25 01:58:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enclosure-kit-slide-door-24-x-60-x-18-dark-yellow>: HTTP status code is not handled or not allowed 2026-01-25 01:58:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-heavy-duty-boltless-shelving-add-on-48inw-x-18ind-x-72inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 01:58:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/rack_protectors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:58:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2291238-protection-cap-for-garland-range>: HTTP status code is not handled or not allowed 2026-01-25 01:58:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 50051 pages (at 94 pages/min), scraped 25397 items (at 58 items/min) 2026-01-25 01:58:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uf-b-underground-feeder-cable-8-3-awg>: HTTP status code is not handled or not allowed 2026-01-25 01:58:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thhn-8-gauge-building-wire-stranded-type-gray>: HTTP status code is not handled or not allowed 2026-01-25 01:58:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-high-capacity-bulk-rack-with-wire-decking-96x36x120-add-on>: HTTP status code is not handled or not allowed 2026-01-25 01:58:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/rack_protectors>: HTTP status code is not handled or not allowed 2026-01-25 01:58:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coveralls landed on page that is not a product page. 2026-01-25 01:58:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-1-4-npt-double-solenoid-valve-pilot-110v-ac-5va-coil-black-connection>: HTTP status code is not handled or not allowed 2026-01-25 01:58:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-add-on-72inw-x-30ind-x-72inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 01:58:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-in-w-x-76-in-h-panel-with-electric-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:58:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-in-w-x-46-in-h-panel-with-electric-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:58:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-open-single-solenoid-valve-ext-pilot-g-1-8-220v-ac-5va-coil-led-connection>: HTTP status code is not handled or not allowed 2026-01-25 01:58:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drink-mat-13984-nfl-denver-broncos-3-1-4-x-24-x-1>: HTTP status code is not handled or not allowed 2026-01-25 01:58:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/bar_restaurant_equipment/bartending_supplies/bar_mats_shelf_liners already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:58:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-hose-double-male-hose-nipple-358-2562521-2-1-2-npt-x-2-1-2-nh-brass>: HTTP status code is not handled or not allowed 2026-01-25 01:58:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-closed-single-solenoid-valve-ext-pilot-g-1-4-220v-ac-5va-coil-black-connection>: HTTP status code is not handled or not allowed 2026-01-25 01:58:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2411018-fuse-20a-for-panasonic-microwave>: HTTP status code is not handled or not allowed 2026-01-25 01:58:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kit2cv083x11-8-cal-cm2-arc-flash-kit-with-fr-coverall-3xl-glove-size-11>: HTTP status code is not handled or not allowed 2026-01-25 01:58:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-pressure-cntr-g1-2-dbl-solenoid-valve-pilot-spr-cntred-24v-ac-5va-coil-black-conn>: HTTP status code is not handled or not allowed 2026-01-25 01:58:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/bar_restaurant_equipment/bartending_supplies/bar_mats_shelf_liners landed on page that is not a product page. 2026-01-25 01:58:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sleeve-anchor-3-8-x-5-phillips-slotted-combo-flat-head-steel-zinc-pkg-of-50-277090>: HTTP status code is not handled or not allowed 2026-01-25 01:58:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sleeve-anchor-3-8-x-6-phillips-slotted-combo-flat-head-steel-zinc-pkg-of-50-277100>: HTTP status code is not handled or not allowed 2026-01-25 01:58:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nesting-basket-127-12-chrome-plated-steel-26l-x-10w-x-2h>: HTTP status code is not handled or not allowed 2026-01-25 01:58:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-starter-96inw-x-36ind-x-60inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 01:58:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1241398-coil-evap-blk-w-2-fans-for-russell>: HTTP status code is not handled or not allowed 2026-01-25 01:58:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-hose-reel-cover-138-8-26-x-10-red-vinyl-for-1431-5>: HTTP status code is not handled or not allowed 2026-01-25 01:58:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mesh-with-drawstring-closure-yellow-24-36-heavy>: HTTP status code is not handled or not allowed 2026-01-25 01:58:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/divider-10-7-8-d-x-5-h-premium-stacking-bin-pack-of-6-1>: HTTP status code is not handled or not allowed 2026-01-25 01:58:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gardner-bender-poly-pull8482-pull-line-6500-pl232b>: HTTP status code is not handled or not allowed 2026-01-25 01:58:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-work-shirt-3xl-light-blue-tcg01190228>: HTTP status code is not handled or not allowed 2026-01-25 01:59:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-closed-single-solenoid-valve-pilot-spr-return-g-1-4-220v-ac-5va-coil-led-conn>: HTTP status code is not handled or not allowed 2026-01-25 01:59:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-holder-cartridge-kit-d5028-4-pk>: HTTP status code is not handled or not allowed 2026-01-25 01:59:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-uv-lamp-for-genpure-and-labtower-edi-uv-systems-only>: HTTP status code is not handled or not allowed 2026-01-25 01:59:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-s-brass-pre-rinse-assembly-44-w-24-riser>: HTTP status code is not handled or not allowed 2026-01-25 01:59:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/haws-horizontal-or-vertical-drench-shower-axion-msr-abs-plastic-showerhead>: HTTP status code is not handled or not allowed 2026-01-25 01:59:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 50150 pages (at 99 pages/min), scraped 25464 items (at 67 items/min) 2026-01-25 01:59:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-18-ga-48-wx24-dx85-h-open-clip-style-5-shelf>: HTTP status code is not handled or not allowed 2026-01-25 01:59:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/66102-brew-basket-for-newco-enterprises>: HTTP status code is not handled or not allowed 2026-01-25 01:59:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-open-single-solenoid-valve-pilot-spr-return-1-2-nptf-110v-ac-5va-coil-blk-conn>: HTTP status code is not handled or not allowed 2026-01-25 01:59:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-58-outdoor-ball-fan-208-230v-1075rpm-1hp>: HTTP status code is not handled or not allowed 2026-01-25 01:59:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-cal-flame-resistant-coveralls-2xl-navy-tcg02160873>: HTTP status code is not handled or not allowed 2026-01-25 01:59:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8460-containment-berm-foam-wall-12-feet-by-50-feet-22-ounce-pvc>: HTTP status code is not handled or not allowed 2026-01-25 01:59:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-safety-view-locker-triple-tier-12x18x24-3-doors-assembled-parchment>: HTTP status code is not handled or not allowed 2026-01-25 01:59:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-safety-view-locker-triple-tier-12x18x24-9-doors-assembled-parchment>: HTTP status code is not handled or not allowed 2026-01-25 01:59:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-safety-view-locker-six-tier-12x15x12-18-doors-unassembled-parchment>: HTTP status code is not handled or not allowed 2026-01-25 01:59:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-closed-1-2-nptf-single-solenoid-valve-pilot-spr-return-110v-ac-5va-coil-blk-conn>: HTTP status code is not handled or not allowed 2026-01-25 01:59:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ptfe-syringe-filters-non-sterile-0-45um-25mm-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 01:59:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/282310-compressor-rear-8oz-for-hollymatic>: HTTP status code is not handled or not allowed 2026-01-25 01:59:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-dbl-solenoid-valve-pilot-spring-cntred-closed-cntr-g-1-2-24v-dc-3w-coil-led-conn>: HTTP status code is not handled or not allowed 2026-01-25 01:59:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-unassembled-storage-cabinet-recessed-handle36w-x-24d-x-72h-black>: HTTP status code is not handled or not allowed 2026-01-25 01:59:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pegboard-hobby-craft-organizer-storage-kit-galvanized-black-32-x-32-x-9>: HTTP status code is not handled or not allowed 2026-01-25 01:59:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-rack-slotted-upright-frame-3-channel-36x192>: HTTP status code is not handled or not allowed 2026-01-25 01:59:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-starter-48inw-x-48ind-x-96inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 01:59:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-starter-60inw-x-36ind-x-84inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 01:59:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-shelf-heavy-duty-boltless-shelving-add-on-72inw-x-30ind-x-36inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 01:59:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-backless-counter-height-stool-metal-square-crystal-teal-blue>: HTTP status code is not handled or not allowed 2026-01-25 01:59:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/five-tier-box-locker-with-legs-3-wide-12w-x-18d-x-12h-assembled-putty>: HTTP status code is not handled or not allowed 2026-01-25 01:59:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2561024-body-valve-for-silver-king>: HTTP status code is not handled or not allowed 2026-01-25 01:59:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-double-solenoid-valve-ext-pilot-1-4-nptf-24v-dc-2w-coil-black-connection>: HTTP status code is not handled or not allowed 2026-01-25 01:59:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-2-single-solenoid-valve-ext-pilot-spring-return-1-8-nptf-24v-dc-2w-coil-led-conn>: HTTP status code is not handled or not allowed 2026-01-25 01:59:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-add-on-60inw-x-36ind-x-60inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 01:59:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/laser-ink-jet-insert-letter-pref-1-5-8-in-8-in-300-per-package>: HTTP status code is not handled or not allowed 2026-01-25 01:59:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/starbucks-caffe-verona-ground-coffee-032-oz-pouch-pack-of-76>: HTTP status code is not handled or not allowed 2026-01-25 01:59:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/76-662-1-light-pendant-warehouse-old-bronze-16w-x-8-125h>: HTTP status code is not handled or not allowed 2026-01-25 01:59:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 01:59:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-king-sk3000-pallet-rack-108w-x-42d-x-96h-4040-lbs-cap-per-level-starter-unit>: HTTP status code is not handled or not allowed 2026-01-25 02:00:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14ab-x-1-1-4-security-sheet-metal-screw-flat-torx-head-302hq-stainless-steel-usa-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 02:00:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:00:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12ab-x-1-2-security-sheet-metal-screw-button-torx-head-302hq-stainless-steel-usa-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 02:00:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10ab-x-1-1-2-security-sheet-metal-screw-flat-hex-socket-head-302hq-18-8-ss-usa-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 02:00:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits>: HTTP status code is not handled or not allowed 2026-01-25 02:00:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usb-serial-adapter-usb-db9m>: HTTP status code is not handled or not allowed 2026-01-25 02:00:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8000va-smart-online-ups-rack-tower-puresine-208v-8u-18-outlets-8kva>: HTTP status code is not handled or not allowed 2026-01-25 02:00:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 50245 pages (at 95 pages/min), scraped 25524 items (at 60 items/min) 2026-01-25 02:00:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits landed on page that is not a product page. 2026-01-25 02:00:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-inch-buna-n-cam-and-groove-gasket>: HTTP status code is not handled or not allowed 2026-01-25 02:00:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-316-ballerina-2-light-vanity-wall-alabaster-polished-chrome-18w-x-7-625h>: HTTP status code is not handled or not allowed 2026-01-25 02:00:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2721313-sprocket-clutched-13-tooth-for-nieco-corp>: HTTP status code is not handled or not allowed 2026-01-25 02:00:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/76-501-add-on-motion-sensor-black>: HTTP status code is not handled or not allowed 2026-01-25 02:00:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fr-mesh-vest-c2-lime-l-xl>: HTTP status code is not handled or not allowed 2026-01-25 02:00:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8014233-blade-type-a-set-of-32>: HTTP status code is not handled or not allowed 2026-01-25 02:00:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-level-wide-span-rack-96x36-wood-deck-800-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 02:00:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wheel-chock-hanger-14-in-w-x-5-in-d-x-3-in-h>: HTTP status code is not handled or not allowed 2026-01-25 02:00:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-t-shirt-long-sleeve-1-pocket-fl-lime-xl>: HTTP status code is not handled or not allowed 2026-01-25 02:00:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/omega-womens-safety-work-boots-steel-toe-6h-size-95-black>: HTTP status code is not handled or not allowed 2026-01-25 02:00:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aware-wear-ansi-107-class-headwear-61588-orange>: HTTP status code is not handled or not allowed 2026-01-25 02:00:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:00:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gravity-womens-safety-work-boots-steel-toe-4h-size-7w-black>: HTTP status code is not handled or not allowed 2026-01-25 02:00:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-t-shirt-short-sleeve-1-pocket-fl-lime-5xl>: HTTP status code is not handled or not allowed 2026-01-25 02:00:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2661009-brush-double-ended-for-taylor-freezer>: HTTP status code is not handled or not allowed 2026-01-25 02:00:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/garden-lead-in-hose-58-id-x-5l-zilla-green>: HTTP status code is not handled or not allowed 2026-01-25 02:00:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/thermometers_temperature_instruments already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:00:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aware-wear-non-ansi-vest-61320-lime>: HTTP status code is not handled or not allowed 2026-01-25 02:00:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-steel-wheel-riser-60-x-18-x-10>: HTTP status code is not handled or not allowed 2026-01-25 02:00:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aware-wear-non-ansi-hi-viz-t-shirt-14211-lime>: HTTP status code is not handled or not allowed 2026-01-25 02:00:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/floor_machine_replacement_parts/vacuum_hoses already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:00:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-25 02:00:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gravity-womens-safety-work-boots-steel-toe-4h-size-65-cognac-brown>: HTTP status code is not handled or not allowed 2026-01-25 02:00:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20v-max-cordless-2-tool-kit-including-hammer-drill-driver-with-flexvolt-advantage>: HTTP status code is not handled or not allowed 2026-01-25 02:00:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-609-polaris-2-light-ceiling-semi-flush-opal-white-brushed-nickel-13-75w-x-8h>: HTTP status code is not handled or not allowed 2026-01-25 02:00:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/77-099-2-light-ceiling-flush-mount-large-crystal-white-drum-white-10w-x-7h>: HTTP status code is not handled or not allowed 2026-01-25 02:00:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-3147-3-light-ceiling-flush-mount-frosted-white-mahogany-bronze-15-25w-x-6h>: HTTP status code is not handled or not allowed 2026-01-25 02:00:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buckhorn-tl4844030051000-48x44-agricultural-bulk-box-lid-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:00:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/vacuums/canister_vacuums already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:00:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/testing_inspection_instruments/thermometers_temperature_instruments landed on page that is not a product page. 2026-01-25 02:00:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/floor_machine_replacement_parts/vacuum_hoses>: HTTP status code is not handled or not allowed 2026-01-25 02:00:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011227-cap-tube-042idx-093od-x-144-inch-for-beverage-air>: HTTP status code is not handled or not allowed 2026-01-25 02:00:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-use-table-transport-dolly-cart-brown>: HTTP status code is not handled or not allowed 2026-01-25 02:00:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/881310-refrigeration-hard-startkit>: HTTP status code is not handled or not allowed 2026-01-25 02:00:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-boltless-shelving-add-on-48inw-x-24ind-x-72inh-no-deck>: HTTP status code is not handled or not allowed 2026-01-25 02:00:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-472-car-port-2-light-flush-mount-texture-frosted-old-bronze-9-25w>: HTTP status code is not handled or not allowed 2026-01-25 02:00:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011301-drain-fitting-for-ice-o-matic>: HTTP status code is not handled or not allowed 2026-01-25 02:00:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/vacuums/canister_vacuums landed on page that is not a product page. 2026-01-25 02:00:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/starter-unit-legal-31-inch-45-1-2-inch-61-inch-5-opening-black>: HTTP status code is not handled or not allowed 2026-01-25 02:00:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1381153-probe-fryvat-k-type-for-cooper-atkins>: HTTP status code is not handled or not allowed 2026-01-25 02:00:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9618a-replacement-blower-motor>: HTTP status code is not handled or not allowed 2026-01-25 02:00:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/automotive-t-style-coupler-3-8-x-3-8-female-npt-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 02:01:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ego-lm2102sp-g3-1p-2-5ah-battery>: HTTP status code is not handled or not allowed 2026-01-25 02:01:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-boltless-shelving-add-on-36inw-x-18ind-x-72inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 02:01:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-way-t-style-air-manifold-with-industrial-6-ball-brass-couplers-1-4>: HTTP status code is not handled or not allowed 2026-01-25 02:01:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/266297-roller-1-inch-od-1-4-inch-id-nylon-tire>: HTTP status code is not handled or not allowed 2026-01-25 02:01:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3m-matic-uniform-case-sealer-700a>: HTTP status code is not handled or not allowed 2026-01-25 02:01:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/humidifiers_dehumidifiers/dehumidifiers_desiccators already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:01:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-boltless-shelving-starter-48inw-x-12ind-x-36inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 02:01:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 50338 pages (at 93 pages/min), scraped 25572 items (at 48 items/min) 2026-01-25 02:01:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-boltless-shelving-starter-48inw-x-24ind-x-60inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 02:01:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-ft-30-50-lbs-tool-balancer-with-cable>: HTTP status code is not handled or not allowed 2026-01-25 02:01:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inside-stational-deck-dome-oven-pentagonal-exterior-39-gas-92500btu-120v-digital-control>: HTTP status code is not handled or not allowed 2026-01-25 02:01:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/humidifiers_dehumidifiers/dehumidifiers_desiccators landed on page that is not a product page. 2026-01-25 02:01:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/material-handling/casters_wheels/wheels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:01:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012436-open-top-burner-rest-for-garland-range>: HTTP status code is not handled or not allowed 2026-01-25 02:01:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-handtruck-fbal14-10fpn-loop-handle-pneumatic-wheels-600-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 02:01:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drum-with-epoxy-phenolic-lining-open-head-2>: HTTP status code is not handled or not allowed 2026-01-25 02:01:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-boltless-shelving-add-on-48inw-x-24ind-x-72inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 02:01:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-boltless-shelving-add-on-36inw-x-24ind-x-84inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 02:01:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/solid-blue-curtain-wall-partition-6-x-8>: HTTP status code is not handled or not allowed 2026-01-25 02:01:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/material-handling/casters_wheels/wheels landed on page that is not a product page. 2026-01-25 02:01:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/561391-self-tapping-water-valve>: HTTP status code is not handled or not allowed 2026-01-25 02:01:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safe-b-rate-vault-door-digital-lock-40-w-x-81-h-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:01:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/outdoor_lighting_fixtures/wall_pack_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:01:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/security_access_control/safes_lockboxes/security_safes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:01:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-3-shelf-boltless-shelving-starter-48inw-x-24ind-x-72inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 02:01:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-style-c-j-hook>: HTTP status code is not handled or not allowed 2026-01-25 02:01:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-ft-square-outdoor-tree-bench-expanded-metal-surface-mount-green>: HTTP status code is not handled or not allowed 2026-01-25 02:01:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-a-frame-bin-cart-2-pegboard-panels-48w-x-30d-x-62h-red>: HTTP status code is not handled or not allowed 2026-01-25 02:01:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/outdoor_lighting_fixtures/wall_pack_lights>: HTTP status code is not handled or not allowed 2026-01-25 02:01:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-safety-view-locker-triple-tier-12x15x24-3-doors-unassembled-tan>: HTTP status code is not handled or not allowed 2026-01-25 02:01:45 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/security_access_control/safes_lockboxes/security_safes landed on page that is not a product page. 2026-01-25 02:01:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-safety-view-locker-4-person-wall-12x18x12-4-doors-unassembled-tan>: HTTP status code is not handled or not allowed 2026-01-25 02:01:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/342295-element-calrod-58-inch-125w120v>: HTTP status code is not handled or not allowed 2026-01-25 02:01:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aaa-size-duracell-procell-battery>: HTTP status code is not handled or not allowed 2026-01-25 02:01:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-outside-tuck-mailers-14-12l-x-7-14w-x-5h-white-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-25 02:01:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-mailers-6l-x-5w-x-2h-kraft-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-25 02:01:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010182-switch-micro-250f>: HTTP status code is not handled or not allowed 2026-01-25 02:01:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012738-microswitch-for-merry-chef>: HTTP status code is not handled or not allowed 2026-01-25 02:01:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torque-limiting-extension-1-2-drive-100-ft-lbs>: HTTP status code is not handled or not allowed 2026-01-25 02:01:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:01:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/legal-size-fireproof-file-cabinet-21-w-x-25-d-x-28-h-black>: HTTP status code is not handled or not allowed 2026-01-25 02:02:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/legal-size-fireproof-file-cabinet-21-w-x-25-d-x-28-h-sand>: HTTP status code is not handled or not allowed 2026-01-25 02:02:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/wall_corner_bumper_guards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:02:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/federal-signal-explosion-proof-strobe-light-120-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:02:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/garage-creeper>: HTTP status code is not handled or not allowed 2026-01-25 02:02:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats>: HTTP status code is not handled or not allowed 2026-01-25 02:02:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/horizontal-baby-changing-table-light-gray-stainless-5240259>: HTTP status code is not handled or not allowed 2026-01-25 02:02:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 50430 pages (at 92 pages/min), scraped 25630 items (at 58 items/min) 2026-01-25 02:02:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/wall_corner_bumper_guards>: HTTP status code is not handled or not allowed 2026-01-25 02:02:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-stand-pin-style-22-ton>: HTTP status code is not handled or not allowed 2026-01-25 02:02:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7a19-50-7w-compact-fluorescent-w-medium-base-natural-light>: HTTP status code is not handled or not allowed 2026-01-25 02:02:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wheel-semi-steel-8x2-with-1-2-axle>: HTTP status code is not handled or not allowed 2026-01-25 02:02:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grease-gun-standard>: HTTP status code is not handled or not allowed 2026-01-25 02:02:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bridges_returns already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:02:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hydraulic-axle-jack-25-10-ton-2-stage-air>: HTTP status code is not handled or not allowed 2026-01-25 02:02:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fido-me-fountain-pbark-498>: HTTP status code is not handled or not allowed 2026-01-25 02:02:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f8t5-ww-8w-fluorescent-w-minature-bi-pin-base-warm-white>: HTTP status code is not handled or not allowed 2026-01-25 02:02:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cfd13w-4p-830-13w-compact-fluorescent-w-g24vq-1-base-warm>: HTTP status code is not handled or not allowed 2026-01-25 02:02:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s2500-otg-anti-fog-safety-glasses-black-frame-clear-lens-scratch-resistant>: HTTP status code is not handled or not allowed 2026-01-25 02:02:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tilt-out-storage-bin-2-compartments-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:02:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/compact_fluorescent_light_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:02:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/separable-roller-follower-metric-nast30zzuu-double-sealed-30-mm-bore-62-mm-od>: HTTP status code is not handled or not allowed 2026-01-25 02:02:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roller-follower-full-comp-inch-cry40vuu-double-sealed-2-12-od>: HTTP status code is not handled or not allowed 2026-01-25 02:02:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rim-wit-truck-wheel-and-rim-removal-tool-for-use-with-22-1-2-24-1-2-tires>: HTTP status code is not handled or not allowed 2026-01-25 02:02:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bridges_returns landed on page that is not a product page. 2026-01-25 02:02:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/long-cardboard-corrugated-boxes-32l-x-10w-x-6-12h-kraft-pack-of-20>: HTTP status code is not handled or not allowed 2026-01-25 02:02:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-ton-ironworker-3-phase-460v-iw60-3p460>: HTTP status code is not handled or not allowed 2026-01-25 02:02:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20v-max-compact-power-tool-battery-4-0ah-lithium-ion>: HTTP status code is not handled or not allowed 2026-01-25 02:02:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/travira-32-bistro-table-tekwood-natural>: HTTP status code is not handled or not allowed 2026-01-25 02:02:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/compact_fluorescent_light_bulbs landed on page that is not a product page. 2026-01-25 02:02:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20v-max-xr-fencing-stapler-9-gauge-for-use-with-1-2-w-fencing-staples-35-capacity>: HTTP status code is not handled or not allowed 2026-01-25 02:02:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_seating/patio_dining_chairs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:02:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42x24-wd-abrasive-blast-cabinet-direct-pressure-101696wd-a>: HTTP status code is not handled or not allowed 2026-01-25 02:02:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-double-square-decorative-light-fixture-20w-304050k-120v-13l-blackgold>: HTTP status code is not handled or not allowed 2026-01-25 02:02:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/51-1429-baso-coaxial-thermocouple-36-inch>: HTTP status code is not handled or not allowed 2026-01-25 02:02:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cut-resistant-gloves-micro-foam-nitrile-coated-ansi-a2-xl-hi-vis-lime>: HTTP status code is not handled or not allowed 2026-01-25 02:02:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tote-box-accessory-long-divider-16>: HTTP status code is not handled or not allowed 2026-01-25 02:02:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-1-thick-x-2-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 02:02:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bicycle-parking-rack-adjustable-5-bike-capacity-single-sided-version>: HTTP status code is not handled or not allowed 2026-01-25 02:02:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/481166-bimetal-hi-limit-for-duke-manufacturing>: HTTP status code is not handled or not allowed 2026-01-25 02:02:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rlph1-long-sleeve-pullover-hoodie-with-uv-protection-class-3-5xl-hi-vis-lime>: HTTP status code is not handled or not allowed 2026-01-25 02:02:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/x10t3-1-4-10w-miniature-sealed-beams-w-festoon-base-12v>: HTTP status code is not handled or not allowed 2026-01-25 02:02:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-filtered-exhaust-fan-direct-drive-totally-enclosed-3-phase-1-hp>: HTTP status code is not handled or not allowed 2026-01-25 02:02:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-filtered-exhaust-fan-direct-drive-totally-enclosed-1-phase-1-hp>: HTTP status code is not handled or not allowed 2026-01-25 02:02:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/strain-relief-cord-connectors-3>: HTTP status code is not handled or not allowed 2026-01-25 02:02:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-mat-shelf-21-in-x-48-in-nexelon>: HTTP status code is not handled or not allowed 2026-01-25 02:02:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009292-ignition-module-for-southbend-range>: HTTP status code is not handled or not allowed 2026-01-25 02:02:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-mat-shelf-21-in-x-72-in-nexelon>: HTTP status code is not handled or not allowed 2026-01-25 02:02:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_seating/patio_dining_chairs landed on page that is not a product page. 2026-01-25 02:02:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uhmw-polyethylene-plastic-sheet-3-4-thick-x-24-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 02:02:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/turbo-lite-mini-torch-yellow>: HTTP status code is not handled or not allowed 2026-01-25 02:02:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tripower-classical-section-molded-notch-vbelt-gates-ax66>: HTTP status code is not handled or not allowed 2026-01-25 02:02:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-pin-compact-fluorescent-light-bulb-2g11-base-3500k-40w-neutral-white-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 02:02:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pedestrian-strip-door-curtain-7-w-x-8-h>: HTTP status code is not handled or not allowed 2026-01-25 02:02:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-strip-door-curtain-9-w-x-9-h>: HTTP status code is not handled or not allowed 2026-01-25 02:02:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drip-dam-leak-diverter-12-x-12-x-1-4-461212-ye>: HTTP status code is not handled or not allowed 2026-01-25 02:02:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibco-silent-pneumatic-turbine-vibrator-mltss-32-316l>: HTTP status code is not handled or not allowed 2026-01-25 02:02:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/snap-up-stinger-berm-compact-12-x-20-x-1-48-12201-bk-su>: HTTP status code is not handled or not allowed 2026-01-25 02:02:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uhmw-polyethylene-plastic-bar-1-thick-x-4-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 02:02:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poly-spill-pad-with-grate-48-x-48-x-2-5620-ye>: HTTP status code is not handled or not allowed 2026-01-25 02:03:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t8-fluorescent-bulb-g13-base-32w-3050-lumens-5000k-daylight-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 02:03:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/tool_holders/tapping_heads_holders_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:03:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-linea-l-desk-and-electric-height-adjustable-table-white>: HTTP status code is not handled or not allowed 2026-01-25 02:03:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uhmw-polyethylene-plastic-bar-3-8-thick-x-1-1-2-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 02:03:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/250-3-4-dia-6x19-improved-plow-stee-galvanized-wire-rope-independent-wire-rope-center>: HTTP status code is not handled or not allowed 2026-01-25 02:03:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/omni-pir-ceiling-low-voltage-sensor-with-1500-sq-ft-range-off-white>: HTTP status code is not handled or not allowed 2026-01-25 02:03:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 50525 pages (at 95 pages/min), scraped 25671 items (at 41 items/min) 2026-01-25 02:03:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-flush-mount-ceiling-light-fixture-60w-7-1516-dia-white>: HTTP status code is not handled or not allowed 2026-01-25 02:03:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/fluorescent_linear_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:03:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/tool_holders/tapping_heads_holders_accessories landed on page that is not a product page. 2026-01-25 02:03:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-safety-knife-spring-loaded-with-6-blades>: HTTP status code is not handled or not allowed 2026-01-25 02:03:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/fluorescent_linear_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:03:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-flush-mount-ceiling-light-fixture-60w-11-14-dia-brushed-nickel>: HTTP status code is not handled or not allowed 2026-01-25 02:03:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:03:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ushio-f17t8-850-24in86cri>: HTTP status code is not handled or not allowed 2026-01-25 02:03:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milton-s-1803w-p-style-automotive-coupler-1-4-fnpt>: HTTP status code is not handled or not allowed 2026-01-25 02:03:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-welded-steel-container-unpainted-18>: HTTP status code is not handled or not allowed 2026-01-25 02:03:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-1-2-thick-x-4-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 02:03:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milton-s-716w-m-style-industrial-coupler-1-4-mnpt>: HTTP status code is not handled or not allowed 2026-01-25 02:03:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/fluorescent_linear_bulbs landed on page that is not a product page. 2026-01-25 02:03:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-container-forkli-able-yellow-1>: HTTP status code is not handled or not allowed 2026-01-25 02:03:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powerlift-hitch-hauler-for-pl65-and-pl20-maintenance-platforms>: HTTP status code is not handled or not allowed 2026-01-25 02:03:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 02:03:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milton-s-1818w-g-style-industrial-plug-1-2-fnpt>: HTTP status code is not handled or not allowed 2026-01-25 02:03:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/incandescent-bulb-with-e12-candelabra-base-7w-120v-white-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-25 02:03:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/881319-filter-drier-liquid-line-for-emerson-radio-corp>: HTTP status code is not handled or not allowed 2026-01-25 02:03:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/fluorescent_linear_bulbs landed on page that is not a product page. 2026-01-25 02:03:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-rack-accessory-bin-5>: HTTP status code is not handled or not allowed 2026-01-25 02:03:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-par16-recessed-floodlight-bulb-e26-base-6w-500-lumens-30k-warm-white-pk-of-6>: HTTP status code is not handled or not allowed 2026-01-25 02:03:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-b13-torpedo-clear-light-bulb-e26-base-7w-500-lumens-30k-warm-white-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 02:03:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-mobile-wire-bin-rack-48x18x69>: HTTP status code is not handled or not allowed 2026-01-25 02:03:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-g2sc-275-b>: HTTP status code is not handled or not allowed 2026-01-25 02:03:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chain-bag-assembly-jlc-hoist-35-ft>: HTTP status code is not handled or not allowed 2026-01-25 02:03:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wireway-pallet-rack-tear-drop-upright-frame-96x48>: HTTP status code is not handled or not allowed 2026-01-25 02:03:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wireway-pallet-rack-tear-drop-upright-frame-120x42>: HTTP status code is not handled or not allowed 2026-01-25 02:03:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fifo-flow-rack-m53-roller-top-rear-extension-esd-50-7-20-w-x-30-4-25-x-4-10-77-h-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:03:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2721297-valve-solenoid-24vac-for-nieco-corp>: HTTP status code is not handled or not allowed 2026-01-25 02:03:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1381259-label-temperature-160f-for-taylor-precision-products-lp>: HTTP status code is not handled or not allowed 2026-01-25 02:03:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0353rwhtxlg-ergo-insulated-goatskin-gloves-xl>: HTTP status code is not handled or not allowed 2026-01-25 02:03:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grip-16-w-1-step-steel-rolling-ladder-10-d-top-step-2>: HTTP status code is not handled or not allowed 2026-01-25 02:03:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grip-16-w-3-step-steel-rolling-ladder-10-d-top-step-2>: HTTP status code is not handled or not allowed 2026-01-25 02:03:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/self_luminous_emergency_exit_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:03:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-inch-long-steel-back-white-deck-scrub-with-stiff-polypropylene-bristles>: HTTP status code is not handled or not allowed 2026-01-25 02:03:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nsf-fabricated-1-compartment-sink-18l-x-18w-bowl-9-splash-18h-left-drainboard-18ga>: HTTP status code is not handled or not allowed 2026-01-25 02:03:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-set-screw-collar-gmc-05-b>: HTTP status code is not handled or not allowed 2026-01-25 02:03:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1355-xl-3-8-composite-xtreme-torque-twin-hammer-impact-wrench>: HTTP status code is not handled or not allowed 2026-01-25 02:03:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-inch-w-push-broom-head-with-stiff-center-and-fine-border-polypropylene-bristles>: HTTP status code is not handled or not allowed 2026-01-25 02:03:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-leg-telescoping-easel-with-pad-retainer-adjusts-34-to-64-aluminum-black>: HTTP status code is not handled or not allowed 2026-01-25 02:03:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/self_luminous_emergency_exit_signs>: HTTP status code is not handled or not allowed 2026-01-25 02:03:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-x-24-adjustable-height-9-14-steel-work-platform>: HTTP status code is not handled or not allowed 2026-01-25 02:03:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-x-24-adjustable-height-9-14-steel-work-platform>: HTTP status code is not handled or not allowed 2026-01-25 02:03:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/71-x-24-adjustable-height-9-14-steel-work-platform>: HTTP status code is not handled or not allowed 2026-01-25 02:03:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-linear-vapor-tight-fixture-surface-mount-253850w-120-277v-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:03:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-g1sc-175-ss>: HTTP status code is not handled or not allowed 2026-01-25 02:03:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ulti-mate-garage-2-0-series-10-piece-cabinet-set-213w-x-21-14d-wall-x-80h-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:04:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/florsheimfe690-mens-quick-release-6-metatarsal-work-boot-black-size-11-d-medium>: HTTP status code is not handled or not allowed 2026-01-25 02:04:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-large-1-3-cut-file-folder-labels-15-16-x-3-7-16-white-450-pk>: HTTP status code is not handled or not allowed 2026-01-25 02:04:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/faultless-rigid-plate-caster-5-polyurethane-wheel-2>: HTTP status code is not handled or not allowed 2026-01-25 02:04:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/511286-pilot-assembly-for-garland-range>: HTTP status code is not handled or not allowed 2026-01-25 02:04:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ulti-mate-garage-2-0-series-5-piece-cabinet-set-106-5w-x-21d-x-80h-white>: HTTP status code is not handled or not allowed 2026-01-25 02:04:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-hand-ctrl-for-electrical-actuation-of-solenoid-valve>: HTTP status code is not handled or not allowed 2026-01-25 02:04:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-valve-solenoid-1-8-nptf-3-port-2-pos-normally-clsd-12vdc-free-to-atmosphere-exh>: HTTP status code is not handled or not allowed 2026-01-25 02:04:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/styleview-emr-electronic-medical-record-medical-cart-with-lcd-arm-sv41-6200-0>: HTTP status code is not handled or not allowed 2026-01-25 02:04:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96-w-x-30-d-plastic-laminate-square-edge-with-drawer-riser>: HTTP status code is not handled or not allowed 2026-01-25 02:04:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 50618 pages (at 93 pages/min), scraped 25711 items (at 40 items/min) 2026-01-25 02:04:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd-4c-tc-hss-4-flute-ticn-square-double-end-mill-1-2-x-1-2-x-1-x-4-1-8>: HTTP status code is not handled or not allowed 2026-01-25 02:04:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96-w-x-36-d-maple-butcher-block-square-edge-with-drawer-riser>: HTTP status code is not handled or not allowed 2026-01-25 02:04:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-g1sc-062-b>: HTTP status code is not handled or not allowed 2026-01-25 02:04:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ses030822-3-hp-oil-less-scroll-compressor-13-gallon-horizontal-116-psi-3-phase-208v>: HTTP status code is not handled or not allowed 2026-01-25 02:04:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-compartment-sink-t304-ss-24l-x-24w-x-14d-bowl-1-24-right-drainboard>: HTTP status code is not handled or not allowed 2026-01-25 02:04:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-vertical-file-letter-size-5-drawer-light-gray-15w-x-26-1-2d-x-64h>: HTTP status code is not handled or not allowed 2026-01-25 02:04:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-valve-5-port-2-pos-man-valve-1-8-nptf-port-red-extended-head-actr-air-spring-ret>: HTTP status code is not handled or not allowed 2026-01-25 02:04:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aah-48rbmag3-wal-storage-cabinet-all-welded-48-wx24-dx78-h-walnut>: HTTP status code is not handled or not allowed 2026-01-25 02:04:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-valve-5-port-2-pos-mech-valve-1-8-nptf-port-adjustable-roller-leaf-actr-air-spring-ret>: HTTP status code is not handled or not allowed 2026-01-25 02:04:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-vertical-file-legal-size-4-drawer-light-gray-18w-x-26-1-2d-x-52h>: HTTP status code is not handled or not allowed 2026-01-25 02:04:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/38-1328-high-temperature-wire-16-gauge-stranded-ptfe-black-50-roll>: HTTP status code is not handled or not allowed 2026-01-25 02:04:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/venture-patio-umbrella-with-bronze-pole-108w-olefin-red-fabric>: HTTP status code is not handled or not allowed 2026-01-25 02:04:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-vertical-file-letter-size-4-drawer-desert-putty-15w-x-25d-x-52h>: HTTP status code is not handled or not allowed 2026-01-25 02:04:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-vertical-file-legal-size-2-drawer-desert-putty-18w-x-25d-x-29h>: HTTP status code is not handled or not allowed 2026-01-25 02:04:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-step-steel-stock-picking-ladder-abrasive-tread-w-cal-osha-handrail-cal-spl-6-14nr>: HTTP status code is not handled or not allowed 2026-01-25 02:04:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011459-thermostat-for-vulcan-hart>: HTTP status code is not handled or not allowed 2026-01-25 02:04:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-valve-5-port-2-pos-dbl-solenoid-1-8-nptf-port-solenoid-ret-120vac>: HTTP status code is not handled or not allowed 2026-01-25 02:04:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-step-steel-stock-picking-ladder-serrated-tread-w-cal-osha-handrail-cal-spl-7-14g>: HTTP status code is not handled or not allowed 2026-01-25 02:04:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-step-steel-safety-stairway-slope-rolling-ladder-weight-actuated-lock-16-w-perf-step-w-cal-osha-handrail-cal-wa-sw-62414p>: HTTP status code is not handled or not allowed 2026-01-25 02:04:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-coupling-gscc-137-137>: HTTP status code is not handled or not allowed 2026-01-25 02:04:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-box-cutter-blades-pointed-tips-10408-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-25 02:04:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/newport-patio-umbrella-with-silver-pole-108w-sunbrella-1a-black-fabric>: HTTP status code is not handled or not allowed 2026-01-25 02:04:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:04:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/made-in-usa-hss-zero-flute-countersink-deburring-tool-set-90-0-4>: HTTP status code is not handled or not allowed 2026-01-25 02:04:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heritage-bench-recycled-plastic-6-ft-green-frame-cedar>: HTTP status code is not handled or not allowed 2026-01-25 02:04:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bca-368418-r-bookcase-all-welded-36-wx18-dx84-h-red>: HTTP status code is not handled or not allowed 2026-01-25 02:04:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-large-adj-height-steel-av-cart-w-cabinet-3-shelves-blk-electric-24-to-42-h>: HTTP status code is not handled or not allowed 2026-01-25 02:04:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conntek-30221-yw-15-to-20-amp-locking-adapter-with-nema-5-15p-to-l5-20r-yellow>: HTTP status code is not handled or not allowed 2026-01-25 02:04:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-safety-view-locker-five-tier-12x15x12-5-doors-assembled-tan>: HTTP status code is not handled or not allowed 2026-01-25 02:04:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-25 02:04:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/four-compartment-cutlery-box-20-7-16-l-x-11-3-8-w-x-3-3-4-h-polypropylene-brown>: HTTP status code is not handled or not allowed 2026-01-25 02:04:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bca-364218-bl-bookcase-all-welded-36-wx18-dx42-h-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:04:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/umbrellas_awnings_shades/outdoor_umbrellas_bases/umbrellas already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:04:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-hopper-stack-bin-pallet-base-50-compartments-37-w-x-15-1-2-d-x-57-1-2-h-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:04:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zip-a-duct-12in-gray-straight-section-with-vents-600-cfm>: HTTP status code is not handled or not allowed 2026-01-25 02:04:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/floor-stop-bollard-with-removable-caps-36height>: HTTP status code is not handled or not allowed 2026-01-25 02:04:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zip-a-duct-16in-gray-straight-section-without-vents-5ft-long>: HTTP status code is not handled or not allowed 2026-01-25 02:04:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-step-steel-safety-rolling-ladder-w-weight-actuated-lock-24-w-serrated-step-orange-w-cal-osha-handrail-cal-wa053214g-o>: HTTP status code is not handled or not allowed 2026-01-25 02:04:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-vis-fr-control-2-0-short-sleeve-tee-type-r-class-2-2xt>: HTTP status code is not handled or not allowed 2026-01-25 02:04:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-safety-cone-scwf436-1>: HTTP status code is not handled or not allowed 2026-01-25 02:04:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gondolashelving-8x16basketwiredivider-796494>: HTTP status code is not handled or not allowed 2026-01-25 02:04:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-step-steel-safety-rolling-ladder-w-weight-actuated-lock-step-16-w-serrated-step-w-cal-osha-handrail-cal-wa072414g>: HTTP status code is not handled or not allowed 2026-01-25 02:04:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ball-crown-bellman-cart-40x25-satin-brass-gray-carpet-1>: HTTP status code is not handled or not allowed 2026-01-25 02:04:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-step-16-wx65-d-stainless-steel-rolling-safety-ladder-serrated-grating-w-cal-osha-handrail-cal-ss072421g>: HTTP status code is not handled or not allowed 2026-01-25 02:04:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-aluminum-platform-step-ladder>: HTTP status code is not handled or not allowed 2026-01-25 02:04:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/outdoor_furniture/umbrellas_awnings_shades/outdoor_umbrellas_bases/umbrellas>: HTTP status code is not handled or not allowed 2026-01-25 02:04:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-step-stock-picking-ladder-grip-step>: HTTP status code is not handled or not allowed 2026-01-25 02:04:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/residential-air-handler-5-ton-208230v>: HTTP status code is not handled or not allowed 2026-01-25 02:04:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-hand-held-shower-spray-massager-with-three-massaging-options>: HTTP status code is not handled or not allowed 2026-01-25 02:04:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yellow-striper-premium-spray-paint>: HTTP status code is not handled or not allowed 2026-01-25 02:04:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-outward-swing-partition-door-w-hardware-26w-dark-khaki>: HTTP status code is not handled or not allowed 2026-01-25 02:04:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-w-x-16-h-x-2-d-pleated-merv-11-high-capacity-air-filter-global-industrial>: HTTP status code is not handled or not allowed 2026-01-25 02:05:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-step-24-w-steel-safety-angle-orange-rolling-ladder-w-handrails-perforated-tread-w-cal-osha-handrail-cal-sw932p-o>: HTTP status code is not handled or not allowed 2026-01-25 02:05:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carry-holster-for-tlk100-portable-radios>: HTTP status code is not handled or not allowed 2026-01-25 02:05:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-inch-cruiser-iii-flip-back-detachable-full-arm-swing-away-footrest>: HTTP status code is not handled or not allowed 2026-01-25 02:05:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oxygen-cylinder-carts-use-w-12-cylinder-e-d-c-or-m9-cart>: HTTP status code is not handled or not allowed 2026-01-25 02:05:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11-step-steel-stock-picking-ladder-perforated-tread-w-cal-osha-handrail-cal-spl-11-14p>: HTTP status code is not handled or not allowed 2026-01-25 02:05:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sk12ne-x03s-secret-service-style-1-wire-surveillance-kit-earpiece>: HTTP status code is not handled or not allowed 2026-01-25 02:05:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qub-series-powered-armless-l-sectional-26-38x26-38x30-12-black>: HTTP status code is not handled or not allowed 2026-01-25 02:05:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 50716 pages (at 98 pages/min), scraped 25752 items (at 41 items/min) 2026-01-25 02:05:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hs65nr-x03s-high-noise-reduction-two-way-radio-headset-over-the-head-dual-muff>: HTTP status code is not handled or not allowed 2026-01-25 02:05:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s201000-3-poly-web-sling-endless-2120-2650-5300-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-25 02:05:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galvanized-steel-oval-raised-planter-bed-31d-x-94w-x-32h-silver>: HTTP status code is not handled or not allowed 2026-01-25 02:05:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s102042-wire-rope-sling-5-8-x-6-eye-and-eye-5800-7800-15600-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-25 02:05:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/sinks/wash_fountains already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:05:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-piece-hand-evaluation-set-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:05:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-closed-linear-bearing-wresin-retainer-sws4g-14id-075l>: HTTP status code is not handled or not allowed 2026-01-25 02:05:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-strapping-seals-3-4-packed-1000>: HTTP status code is not handled or not allowed 2026-01-25 02:05:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-outward-swing-partition-door-w-hardware-24w-dark-khaki>: HTTP status code is not handled or not allowed 2026-01-25 02:05:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hangvu-hang-style-label-holder-with-holes-6l-x-4h-matte-clear-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-25 02:05:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-extinguisher-carbon-dioxide-20-lb>: HTTP status code is not handled or not allowed 2026-01-25 02:05:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forney-three-flint-striker>: HTTP status code is not handled or not allowed 2026-01-25 02:05:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dispenser-rwd2020-for-steel-strapping-fits-3-ribbon-wound-coils>: HTTP status code is not handled or not allowed 2026-01-25 02:05:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-pilaster-w-shoe-5w-x-82h-almond>: HTTP status code is not handled or not allowed 2026-01-25 02:05:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1802-33-cobalt-heavy-duty-straw-135-split-point-jobber-length-drill>: HTTP status code is not handled or not allowed 2026-01-25 02:05:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-piece-hand-evaluation-set-black>: HTTP status code is not handled or not allowed 2026-01-25 02:05:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/plumbing/sinks/wash_fountains landed on page that is not a product page. 2026-01-25 02:05:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/plastic_laminate_bathroom_partitions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:05:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1821-3-4-8in-oal-hss-h-d-sand-blasted-118-point-carbide-tipped-sds-plus-2-masonry-drill>: HTTP status code is not handled or not allowed 2026-01-25 02:05:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plain-magnetic-roll-stock-1200l-x-1w-x-003-thick-black>: HTTP status code is not handled or not allowed 2026-01-25 02:05:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flipak-attached-lid-container-26-7-8-17x12-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:05:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-signs-caution-do-not-operate-vinyl-10-h-x-14-w>: HTTP status code is not handled or not allowed 2026-01-25 02:05:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacment-squeegee-set-for-nilfisk-advance-11338a>: HTTP status code is not handled or not allowed 2026-01-25 02:05:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b151124-12-single-leg-chain-sling-w-sling-hook>: HTTP status code is not handled or not allowed 2026-01-25 02:05:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacment-squeegee-set-for-factory-cat-25-770l-factory-cat-25-754l-factory-cat-25-755l>: HTTP status code is not handled or not allowed 2026-01-25 02:05:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/plastic_laminate_bathroom_partitions>: HTTP status code is not handled or not allowed 2026-01-25 02:05:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-tier-6-door-digital-locker-12inwx15indx78inh-gray-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 02:05:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b152014-12-quad-leg-chain-sling-w-sling-hook>: HTTP status code is not handled or not allowed 2026-01-25 02:05:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1818-1-6in-oal-hss-heavy-duty-sand-blasted-118-point-carbide-tipped-masonry-drill>: HTTP status code is not handled or not allowed 2026-01-25 02:05:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/snap-lock-hard-hat-white>: HTTP status code is not handled or not allowed 2026-01-25 02:05:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-wall-exhaust-shutter-front-flange-22-easr-mf-22>: HTTP status code is not handled or not allowed 2026-01-25 02:05:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7111-01-flexrist-wrist-support-black-regular>: HTTP status code is not handled or not allowed 2026-01-25 02:05:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7195-05-deluxe-spanbak-back-support-xx-large>: HTTP status code is not handled or not allowed 2026-01-25 02:05:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-partition-panel-wo-brackets-5412w-white>: HTTP status code is not handled or not allowed 2026-01-25 02:05:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/photoluminescent-anti-slip-tape-with-self-adhesive-backing-2-inch-by-52-feet-ul1994-listed-83-0780>: HTTP status code is not handled or not allowed 2026-01-25 02:05:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/locking-post-collar-bag-4>: HTTP status code is not handled or not allowed 2026-01-25 02:05:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacment-squeegee-set-for-nobles-tennant-222424>: HTTP status code is not handled or not allowed 2026-01-25 02:05:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cluster-box-unit-8-a-size-doors-type-i-white-usps-access>: HTTP status code is not handled or not allowed 2026-01-25 02:05:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-drain-hose-w-drain-cap-full-assembly-for-nobles-tennant-1017380>: HTTP status code is not handled or not allowed 2026-01-25 02:05:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dunnage-shelf-18x48-2>: HTTP status code is not handled or not allowed 2026-01-25 02:06:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-rectangular-strip-carbide-blank-14x14x12-stb88l>: HTTP status code is not handled or not allowed 2026-01-25 02:06:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-squeegee-rear-a-for-ipc-eagle-mpvr05806>: HTTP status code is not handled or not allowed 2026-01-25 02:06:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/science-table-with-casters-phenolic-top-adjustable-height-24-x-72-black-oak>: HTTP status code is not handled or not allowed 2026-01-25 02:06:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-latch-soft-rubber-w-keep-for-betco-e1263300>: HTTP status code is not handled or not allowed 2026-01-25 02:06:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/159-11-64-hss-heavy-duty-steam-oxide-135-split-point-screw-machine-drill>: HTTP status code is not handled or not allowed 2026-01-25 02:06:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/polymer_bathroom_partitions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:06:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/150asp-61-hss-heavy-duty-steam-oxide-135-point-point-jobber-length-drill>: HTTP status code is not handled or not allowed 2026-01-25 02:06:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-vis-flame-resistant-safety-cargo-pant-30-waist-x-32l-inseam-navy>: HTTP status code is not handled or not allowed 2026-01-25 02:06:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 50811 pages (at 95 pages/min), scraped 25801 items (at 49 items/min) 2026-01-25 02:06:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ppb-1g-pour-w-green-spout-lid-1-qt>: HTTP status code is not handled or not allowed 2026-01-25 02:06:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-carbon-fiber-tube-unidirectional-1-4-id-x-5-16-od-x-4-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 02:06:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-x-10-x-7-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-25 02:06:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/polymer_bathroom_partitions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:06:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-g-10-fr-4-garolite-sheet-1-32-thick-x-24-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 02:06:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slotted-cap-with-lamp-bracket--cord-holder-class-c-staz-on-suspension-white>: HTTP status code is not handled or not allowed 2026-01-25 02:06:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/composites/composite_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:06:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/polymer_bathroom_partitions>: HTTP status code is not handled or not allowed 2026-01-25 02:06:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/abc-industries-enviroduct-15wk-portable-heater-duct-12-dia-x-25l-x-7-pitch-black>: HTTP status code is not handled or not allowed 2026-01-25 02:06:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/abc-industries-enviroduct-15wk-portable-heater-duct-16-dia-x-25l-x-7-pitch-black>: HTTP status code is not handled or not allowed 2026-01-25 02:06:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milwaukee-packout-48-22-8436-low-profile-organizer>: HTTP status code is not handled or not allowed 2026-01-25 02:06:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triple-stream-satin-aluminum-30-gal-bottles-cans-trash-bct-2032sa-sa>: HTTP status code is not handled or not allowed 2026-01-25 02:06:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-x-8-x-6-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-25 02:06:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/skullerz-8970led-hard-hat-cap-style-ratchet-suspension-led-light-class-e-orange>: HTTP status code is not handled or not allowed 2026-01-25 02:06:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ppc-2-diameter-blade-pizza-cutter-with-plastic-handle>: HTTP status code is not handled or not allowed 2026-01-25 02:06:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/composites/composite_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-25 02:06:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/polymer_bathroom_partitions landed on page that is not a product page. 2026-01-25 02:06:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sweeper-dry-43-wet-starter-kit-46handle-10-x-8-head-silvergreen-6carton>: HTTP status code is not handled or not allowed 2026-01-25 02:06:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/psw-24r-wide-mouth-squeeze-bottles-24-oz>: HTTP status code is not handled or not allowed 2026-01-25 02:06:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-wire-4-inch-submersible-well-pump-230-volts-1-hp-20-gpm>: HTTP status code is not handled or not allowed 2026-01-25 02:06:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-recessed-screw-climax-h1c-200-s>: HTTP status code is not handled or not allowed 2026-01-25 02:06:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/contouring-roof-fabric-40-x-81-roll>: HTTP status code is not handled or not allowed 2026-01-25 02:06:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ppok-113-measuring-pourer-1-1-8-oz>: HTTP status code is not handled or not allowed 2026-01-25 02:06:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arsenal-5820-gear-and-tool-storage-replacement-shoulder-strap-black>: HTTP status code is not handled or not allowed 2026-01-25 02:06:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-laminate-pilaster-w-shoe-8w-x-82h-black>: HTTP status code is not handled or not allowed 2026-01-25 02:06:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h1-safety-helmet-novent-fas-trac-iii-pivot-ansi-csa-en397-red>: HTTP status code is not handled or not allowed 2026-01-25 02:06:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-with-keyway-2c-kw-series-climax-2c-125-s-kw>: HTTP status code is not handled or not allowed 2026-01-25 02:06:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hand-scoop-82-oz-yellow>: HTTP status code is not handled or not allowed 2026-01-25 02:06:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hand-scoop-32-oz-purple>: HTTP status code is not handled or not allowed 2026-01-25 02:06:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wbkg-15-oblong-black-wire-bread-basket>: HTTP status code is not handled or not allowed 2026-01-25 02:06:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-canvas-ce-phenolic-sheet-0-093-thick-48-x-96>: HTTP status code is not handled or not allowed 2026-01-25 02:06:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spjl-202-steam-pan-half-size-2-1-2h>: HTTP status code is not handled or not allowed 2026-01-25 02:06:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-linen-le-phenolic-sheet-0-750-thick-36-x-48>: HTTP status code is not handled or not allowed 2026-01-25 02:06:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ductless-mini-split-ac-heat-pump-wi-fi-control-18000-btu-22-seer-115v-16-lineset-white>: HTTP status code is not handled or not allowed 2026-01-25 02:06:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-climax-1c-287>: HTTP status code is not handled or not allowed 2026-01-25 02:06:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/packaging/stretch_wrap_shrink_wrap/shrink_wrap already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:06:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/phenolic-black-core-pilaster-w-shoe-6w-x-82h-almond>: HTTP status code is not handled or not allowed 2026-01-25 02:06:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelf-vented-sides-24wx24dx76h-red-all-welded>: HTTP status code is not handled or not allowed 2026-01-25 02:06:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/phenolic-black-core-pilaster-w-shoe-12w-x-82h-almond>: HTTP status code is not handled or not allowed 2026-01-25 02:06:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-climax-1c-131-z>: HTTP status code is not handled or not allowed 2026-01-25 02:06:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iv5076055-5-hp-two-stage-piston-compressor-60-gal-vertical-175-psi-1-phase-230v>: HTTP status code is not handled or not allowed 2026-01-25 02:06:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/k500-super-turf-tire-20508-4tf-i-20-x-10-50-8-4-ply-tubeless>: HTTP status code is not handled or not allowed 2026-01-25 02:06:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelf-vented-sides-24wx21dx72h-blue-all-welded>: HTTP status code is not handled or not allowed 2026-01-25 02:06:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/packaging/stretch_wrap_shrink_wrap/shrink_wrap>: HTTP status code is not handled or not allowed 2026-01-25 02:06:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vx-034-0197rp-pressure-switch-100-130-psi-4-port-with-bleeder>: HTTP status code is not handled or not allowed 2026-01-25 02:06:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/patriot-fully-framed-duty-locker-24wx24dx90h-gray-all-welded>: HTTP status code is not handled or not allowed 2026-01-25 02:06:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-soil-dishtable-right-to-left-72-long-16304-ss>: HTTP status code is not handled or not allowed 2026-01-25 02:07:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4th-level-add-on-for-straight-transport-riser-gray-carpet-with-black-aluminum-frame>: HTTP status code is not handled or not allowed 2026-01-25 02:07:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/patriot-fully-framed-ta-50-locker-wperforated-doorexpanded-side-48x24x78blall-welded>: HTTP status code is not handled or not allowed 2026-01-25 02:07:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/plastic_laminate_bathroom_partitions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:07:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-shop-desk-with-pigeonhole-compartment-riser-34-1-2-w-x-30-d-x-38-h-sloped-surface-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:07:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stretch-wrap-with-extended-core-handle-cast-90-gauge-15w-x-1000l-clear-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-25 02:07:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-paper-phenolic-sheet-0-250-thick-48-x-96>: HTTP status code is not handled or not allowed 2026-01-25 02:07:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/atch-46-table-card-holder-acrylic>: HTTP status code is not handled or not allowed 2026-01-25 02:07:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qf-pipe-4-dia-304-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 02:07:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-set-screw-collar-climax-mc-25>: HTTP status code is not handled or not allowed 2026-01-25 02:07:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-set-screw-collar-climax-mc-28>: HTTP status code is not handled or not allowed 2026-01-25 02:07:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 50904 pages (at 93 pages/min), scraped 25842 items (at 41 items/min) 2026-01-25 02:07:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-linen-le-phenolic-tube-1-125-id-x-1-250-od-x-48-000>: HTTP status code is not handled or not allowed 2026-01-25 02:07:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bw-ss1-solid-spoon-black-nylon>: HTTP status code is not handled or not allowed 2026-01-25 02:07:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelf-24wx21dx72h-gray-all-welded>: HTTP status code is not handled or not allowed 2026-01-25 02:07:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/plastic_laminate_bathroom_partitions landed on page that is not a product page. 2026-01-25 02:07:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qf-clamp-with-pin-14-dia-galvanized-steel>: HTTP status code is not handled or not allowed 2026-01-25 02:07:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnetic-glass-dry-erase-board-36-x-24-seafoam>: HTTP status code is not handled or not allowed 2026-01-25 02:07:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-set-screw-collar-climax-mc-45-s>: HTTP status code is not handled or not allowed 2026-01-25 02:07:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qf-adjustable-nipple-with-o-ring-10-dia-galvanized>: HTTP status code is not handled or not allowed 2026-01-25 02:07:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-grate-guard-with-overflows-sediment-high-flow-28l-x-28w-x-4h>: HTTP status code is not handled or not allowed 2026-01-25 02:07:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelf-footlocker-24wx24dx76h-red-all-welded>: HTTP status code is not handled or not allowed 2026-01-25 02:07:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rigid-coupling-4-set-screws-2-90-rc4h-series-climax-rc-075-kw4h90>: HTTP status code is not handled or not allowed 2026-01-25 02:07:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelf-footlocker-24wx18dx72h-gray-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 02:07:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/floor_machine_accessories_supplies/vacuum_attachments_tools already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:07:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0385r-iron-tuff-high-bib-overall-tall-navy-4xl>: HTTP status code is not handled or not allowed 2026-01-25 02:07:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/om1-power-degreaser-lemon-grass-scent-32-oz-bottle-9-case-om103-qps9>: HTTP status code is not handled or not allowed 2026-01-25 02:07:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0385r-iron-tuff-high-bib-overall-regular-navy-5xl>: HTTP status code is not handled or not allowed 2026-01-25 02:07:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/afp-10ns-fry-pan>: HTTP status code is not handled or not allowed 2026-01-25 02:07:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poly-z-brite-wire-shelf-14x72>: HTTP status code is not handled or not allowed 2026-01-25 02:07:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wsecurity-box-footlocker-18wx18dx72hgrall-welded>: HTTP status code is not handled or not allowed 2026-01-25 02:07:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tear-resistant-squeegee-blade-for-b-150-r-bp-brush-head-r-75-kira-b-50-red>: HTTP status code is not handled or not allowed 2026-01-25 02:07:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bus-smoking-shelter-flat-roof-3-side-open-front-w-terracotta-5-gallon-outdoor-ashtray-6-5-x3-8-x7>: HTTP status code is not handled or not allowed 2026-01-25 02:07:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/floor_machine_accessories_supplies/vacuum_attachments_tools>: HTTP status code is not handled or not allowed 2026-01-25 02:07:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0485-chillbreaker-high-bib-overall-regular-navy-large>: HTTP status code is not handled or not allowed 2026-01-25 02:07:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bspb-15-perforated-basting-spoon-w-bakelite-handle-15l-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 02:07:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelf-footlocker-24wx21dx78h-gray-all-welded>: HTTP status code is not handled or not allowed 2026-01-25 02:07:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-vision-link-qr-lever-assembly>: HTTP status code is not handled or not allowed 2026-01-25 02:07:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-t200-ss-t-link-hood-head-harness-px5>: HTTP status code is not handled or not allowed 2026-01-25 02:07:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0021-05-continental-dinner-fork>: HTTP status code is not handled or not allowed 2026-01-25 02:07:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelate-silver-epoxy-wire-shelf-42x21>: HTTP status code is not handled or not allowed 2026-01-25 02:07:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=jtm-949evs-230-mill-with-2-axis-acu-rite-g-2-millpwr-cnc already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:07:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0359-iron-tuff-arctic-jacket-regular-navy-4xl>: HTTP status code is not handled or not allowed 2026-01-25 02:07:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0526-cooler-wear-trousers-regular-navy-4xl>: HTTP status code is not handled or not allowed 2026-01-25 02:07:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/azar-250035-single-pocket-wall-file-w-adhesive-tape-14-3-4-w-x-3-d-x-7-h>: HTTP status code is not handled or not allowed 2026-01-25 02:07:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/afp-14ns-h-fry-pan>: HTTP status code is not handled or not allowed 2026-01-25 02:07:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0033-05-oxford-dinner-fork>: HTTP status code is not handled or not allowed 2026-01-25 02:07:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-carry-bag>: HTTP status code is not handled or not allowed 2026-01-25 02:07:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0381r-iron-tuff-minus-50-suit-regular-sage-2xl>: HTTP status code is not handled or not allowed 2026-01-25 02:07:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-one-piece-clamping-collar-climax-m1c-40-s>: HTTP status code is not handled or not allowed 2026-01-25 02:07:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=jtm-949evs-230-mill-with-2-axis-acu-rite-g-2-millpwr-cnc>: HTTP status code is not handled or not allowed 2026-01-25 02:07:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0342r-iron-tuff-jackoat-regular-navy-5xl>: HTTP status code is not handled or not allowed 2026-01-25 02:07:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-ladder-back-swivel-metal-barstool-walnut-wood-seat>: HTTP status code is not handled or not allowed 2026-01-25 02:07:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powerquad153-ned-vacuum-kit-with-1-12-inlet-20-gallon-capacity>: HTTP status code is not handled or not allowed 2026-01-25 02:07:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coveralls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:07:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0322r-iron-tuff-polar-jacket-regular-navy-3xl>: HTTP status code is not handled or not allowed 2026-01-25 02:07:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/azar-250025-single-pocket-wall-file-wallmount-14-3-4-w-x-3-d-x-7-h>: HTTP status code is not handled or not allowed 2026-01-25 02:08:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-comfort-commercial-fabric-and-resin-mesh-folding-chair-black-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-25 02:08:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fold-in-half-plastic-table-96-x-30-x-29-inch-black>: HTTP status code is not handled or not allowed 2026-01-25 02:08:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bash-brass-sledge-hammer-4-lb-head-16-steel-handle>: HTTP status code is not handled or not allowed 2026-01-25 02:08:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/six-inch-brass-interlocking-stencil-letters-thirty-three-piece-set>: HTTP status code is not handled or not allowed 2026-01-25 02:08:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dxs2153-sae-standard-impact-socket-1-14-drive-black>: HTTP status code is not handled or not allowed 2026-01-25 02:08:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coveralls landed on page that is not a product page. 2026-01-25 02:08:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lifting-oz2bca-mr-beam-clamp-2-ton-capacity-fits-beam-flange-9-10>: HTTP status code is not handled or not allowed 2026-01-25 02:08:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lifting-industrial-manual-chain-hoist-1-2-ton-capacity-15-lift>: HTTP status code is not handled or not allowed 2026-01-25 02:08:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 51000 pages (at 96 pages/min), scraped 25888 items (at 46 items/min) 2026-01-25 02:08:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pfb-10g-oval-fast-food-baskets>: HTTP status code is not handled or not allowed 2026-01-25 02:08:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-zlink-zytec-fr-shoulder-cape>: HTTP status code is not handled or not allowed 2026-01-25 02:08:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sun-x-30-71666-sunscreen-lotion-spf-30-4oz-bottle>: HTTP status code is not handled or not allowed 2026-01-25 02:08:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-double-wide-climax-d2c-012>: HTTP status code is not handled or not allowed 2026-01-25 02:08:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ssc-2-utility-scoop-64-oz-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 02:08:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n23s-quick-disconnect-hose-adapter-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 02:08:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/twp-40-scraper-4-1-2l-blade-4w-white-polypropylene-handle>: HTTP status code is not handled or not allowed 2026-01-25 02:08:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-climax-c-025>: HTTP status code is not handled or not allowed 2026-01-25 02:08:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/str-15-steamer-rack-14-3-4d>: HTTP status code is not handled or not allowed 2026-01-25 02:08:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-inch-x-5-yards-self-adhesive-bandage-non-latex-kelly-green-pack-of-20>: HTTP status code is not handled or not allowed 2026-01-25 02:08:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/icop-24-disher-w-single-piece-handle-size-24-red>: HTTP status code is not handled or not allowed 2026-01-25 02:08:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-30-maple-top-pedestal-workbench-w-8-drawers>: HTTP status code is not handled or not allowed 2026-01-25 02:08:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/surface-mounted-alarmed-cabinet-100bd-alarm-powder-coated-white>: HTTP status code is not handled or not allowed 2026-01-25 02:08:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/twp-51-pie-server-5-1-2l-blade-white-polypropylene-handle>: HTTP status code is not handled or not allowed 2026-01-25 02:08:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-inch-x-5-yards-self-adhesive-bandage-non-latex-black-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-25 02:08:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-30-maple-top-pedestal-workbench-w-7-drawers>: HTTP status code is not handled or not allowed 2026-01-25 02:08:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0033-wick-sock-white-small-medium>: HTTP status code is not handled or not allowed 2026-01-25 02:08:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-industry-standard-clamping-couplings-w-keyway-climax-iscc-125-125skw>: HTTP status code is not handled or not allowed 2026-01-25 02:08:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/surge_protectors_extension_cords/extension_cords already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:08:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glc-01056-amber-glass-screw-thread-sample-vials-with-green-caps-5-dram-20ml-case-of-144>: HTTP status code is not handled or not allowed 2026-01-25 02:08:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glc-01008-clear-glass-screw-thread-vials-with-green-caps-8-dram-30ml-case-of-144>: HTTP status code is not handled or not allowed 2026-01-25 02:08:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-nova-3-air-inlet-kit>: HTTP status code is not handled or not allowed 2026-01-25 02:08:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-inch-x-5-yards-self-adhesive-bandage-non-latex-kelly-green>: HTTP status code is not handled or not allowed 2026-01-25 02:08:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-rdb-175-hydraulic-rotary-draw-tube-pipe-bender-with-auto-stop-110v>: HTTP status code is not handled or not allowed 2026-01-25 02:08:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bench-yoke-vise-1-8-3-pipe-capacity>: HTTP status code is not handled or not allowed 2026-01-25 02:08:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mst-10s-fine-mesh-10-1-4-diameter-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 02:08:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cmt-1318-oval-serving-tray-18-3-4l-chrome-gadroon-edge-w-engraving>: HTTP status code is not handled or not allowed 2026-01-25 02:08:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-mobile-pizza-oven>: HTTP status code is not handled or not allowed 2026-01-25 02:08:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-inch-x-5-yards-self-adhesive-bandage-non-latex-black>: HTTP status code is not handled or not allowed 2026-01-25 02:08:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-green-3-tier-wire-shelving-starter-unit-72-w-x-18-d-x-34-h>: HTTP status code is not handled or not allowed 2026-01-25 02:08:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s-1173-mini-disposable-air-line-desiccant-filter-polycarbonate-bowl-14-npt-125-psi>: HTTP status code is not handled or not allowed 2026-01-25 02:08:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/surge_protectors_extension_cords/extension_cords>: HTTP status code is not handled or not allowed 2026-01-25 02:08:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1450150-airflow-conveyor-chain-af-24d-60065>: HTTP status code is not handled or not allowed 2026-01-25 02:08:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-outdoor-steel-picnic-bench-with-backrest-perforated-metal-red>: HTTP status code is not handled or not allowed 2026-01-25 02:08:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/epdm-rubber-strip-no-adhesive-60a-14-thick-x-12-wide-x-10-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 02:08:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gh-1860zx-large-spindle-bore-lathe-acu-rite-203-dro-collet-closer-7-12-hp-3ph-460v>: HTTP status code is not handled or not allowed 2026-01-25 02:08:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-shelf-36-x12>: HTTP status code is not handled or not allowed 2026-01-25 02:08:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-shelf-48-x12>: HTTP status code is not handled or not allowed 2026-01-25 02:08:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/replacement_shelves_decks/file_additional_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:08:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e-1236vs-elite-evs-lathe-with-newall-dp700-dro-taper-attachment-2-hp-3ph-230v>: HTTP status code is not handled or not allowed 2026-01-25 02:08:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:08:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/surge_protectors_extension_cords/extension_cords already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:08:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-blade-for-24in-flex-blade-floor-squeegee-1059>: HTTP status code is not handled or not allowed 2026-01-25 02:08:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-double-wide-climax-d2c-037-a>: HTTP status code is not handled or not allowed 2026-01-25 02:08:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-sheet-with-acrylic-adhesive-50a-116-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 02:08:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-climax-c-150-bo>: HTTP status code is not handled or not allowed 2026-01-25 02:08:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:08:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-climax-c-156-dt>: HTTP status code is not handled or not allowed 2026-01-25 02:08:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/shelving/replacement_shelves_decks/file_additional_shelves>: HTTP status code is not handled or not allowed 2026-01-25 02:08:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-roll-no-adhesive-60a-38-thick-x-36-wide-x-6-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 02:08:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips>: HTTP status code is not handled or not allowed 2026-01-25 02:08:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/surge_protectors_extension_cords/extension_cords>: HTTP status code is not handled or not allowed 2026-01-25 02:08:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-tier-shelving-add-on-unit-solid-galvanized-steel-54w-x-24d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 02:08:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-green-2-tier-wire-shelving-add-on-unit-60-w-x-14-d-x-34-h>: HTTP status code is not handled or not allowed 2026-01-25 02:08:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/sheet_metal_cutting_forming_tools already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:08:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sssp-6-sauce-pan-w-cover-6-qt>: HTTP status code is not handled or not allowed 2026-01-25 02:08:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-tier-shelving-starter-unit-solid-galvanized-steel-36w-x-24d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 02:08:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/35152-psa-disc-10-inch-dia-40-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-25 02:08:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/lathes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:08:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrosion-resistant-set-screw-collar-cr-climax-crc-018-s>: HTTP status code is not handled or not allowed 2026-01-25 02:08:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls>: HTTP status code is not handled or not allowed 2026-01-25 02:08:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrosion-resistant-set-screw-collar-cr-climax-crc-025>: HTTP status code is not handled or not allowed 2026-01-25 02:09:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/highlighters_markers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:09:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-wjt-ab864uw-16-angle-block-8-34l-x-6w-x-4h>: HTTP status code is not handled or not allowed 2026-01-25 02:09:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72316-ovation-attacker-flap-disc-t27-5-inch-x-5-8-11-inch-60-grit-zirconium>: HTTP status code is not handled or not allowed 2026-01-25 02:09:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:09:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-punch-die-set-18>: HTTP status code is not handled or not allowed 2026-01-25 02:09:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-industry-standard-clamping-couplings-w-keyway-climax-iscc-062-062-kw>: HTTP status code is not handled or not allowed 2026-01-25 02:09:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/machining/machinery/lathes>: HTTP status code is not handled or not allowed 2026-01-25 02:09:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-wjt-4848uw-16-you-weld-welding-table-48l-x-48w-x-36h>: HTTP status code is not handled or not allowed 2026-01-25 02:09:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-industry-standard-clamping-couplings-w-keyway-climax-iscc-087-087skw>: HTTP status code is not handled or not allowed 2026-01-25 02:09:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 51086 pages (at 86 pages/min), scraped 25912 items (at 24 items/min) 2026-01-25 02:09:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/highlighters_markers>: HTTP status code is not handled or not allowed 2026-01-25 02:09:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dock-light-arm-w-mounting-kit-90-l-heavy-duty-1-5-square-tubing>: HTTP status code is not handled or not allowed 2026-01-25 02:09:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-punch-die-set-2332>: HTTP status code is not handled or not allowed 2026-01-25 02:09:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/viton-rubber-strip-with-high-temp-adhesive-75a-132-thick-x-14-wide-x-3-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 02:09:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/machining/machinery>: HTTP status code is not handled or not allowed 2026-01-25 02:09:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pla-lids-for-fiber-containers-8-8-x-6-9-x-0-8-clear-400-carton>: HTTP status code is not handled or not allowed 2026-01-25 02:09:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/viton-rubber-strip-with-high-temp-adhesive-75a-18-thick-x-14-wide-x-3-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 02:09:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-handle-tool-grip-cap-for-rubbermaid-cleaning-cart>: HTTP status code is not handled or not allowed 2026-01-25 02:09:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/34990-psa-disc-1-inch-dia-50-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-25 02:09:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/35082-psa-disc-5-inch-dia-40-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-25 02:09:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-roll-no-adhesive-50a-18-thick-x-36-wide-x-6-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 02:09:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cat-stainless-steel-wet-dry-vacuum-16-gallon-cap>: HTTP status code is not handled or not allowed 2026-01-25 02:09:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-caster-w-expansion-swivel-stem-and-brake-2-13-16h-stem-325-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 02:09:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/l-320an-s-c-alloy-hook-eye-w-latch-22t-wll>: HTTP status code is not handled or not allowed 2026-01-25 02:09:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-strip-with-acrylic-adhesive-60a-12-thick-x-6-wide-x-5-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 02:09:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/81520-da-f-shop-roll-2-inch-x-50-yds-150-grit-handy-roll-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-25 02:09:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrosion-resistant-set-screw-collar-cr-climax-crc-100>: HTTP status code is not handled or not allowed 2026-01-25 02:09:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-bv-4m-three-way-clamping-vise-4-116-jaw-width>: HTTP status code is not handled or not allowed 2026-01-25 02:09:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-roll-no-adhesive-60a-14-thick-x-36-wide-x-8-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 02:09:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-roll-no-adhesive-50a-18-thick-x-36-wide-x-10-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 02:09:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s-2130-s-c-carbon-shackle-bta-3-4-4-3-4t-wll>: HTTP status code is not handled or not allowed 2026-01-25 02:09:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50274-z-fiber-disc-4-1-2-inch-x-7-8-inch-36-grit-zirconium>: HTTP status code is not handled or not allowed 2026-01-25 02:09:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-of-boxed-serrated-jaw-inserts-with-4-screws-for-1765-vise>: HTTP status code is not handled or not allowed 2026-01-25 02:09:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-curtain-47-w-with-remote-control>: HTTP status code is not handled or not allowed 2026-01-25 02:09:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/bins_totes_containers/baskets_organizers/baskets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:09:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-climax-c-143-a>: HTTP status code is not handled or not allowed 2026-01-25 02:09:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-recessed-screw-climax-h2c-243>: HTTP status code is not handled or not allowed 2026-01-25 02:09:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-step-16-w-perforated-safety-angle-ladder>: HTTP status code is not handled or not allowed 2026-01-25 02:09:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gn-251-6-setting-bolts-steel-w-retaining-magnet-zinc-pltd-m10-11-16-mag-dia-2-l>: HTTP status code is not handled or not allowed 2026-01-25 02:09:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-step-24-w-grip-safety-angle-ladder>: HTTP status code is not handled or not allowed 2026-01-25 02:09:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n-7269-bi-fold-door-topbottom-pivot-and-guide-set>: HTTP status code is not handled or not allowed 2026-01-25 02:09:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/l100-1500wo-15-hand-chain-hoist-with-overload-protection-15-ton-capacity-15-lift>: HTTP status code is not handled or not allowed 2026-01-25 02:09:42 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/bins_totes_containers/baskets_organizers/baskets landed on page that is not a product page. 2026-01-25 02:09:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/211-quad-ring-xring-1316id-x-1116od-70-duro-round-black-pkg-100>: HTTP status code is not handled or not allowed 2026-01-25 02:09:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-cart-poly-green-24l-x-18w-x-40h-polyurethane-rigid-casters>: HTTP status code is not handled or not allowed 2026-01-25 02:09:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gd-12316-garage-door-torsion-spring-207-in-x-2-in-x-20-in-orange-left-hand-wind>: HTTP status code is not handled or not allowed 2026-01-25 02:09:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n-6775-closet-door-guide-white-plasticpack-of-2>: HTTP status code is not handled or not allowed 2026-01-25 02:09:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gn-749-threaded-pipe-plugs-steel-zinc-plated-with-nbr-synthetic-rubber-seal-m-33-x-2>: HTTP status code is not handled or not allowed 2026-01-25 02:09:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/general_purpose_electric_motors/three_phase_motors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:09:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leeson-dc-controls-scr-series-pwm-series-nema-4-1ph-3hp-230v>: HTTP status code is not handled or not allowed 2026-01-25 02:09:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:09:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-dolly-42-x-48-floor-locks>: HTTP status code is not handled or not allowed 2026-01-25 02:09:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-roll-no-adhesive-70a-316-thick-x-36-wide-x-10-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 02:09:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/160-4mm-standard-length-ball-end-blade>: HTTP status code is not handled or not allowed 2026-01-25 02:09:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-mesh-tarp-5-1-3-ft-x-9-1-2-ft-3014677>: HTTP status code is not handled or not allowed 2026-01-25 02:09:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-sided-truck-2-shelves-slat-sides-24-x-48>: HTTP status code is not handled or not allowed 2026-01-25 02:09:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:09:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/general_purpose_electric_motors/three_phase_motors>: HTTP status code is not handled or not allowed 2026-01-25 02:09:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:10:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0-3-hp-1-4-inch-capacity-26000-rpm-light-duty-die-grinder>: HTTP status code is not handled or not allowed 2026-01-25 02:10:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/viton-rubber-sheet-no-adhesive-75a-116-thick-x-36-wide-x-36-long>: HTTP status code is not handled or not allowed 2026-01-25 02:10:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-25 02:10:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-speed-electric-tarp-gear-motor-5541065>: HTTP status code is not handled or not allowed 2026-01-25 02:10:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-beveling-machine-1-hp-single-phase-110v-cm-10p>: HTTP status code is not handled or not allowed 2026-01-25 02:10:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=manually-powered-ring-and-angle-roll-bending-machine-3-rolls-1-4-bends-curves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:10:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-ton-air-hand-operated-h-frame-press-7-1-2-stoke-ce-approved>: HTTP status code is not handled or not allowed 2026-01-25 02:10:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-lid-114004>: HTTP status code is not handled or not allowed 2026-01-25 02:10:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leeson-single-phase-general-purpose-motor-50hz-1-1hp-1-1kw-1425rpm-56h-ip54-1-0sf-rigid>: HTTP status code is not handled or not allowed 2026-01-25 02:10:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-25 02:10:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anti-sail-tarp-retention-bow-adjustable-dtb102>: HTTP status code is not handled or not allowed 2026-01-25 02:10:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rock-door-silencer-grey>: HTTP status code is not handled or not allowed 2026-01-25 02:10:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 51180 pages (at 94 pages/min), scraped 25950 items (at 38 items/min) 2026-01-25 02:10:09 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips landed on page that is not a product page. 2026-01-25 02:10:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leeson-single-phase-explosion-proof-motor-3-4hp-1725rpm-56-epfc-60hz-automatic-1-0sf-rigid>: HTTP status code is not handled or not allowed 2026-01-25 02:10:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-sided-adjustable-truck-slat-sides-24-x-48>: HTTP status code is not handled or not allowed 2026-01-25 02:10:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rock-roller-latch-1-1-8-x-2-3-4-strike-085862>: HTTP status code is not handled or not allowed 2026-01-25 02:10:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pem-aluminum-saddle-threshold-36-l-x-6-w-1-4-h-mill-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 02:10:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=manually-powered-ring-and-angle-roll-bending-machine-3-rolls-1-4-bends-curves>: HTTP status code is not handled or not allowed 2026-01-25 02:10:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pipe-welding-positioner-3-phase-220v-rwp-110-1-0>: HTTP status code is not handled or not allowed 2026-01-25 02:10:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/saw-muzzle-dust-collector-7-8-skil-and-bosch-worm-drive-circular-saws>: HTTP status code is not handled or not allowed 2026-01-25 02:10:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/saw-muzzle-gp-dust-collector-12-14-makita-cut-off-saws>: HTTP status code is not handled or not allowed 2026-01-25 02:10:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-sheet-no-adhesive-70a-332-thick-x-36-wide-x-36-long>: HTTP status code is not handled or not allowed 2026-01-25 02:10:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=heavy-duty-manually-operated-hand-straight-brake-10-l-16-gauge already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:10:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=sheet-metal-notcher-4-hp-3-phase-220v-sn-f04-ms already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:10:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-sheet-no-adhesive-70a-332-thick-x-36-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 02:10:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/311-carton-sealing-tape-3-x-110-yds-205-mil-clear-24pack>: HTTP status code is not handled or not allowed 2026-01-25 02:10:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/355-carton-sealing-tape-2-x-55-yds-35-mil-clear-36pack>: HTTP status code is not handled or not allowed 2026-01-25 02:10:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-roll-no-adhesive-60a-132-thick-x-36-wide-x-5-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 02:10:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smi6-general-purpose-laboratory-incubator-5-7-cu-ft-162-l-115v>: HTTP status code is not handled or not allowed 2026-01-25 02:10:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:10:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smi39-reach-in-large-capacity-laboratory-incubator-38-6-cu-ft-1092-l-115v>: HTTP status code is not handled or not allowed 2026-01-25 02:10:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-intensity-infrared-patio-heater-with-transformer-natural-gas-140000-btu>: HTTP status code is not handled or not allowed 2026-01-25 02:10:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smi12-general-purpose-laboratory-incubator-dual-stacked-11-4-cu-ft-324-l-115v>: HTTP status code is not handled or not allowed 2026-01-25 02:10:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-sheet-no-adhesive-60a-116-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 02:10:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=heavy-duty-manually-operated-hand-straight-brake-10-l-16-gauge landed on page that is not a product page. 2026-01-25 02:10:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=sheet-metal-notcher-4-hp-3-phase-220v-sn-f04-ms>: HTTP status code is not handled or not allowed 2026-01-25 02:10:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/work-height-platform-truck-30-x-60-polyurethane-wheels>: HTTP status code is not handled or not allowed 2026-01-25 02:10:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:10:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/373-carton-sealing-tape-2-x-55-yds-25-mil-yellow-6pack>: HTTP status code is not handled or not allowed 2026-01-25 02:10:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets>: HTTP status code is not handled or not allowed 2026-01-25 02:10:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-part-stub-style-inventory-tag-pre-wired-3000-3999-8-6-14l-x-3-18w-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 02:10:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vpkq130-circuit-breaker-type-qp-1-pole-30a-clamshell-packaged>: HTTP status code is not handled or not allowed 2026-01-25 02:10:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carton-sealing-tape-mixed-merchandise-2-x-55-yds-22-mil-redwhite-18pack>: HTTP status code is not handled or not allowed 2026-01-25 02:10:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-shelf-truck-3-flush-shelves-24-x-48>: HTTP status code is not handled or not allowed 2026-01-25 02:10:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flush-handle-instrument-cart-lip-shelves-30-x-48>: HTTP status code is not handled or not allowed 2026-01-25 02:10:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ubitba230-circuit-breaker-type-a-2-pole-30a>: HTTP status code is not handled or not allowed 2026-01-25 02:10:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-welded-service-cart-2-flush-shelves-18-x-32>: HTTP status code is not handled or not allowed 2026-01-25 02:10:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ited230-classified-circuit-breaker-type-qd-replacement-for-square-d-type-qo-2-pole-30a>: HTTP status code is not handled or not allowed 2026-01-25 02:10:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/55-cup-coffee-urn-120v-1500w-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 02:10:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-25 02:10:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-shelving-closed-shelving-36x12x75-8>: HTTP status code is not handled or not allowed 2026-01-25 02:10:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-part-carbon-style-inventory-tag-2500-2999-8-6-14l-x-3-18w-500pack>: HTTP status code is not handled or not allowed 2026-01-25 02:10:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-general-security-weather-resistant-covered-laminated-padlocks-no-312>: HTTP status code is not handled or not allowed 2026-01-25 02:10:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/j26112-bomber-ii-hooded-jacket-fluorescent-yellowgreenblack-md>: HTTP status code is not handled or not allowed 2026-01-25 02:10:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-industrial-spray-nozzle-white>: HTTP status code is not handled or not allowed 2026-01-25 02:10:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hytrol-10-ft-12-inch-w-skatewheel-section-12-inch-oaw-10-wpf>: HTTP status code is not handled or not allowed 2026-01-25 02:10:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hytrol-24-inch-w-skatewheel-curve-24-inch-oaw-45-degree>: HTTP status code is not handled or not allowed 2026-01-25 02:10:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-x-5-hatch-net-high-vis-orange-webbing-free-placement-stainless-brackets-snap-hooks>: HTTP status code is not handled or not allowed 2026-01-25 02:10:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:10:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hytrol-5-ft-16-inch-w-1-9-inch-dia-galvanized-rollers-13-inch-between-rail-3-inch-rollers-centers>: HTTP status code is not handled or not allowed 2026-01-25 02:10:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluidity-02f-direct-acting-solenoid-valve-2-2-nc-fkm-seal-3-8-nptf-3-mm-110v-ac>: HTTP status code is not handled or not allowed 2026-01-25 02:10:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/capital-locker-single-tier-12x18x72-1-door-assembled-tan>: HTTP status code is not handled or not allowed 2026-01-25 02:10:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/355-carton-sealing-tape-2-x-55-yds-35-mil-clear-6pack>: HTTP status code is not handled or not allowed 2026-01-25 02:10:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paramount-locker-single-tier-12x12x60-1-door-assembled-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:10:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-part-carbonless-inventory-tag-pre-wired-0000-0499-8-6-14l-x-3-18w-500pack>: HTTP status code is not handled or not allowed 2026-01-25 02:10:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pinnacle-rubber-wall-base-coil-6-in-x-125-in-x-120-ft-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:11:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/220-acrylic-carton-sealing-tape-3-x-110-yds-22-mil-clear-6pack>: HTTP status code is not handled or not allowed 2026-01-25 02:11:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-delrin-homopolymer-sheet-0-500-thick-x-12-w-x-12-l>: HTTP status code is not handled or not allowed 2026-01-25 02:11:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-single-tab-white-helvetica-3-4-letters>: HTTP status code is not handled or not allowed 2026-01-25 02:11:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-25 02:11:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-300-nipple-3-8-hose-barb-aro-210-standard>: HTTP status code is not handled or not allowed 2026-01-25 02:11:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-sheet-with-acrylic-adhesive-60a-14-thick-x-12-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 02:11:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/j21207-eagle-storm-fly-front-jacket-yellow-hood-snaps-md>: HTTP status code is not handled or not allowed 2026-01-25 02:11:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vinyl-heavy-duty-ribbed-stair-tread-square-nose-125-x-48-pewter>: HTTP status code is not handled or not allowed 2026-01-25 02:11:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-af-blend-sheet-0-125-thick-x-24-w-x-24-l>: HTTP status code is not handled or not allowed 2026-01-25 02:11:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluidity-03f-hung-diaphragm-solenoid-valve-2-2-nc-epdm-3-4-nptf-18mm-110vac-31va>: HTTP status code is not handled or not allowed 2026-01-25 02:11:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pinnacle-rubber-wall-base-4-in-x-125-in-x-48-in-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:11:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wireway-husky-tear-drop-pallet-rack-add-on-96x48x144>: HTTP status code is not handled or not allowed 2026-01-25 02:11:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-raised-circular-stair-tread-square-nose-12063-x-60-brown>: HTTP status code is not handled or not allowed 2026-01-25 02:11:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 51274 pages (at 94 pages/min), scraped 25983 items (at 33 items/min) 2026-01-25 02:11:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/w-operated-pickup-trailer-truck-jib-crane-1000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 02:11:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-delrin-homopolymer-sheet-1-250-thick-x-24-w-x-24-l>: HTTP status code is not handled or not allowed 2026-01-25 02:11:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-sheet-no-adhesive-50a-132-thick-x-36-wide-x-36-long>: HTTP status code is not handled or not allowed 2026-01-25 02:11:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3m-ss-316316l-18-half-coupling-npt-female-plain>: HTTP status code is not handled or not allowed 2026-01-25 02:11:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vented-plastic-shelving-48x24x74-nexelon-finish>: HTTP status code is not handled or not allowed 2026-01-25 02:11:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-heavy-duty-double-shift-cabinet-12-gauge-36-w-x-24-d-x-66-h>: HTTP status code is not handled or not allowed 2026-01-25 02:11:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-315-coupling-1-4-male-npt-asian-interchange>: HTTP status code is not handled or not allowed 2026-01-25 02:11:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3779-carton-sealing-tape-check-seal-before-accepting-2-x-110-yds-clearred-6pk>: HTTP status code is not handled or not allowed 2026-01-25 02:11:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-delrin-homopolymer-sheet-3-000-thick-x-24-w-x-48-l>: HTTP status code is not handled or not allowed 2026-01-25 02:11:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-eyewash-bottle-mount-for-up-to-32-oz-bottles>: HTTP status code is not handled or not allowed 2026-01-25 02:11:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-gas-storage-caddy>: HTTP status code is not handled or not allowed 2026-01-25 02:11:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-bottle-eyewash-station-32-oz>: HTTP status code is not handled or not allowed 2026-01-25 02:11:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-circle-tag-pre-wired-2-dia-green-100pack>: HTTP status code is not handled or not allowed 2026-01-25 02:11:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conductive-dividable-grid-label-holder-8x2-sold-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 02:11:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-delrin-homopolymer-sheet-1-000-thick-x-12-w-x-24-l>: HTTP status code is not handled or not allowed 2026-01-25 02:11:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-2-in-line-vac-only-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 02:11:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o41008-safetyflex-plain-front-overall-green-4xl>: HTTP status code is not handled or not allowed 2026-01-25 02:11:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-delrin-homopolymer-rod-4-000-dia>: HTTP status code is not handled or not allowed 2026-01-25 02:11:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:11:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-in-super-air-knife-only-303ss>: HTTP status code is not handled or not allowed 2026-01-25 02:11:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/j53129-comfortbrite-jacket-fluorescent-orange-md>: HTTP status code is not handled or not allowed 2026-01-25 02:11:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-in-super-air-knife-only-303ss>: HTTP status code is not handled or not allowed 2026-01-25 02:11:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-delrin-homopolymer-rod-1-750-dia>: HTTP status code is not handled or not allowed 2026-01-25 02:11:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/box-patners-cool-shield-thermal-bubble-roll-60w-x-125l-x-316-bubble-silver-1-roll>: HTTP status code is not handled or not allowed 2026-01-25 02:11:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v70639-job-sight-class-2-vest-fluorescent-orange-polyester-mesh-sm>: HTTP status code is not handled or not allowed 2026-01-25 02:11:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:11:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-sheet-no-adhesive-50a-12-thick-x-36-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 02:11:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-clear-plastic-box-12-1-2-gallon-18x26x9>: HTTP status code is not handled or not allowed 2026-01-25 02:11:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1748s05-osha-stairway-48width-5-stairs>: HTTP status code is not handled or not allowed 2026-01-25 02:11:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-25 02:11:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/physicianscare-first-responder-fluid-spill-kit>: HTTP status code is not handled or not allowed 2026-01-25 02:11:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-ldpe-rod-3-000-dia>: HTTP status code is not handled or not allowed 2026-01-25 02:11:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-delrin-homopolymer-sheet-2-000-thick-x-12-w-x-12-l>: HTTP status code is not handled or not allowed 2026-01-25 02:11:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-x-24-18-gauge-stainless-steel-workbench-w-backsplash>: HTTP status code is not handled or not allowed 2026-01-25 02:11:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-bench-rack-with-12-bins-8-wide-33x12x21-1>: HTTP status code is not handled or not allowed 2026-01-25 02:11:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slatwall-black-shelves-3-4-x8-x2-1-2-finished-on-2-sides-and-3-edges>: HTTP status code is not handled or not allowed 2026-01-25 02:11:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dewalt-dxh70cfa-forced-air-propane-heater-68000-btu-20v-cordless>: HTTP status code is not handled or not allowed 2026-01-25 02:11:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-shelf-bench-rack-with-128-bins-4-wide-1>: HTTP status code is not handled or not allowed 2026-01-25 02:11:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls>: HTTP status code is not handled or not allowed 2026-01-25 02:11:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slatwall-h-display-fixture-cherry-w-casters>: HTTP status code is not handled or not allowed 2026-01-25 02:11:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-tag-pre-wired-12-8l-x-4w-orange-500pack>: HTTP status code is not handled or not allowed 2026-01-25 02:11:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fda-silicone-rubber-strip-with-high-temp-adhesive-50a-116-thick-x-2-wide-x-10-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 02:11:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-kynar-740-pvdf-sheet-0-750-thick-x-12-w-x-24-l>: HTTP status code is not handled or not allowed 2026-01-25 02:11:45 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-25 02:11:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-drawer-steel-junior-tall-medical-bedside-cart-push-button-lock-spring-green>: HTTP status code is not handled or not allowed 2026-01-25 02:11:46 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:26 2026-01-25 02:11:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96-x-24-18-gauge-stainless-steel-workbench-w-backsplash>: HTTP status code is not handled or not allowed 2026-01-25 02:11:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/machine_mounts_pads_vibration_dampers/vibration_isolating_spring_floor_mounts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:11:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-1-pneumatic-oil-pump-16-gal>: HTTP status code is not handled or not allowed 2026-01-25 02:11:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fda-silicone-rubber-sheet-no-adhesive-40a-38-thick-x-36-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 02:11:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:11:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/j67113-stormflex-zipper-front-hooded-jacket-black-xl>: HTTP status code is not handled or not allowed 2026-01-25 02:11:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ahf1-refrigerator-freezer-thermometer-nsf-listed>: HTTP status code is not handled or not allowed 2026-01-25 02:11:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tygothane-c-544-a-innerbraid-tube-azy02038-0-500-id-x-750-od-x-100-l>: HTTP status code is not handled or not allowed 2026-01-25 02:11:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fda-silicone-rubber-roll-no-adhesive-40a-132-thick-x-36-wide-x-8-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 02:11:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/expandable-starter-rack-48x18x84-gray-with-3-level-wood-deck-1500lb-cap-per-deck>: HTTP status code is not handled or not allowed 2026-01-25 02:11:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/expandable-starter-rack-48x24x84-gray-with-3-level-wood-deck-1500lb-cap-per-deck>: HTTP status code is not handled or not allowed 2026-01-25 02:11:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-tag-3-3-34l-x-1-78w--fluorescent-pink-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 02:11:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/key-tags-white-pkg-1000>: HTTP status code is not handled or not allowed 2026-01-25 02:11:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tygon-e-3603-tube-acf02017-0-250-id-x-375-od-x-100-l>: HTTP status code is not handled or not allowed 2026-01-25 02:11:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/machine_mounts_pads_vibration_dampers/vibration_isolating_spring_floor_mounts landed on page that is not a product page. 2026-01-25 02:11:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/neoprene-rubber-strip-no-adhesive-70a-14-thick-x-2-wide-x-10-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 02:11:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/p56007-durascrim-plain-front-pants-yellow-lg>: HTTP status code is not handled or not allowed 2026-01-25 02:12:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/three-m-ite-roloc-roll-on-coated-polyester-disc-14>: HTTP status code is not handled or not allowed 2026-01-25 02:12:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-25 02:12:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-basket-truck-24-bu-green-vinyl-wood-base-2-rg-2-sw>: HTTP status code is not handled or not allowed 2026-01-25 02:12:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-oil-filled-cast-nylon-sheet-0-750-thick-x-24-w-x-48-l>: HTTP status code is not handled or not allowed 2026-01-25 02:12:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3m-s-b-8x1-2x1-2-2s048011-15844>: HTTP status code is not handled or not allowed 2026-01-25 02:12:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/61500017985-belt-3-1-2w-x-15-1-2l-crs-grit>: HTTP status code is not handled or not allowed 2026-01-25 02:12:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tower-drawer-cart>: HTTP status code is not handled or not allowed 2026-01-25 02:12:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-roloc-type-t27-zir-reg-40-grit-flap-disc>: HTTP status code is not handled or not allowed 2026-01-25 02:12:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14x1-1-2x5-t1-az60-h8-v32agrinding-wheels>: HTTP status code is not handled or not allowed 2026-01-25 02:12:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tyvek174-shipping-tag-pre-wired-5-34l-x-2-78w-blue-100pack>: HTTP status code is not handled or not allowed 2026-01-25 02:12:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/expandable-starter-rack-36x24x84-gray-with-3-levels-no-deck-1500lb-cap-per-level>: HTTP status code is not handled or not allowed 2026-01-25 02:12:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-wire-truck-40-cu-ft-gray-vinyl>: HTTP status code is not handled or not allowed 2026-01-25 02:12:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-oil-filled-cast-nylon-rod-5-dia>: HTTP status code is not handled or not allowed 2026-01-25 02:12:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 51369 pages (at 95 pages/min), scraped 26008 items (at 25 items/min) 2026-01-25 02:12:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/expandable-add-on-rack-72x36x84-gray-with-3-levels-no-deck-750-lb-cap-per-level>: HTTP status code is not handled or not allowed 2026-01-25 02:12:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3m-s-b-3-a-vfn048011-05530>: HTTP status code is not handled or not allowed 2026-01-25 02:12:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-sheet-no-adhesive-50a-38-thick-x-36-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 02:12:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-step-industrial-access-stairway-ladder-grip-strut>: HTTP status code is not handled or not allowed 2026-01-25 02:12:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-step-industrial-access-stairway-ladder-perforated>: HTTP status code is not handled or not allowed 2026-01-25 02:12:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-step-industrial-access-stairway-ladder-perforated>: HTTP status code is not handled or not allowed 2026-01-25 02:12:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3m-cpd5-s-4x1-25-tan048011-14779>: HTTP status code is not handled or not allowed 2026-01-25 02:12:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-step-heavy-duty-600-lb-cap-safety-angle-steel-rolling-ladder-perfor>: HTTP status code is not handled or not allowed 2026-01-25 02:12:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/expandable-starter-rack-96x48x84-gray-with-3-level-wire-deck-800lb-cap-per-deck>: HTTP status code is not handled or not allowed 2026-01-25 02:12:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/del-polyester-filter-class-m-19-7-10-l-x-19-7-10-w-x-15-7-10-h>: HTTP status code is not handled or not allowed 2026-01-25 02:12:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/welded-workbench-butcher-block-top-fixed-height-24-x-48>: HTTP status code is not handled or not allowed 2026-01-25 02:12:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wide-span-rack-60x36x84-gray-with-3-shelves-laminated-deck-1200-lb-cap-per-level>: HTTP status code is not handled or not allowed 2026-01-25 02:12:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kapton-hn-film-0-005-thick-x-24-w-x-24-000-l>: HTTP status code is not handled or not allowed 2026-01-25 02:12:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wide-span-rack-60x48x96-gray-with-3-shelves-laminated-deck-1200-lb-cap-per-level>: HTTP status code is not handled or not allowed 2026-01-25 02:12:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/del-square-brush-antistatic>: HTTP status code is not handled or not allowed 2026-01-25 02:12:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mb924b-neoprene-steel-toe-snugleg-boots-brick-redbrown-size-6>: HTTP status code is not handled or not allowed 2026-01-25 02:12:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mb924b-neoprene-steel-toe-snugleg-boots-brick-redbrown-size-7>: HTTP status code is not handled or not allowed 2026-01-25 02:12:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gss-saftey-8505-3-in-1-waterproof-parka-class-3-lime-black-5xl>: HTTP status code is not handled or not allowed 2026-01-25 02:12:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pedestal-stroller-for-high-velocity-dock-fans>: HTTP status code is not handled or not allowed 2026-01-25 02:12:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-30-glass-filled-nylon-6-6-rod-0-375-dia>: HTTP status code is not handled or not allowed 2026-01-25 02:12:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-cas-761-durostone-sheet-0-472-thick-x-48-000-w-x-96-000-l>: HTTP status code is not handled or not allowed 2026-01-25 02:12:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/bins_totes_containers/bins/cubby_bins_trays already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:12:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cubby-locker-welded-34-1-2w-x-13-1-2d-x-52h-12-compartments-medium-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:12:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gloveworks-hd-diamond-textured-nitrile-industrial-gloves-xl-green-100-per-box-10-box-per-cs>: HTTP status code is not handled or not allowed 2026-01-25 02:12:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/retail_fixtures_displays_equipment/displays_display_racks/gondola_shelving already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:12:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torlon-4203-balls-0-250-dia-250-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:12:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-4-shelf-unit-60-x-36-x-13-1-2-9373-3-03>: HTTP status code is not handled or not allowed 2026-01-25 02:12:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-adjustable-4-shelf-unit-60-x-42-x-16-9364-3-03>: HTTP status code is not handled or not allowed 2026-01-25 02:12:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-kit-for-double-sided-12-and-16-rail-units>: HTTP status code is not handled or not allowed 2026-01-25 02:12:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-wire-shelf-truck-72x18x69-1200-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 02:12:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eva-tubing-shrink-wrapped-coil-w-label-3-4-i-d-x-1-8-wall-25-feet>: HTTP status code is not handled or not allowed 2026-01-25 02:12:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eva-tubing-shrink-wrapped-coil-w-label-1-i-d-x-1-8-wall-25-feet>: HTTP status code is not handled or not allowed 2026-01-25 02:12:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided16-rail-unit-with-96-qus230-ultra-stack-hang-bins-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:12:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eva-tubing-w-nylon-braid-reinforcement-bulk-1-4-i-d-300-feet>: HTTP status code is not handled or not allowed 2026-01-25 02:12:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eva-tubing-w-nylon-braid-reinforcement-bulk-3-8-i-d-300-feet>: HTTP status code is not handled or not allowed 2026-01-25 02:12:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/bins_totes_containers/bins/cubby_bins_trays landed on page that is not a product page. 2026-01-25 02:12:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-welded-storage-locker-30-x-48>: HTTP status code is not handled or not allowed 2026-01-25 02:12:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/retail_fixtures_displays_equipment/displays_display_racks/gondola_shelving>: HTTP status code is not handled or not allowed 2026-01-25 02:12:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alkaline-cleaner-and-degreaser-gallon-bottle-4-bottles>: HTTP status code is not handled or not allowed 2026-01-25 02:12:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/906-ni-m22x1-5-a-stainless-threaded-plug-with-m22-x-1-5-tapered-thread>: HTTP status code is not handled or not allowed 2026-01-25 02:12:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gift-boxes-3l-x-3w-x-3h-kraft-100pack>: HTTP status code is not handled or not allowed 2026-01-25 02:12:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-top-machine-table-18-x-24-x-36-swivel-casters-w-brakes>: HTTP status code is not handled or not allowed 2026-01-25 02:12:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-sided-16-rail-unit-with-48-qus230-ultra-stack-hang-bins-black>: HTTP status code is not handled or not allowed 2026-01-25 02:12:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/252-m20x1-5-a-steel-threaded-plug-m20-x-1-5-thread>: HTTP status code is not handled or not allowed 2026-01-25 02:12:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g5-145t-ap-7-5hp-oil-injected-rotary-screw-71-gallon-tank-mount-145psi-1ph-230v>: HTTP status code is not handled or not allowed 2026-01-25 02:12:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-wall-container-24x15x7-1-2>: HTTP status code is not handled or not allowed 2026-01-25 02:12:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/great-lakes-tin-jamestown-2-x-2-nail-up-tin-ceiling-tile-in-antique-white-t51-02>: HTTP status code is not handled or not allowed 2026-01-25 02:12:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/office_furniture/literature_mail_sorters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:12:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/welders-table-with-drawer>: HTTP status code is not handled or not allowed 2026-01-25 02:12:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-magnetic-write-on-wipe-off-tape-rolls-2x-50-ft-fm2118>: HTTP status code is not handled or not allowed 2026-01-25 02:12:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g4-145t-aff-5hp-oil-injected-rotary-screw-71-gal-tank-145psi-208-230-460v-w-dryer>: HTTP status code is not handled or not allowed 2026-01-25 02:12:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/products-crossover-truck-box-27x71x23-matte-black>: HTTP status code is not handled or not allowed 2026-01-25 02:12:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-z-brite-wire-add-on-30wx14dx54h>: HTTP status code is not handled or not allowed 2026-01-25 02:12:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/great-lakes-tin-syracuse-2-x-2-nail-up-tin-ceiling-tile-in-bronze-burst-t50-06>: HTTP status code is not handled or not allowed 2026-01-25 02:12:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/54h-nexel-chrome-wire-shelving-add-on-60wx21d>: HTTP status code is not handled or not allowed 2026-01-25 02:12:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/63h-nexel-chrome-wire-shelving-add-on-54wx21d>: HTTP status code is not handled or not allowed 2026-01-25 02:12:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/synthetic-extra-lightweight-oil-4-oz-bottle-iso-46-clear>: HTTP status code is not handled or not allowed 2026-01-25 02:12:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/office_furniture/literature_mail_sorters landed on page that is not a product page. 2026-01-25 02:12:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/health-gards-green-seal-recycled-toilet-seat-covers-250-pack-4-packs-case>: HTTP status code is not handled or not allowed 2026-01-25 02:12:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/49-56-9667-2-bi-metal-hole-saw-w-3-8-arbor>: HTTP status code is not handled or not allowed 2026-01-25 02:13:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/63h-nexel-chrome-wire-shelving-add-on-30wx21d>: HTTP status code is not handled or not allowed 2026-01-25 02:13:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-z-brite-wire-shelving-54wx24dx54h>: HTTP status code is not handled or not allowed 2026-01-25 02:13:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-32-4512-12-magnetic-bit-tip-holder>: HTTP status code is not handled or not allowed 2026-01-25 02:13:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnum-300-welding-gun-035-045-liner-15l-cable>: HTTP status code is not handled or not allowed 2026-01-25 02:13:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-z-brite-wire-shelving-42wx24dx54h>: HTTP status code is not handled or not allowed 2026-01-25 02:13:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/great-lakes-tin-ludington-2-x-2-lay-in-tin-ceiling-tile-in-penny-vein-y57-05>: HTTP status code is not handled or not allowed 2026-01-25 02:13:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-gallon-plastic-liner-black-1>: HTTP status code is not handled or not allowed 2026-01-25 02:13:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/libman-commercial-kitchen-brush>: HTTP status code is not handled or not allowed 2026-01-25 02:13:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gxt3000-gmrs-two-way-radio-36-miles-black-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-25 02:13:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultrasonic-table-top-parts-cleaning-system-digital-timer-heat-power-control-7-gal-45-khz-230v>: HTTP status code is not handled or not allowed 2026-01-25 02:13:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 51463 pages (at 94 pages/min), scraped 26034 items (at 26 items/min) 2026-01-25 02:13:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/86h-nexel-chrome-wire-shelving-48wx21d>: HTTP status code is not handled or not allowed 2026-01-25 02:13:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forkli-maintenance-platform-all-welded-48x40>: HTTP status code is not handled or not allowed 2026-01-25 02:13:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/great-lakes-tin-syracuse-2-x-2-lay-in-tin-ceiling-tile-in-penny-vein-y50-05>: HTTP status code is not handled or not allowed 2026-01-25 02:13:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-optima-no-stick-aluminum-fry-pan>: HTTP status code is not handled or not allowed 2026-01-25 02:13:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4c-horizontal-mailbox-37-1-2h-double-9-mb1-2-pl-doors-front-load-aluminum-usps>: HTTP status code is not handled or not allowed 2026-01-25 02:13:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/probite-174-1-2-x-1-2-mnpt-lead-free-brass-straight-male-coupling>: HTTP status code is not handled or not allowed 2026-01-25 02:13:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/panel-kit-72-w-workbench-with-18-w-36-w-whiteboard-mounting-rail-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:13:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/key-5513028>: HTTP status code is not handled or not allowed 2026-01-25 02:13:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2221vs-mag-pickup-bracket-5517384>: HTTP status code is not handled or not allowed 2026-01-25 02:13:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inner-metal-wire-basket-w-handle-11-dia-7-1-2h>: HTTP status code is not handled or not allowed 2026-01-25 02:13:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reechcraft-powerlift-mast-with-height-control-48h>: HTTP status code is not handled or not allowed 2026-01-25 02:13:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reechcraft-pl-xm-outrigger-base-powerlift-242-38-platform-height-350-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 02:13:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bitk1616-16-x-16-exposed-flange-knurled-knob>: HTTP status code is not handled or not allowed 2026-01-25 02:13:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/prf072wwg-0-white-diamond-glass-door-refrigerator-freezer-combo-7-cu-ft>: HTTP status code is not handled or not allowed 2026-01-25 02:13:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cut-true-fully-automatic-hydraulic-guillotine-paper-cutter>: HTTP status code is not handled or not allowed 2026-01-25 02:13:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/embassy-34-element-60-panel-track-heaters>: HTTP status code is not handled or not allowed 2026-01-25 02:13:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/embassy-scc-r-right-end-cap>: HTTP status code is not handled or not allowed 2026-01-25 02:13:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glance-powerized-glass-and-surface-cleaner-liquid-gallon-bottle-2-case>: HTTP status code is not handled or not allowed 2026-01-25 02:13:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fabric-softener-sheets-outdoor-fresh-15-sheets-box-15-box-case>: HTTP status code is not handled or not allowed 2026-01-25 02:13:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulldog-5500-engine-driven-welder-kohler-100-199a-60hz-20-25v>: HTTP status code is not handled or not allowed 2026-01-25 02:13:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2221vs-key-5510093>: HTTP status code is not handled or not allowed 2026-01-25 02:13:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2232ac-seal-oil-5510184>: HTTP status code is not handled or not allowed 2026-01-25 02:13:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keyguard-lab-coat-3-pockets-elastic-wrists-snap-front-single-collar-white-md-30case>: HTTP status code is not handled or not allowed 2026-01-25 02:13:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/food-safe-motor-cfswdm3538-e-ip69-5-hp-1765-rpm-3-ph-60-hz-56-c-3516m-tenv-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:13:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/element-kit-for-arrow-inline-tool-filter-9074>: HTTP status code is not handled or not allowed 2026-01-25 02:13:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/click-gel-automatic-toilet-bowl-cleaner-lavender-fields-6-box-4-boxes-case>: HTTP status code is not handled or not allowed 2026-01-25 02:13:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-coverallbunny-suit-elastic-wrists-attached-hood-boots-zipper-front-2xl-25cs>: HTTP status code is not handled or not allowed 2026-01-25 02:13:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-element-kit-for-arrow-mini-particulate-filters>: HTTP status code is not handled or not allowed 2026-01-25 02:13:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arsenal-5700-16-pocket-apron>: HTTP status code is not handled or not allowed 2026-01-25 02:13:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-soft-facial-tissue-2-ply-white-56-sheets-box-4-boxes-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:13:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd-polypropylene-lab-coat-no-pockets-elastic-wrists-snap-front-single-collar-white-3xl-30cs>: HTTP status code is not handled or not allowed 2026-01-25 02:13:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-storage-drawer-cabinet-30x27x32-12>: HTTP status code is not handled or not allowed 2026-01-25 02:13:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dispatcher-locker-triple-front-door-triple-rear-door-24w-x-25-1-2d-x-79-1-2h>: HTTP status code is not handled or not allowed 2026-01-25 02:13:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd-polypropylene-coverall-elastic-wrists-ankles-zipper-front-single-collar-white-4xl-25cs>: HTTP status code is not handled or not allowed 2026-01-25 02:13:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/odorshield-tall-kitchen-drawstring-bags-13-gal-0-95-mil-lavendar-white-240-case>: HTTP status code is not handled or not allowed 2026-01-25 02:13:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/160-expanding-steel-aluminum-barricade-yellow-black-37h-x-16-l-fpa160yb>: HTTP status code is not handled or not allowed 2026-01-25 02:13:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/750-retracting-belt-barrier-wall-mounted-black-housing-yellow-black-belt-75-l>: HTTP status code is not handled or not allowed 2026-01-25 02:13:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/upper-wheel-shaft-5782551>: HTTP status code is not handled or not allowed 2026-01-25 02:13:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-x-1-1-8-drywall-screw-phillips-bugle-head-steel-black-phosphate-sharp-point-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 02:13:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/viking153-3350-welding-helmet-5-13-shade-twisted-metal>: HTTP status code is not handled or not allowed 2026-01-25 02:13:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-drawer-unit-divider-kit-12-compartment-13>: HTTP status code is not handled or not allowed 2026-01-25 02:13:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylon-beard-net-honeycomb-116-hole-100-latex-free-black-18-100bag->: HTTP status code is not handled or not allowed 2026-01-25 02:13:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-outdoor-drinking-fountain-bottle-filler-black>: HTTP status code is not handled or not allowed 2026-01-25 02:13:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-x-2-1-4-drywall-screw-phillips-bugle-head-steel-black-phosphate-sharp-point-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:13:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stretch-n-dust-cloths-12-3-5-x-17-yellow-400-case>: HTTP status code is not handled or not allowed 2026-01-25 02:13:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-outdoor-pedestal-bottle-filling-station-ss>: HTTP status code is not handled or not allowed 2026-01-25 02:13:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-drawer-unit-divider-kit-20-compartment-3>: HTTP status code is not handled or not allowed 2026-01-25 02:13:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-drawer-unit-divider-kit-20-compartment-5>: HTTP status code is not handled or not allowed 2026-01-25 02:13:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/410-airug-anti-fatigue-mat-38-thick-4w-x-60l-blackyellow-border>: HTTP status code is not handled or not allowed 2026-01-25 02:13:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/advanced-jumbo-bath-tissue-septic-safe-2-ply-white-1600-ft-roll-6-rolls-case>: HTTP status code is not handled or not allowed 2026-01-25 02:13:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-drawer-unit-divider-kit-20-compartment-18>: HTTP status code is not handled or not allowed 2026-01-25 02:13:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/410-airug-anti-fatigue-mat-38-thick-3w-x-5l-black>: HTTP status code is not handled or not allowed 2026-01-25 02:13:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7060-shaft-5512282>: HTTP status code is not handled or not allowed 2026-01-25 02:13:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sms-coverall-elastic-wrists-ankles-attached-hood-zipper-front-white-4xl-25case>: HTTP status code is not handled or not allowed 2026-01-25 02:13:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ep-ms1a-c-earpiece-1a-tubeless-listen-only-3-5mm-clear>: HTTP status code is not handled or not allowed 2026-01-25 02:13:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8205hl-class-2-super-econo-vest-orange-4xl-5xl>: HTTP status code is not handled or not allowed 2026-01-25 02:13:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/am73915mzt-edge-handheld-digital-microscope-with-polarizer-5mp-20x-220x>: HTTP status code is not handled or not allowed 2026-01-25 02:13:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sms-lab-coat-no-pockets-elastic-wrists-snap-front-single-collar-blue-3xl-30case>: HTTP status code is not handled or not allowed 2026-01-25 02:13:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30k-btu-dual-zone-heat-pump-split-system-with-12k-and-18k-indoor-high-wall-units>: HTTP status code is not handled or not allowed 2026-01-25 02:13:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8229z-class-2-economy-two-tone-vest-orange-l-xl>: HTTP status code is not handled or not allowed 2026-01-25 02:13:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-gallon-hazmat-spill-kit-spkhz-10>: HTTP status code is not handled or not allowed 2026-01-25 02:13:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/small-red-bottle-tote-safety-carrier-500ml-and-1000ml-bottles-capacity>: HTTP status code is not handled or not allowed 2026-01-25 02:13:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bullet-office-desk-71wx35d-driftwood>: HTTP status code is not handled or not allowed 2026-01-25 02:14:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelite-shelf-divider-24>: HTTP status code is not handled or not allowed 2026-01-25 02:14:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-adjustable-solid-brute-shelf-15-wx60-l>: HTTP status code is not handled or not allowed 2026-01-25 02:14:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/380947-mini-clamp-meter-greenorange>: HTTP status code is not handled or not allowed 2026-01-25 02:14:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sunset-stacking-outdoor-armchair-solid-gray-sling-on-platinum-gray-frame-pk-of-4>: HTTP status code is not handled or not allowed 2026-01-25 02:14:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelite-shelf-ledge-54>: HTTP status code is not handled or not allowed 2026-01-25 02:14:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/handle-assembly-for-manual-pallet-jack-truck-mo-18e1-fits-model-eco-i-55-newer>: HTTP status code is not handled or not allowed 2026-01-25 02:14:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/relief-strain-4224-6295729>: HTTP status code is not handled or not allowed 2026-01-25 02:14:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/382252-earth-ground-resistance-tester-kit-200-ac-voltage>: HTTP status code is not handled or not allowed 2026-01-25 02:14:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-32w-x-42l-2-mil-clear-200pack>: HTTP status code is not handled or not allowed 2026-01-25 02:14:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jet-jtm-949evs-mill-acu-rite-200s-dro-x-and-y-axis-powerfeeds-698177>: HTTP status code is not handled or not allowed 2026-01-25 02:14:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 51564 pages (at 101 pages/min), scraped 26062 items (at 28 items/min) 2026-01-25 02:14:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/81-h-kiosk-unit-beige>: HTTP status code is not handled or not allowed 2026-01-25 02:14:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-12-vermont-gage-hss-extra-long-drill-blank>: HTTP status code is not handled or not allowed 2026-01-25 02:14:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/27-5-x-140mm-15-dram-clear-vial-w-24-410-green-pp-hole-cap-ptfe-silicone-septa-72pk>: HTTP status code is not handled or not allowed 2026-01-25 02:14:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/17-x-60mm-2-dram-vial-w-15-425-green-thermoset-f217-ptfe-lined-cap-cleaned-144pk>: HTTP status code is not handled or not allowed 2026-01-25 02:14:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kennedy-174-275xb-52611b-27-w-x-18-d-x-53-h-16-drawer-roller-cabinet-machinest-chest-combo>: HTTP status code is not handled or not allowed 2026-01-25 02:14:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/food_beverages/snacks_candies/nutrition_bars_mixes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:14:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-step-steel-orange-tilt-and-roll-ladder-with-serrated-grating-tr-5-g-o>: HTTP status code is not handled or not allowed 2026-01-25 02:14:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-step-steel-orange-tilt-and-roll-ladder-with-perforated-tread-tr-2-p-o>: HTTP status code is not handled or not allowed 2026-01-25 02:14:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-7w-x-16l-1-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 02:14:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n-ferno-6870-2-layer-cold-series-mouth-piece>: HTTP status code is not handled or not allowed 2026-01-25 02:14:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8210z-class-2-economy-vest-lime-l-xl>: HTTP status code is not handled or not allowed 2026-01-25 02:14:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:14:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wallpro-twin-black-post-retracting-belt-barrier-75-ft-red-authorized-belt>: HTTP status code is not handled or not allowed 2026-01-25 02:14:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/28-400-black-ribbed-polypropylene-unlined-cap-packed-in-bags-of-100-500pk>: HTTP status code is not handled or not allowed 2026-01-25 02:14:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/food_beverages/snacks_candies/nutrition_bars_mixes>: HTTP status code is not handled or not allowed 2026-01-25 02:14:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/neoprene-cord-pad-2-x-2-x-1>: HTTP status code is not handled or not allowed 2026-01-25 02:14:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-use-speciman-containers-with-sterile-fluid-pathway-4-oz-100-case-dynd30330>: HTTP status code is not handled or not allowed 2026-01-25 02:14:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-cantilever-t-bar-shelf-18-wx48-l>: HTTP status code is not handled or not allowed 2026-01-25 02:14:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/long-sleeve-t-shirt-segmented-tape-hi-vis-yellow-4xl>: HTTP status code is not handled or not allowed 2026-01-25 02:14:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-4w-x-8l-1-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 02:14:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/long-sleeve-wicking-birdseye-t-shirt-w-pocket-hi-vis-yellow-4xl>: HTTP status code is not handled or not allowed 2026-01-25 02:14:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22-x-56mm-5-dram-clear-ps-snap-cap-vial-with-29mm-white-pp-linerless-snap-cap-144pk>: HTTP status code is not handled or not allowed 2026-01-25 02:14:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gray-42x15x84-steel-boltless-shelving-with-10-hopper-bins-red>: HTTP status code is not handled or not allowed 2026-01-25 02:14:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions landed on page that is not a product page. 2026-01-25 02:14:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/weathermaster-xtra-yellow-post-belt-barrier-3w-x-11-ft-yellow-authorized-belt>: HTTP status code is not handled or not allowed 2026-01-25 02:14:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8008-hi-vis-leg-gaiters-orange-osfa>: HTTP status code is not handled or not allowed 2026-01-25 02:14:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/le-corner-desk-dark-cherry-office-furniture-groupings-1>: HTTP status code is not handled or not allowed 2026-01-25 02:14:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-parka-hi-viz-yellow-medium>: HTTP status code is not handled or not allowed 2026-01-25 02:14:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-parka-hi-viz-yellow-xl>: HTTP status code is not handled or not allowed 2026-01-25 02:14:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-unit-with-one-enclosure-chrome-18w-x-36l-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 02:14:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-flat-cap-gel-knee-pads-black>: HTTP status code is not handled or not allowed 2026-01-25 02:14:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-4000-single-strap-wrist-support-black-large-right>: HTTP status code is not handled or not allowed 2026-01-25 02:14:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xd-smooth-aluminum-underbody-with-barn-door-18x24x48-1705425>: HTTP status code is not handled or not allowed 2026-01-25 02:14:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-washer-5625591>: HTTP status code is not handled or not allowed 2026-01-25 02:14:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-8-manual-combination-strapping-tool>: HTTP status code is not handled or not allowed 2026-01-25 02:14:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-two-pocket-portfolios-w-tang-fasteners-11-x-8-1-2-dark-blue-25-box>: HTTP status code is not handled or not allowed 2026-01-25 02:14:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/898-filament-tape-34-x-60-yds-66-mil-clear-6pack>: HTTP status code is not handled or not allowed 2026-01-25 02:14:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-economy-scissors-8-length-bent-handle-stainless-steel-black>: HTTP status code is not handled or not allowed 2026-01-25 02:14:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/water-wall-72-inch-high-connector-pin-45032-wwf-cp>: HTTP status code is not handled or not allowed 2026-01-25 02:14:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/breathable-foul-weather-coat-hi-viz-yellow-medium>: HTTP status code is not handled or not allowed 2026-01-25 02:14:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insul-sheet-s2s-3-83648>: HTTP status code is not handled or not allowed 2026-01-25 02:14:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acrylic-safety-ppe-dispenser-visitor-specs-deluxe-by-e-glasg-d>: HTTP status code is not handled or not allowed 2026-01-25 02:14:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/recess-kit-to-inset-xlerator-xleratorecohand-dryers-40502>: HTTP status code is not handled or not allowed 2026-01-25 02:14:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-8-dia-3-8-shank-3-8-loc-3-1-16-oal-2-flute-solid-carbide-double-end-mill-altin-flat>: HTTP status code is not handled or not allowed 2026-01-25 02:14:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-one-aluminum-document-box-2-5-capacity-holds-8-1-2w-x-11h-silver>: HTTP status code is not handled or not allowed 2026-01-25 02:14:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-shelf-truck-with-dolly-base-36x24x70-1600-pound-capacity>: HTTP status code is not handled or not allowed 2026-01-25 02:14:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/laser-printer-permanent-labels-812-x-11-white-100box>: HTTP status code is not handled or not allowed 2026-01-25 02:14:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speed-collection-premium-cold-weather-bib-pants-hi-viz-yellow-2xl>: HTTP status code is not handled or not allowed 2026-01-25 02:15:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anchor-6020m-grey-nylonknit-dk-grey-foam-palm-6020-m>: HTTP status code is not handled or not allowed 2026-01-25 02:15:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-visibility-premium-wicking-hoodie-hi-viz-orange-m>: HTTP status code is not handled or not allowed 2026-01-25 02:15:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/selfseal-catalog-envelope-12-x-15-12-white-100box>: HTTP status code is not handled or not allowed 2026-01-25 02:15:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-15w-x-20l-3-mil-clear-500pack>: HTTP status code is not handled or not allowed 2026-01-25 02:15:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/laser-printer-permanent-labels-2-x-4-white-1000box>: HTTP status code is not handled or not allowed 2026-01-25 02:15:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stop-flange-hbs-1321w-1321w-175>: HTTP status code is not handled or not allowed 2026-01-25 02:15:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/copy-paper-convenience-carton-92-brightness-20lb-812-x-11-white-2500ctn>: HTTP status code is not handled or not allowed 2026-01-25 02:15:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-selfstick-notes-112-x-2-yellow-12-100sheet-padpack>: HTTP status code is not handled or not allowed 2026-01-25 02:15:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/colored-paper-20lb-812-x-11-pink-500-sheetsream>: HTTP status code is not handled or not allowed 2026-01-25 02:15:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/copy_printer_paper already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:15:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classic-standard-wicking-birdseye-t-shirt-with-pocket-hi-viz-yellow-large>: HTTP status code is not handled or not allowed 2026-01-25 02:15:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exit-10x14-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 02:15:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 51660 pages (at 96 pages/min), scraped 26102 items (at 40 items/min) 2026-01-25 02:15:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fanfolded-popup-notes-3-x-3-5-colors-12-100sheet-padspack>: HTTP status code is not handled or not allowed 2026-01-25 02:15:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-38w-x-48l-4-mil-clear-100pack>: HTTP status code is not handled or not allowed 2026-01-25 02:15:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/411kas3blk-safety-padlock-1-12w-x-1-12h-black-3set>: HTTP status code is not handled or not allowed 2026-01-25 02:15:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-7w-x-7l-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 02:15:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-ball-joint-extractor-lever-type-w-short-hydrualic-spindle-1-5-8-40-mm-opening>: HTTP status code is not handled or not allowed 2026-01-25 02:15:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-regulator-jsg-0716n-302-0019>: HTTP status code is not handled or not allowed 2026-01-25 02:15:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/sticky_notes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:15:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-48w-x-60l-3-mil-clear-50pack>: HTTP status code is not handled or not allowed 2026-01-25 02:15:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/notice-monitored-by-video-camera-10x14-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 02:15:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/no-parking-18x12-063-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 02:15:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/paper_stationery/copy_printer_paper>: HTTP status code is not handled or not allowed 2026-01-25 02:15:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/notebooks_pads already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:15:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/411kas3ylw-safety-padlock-1-12w-x-1-12h-yellow-3set>: HTTP status code is not handled or not allowed 2026-01-25 02:15:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-mid-length-twill-w-mouthpiece-navy-6-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:15:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-jaw-self-centering-puller-75-ton-capacity-7-8-to-9-7-8-open-7-7-8-reach>: HTTP status code is not handled or not allowed 2026-01-25 02:15:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mesh-swivel-ergonomic-task-office-chair-flip-up-arms-gray-mid-back>: HTTP status code is not handled or not allowed 2026-01-25 02:15:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sparkplugs-disposable-ear-plug-vinyl-corded-6615>: HTTP status code is not handled or not allowed 2026-01-25 02:15:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/paper_stationery/sticky_notes>: HTTP status code is not handled or not allowed 2026-01-25 02:15:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speed-collection-premium-breathable-rain-pants-hi-viz-yellow-4xl>: HTTP status code is not handled or not allowed 2026-01-25 02:15:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bearing-fitting-kit-3-8-to-2-dia-33-impact-rings-3-strike-pipes-ha-mmer>: HTTP status code is not handled or not allowed 2026-01-25 02:15:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/paper_stationery/notebooks_pads>: HTTP status code is not handled or not allowed 2026-01-25 02:15:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anchor-beam-stationary-2108406>: HTTP status code is not handled or not allowed 2026-01-25 02:15:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/defender-respirator-cartridge-w-p100-filter-75scp100>: HTTP status code is not handled or not allowed 2026-01-25 02:15:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chain-plastic-2-links-safety-orange-100-feet-trade-size-8>: HTTP status code is not handled or not allowed 2026-01-25 02:15:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pathway-hd-code-compliant-wheelchair-ramp-kit-8-x-48-w-guards-2-pk>: HTTP status code is not handled or not allowed 2026-01-25 02:15:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pathway-hd-code-compliant-wheelchair-ramp-kit-24-x-48-w-guards-3-pk>: HTTP status code is not handled or not allowed 2026-01-25 02:15:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metaltech-pig-tail-lock>: HTTP status code is not handled or not allowed 2026-01-25 02:15:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/406kas12prp-safety-padlock-purple-12set>: HTTP status code is not handled or not allowed 2026-01-25 02:15:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pathway-hd-code-compliant-wheelchair-ramp-kit-36-x-48-w-turnback-guards-3-pk>: HTTP status code is not handled or not allowed 2026-01-25 02:15:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manyard-w-locking-snaps6-ft-shock-absor-216wls-6ftyl>: HTTP status code is not handled or not allowed 2026-01-25 02:15:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stop-cs275-45>: HTTP status code is not handled or not allowed 2026-01-25 02:15:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-11w-x-24l-3-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 02:15:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plus-nutrition-boost-bar-cranberry-almond-1-4-oz>: HTTP status code is not handled or not allowed 2026-01-25 02:15:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fiberglass-stacking-wash-box-23-3-8-x-12-x-4-3-8-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:15:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-safety-drum-with-high-intensity-sheeting>: HTTP status code is not handled or not allowed 2026-01-25 02:15:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blowers-68-1463-fwez-600-2065>: HTTP status code is not handled or not allowed 2026-01-25 02:15:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-twister-hardwood-handle>: HTTP status code is not handled or not allowed 2026-01-25 02:15:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s1-safety-quick-coupling-1-4-aro-210-interchange-with-1-2-mnpt-connection>: HTTP status code is not handled or not allowed 2026-01-25 02:15:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-8w-x-9l-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 02:15:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/45-266-216-sit-stand-combo-arm-white>: HTTP status code is not handled or not allowed 2026-01-25 02:15:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-plug-body-with-industrial-profile-1-2-hose-barb-connection-used-with-isi-coupling>: HTTP status code is not handled or not allowed 2026-01-25 02:15:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electronic-ethernet-time-and-attendance-system>: HTTP status code is not handled or not allowed 2026-01-25 02:15:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-unit-5-tier-nexelon-24w-x-72l-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 02:15:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-15w-x-18l-6-mil-clear-250pack>: HTTP status code is not handled or not allowed 2026-01-25 02:15:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/13-step-24in-w-steel-safety-angle-rolling-ladder-grip-strut-gray-kdec113242>: HTTP status code is not handled or not allowed 2026-01-25 02:15:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drill-chuck-arbor-mt3-to-jt2>: HTTP status code is not handled or not allowed 2026-01-25 02:15:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-step-24in-w-steel-safety-angle-rolling-ladder-grip-strut-gray-kdec105242>: HTTP status code is not handled or not allowed 2026-01-25 02:15:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-jaw-solid-combination-chuck-plain-back-6-1-4-inch>: HTTP status code is not handled or not allowed 2026-01-25 02:16:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/lathe_chucks_accessories/lathe_chuck_parts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:16:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-poly-z-brite-24w-x-54l-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 02:16:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bull-nose-and-pipe-live-center-mt5-head-diameter-6-61-inch-t-i-r-0008>: HTTP status code is not handled or not allowed 2026-01-25 02:16:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/penco-locker-single-tier-12x12x72-3-door-assembled-champagne>: HTTP status code is not handled or not allowed 2026-01-25 02:16:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/lathe_chucks_accessories/lathe_chuck_parts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:16:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scroll-plate-for-5-inch-5c-collet-plain-back-chucks>: HTTP status code is not handled or not allowed 2026-01-25 02:16:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hard-top-jaws-for-scroll-chuck-10-inch-3-jaw-3-piece-set>: HTTP status code is not handled or not allowed 2026-01-25 02:16:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elements-34-1595-imerpc13-093>: HTTP status code is not handled or not allowed 2026-01-25 02:16:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-tpi-portable-blower-fan-belt-drive-hazardous-location>: HTTP status code is not handled or not allowed 2026-01-25 02:16:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 51754 pages (at 94 pages/min), scraped 26142 items (at 40 items/min) 2026-01-25 02:16:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-jaw-2-piece-scroll-chuck-forged-steel-body-camlock-d-6-inch-d1-4>: HTTP status code is not handled or not allowed 2026-01-25 02:16:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/lathe_chucks_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:16:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-1-2-body-plug-with-iso-6150-profile-3-8-fnpt-used-with-csm-coupler>: HTTP status code is not handled or not allowed 2026-01-25 02:16:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/counter-top-crt-security-computer-cabinet-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:16:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reel-rack-starter-unit-24-w-x-24-d-x-120-h>: HTTP status code is not handled or not allowed 2026-01-25 02:16:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/lathe_chucks_accessories/lathe_chuck_parts>: HTTP status code is not handled or not allowed 2026-01-25 02:16:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/877-00-knee-pad-fire-retardant-cover-hinged-hard-shell-ribbed-cap-premium-foam-pad>: HTTP status code is not handled or not allowed 2026-01-25 02:16:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f22guc-u-connector-type-1-fits-250x250-steelgray>: HTTP status code is not handled or not allowed 2026-01-25 02:16:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cleaning-station-shadow-board-combo-kit-yellow-black-72-x-36-acp-composite>: HTTP status code is not handled or not allowed 2026-01-25 02:16:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ram-assembly-jasj22c-103>: HTTP status code is not handled or not allowed 2026-01-25 02:16:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/lathe_chucks_accessories>: HTTP status code is not handled or not allowed 2026-01-25 02:16:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/vacuums/wet_dry_vacuums/specialty_shop_vacuums already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:16:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-parts-bags-with-hang-hole-6w-x-9l-4-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 02:16:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spd-label-jmlvs-109>: HTTP status code is not handled or not allowed 2026-01-25 02:16:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cleaning-station-shadow-board-combo-kit-red-white-72-x-36-acp-composite>: HTTP status code is not handled or not allowed 2026-01-25 02:16:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-inch-ready-set-staple-belt-lacing-galvanized-rs125j36>: HTTP status code is not handled or not allowed 2026-01-25 02:16:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/balldriver-l-wrench-key-sets-10999>: HTTP status code is not handled or not allowed 2026-01-25 02:16:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-step-24in-w-30in-d-top-step-steel-rolling-ladder-perforated-tread-36in-handrail-kdsr107246-d3>: HTTP status code is not handled or not allowed 2026-01-25 02:16:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/vacuums/wet_dry_vacuums/specialty_shop_vacuums landed on page that is not a product page. 2026-01-25 02:16:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/wire_shelving/poly_zbrite_wire_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:16:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-18w-x-18l-6-mil-clear-250pack>: HTTP status code is not handled or not allowed 2026-01-25 02:16:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-ventilated-locker-six-tier-12x15x12-6-door-assembled-1>: HTTP status code is not handled or not allowed 2026-01-25 02:16:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-step-aluminum-rolling-ladder-24in-w-ribbed-tread-14in-d-top-step-36in-handrails-wlar112244>: HTTP status code is not handled or not allowed 2026-01-25 02:16:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/name-plate-jlh80-34>: HTTP status code is not handled or not allowed 2026-01-25 02:16:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ck28-311-contactor-3-pole-24v>: HTTP status code is not handled or not allowed 2026-01-25 02:16:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ck28-440-contactor-4-pole-120v>: HTTP status code is not handled or not allowed 2026-01-25 02:16:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/switch-sticks-adjustable-folding-walking-cane-and-walking-stick-collapses-and-adjusts-from-32-to-37-inches-engraved-pearl-gold>: HTTP status code is not handled or not allowed 2026-01-25 02:16:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/shelving/wire_shelving/poly_zbrite_wire_shelves>: HTTP status code is not handled or not allowed 2026-01-25 02:16:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-gas-infrared-heater-092030ngs-straight-configuration-30-l-50000-btu>: HTTP status code is not handled or not allowed 2026-01-25 02:16:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/seat-cushion-for-wheelchairs-mobility-scooters-office-and-kitchen-chairs-or-car-seats-to-add-support-and-comfort-while-reducing-pressure-and-stress-on-back-3-inch-thick-16-x-18-navy-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:16:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-poly-z-brite-18w-x-60l-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 02:16:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f66lsag-seal-plate-90-deg-600x600-steelzinc>: HTTP status code is not handled or not allowed 2026-01-25 02:16:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hanging-chain-kit-1800csu05-for-u-configuration-infrared-heaters-5-l>: HTTP status code is not handled or not allowed 2026-01-25 02:16:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regulator-sprng-jsm-401-sm-40146>: HTTP status code is not handled or not allowed 2026-01-25 02:16:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-shelving-48x24x72-with-4-shelves-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 02:16:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1300-strapping-tape-12-x-60-yds-39-mil-clear-72pack>: HTTP status code is not handled or not allowed 2026-01-25 02:16:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-poly-z-brite-18w-x-30l-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 02:16:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cca22128-compact8482-cast-junction-box-type-4x-220x122x80mm>: HTTP status code is not handled or not allowed 2026-01-25 02:16:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corner-bumper-guard-w-39-inches-long-by-2inches-wide-black-and-yellow-60-6902>: HTTP status code is not handled or not allowed 2026-01-25 02:16:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-chrome-24w-x-30l-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 02:16:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-poly-z-brite-21w-x-72l-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 02:16:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/key-516x5-50-3388100>: HTTP status code is not handled or not allowed 2026-01-25 02:16:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30mm-non-metallic-illuminated-extended-head-momentary-op-full-volt-push-button-white>: HTTP status code is not handled or not allowed 2026-01-25 02:16:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-polypropylene-coverall-open-wrists-ankles-white-x-large-25-case>: HTTP status code is not handled or not allowed 2026-01-25 02:16:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f4490ein-90-degree-elbow-type-1-inside-fits-400x400-steelgray>: HTTP status code is not handled or not allowed 2026-01-25 02:16:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/din-rail-1-meter-standard-35mm>: HTTP status code is not handled or not allowed 2026-01-25 02:16:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f44t48hc-wire-trough-hinged-cover-400x400x4800-steelgray>: HTTP status code is not handled or not allowed 2026-01-25 02:16:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/model-no-e-110-hex-wrench-9-12-1-18-to-2-58-pipe-capacity>: HTTP status code is not handled or not allowed 2026-01-25 02:16:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c06-310-9-amp-mini-contactor-24v-coil>: HTTP status code is not handled or not allowed 2026-01-25 02:16:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-nexelon-18w-x-54l-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 02:16:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bunan-o-ring-10-inch-diameter>: HTTP status code is not handled or not allowed 2026-01-25 02:16:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proto-174-1-2-drive-ratcheting-head-micrometer-torque-wrench-16-80-lbs-asme>: HTTP status code is not handled or not allowed 2026-01-25 02:16:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-cork-roll-48-x-72>: HTTP status code is not handled or not allowed 2026-01-25 02:16:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/itm-12-6-14-sds-plus-carbide-hammer-drill-bit>: HTTP status code is not handled or not allowed 2026-01-25 02:16:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/itm-carbide-hole-saw-maintenance-kit>: HTTP status code is not handled or not allowed 2026-01-25 02:17:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/table-4-chair-set-black-polypropylene-cafe-chairs-36w-x-29h-round-graphite-table>: HTTP status code is not handled or not allowed 2026-01-25 02:17:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flammable-cabinet-with-manual-close-double-door-28-gallon>: HTTP status code is not handled or not allowed 2026-01-25 02:17:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/120-lb-keg-super-lube-synthetic-grease-nlgi-1>: HTTP status code is not handled or not allowed 2026-01-25 02:17:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11-oz-super-lube-penetrant-aerosol>: HTTP status code is not handled or not allowed 2026-01-25 02:17:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-stacking-bins-16-1-2-x-30-x-15-yellow>: HTTP status code is not handled or not allowed 2026-01-25 02:17:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 51849 pages (at 95 pages/min), scraped 26181 items (at 39 items/min) 2026-01-25 02:17:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-x-24-x-74-chrome-wire-shelving-with-10-24-d-bins-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:17:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/44040-12v-heavy-duty-electric-fuel-pump-explosion-proof-ul-approved-15-gpm>: HTTP status code is not handled or not allowed 2026-01-25 02:17:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gravity-flow-rack-add-on-1>: HTTP status code is not handled or not allowed 2026-01-25 02:17:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elite-workbench-2500c-sm04-cabine-accessory-kit>: HTTP status code is not handled or not allowed 2026-01-25 02:17:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guide-pltwscrs-1435-45-60-2900910>: HTTP status code is not handled or not allowed 2026-01-25 02:17:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/100-cfm-breather-box-8-outlets-schrader-fitting-bb100-co8>: HTTP status code is not handled or not allowed 2026-01-25 02:17:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-full-brim-4-point-ratchet-sleek-shell-hard-hat>: HTTP status code is not handled or not allowed 2026-01-25 02:17:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/personal_care_hygiene/hand_soap_cleaners already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:17:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-cabinets-fs-30-g-storage-cabinet-assembled-30x18x72-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:17:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/molded-urethane-cable-hose-protection-bridges-36-x-24-x-4>: HTTP status code is not handled or not allowed 2026-01-25 02:17:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-chain-1-link-100-l-hdpe-safety-orange>: HTTP status code is not handled or not allowed 2026-01-25 02:17:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-cabinets-bca-364213-g-bookcase-assembled-36x13x42-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:17:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-magnetic-clip-dispenser-two-compartments-plastic-2-1-2-x-2-1-2-x-3>: HTTP status code is not handled or not allowed 2026-01-25 02:17:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/computer-cabinet-side-car-black-1>: HTTP status code is not handled or not allowed 2026-01-25 02:17:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/personal_care_hygiene/hand_soap_cleaners landed on page that is not a product page. 2026-01-25 02:17:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nose-jsg-91219121ns-731009>: HTTP status code is not handled or not allowed 2026-01-25 02:17:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-pocket-oak-wall-rack-medium-oak>: HTTP status code is not handled or not allowed 2026-01-25 02:17:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6876-winter-hard-hat-liner-kit-fr-2-layer-mouthpiece-black>: HTTP status code is not handled or not allowed 2026-01-25 02:17:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/074-841-10bl-cat-6-utp-mold-injection-snagless-patch-cord-10-ft-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:17:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/074-822-5bk-cat-6-utp-mold-injection-snagless-patch-cord-5-ft-black>: HTTP status code is not handled or not allowed 2026-01-25 02:17:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electronic-packaging-workbench-esd-square-edge-72-x-30-with-riser-kit>: HTTP status code is not handled or not allowed 2026-01-25 02:17:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-foot-x-20-foot-heavy-duty-white-tarp>: HTTP status code is not handled or not allowed 2026-01-25 02:17:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-x-24-x-30-drawer-shelf-shop-stand-maple-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:17:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clean-and-strip-xt-pro-disc-t27-7-x-7-8-in-s-xcs-5-per-case>: HTTP status code is not handled or not allowed 2026-01-25 02:17:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fibre-disc-787c-120-4-1-2-x-7-8-in-die-450e-25-per-inner-100-per-case>: HTTP status code is not handled or not allowed 2026-01-25 02:17:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/surface-conditioning-disc-tn-quick-change-7-x-nh-a-med-25-per-case>: HTTP status code is not handled or not allowed 2026-01-25 02:17:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-x-24-x-30-double-shelf-shop-stand-maple-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:17:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f9469pc-double-sided-adhesive-transfer-tape-1-x-5-yds-5-mil-clear>: HTTP status code is not handled or not allowed 2026-01-25 02:17:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/27-0513b-9l-cut-resistant-gloves-nitrile-micro-foam-coating-a7-f-size-9>: HTTP status code is not handled or not allowed 2026-01-25 02:17:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flap-disc-769f-05926-120-7-x-7-8-in-type-27-5-per-case>: HTTP status code is not handled or not allowed 2026-01-25 02:17:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rivet-jvm-836-vm-l7-1>: HTTP status code is not handled or not allowed 2026-01-25 02:17:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-4w-x-7l-4-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 02:17:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-precision-ansi-stainless-steel-roller-chain-35-1ss-3-8-pitch-offset-link>: HTTP status code is not handled or not allowed 2026-01-25 02:17:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/feed-cover-jmd18-108>: HTTP status code is not handled or not allowed 2026-01-25 02:17:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/purofort-thermo-full-safety-mens-work-boots-size-14-green>: HTTP status code is not handled or not allowed 2026-01-25 02:17:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rocia-focus-led-spot-light-articulating-arm-12-28v-ac-12-40v-dc-85w-40-deg-beam-angle>: HTTP status code is not handled or not allowed 2026-01-25 02:17:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-tube-light-rl70le-84-n-led-22-26v-dc-42w-acrylic-42>: HTTP status code is not handled or not allowed 2026-01-25 02:17:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-kit-for-manual-pallet-jack-gwk-4123-lw-fits-uline-model-h-4123>: HTTP status code is not handled or not allowed 2026-01-25 02:17:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30020su-f30t12cw-30w-fluorescent-t12-bulb-medium-bipin-cool-white>: HTTP status code is not handled or not allowed 2026-01-25 02:17:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triango-universal-led-treatment-light-100-3-f-dimming-color-changing-floor-stand>: HTTP status code is not handled or not allowed 2026-01-25 02:17:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:18:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crossover-bridge-30-inch-l-x-24-inch-w-perforated>: HTTP status code is not handled or not allowed 2026-01-25 02:18:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cantilever-rack-single-sided-upright-21>: HTTP status code is not handled or not allowed 2026-01-25 02:18:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poly-tubing-4w-x-725l-4-mil-black-1-roll>: HTTP status code is not handled or not allowed 2026-01-25 02:18:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:18:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/05571su-ap2330k-air-purification-spiral-cfl-light-bulb-medium-base-warm-white>: HTTP status code is not handled or not allowed 2026-01-25 02:18:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30430su-f54t5830ho-54w-fluorescent-t5-mini-bipin-bulb-warm-white>: HTTP status code is not handled or not allowed 2026-01-25 02:18:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/complete-wheel-kit-for-manual-pallet-jack-gwk-hj55-ck-fits-total-source-model-hj55>: HTTP status code is not handled or not allowed 2026-01-25 02:18:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits>: HTTP status code is not handled or not allowed 2026-01-25 02:18:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 51944 pages (at 95 pages/min), scraped 26230 items (at 49 items/min) 2026-01-25 02:18:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40252-window-underlay-120-x-24-polystyrene>: HTTP status code is not handled or not allowed 2026-01-25 02:18:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-cal-diamond-grip-rubber-runner-mats-2mm-x-4-x-13-rubber-flooring-rolls-black>: HTTP status code is not handled or not allowed 2026-01-25 02:18:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40371-siphon-hose-rubber>: HTTP status code is not handled or not allowed 2026-01-25 02:18:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits>: HTTP status code is not handled or not allowed 2026-01-25 02:18:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-pedal-20-5x30x10-641244-641264-floor-scrubbers>: HTTP status code is not handled or not allowed 2026-01-25 02:18:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shaded-pole-blower-1-25-hp-60-cfm>: HTTP status code is not handled or not allowed 2026-01-25 02:18:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40402-top-open-blast-cabinet-w-dust-collector-steel>: HTTP status code is not handled or not allowed 2026-01-25 02:18:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speedpress-4-6-rhino-cutting-mat-grid>: HTTP status code is not handled or not allowed 2026-01-25 02:18:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-pressure-switch-150x135x5-n-o-641407-641245-641327-floor-scrubbers-sweepers>: HTTP status code is not handled or not allowed 2026-01-25 02:18:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speakman-anystream-icon-8-jet-showerhead-polished-brass-finish-2-5-gpm>: HTTP status code is not handled or not allowed 2026-01-25 02:18:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wypall-waterless-hand-wipes-herbal-fragrance-75-per-bucket>: HTTP status code is not handled or not allowed 2026-01-25 02:18:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pva-precision-material>: HTTP status code is not handled or not allowed 2026-01-25 02:18:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss-single-mesh-strainer-6-1-4-fine>: HTTP status code is not handled or not allowed 2026-01-25 02:18:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/method-build-tape-kit>: HTTP status code is not handled or not allowed 2026-01-25 02:18:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speakman-t-handle-adjustable-spray-institutional-showerhead-2-5-gpm>: HTTP status code is not handled or not allowed 2026-01-25 02:18:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40001-65-lb-portable-pressure-blaster-deadman-steel>: HTTP status code is not handled or not allowed 2026-01-25 02:18:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-ss-mixing-paddle>: HTTP status code is not handled or not allowed 2026-01-25 02:18:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8210hl-xs-lime-type-r-class-2-economy-mesh-vest>: HTTP status code is not handled or not allowed 2026-01-25 02:18:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/packing-workbench-plastic-square-edge-72-x-36-with-lower-shelf-kit>: HTTP status code is not handled or not allowed 2026-01-25 02:18:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rotary-waste-valve-with-twist-handle-male-adapter>: HTTP status code is not handled or not allowed 2026-01-25 02:18:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/office_machines/office_machine_accessories/3d_printer_filaments_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:18:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-athletic-sneakers-size-95m-black>: HTTP status code is not handled or not allowed 2026-01-25 02:18:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-athletic-sneakers-size-9m-black>: HTTP status code is not handled or not allowed 2026-01-25 02:18:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hndl-o-ring-p9-131101007-mg1b-03>: HTTP status code is not handled or not allowed 2026-01-25 02:18:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-beige-carpet-pad-5-per-case>: HTTP status code is not handled or not allowed 2026-01-25 02:18:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swing-nozzle-12-1-8in-length-5-11-16in-height>: HTTP status code is not handled or not allowed 2026-01-25 02:18:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/csd363016-concept-wall-mount-encl-3600x3000x1600-steelgray>: HTTP status code is not handled or not allowed 2026-01-25 02:18:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/complete-mobile-electric-packing-workbench-maple-butcher-block-square-edge-72-x-36>: HTTP status code is not handled or not allowed 2026-01-25 02:18:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/diamond-polishing-20-white-800-grit-deep-cleaning-pad-2-per-case>: HTTP status code is not handled or not allowed 2026-01-25 02:18:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/office_machines/office_machine_accessories/3d_printer_filaments_accessories>: HTTP status code is not handled or not allowed 2026-01-25 02:18:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-storage-tote-with-lid-56-quart>: HTTP status code is not handled or not allowed 2026-01-25 02:18:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/food-grade-55-gallon-double-action-drum-pump>: HTTP status code is not handled or not allowed 2026-01-25 02:18:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/500-gallon-fuelcube-diesel-fuel-tank>: HTTP status code is not handled or not allowed 2026-01-25 02:18:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18vx2-36v-lxt-174-li-ion-brushless-cordless-string-trimmer-15-max-cut-dia-tool-only>: HTTP status code is not handled or not allowed 2026-01-25 02:18:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hndl-jsg-0522-ap-70047>: HTTP status code is not handled or not allowed 2026-01-25 02:18:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/minigrip174-reclosable-poly-bags-12w-x-15l-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 02:18:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a42hs3112lp-disc-encl-wclamps-type-4-4200x3138x1200-steelgray>: HTTP status code is not handled or not allowed 2026-01-25 02:18:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/minigrip174-reclosable-poly-bags-4w-x-6l-4-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 02:18:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mattress-bags-fits-double-size-54w-x-7d-x-84l-11-mil-clear-150roll>: HTTP status code is not handled or not allowed 2026-01-25 02:18:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gib-knife-6292579>: HTTP status code is not handled or not allowed 2026-01-25 02:18:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/threaded-end-hardwood-broom-handle>: HTTP status code is not handled or not allowed 2026-01-25 02:18:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/csd30208-concept-wall-mount-encl-3000x2000x800-steelgray>: HTTP status code is not handled or not allowed 2026-01-25 02:18:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/packing-mobile-electric-workbench-maple-butcher-block-square-edge-60-x-36-with-lower-shelf-kit>: HTTP status code is not handled or not allowed 2026-01-25 02:18:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/front-end-plate-jsm5130-19>: HTTP status code is not handled or not allowed 2026-01-25 02:18:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/key-jsm-6196195b-sm-61934>: HTTP status code is not handled or not allowed 2026-01-25 02:18:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72x-36-steel-top-fixed-leg-work-table>: HTTP status code is not handled or not allowed 2026-01-25 02:18:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slurry-vacuum-system-30-gallon-capacity>: HTTP status code is not handled or not allowed 2026-01-25 02:18:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/afe8x8-flush-cover-for-pull-box-fits-800x800-950-x-950-steelgray>: HTTP status code is not handled or not allowed 2026-01-25 02:18:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/125-lf-3-2-12-hex-bushing-npt-male-female>: HTTP status code is not handled or not allowed 2026-01-25 02:18:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-enzymatic-chute-dumpster-wash-plus-orange-scent-gallon-bottle-4-case>: HTTP status code is not handled or not allowed 2026-01-25 02:19:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exq-pro-extreme-power-workstation-hutch-72w-x-30d-x-26-38h-orange-w-black-trim>: HTTP status code is not handled or not allowed 2026-01-25 02:19:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-ss-304-cast-welding-spud-1-12-npt-female>: HTTP status code is not handled or not allowed 2026-01-25 02:19:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e30pby25m-push-button-encl-extra-lgpanel-30pb-x-225mm>: HTTP status code is not handled or not allowed 2026-01-25 02:19:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-ss-304-cast-welding-spud-4-npt-female>: HTTP status code is not handled or not allowed 2026-01-25 02:19:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/wood_cleaners_polishes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:19:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waterhog-fashion-diamond-mat-charcoal-4-x-6>: HTTP status code is not handled or not allowed 2026-01-25 02:19:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ductless-mini-split-ac-with-heat-pump-wi-fi-control-22000-btu-25-installation-kit-white>: HTTP status code is not handled or not allowed 2026-01-25 02:19:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gold-fibre-7x5-double-wire-personal-130-sheet-pocket-notebook-classic-green>: HTTP status code is not handled or not allowed 2026-01-25 02:19:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nilozyme-153-trap-drain-treatment-fresh-scent-gallon-bottle-4-bottles-case>: HTTP status code is not handled or not allowed 2026-01-25 02:19:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chain-1632-6293359>: HTTP status code is not handled or not allowed 2026-01-25 02:19:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 52038 pages (at 94 pages/min), scraped 26266 items (at 36 items/min) 2026-01-25 02:19:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pop-up-notes-super-sticky-3-x-3-assorted-neon-colors-6-pk>: HTTP status code is not handled or not allowed 2026-01-25 02:19:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ingersoll-rand-2135qxpa-1-2-in-impactool-quiet>: HTTP status code is not handled or not allowed 2026-01-25 02:19:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ingersoll-rand-2135qxpa-2-1-2-in-impactool-quiet-2-in-extended-anvil>: HTTP status code is not handled or not allowed 2026-01-25 02:19:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-clips-for-clear-wall-seperators>: HTTP status code is not handled or not allowed 2026-01-25 02:19:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m40-non-powered-mobile-lcd-cart>: HTTP status code is not handled or not allowed 2026-01-25 02:19:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/wood_cleaners_polishes landed on page that is not a product page. 2026-01-25 02:19:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speedi-boot-square-to-round-adaptor-register-vent-boot-adjustable-hangers-for-hvac-duct-work-8-x-8x-8>: HTTP status code is not handled or not allowed 2026-01-25 02:19:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brass-breather-valve-low-profile-m10-x-1-0-thread-883-m10x1-160-a-ms>: HTTP status code is not handled or not allowed 2026-01-25 02:19:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90-176-ss-corner-guard-16-ga-3m-two-sided-tape-sq-corners-60-lx2-wx2-h>: HTTP status code is not handled or not allowed 2026-01-25 02:19:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quick-coupler-plug-industrial-interchange-m-style-14-fnpt-14-mnpt-6-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:19:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exq-pro-triple-bank-roller-cabinet-72w-x-30d-x-46-58horange-wblack-drawer-pulls>: HTTP status code is not handled or not allowed 2026-01-25 02:19:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-unit-5-tier-48w-x-24d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 02:19:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speedi-grille-return-air-vent-grille-vent-cover-20-x-20>: HTTP status code is not handled or not allowed 2026-01-25 02:19:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3m-duct-tape-3900-yellow-48-mm-x-54-8-m-7-7-mil-24-per-case>: HTTP status code is not handled or not allowed 2026-01-25 02:19:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/asg16x16x6nk-pull-box-screw-cover-1600x1600x600-galvanized>: HTTP status code is not handled or not allowed 2026-01-25 02:19:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galvanized-steel-load-stabilizer-1-1-2-o-d>: HTTP status code is not handled or not allowed 2026-01-25 02:19:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/filter-elements-12-npt-for-use-with-motorguard-m30-and-m60>: HTTP status code is not handled or not allowed 2026-01-25 02:19:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/solarguard-storage-master-building-14w-x-14h-x-28l-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:19:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-green-wire-shelving-starter-unit-5-tier-60w-x-18d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 02:19:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26x12x72-solar-star-greenhouse-w-poly-ends-and-drop-down-sides-gas-heater>: HTTP status code is not handled or not allowed 2026-01-25 02:19:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g-km-thermaflex-flexible-hvac-duct-8-diameter-r4-2>: HTTP status code is not handled or not allowed 2026-01-25 02:19:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elite-wall-desk-full-size-manual-close-combination-lock-oak>: HTTP status code is not handled or not allowed 2026-01-25 02:19:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/armorultra-job-on-pallet-2-layer-epoxy-system-2500-sq-ft-white>: HTTP status code is not handled or not allowed 2026-01-25 02:19:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bollard-safety-net-4-foot-l-by-26-foot-w-fits-3-1-2-inches-to-4-1-2-inches-dia-brsn-426-5>: HTTP status code is not handled or not allowed 2026-01-25 02:19:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slimline-wall-desk-full-size-manual-close-keyed-lock-beige>: HTTP status code is not handled or not allowed 2026-01-25 02:19:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/machinery_parts_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:19:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-solid-galvanized-steel-shelving-add-on-unit-5-tier-48w-x-18d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 02:19:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/adhesives_sealants_tape/caulk_sealants/concrete_fillers_patchers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:19:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/screw-set-38-16x38-6295598>: HTTP status code is not handled or not allowed 2026-01-25 02:19:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60440106825-roll-314d-1w-x-50-yd-p180>: HTTP status code is not handled or not allowed 2026-01-25 02:19:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/armorultra-commercial-grade-100-solids-epoxy-15-gallon-pail-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:19:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/check-holder-tip-tray-7-34-x-4-38black>: HTTP status code is not handled or not allowed 2026-01-25 02:19:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-box-locker-11-1-2-x-18-x-44-three-clear-door-with-hasp-lock>: HTTP status code is not handled or not allowed 2026-01-25 02:19:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cast-acrylic-bar-12-l-x-2-w-x-1-8-thick-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:19:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bollard-safety-net-4-foot-l-by-9-foot-w-fits-7-3-4-inches-to-8-5-8-inches-dia-brsn-49-9>: HTTP status code is not handled or not allowed 2026-01-25 02:19:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/machining/machinery/machinery_parts_accessories>: HTTP status code is not handled or not allowed 2026-01-25 02:19:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/adhesives_sealants_tape/caulk_sealants/concrete_fillers_patchers>: HTTP status code is not handled or not allowed 2026-01-25 02:19:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shaft-model-201planer-6012249>: HTTP status code is not handled or not allowed 2026-01-25 02:19:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hydraulic-cylinder-719t-156>: HTTP status code is not handled or not allowed 2026-01-25 02:19:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_lockout_identification_tags already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:19:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/krylon-farm-implement-paint-int-l-harvester-red>: HTTP status code is not handled or not allowed 2026-01-25 02:19:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/heaters/patio_heaters/gas_patio_heaters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:19:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shaft-20-planer-mdl-208-6292655>: HTTP status code is not handled or not allowed 2026-01-25 02:19:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/krylon-appliance-epoxy-paint-bisque>: HTTP status code is not handled or not allowed 2026-01-25 02:19:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classical-b-v-belt-b55>: HTTP status code is not handled or not allowed 2026-01-25 02:19:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-view-greenhouse-kit-26w-x-12h-x-36l-propane>: HTTP status code is not handled or not allowed 2026-01-25 02:19:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/105723-padlock-tag-danger-do-not-operate-polyester-3w-x-3h-25pack>: HTTP status code is not handled or not allowed 2026-01-25 02:19:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/krylon-industrial-quik-mark-sb-inverted-marking-paint-apwa-utility-white>: HTTP status code is not handled or not allowed 2026-01-25 02:19:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labels-with-fragile-print-3l-x-2w-fluorescent-red-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-25 02:19:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/narrow-3vx-cogged-v-belt-3vx335>: HTTP status code is not handled or not allowed 2026-01-25 02:19:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_lockout_identification_tags>: HTTP status code is not handled or not allowed 2026-01-25 02:19:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cast-acrylic-bar-24-l-x-2-w-x-3-16-thick-green>: HTTP status code is not handled or not allowed 2026-01-25 02:19:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/heaters/patio_heaters/gas_patio_heaters>: HTTP status code is not handled or not allowed 2026-01-25 02:19:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gear-drying-stand-black>: HTTP status code is not handled or not allowed 2026-01-25 02:19:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spacer-model-201planer-6012164>: HTTP status code is not handled or not allowed 2026-01-25 02:19:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/71565-danger-high-voltage-sign-fiberglass-10w-x-7h>: HTTP status code is not handled or not allowed 2026-01-25 02:20:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cast-acrylic-sheet-48-l-x-48-w-x-1-4-thick-green-ps-cact-94>: HTTP status code is not handled or not allowed 2026-01-25 02:20:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10x10-heavy-duty-commercial-canopy-12-5oz-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:20:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cast-acrylic-bar-48-l-x-1-w-x-3-16-thick-yellow>: HTTP status code is not handled or not allowed 2026-01-25 02:20:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-green-wire-shelving-add-on-unit-5-tier-36w-x-24d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 02:20:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/crowd_control/crowd_control_barricades already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:20:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/painting_supplies/spray_paint_primer already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:20:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paramount-combination-cabinet-easy-assembly-36x18x72-gray-269878gy>: HTTP status code is not handled or not allowed 2026-01-25 02:20:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cover-upper-208-planer-6292670>: HTTP status code is not handled or not allowed 2026-01-25 02:20:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grade-30-proof-coil-chain-3-16-dia-750-lb-cap-10-box-bright-zinc>: HTTP status code is not handled or not allowed 2026-01-25 02:20:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 52131 pages (at 93 pages/min), scraped 26300 items (at 34 items/min) 2026-01-25 02:20:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/padded-mailers-0-6w-x-10l-kraft-250pack>: HTTP status code is not handled or not allowed 2026-01-25 02:20:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelate-silver-epoxy-wire-shelving-add-on-unit-5-tier-48w-x-18d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 02:20:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jumbo-envelopes-14w-x-18l-white-100pack>: HTTP status code is not handled or not allowed 2026-01-25 02:20:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stayflats-plus-peel-seal-mailers-17w-x-21l-kraft-100pack>: HTTP status code is not handled or not allowed 2026-01-25 02:20:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bpu500-basic-universal-pads15w-x-17llight-weightbale100>: HTTP status code is not handled or not allowed 2026-01-25 02:20:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:20:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/crowd_control/crowd_control_barricades>: HTTP status code is not handled or not allowed 2026-01-25 02:20:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/94116-no-parking-sign-redwhite-hip-reflective-sign-aluminum-12w-x-18h>: HTTP status code is not handled or not allowed 2026-01-25 02:20:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/painting_supplies/spray_paint_primer landed on page that is not a product page. 2026-01-25 02:20:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelving-starter-unit-5-tier-24w-x-18d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 02:20:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:20:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelving-starter-unit-5-tier-48w-x-21d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 02:20:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:20:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700505pnk-pegboard-gift-card-display-8w-x-20h-pk-1-pc>: HTTP status code is not handled or not allowed 2026-01-25 02:20:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labels-with-do-not-bend-print-5l-x-3w-redwhite-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-25 02:20:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700460gre-pinwheel-pegboard-display-wrb-135w-x-60h-gn-1-pc>: HTTP status code is not handled or not allowed 2026-01-25 02:20:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_lockout_identification_tags already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:20:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bacterial-oil-spill-eater-10-lb>: HTTP status code is not handled or not allowed 2026-01-25 02:20:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs>: HTTP status code is not handled or not allowed 2026-01-25 02:20:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-duralast-wheel-5x2-12-ball>: HTTP status code is not handled or not allowed 2026-01-25 02:20:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700357yel-pegboard-powerwing-display-16w-x-2025h-yl-1-pc>: HTTP status code is not handled or not allowed 2026-01-25 02:20:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-duralast-wheel-8x2-1-roller>: HTTP status code is not handled or not allowed 2026-01-25 02:20:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700505clr-pegboard-gift-card-display-8w-x-20h-clr-1-pc>: HTTP status code is not handled or not allowed 2026-01-25 02:20:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips landed on page that is not a product page. 2026-01-25 02:20:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets>: HTTP status code is not handled or not allowed 2026-01-25 02:20:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cast-acrylic-sheet-24-l-x-12-w-x-1-4-thick-yellow-ps-cacc-75>: HTTP status code is not handled or not allowed 2026-01-25 02:20:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/self_luminous_emergency_exit_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:20:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/table-insert-wbs-14csos-txt-199037>: HTTP status code is not handled or not allowed 2026-01-25 02:20:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelite-vented-plastic-shelving-starter-unit-5-tier-42w-x-24d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 02:20:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_lockout_identification_tags landed on page that is not a product page. 2026-01-25 02:20:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cast-acrylic-bar-12-l-x-1-w-x-3-16-thick-black>: HTTP status code is not handled or not allowed 2026-01-25 02:20:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:20:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-plastex-wheel-5x2-1-roller>: HTTP status code is not handled or not allowed 2026-01-25 02:20:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/battery-doctor-weatherproof-ato-atc-fuse-holder>: HTTP status code is not handled or not allowed 2026-01-25 02:20:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-nylon-leveling-mount-w-bolt-nylon-58-11-x-6-bolt-2-dia-x-6-78-h-4000-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 02:20:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-12-foot-power-cord-parpcd00110a-for-jetstream-270-and-hurricane-360-115v-20-amp>: HTTP status code is not handled or not allowed 2026-01-25 02:20:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/self_luminous_emergency_exit_signs>: HTTP status code is not handled or not allowed 2026-01-25 02:20:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simonize-liquid-gleam-chlorinated-detergent-machine-dishwashing-gallon>: HTTP status code is not handled or not allowed 2026-01-25 02:20:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 02:20:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cast-acrylic-bar-12-l-x-4-w-x-1-4-thick-black>: HTTP status code is not handled or not allowed 2026-01-25 02:20:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regulated-labels-c-e1-x-1-blackwhite-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-25 02:20:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milton-mk506g-inflator-gauge-m-style-coupler-plug-kit-1-4-npt-6-piece>: HTTP status code is not handled or not allowed 2026-01-25 02:20:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:20:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-heavy-duty-standard-threaded-hex-base-warning-whip-w-o-light-12-x9-orange-triangle-flag>: HTTP status code is not handled or not allowed 2026-01-25 02:20:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_film_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:20:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ring-retainer-rtw-30-5fb-a22-15planer-6284798>: HTTP status code is not handled or not allowed 2026-01-25 02:20:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/112739-vert-l-shaped-acrylic-sign-hlr-25w-x-35h-10pack>: HTTP status code is not handled or not allowed 2026-01-25 02:20:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-heavy-duty-split-pole-warning-whip-w-o-light-no-flag>: HTTP status code is not handled or not allowed 2026-01-25 02:20:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/machinery_parts_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:20:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crowd-control-stanchion-40-in-black-post-red-belt-10-ft-l>: HTTP status code is not handled or not allowed 2026-01-25 02:20:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips>: HTTP status code is not handled or not allowed 2026-01-25 02:20:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-hitch-mount-w-female-quick-disconnect-base-24-power-cord-and-back-up-alarm>: HTTP status code is not handled or not allowed 2026-01-25 02:20:56 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_film_stock landed on page that is not a product page. 2026-01-25 02:20:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-silicone-sheet-6-l-x-6-w-x-1-16-thick-blue-detectable-60a-high-temp-adhesive>: HTTP status code is not handled or not allowed 2026-01-25 02:20:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/556129-display-bucket-4h-clr-20-pc>: HTTP status code is not handled or not allowed 2026-01-25 02:21:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/material-handling/casters_wheels/tires already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:21:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-x-27-kra-stayflat-mailers>: HTTP status code is not handled or not allowed 2026-01-25 02:21:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultem-pei-sheet-48-l-x-12-w-x-3-8-thick-amber>: HTTP status code is not handled or not allowed 2026-01-25 02:21:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-buna-n-sheet-12-l-x-12-w-x-1-16-thick-blue-detectable-60a-acrylic-adhesive>: HTTP status code is not handled or not allowed 2026-01-25 02:21:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-32-x-2-cotter-pin-316-stainless-steel-asme-18-8-1-made-in-usa>: HTTP status code is not handled or not allowed 2026-01-25 02:21:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700292clr-pegboard-floor-stand-24w-x-48h-clr-1-pc>: HTTP status code is not handled or not allowed 2026-01-25 02:21:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-pvc-90-degree-angle-72-l-x-1-1-4-w-gray-bulk-pa-pvc-11>: HTTP status code is not handled or not allowed 2026-01-25 02:21:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rectangular-inventory-paper-labels-7l-x-5w-green-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-25 02:21:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/machinery_parts_accessories landed on page that is not a product page. 2026-01-25 02:21:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-buna-n-strip-36-l-x-7-w-x-1-8-thick-blue-detectable-60a-acrylic-adhesive-bulk-rs-h60md-88>: HTTP status code is not handled or not allowed 2026-01-25 02:21:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scale-20b2-3684210>: HTTP status code is not handled or not allowed 2026-01-25 02:21:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5410l-3-5v-portable-diagnostic-set-led-lamp-black>: HTTP status code is not handled or not allowed 2026-01-25 02:21:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 52215 pages (at 84 pages/min), scraped 26327 items (at 27 items/min) 2026-01-25 02:21:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/material-handling/casters_wheels/tires>: HTTP status code is not handled or not allowed 2026-01-25 02:21:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/datalogic-65l-usb-cable-for-use-with-datalogic-powerscan-rf>: HTTP status code is not handled or not allowed 2026-01-25 02:21:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90a-3-washer-6288903>: HTTP status code is not handled or not allowed 2026-01-25 02:21:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:21:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/116in-w-x80in-d-reception-station-w-non-electric-raceway-gray-counter-gray-panel>: HTTP status code is not handled or not allowed 2026-01-25 02:21:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/warning-label-l510-3408265>: HTTP status code is not handled or not allowed 2026-01-25 02:21:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-dia-round-paper-labels-with-nov-print-fluorescent-green-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-25 02:21:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-steel-revolving-low-pulley-bar>: HTTP status code is not handled or not allowed 2026-01-25 02:21:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b0150s-2-bucket-trap-1-2-in>: HTTP status code is not handled or not allowed 2026-01-25 02:21:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-socket-1-2-drive-6-point-15-16-deep>: HTTP status code is not handled or not allowed 2026-01-25 02:21:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/maryland-hitch-cover-4-1-2-x-3-3-8-15112>: HTTP status code is not handled or not allowed 2026-01-25 02:21:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-socket-3-8-drive-12-point-5-8-short>: HTTP status code is not handled or not allowed 2026-01-25 02:21:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tennessee-hitch-cover-4-1-2-x-3-3-8-15061>: HTTP status code is not handled or not allowed 2026-01-25 02:21:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 02:21:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-coverall-knit-cuffs-and-attached-hood-white-red-3x-large-4565-blk-3xl>: HTTP status code is not handled or not allowed 2026-01-25 02:21:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:21:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-12-l-x-1-w-x-1-thick-black-antistatic>: HTTP status code is not handled or not allowed 2026-01-25 02:21:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/office-partition-panel-with-partial-window-and-raceway-60-1-4w-x-100h-gray-gl>: HTTP status code is not handled or not allowed 2026-01-25 02:21:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-socket-1-2-drive-6-point-1-1-8-deep>: HTTP status code is not handled or not allowed 2026-01-25 02:21:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tapcon-1-4x2-3-4-concrete-anchor-hex-head-75-pk>: HTTP status code is not handled or not allowed 2026-01-25 02:21:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pneumatic-bit-tailpipe-cut-off-tool>: HTTP status code is not handled or not allowed 2026-01-25 02:21:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-x-12-x-4-deluxe-literature-mailers>: HTTP status code is not handled or not allowed 2026-01-25 02:21:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mailing-tube-bags-6w-x-3-12d-x-45l-kraft-100pack>: HTTP status code is not handled or not allowed 2026-01-25 02:21:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-12-l-x-3-w-x-1-4-thick-black-antistatic>: HTTP status code is not handled or not allowed 2026-01-25 02:21:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jiffy-rigi-bag-self-seal-mailers-3-8-12w-x-13l-kraft-200pack>: HTTP status code is not handled or not allowed 2026-01-25 02:21:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:21:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/work-benches/workbenches_worktables/cabinet_pedestal_workbenches/mobile_cabinet_pedestal_workbenches already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:21:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/moo-tel-calf-nursery-w-doors-26w-x-24l-white>: HTTP status code is not handled or not allowed 2026-01-25 02:21:34 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 02:21:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-non-electric-office-partition-panel-with-raceway-60-1-4w-x-101-1-2h-blue-gl>: HTTP status code is not handled or not allowed 2026-01-25 02:21:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/impact-socket-3-8-drive-5-8-6-point-deep>: HTTP status code is not handled or not allowed 2026-01-25 02:21:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labels-w-part-description-qty-print-2l-x-1-14w-blackwhite-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-25 02:21:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:21:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leveling-foot-threaded-4500-lbs-capacity-3-15-base-dia-41-80-1-2x13-125-d3-sk>: HTTP status code is not handled or not allowed 2026-01-25 02:21:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/work-benches/workbenches_worktables/cabinet_pedestal_workbenches/mobile_cabinet_pedestal_workbenches>: HTTP status code is not handled or not allowed 2026-01-25 02:21:42 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-25 02:21:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mstf647vs-caution-sign-not-a-step-10w-x-7h-adhesive-vinyl>: HTTP status code is not handled or not allowed 2026-01-25 02:21:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-x-8-self-seal-white-stayflat-mailers>: HTTP status code is not handled or not allowed 2026-01-25 02:21:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-24-l-x-1-1-2-w-x-1-4-thick-black-antistatic>: HTTP status code is not handled or not allowed 2026-01-25 02:21:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 02:21:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:21:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-outdoor-in-ground-steel-bench-with-backrest-expanded-metal-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:21:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-spill-collector-pallet-jack-style-with-no-drain>: HTTP status code is not handled or not allowed 2026-01-25 02:21:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/recycled-rubber-tackboard-aluminum-trim-46-1-2w-x-36h-black>: HTTP status code is not handled or not allowed 2026-01-25 02:21:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/07157-50815-torx-t15-x-5-thandle>: HTTP status code is not handled or not allowed 2026-01-25 02:21:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spm-1080-mini-accelerator-4-1-2-x-10-120v-50-cord>: HTTP status code is not handled or not allowed 2026-01-25 02:21:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-bit-5-16>: HTTP status code is not handled or not allowed 2026-01-25 02:21:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-ft-x-7-ft-handwoven-striped-jute-blend-area-rug-natural>: HTTP status code is not handled or not allowed 2026-01-25 02:21:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s153-zebra-thermal-transfer-ribbons-3-12w-x-1476l-waxresin-black-24-rollscase>: HTTP status code is not handled or not allowed 2026-01-25 02:21:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/foam/foam_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:21:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22x25-liter-stak-a-cab-acid-cabinet-47w-x-18d-x-18h>: HTTP status code is not handled or not allowed 2026-01-25 02:21:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 02:22:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shag-style-diamond-trellis-area-rug-8-ft-x-10-ft-ivory-gray-polyester>: HTTP status code is not handled or not allowed 2026-01-25 02:22:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/non-slip-rug-pad-for-8-ft-x-10-ft-area-rug-white>: HTTP status code is not handled or not allowed 2026-01-25 02:22:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/starsense-explorer-dx-130az-smartphone-app-enabled-newtonian-reflector-telescope>: HTTP status code is not handled or not allowed 2026-01-25 02:22:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6x10-6-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 02:22:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/foam/foam_rolls>: HTTP status code is not handled or not allowed 2026-01-25 02:22:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5x7-2-mil-anti-static-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 02:22:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wrestling-marial-arts-roll-mats-24-x-6-x-1-5-8-navy-111r-ny>: HTTP status code is not handled or not allowed 2026-01-25 02:22:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 52305 pages (at 90 pages/min), scraped 26365 items (at 38 items/min) 2026-01-25 02:22:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clean-cut-fray-resistant-sleeving-3-8-dia-100-black>: HTTP status code is not handled or not allowed 2026-01-25 02:22:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mfs729-keep-clear-in-front-of-electrical-floor-sign-17-diameter-adhesive-vinyl>: HTTP status code is not handled or not allowed 2026-01-25 02:22:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-shelf-with-new-electric-outlet-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:22:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/socket-1-4-drive-t10-internal-torx>: HTTP status code is not handled or not allowed 2026-01-25 02:22:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6x12-4-mil-anti-static-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 02:22:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-rack-guard-back-panel-with-flush-mount-clip-96l-x-1-14w-x-48h>: HTTP status code is not handled or not allowed 2026-01-25 02:22:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-impact-socket-set-1-2-drive-6-point-deep-12-piece>: HTTP status code is not handled or not allowed 2026-01-25 02:22:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/31-polyolefin-heat-shrink-1-16-dia-25-black>: HTTP status code is not handled or not allowed 2026-01-25 02:22:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-x-6-dri-shieldtm-moisture-barrier-bags>: HTTP status code is not handled or not allowed 2026-01-25 02:22:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-artic-blast-ergonomic-aluminum-snow-shovel>: HTTP status code is not handled or not allowed 2026-01-25 02:22:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-16flexo-pet-general-purpose-sleeving-dia-100-black>: HTTP status code is not handled or not allowed 2026-01-25 02:22:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90-degree-elbow>: HTTP status code is not handled or not allowed 2026-01-25 02:22:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-x14-x36-1-5-mil-gusseted-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 02:22:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gloveworks-powderfree-industrial-grade-latex-gloves-small-100box-10-boxcs>: HTTP status code is not handled or not allowed 2026-01-25 02:22:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gloveworks-powderfree-industrial-grade-nitrile-gloves-large-100box-10-boxcs>: HTTP status code is not handled or not allowed 2026-01-25 02:22:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/impact-socket-3-8-drive-17mm-6-point-deep>: HTTP status code is not handled or not allowed 2026-01-25 02:22:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4000-semi-outdoor-led-mass-notification-sign-full-color-power-over-ethernet-4080rgb>: HTTP status code is not handled or not allowed 2026-01-25 02:22:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fairfield-single-unit-towel-valet-white>: HTTP status code is not handled or not allowed 2026-01-25 02:22:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/er25-2-16mm-x-1mm-15-piece-aa-extreme-precision-collet-sets>: HTTP status code is not handled or not allowed 2026-01-25 02:22:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-socket-1-4-drive-11mm-deep>: HTTP status code is not handled or not allowed 2026-01-25 02:22:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/credenzas already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:22:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jpj-12bhh-12-planerjointer-combo-with-helical-head-cutterhead-3hp-1-phase-230v>: HTTP status code is not handled or not allowed 2026-01-25 02:22:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gorilla-sleeve-high-abrasion-3-8-dia-500-black>: HTTP status code is not handled or not allowed 2026-01-25 02:22:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-socket-1-4-drive-5-5mm-deep>: HTTP status code is not handled or not allowed 2026-01-25 02:22:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:22:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-tab-panel-1>: HTTP status code is not handled or not allowed 2026-01-25 02:22:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pgf8acsl-bi-level-drinking-fountain-filtered-greystone>: HTTP status code is not handled or not allowed 2026-01-25 02:22:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-13-16-x-8-5-16-x-5-lewisbins-divider-box-light-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:22:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powermatic-pm2815bt-benchtop-drill-press-34-hp-1-phase-120v>: HTTP status code is not handled or not allowed 2026-01-25 02:22:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/z-line-additional-perforated-steel-shelf-level-with-clips-42-w-x-24-d-x-1-5-16-h>: HTTP status code is not handled or not allowed 2026-01-25 02:22:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/werner-lock-job-caddy-ac54-jc>: HTTP status code is not handled or not allowed 2026-01-25 02:22:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5x8-1-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 02:22:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/duraflex-woven-high-abrasion-sleeve-286-dia-100-black>: HTTP status code is not handled or not allowed 2026-01-25 02:22:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock landed on page that is not a product page. 2026-01-25 02:22:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pm2000t-10-table-saw-with-armorglide--workbench-table-50-rip-5hp-1ph-230v>: HTTP status code is not handled or not allowed 2026-01-25 02:22:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/credenzas landed on page that is not a product page. 2026-01-25 02:22:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durawrap-1050-ballistic-nylon-wrap-w-velcro-closure-450-dia-150-black>: HTTP status code is not handled or not allowed 2026-01-25 02:22:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50-gallon-rotating-composter>: HTTP status code is not handled or not allowed 2026-01-25 02:22:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g22-150-ff-30hp-oil-injected-rotary-screw-floor-mount-150psi-3ph-208-230-460v>: HTTP status code is not handled or not allowed 2026-01-25 02:22:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-ij-833-8-benchtop-jointer-with-spiral-head-cutter-1-516hp-1ph-110v>: HTTP status code is not handled or not allowed 2026-01-25 02:22:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-bar-12-l-x-2-w-x-1-8-thick-clear-scratch-and-uv-resistant>: HTTP status code is not handled or not allowed 2026-01-25 02:22:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-straight-coupling-2>: HTTP status code is not handled or not allowed 2026-01-25 02:22:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durawrap-1050-ballistic-nylon-wrap-w-velcro-closure-8-dia-25-black>: HTTP status code is not handled or not allowed 2026-01-25 02:22:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cpvc-bar-36-l-x-2-1-2-w-x-1-16-thick-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:22:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-bar-12-l-x-4-w-x-1-4-thick-clear-scratch-and-uv-resistant>: HTTP status code is not handled or not allowed 2026-01-25 02:22:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-heavy-duty-four-hole-square-flange>: HTTP status code is not handled or not allowed 2026-01-25 02:22:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/freestanding-2-panel-corner-room-divider-with-partial-window-48-x-60-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:22:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24x26-1-5-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 02:22:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-freestanding-3-panel-corner-room-divider-with-partial-window-48-x-43-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:22:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4x10-1-5-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 02:22:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-oz-glass-jar-straight-side-clear-poly-vinyl-liner-case-of-12>: HTTP status code is not handled or not allowed 2026-01-25 02:22:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-oz-glass-jars-straight-side-clear-poly-vinyl-liner-case-of-24>: HTTP status code is not handled or not allowed 2026-01-25 02:23:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mechanical-edge-of-dock-leveler-78-usable-w-100-overall-w-25-000-lb-cap-986639>: HTTP status code is not handled or not allowed 2026-01-25 02:23:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hurricane-360-hoist-and-trolley-combo-geared-3-ton-capacity-10-lift>: HTTP status code is not handled or not allowed 2026-01-25 02:23:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hurricane-360-hoist-and-trolley-combo-geared-10-ton-capacity-10-lift>: HTTP status code is not handled or not allowed 2026-01-25 02:23:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_film_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:23:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evoke-glide-e-bike-264-lb-capacity-metallic-greencharcoal>: HTTP status code is not handled or not allowed 2026-01-25 02:23:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jct-1600-benchtop-sand-rammer-14-cfm-1-500-bpm-90-psi>: HTTP status code is not handled or not allowed 2026-01-25 02:23:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/bar_restaurant_equipment/dispensers_organizers/beverage_dispensers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:23:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/omnimed-6-capacity-storage-rack>: HTTP status code is not handled or not allowed 2026-01-25 02:23:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3x4-4-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 02:23:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pycnometer-50ml-no-mercury>: HTTP status code is not handled or not allowed 2026-01-25 02:23:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/w-350ic-14-i-bar-sealer-w-slider-cutter-film-roller-2-7-mm-seal-width>: HTTP status code is not handled or not allowed 2026-01-25 02:23:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 52400 pages (at 95 pages/min), scraped 26401 items (at 36 items/min) 2026-01-25 02:23:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:23:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/w-650i-w-series-26-i-bar-sealer-with-film-roller-round-wire>: HTTP status code is not handled or not allowed 2026-01-25 02:23:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_film_stock>: HTTP status code is not handled or not allowed 2026-01-25 02:23:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/digital-battery-charging-tester-pbt-200>: HTTP status code is not handled or not allowed 2026-01-25 02:23:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torrent-walk-behind-battery-powered-sweeper-28-cleaning-width>: HTTP status code is not handled or not allowed 2026-01-25 02:23:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/bar_restaurant_equipment/dispensers_organizers/beverage_dispensers>: HTTP status code is not handled or not allowed 2026-01-25 02:23:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/safety_cabinets/gas_cylinder_storage_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:23:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/versa-mat-vm5280cl>: HTTP status code is not handled or not allowed 2026-01-25 02:23:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_film_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:23:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-nylon-6-6-bar-36-l-x-3-8-w-x-3-8-thick-black>: HTTP status code is not handled or not allowed 2026-01-25 02:23:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/freestanding-3-panel-corner-room-divider-36-x-60-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:23:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 02:23:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jlp-a-series-lever-hoist-3-ton-20-ft-lift>: HTTP status code is not handled or not allowed 2026-01-25 02:23:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-bar-48-l-x-3-w-x-3-32-thick-clear>: HTTP status code is not handled or not allowed 2026-01-25 02:23:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/window-kit-for-sliding-windows-white-23-1-2-h-exclusively-for-all-micro-split-ac>: HTTP status code is not handled or not allowed 2026-01-25 02:23:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/cabinets/safety_cabinets/gas_cylinder_storage_cabinets>: HTTP status code is not handled or not allowed 2026-01-25 02:23:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_film_stock>: HTTP status code is not handled or not allowed 2026-01-25 02:23:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-21-degree-baume-hydrometer>: HTTP status code is not handled or not allowed 2026-01-25 02:23:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guardair-lazer-pistol-grip-48-alum-ext-alum-nozzle>: HTTP status code is not handled or not allowed 2026-01-25 02:23:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/saf-t-grip-board-mate-cbm1318>: HTTP status code is not handled or not allowed 2026-01-25 02:23:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-freestanding-2-panel-corner-room-divider-48-x-73-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:23:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-wash-bottle-carrier-holds-250-500ml-8-16oz-bottles-11-x-7-x-8>: HTTP status code is not handled or not allowed 2026-01-25 02:23:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guardair-force-5-48-alum-ext-venturi-nozzle>: HTTP status code is not handled or not allowed 2026-01-25 02:23:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classic-entrance-mat-waffle-pattern-3-8-thick-3-x-20-charcoal>: HTTP status code is not handled or not allowed 2026-01-25 02:23:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0-12-degree-brix-sugar-scale-hydrometer>: HTTP status code is not handled or not allowed 2026-01-25 02:23:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/prison-bench-4-foot-solid-steel-seating-without-backrest-black>: HTTP status code is not handled or not allowed 2026-01-25 02:23:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vci-flat-poly-bags-18w-x-24l-4-mil-blue-250pack>: HTTP status code is not handled or not allowed 2026-01-25 02:23:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-x-3-x-25-square-mailing-tubes>: HTTP status code is not handled or not allowed 2026-01-25 02:23:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dn-10030-debris-safety-netting-86-ft-x-150-ft-black-1-roll>: HTTP status code is not handled or not allowed 2026-01-25 02:23:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/prison-bench-3-foot-composite-lumber-seating-with-backrest-chocolate-brown>: HTTP status code is not handled or not allowed 2026-01-25 02:23:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guardair-jet-guard-safety-air-gun>: HTTP status code is not handled or not allowed 2026-01-25 02:23:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wet-zone-shadow-board-green-black-68-x-30-pro-series-acrylic-sb120fg>: HTTP status code is not handled or not allowed 2026-01-25 02:23:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/-5-60g-l-soil-analysis-astm-152h-hydrometer>: HTTP status code is not handled or not allowed 2026-01-25 02:23:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-rod-60-l-x-2-1-2-dia-white>: HTTP status code is not handled or not allowed 2026-01-25 02:23:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:23:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-stack-rack-60-w-x-42-d-x-48-h-2>: HTTP status code is not handled or not allowed 2026-01-25 02:23:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/electrical_power_testing/multimeters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:23:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelate-wire-shelving-36728ep-72w-x-36d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 02:23:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:23:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-hydraulic-jack-65-ton-65-1h>: HTTP status code is not handled or not allowed 2026-01-25 02:23:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelate-wire-shelving-add-on-a36488ep-48w-x-36d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 02:23:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:23:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/covamat-polyethylene-clear-bench-table-liner-50-foot-roll>: HTTP status code is not handled or not allowed 2026-01-25 02:23:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9500-9999-inventory-tag-2-part-carbonless>: HTTP status code is not handled or not allowed 2026-01-25 02:23:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:23:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers>: HTTP status code is not handled or not allowed 2026-01-25 02:23:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-support-stand-8-3-16-dia-base-18-x-5-16-support-rod>: HTTP status code is not handled or not allowed 2026-01-25 02:23:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shuretech-sts-s4-8-bench-system-gloss-black>: HTTP status code is not handled or not allowed 2026-01-25 02:23:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 02:23:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/auto-vent-kit-for-snap-grow-and-nature-greenhouses>: HTTP status code is not handled or not allowed 2026-01-25 02:23:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock>: HTTP status code is not handled or not allowed 2026-01-25 02:23:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/testing_inspection_instruments/electrical_power_testing/multimeters landed on page that is not a product page. 2026-01-25 02:24:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/colony-counter-black-replacement-tip-marker>: HTTP status code is not handled or not allowed 2026-01-25 02:24:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:24:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-3-4-x-2-3-8-5-light-blue-shipping-tag>: HTTP status code is not handled or not allowed 2026-01-25 02:24:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crinkle-paper-10-lbs-sky-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:24:04 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-25 02:24:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:24:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vci-paper-30-lbs-8w-x-8l-kraft-1000-sheets>: HTTP status code is not handled or not allowed 2026-01-25 02:24:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:24:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roundup-2w-8l-endless-poly-roundsling-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:24:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 52488 pages (at 88 pages/min), scraped 26438 items (at 37 items/min) 2026-01-25 02:24:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-hdpe-bar-48-l-x-4-w-x-1-thick-black-marine>: HTTP status code is not handled or not allowed 2026-01-25 02:24:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 02:24:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-epoxy-wire-shelving-add-on-14427gz-42w-x-14d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 02:24:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock>: HTTP status code is not handled or not allowed 2026-01-25 02:24:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:24:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-epoxy-wire-shelving-add-on-14546gz-54w-x-14d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 02:24:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-25 02:24:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-step-on-receptacle-17-gallon-black>: HTTP status code is not handled or not allowed 2026-01-25 02:24:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-72-x-30-adj-height-workbench-w-drawer-black-esd-square-edge-top>: HTTP status code is not handled or not allowed 2026-01-25 02:24:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nace-paper-vacuum-bag-nace-numatic-henryjames-200225235250252260-rsv130200>: HTTP status code is not handled or not allowed 2026-01-25 02:24:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-4-x-2-7-8-7-pink-fluorescent-tag>: HTTP status code is not handled or not allowed 2026-01-25 02:24:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers>: HTTP status code is not handled or not allowed 2026-01-25 02:24:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-96-x-30-adj-height-workbench-w-drawer-riser-black-maple-square-edge-top>: HTTP status code is not handled or not allowed 2026-01-25 02:24:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-72-x-30-adj-height-workbench-w-drawer-riser-bk-plastic-laminate-square-top>: HTTP status code is not handled or not allowed 2026-01-25 02:24:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-gallon-120-lbs-drum-dolly>: HTTP status code is not handled or not allowed 2026-01-25 02:24:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-epoxy-wire-shelving-18245gz-24w-x-18d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 02:24:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-rod-72-l-x-2-dia-white>: HTTP status code is not handled or not allowed 2026-01-25 02:24:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-aker-ceiling-fan-with-light-59302-matte-black>: HTTP status code is not handled or not allowed 2026-01-25 02:24:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-cart-system-w-oil-control-handle>: HTTP status code is not handled or not allowed 2026-01-25 02:24:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-sided-rack-12x36x60-with-72-yellow-drawers>: HTTP status code is not handled or not allowed 2026-01-25 02:24:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storage-container-with-lid-54-1-2-x-11-1-2-x-8-white>: HTTP status code is not handled or not allowed 2026-01-25 02:24:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:24:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storage-container-with-lid-61-1-4-x-5-1-4-x-7-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:24:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buyers-stainless-steel-underbody-box-w-t-handle-black-18x18x48>: HTTP status code is not handled or not allowed 2026-01-25 02:24:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:24:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tempco-finned-strip-240v-t1-19-12l-1000w>: HTTP status code is not handled or not allowed 2026-01-25 02:24:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-12-l-x-5-w-x-1-thick-white>: HTTP status code is not handled or not allowed 2026-01-25 02:24:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/102222-314-amber-centermount-reflector-plastic-backplate-rt90a>: HTTP status code is not handled or not allowed 2026-01-25 02:24:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-x-5-clear-face-invoice-enclosed-panel-face>: HTTP status code is not handled or not allowed 2026-01-25 02:24:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buyers-stainless-steel-topside-w-t-handle-white-16x13x96>: HTTP status code is not handled or not allowed 2026-01-25 02:24:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-ptfe-bar-48-l-x-1-w-x-3-8-thick-off-white-glass-filled>: HTTP status code is not handled or not allowed 2026-01-25 02:24:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-x-5-4-mil-reclosable-poly-bags-w-hang-hole>: HTTP status code is not handled or not allowed 2026-01-25 02:24:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-ptfe-sheet-48-l-x-24-w-x-3-8-thick-off-white-glass-filled>: HTTP status code is not handled or not allowed 2026-01-25 02:24:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock>: HTTP status code is not handled or not allowed 2026-01-25 02:24:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microscope-slide-box-stackable-holds-50-slides>: HTTP status code is not handled or not allowed 2026-01-25 02:24:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-ptfe-bar-36-l-x-1-2-w-x-3-8-thick-black>: HTTP status code is not handled or not allowed 2026-01-25 02:24:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buyers-aluminum-underbody-box-w-t-handle-24x24x30>: HTTP status code is not handled or not allowed 2026-01-25 02:24:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock>: HTTP status code is not handled or not allowed 2026-01-25 02:24:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-high-distressed-metal-indoor-outdoor-bar-stool-with-back-copper>: HTTP status code is not handled or not allowed 2026-01-25 02:24:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spruce-general-use-spray-paint-16-oz-gloss-purple-12-cans-case-98-89>: HTTP status code is not handled or not allowed 2026-01-25 02:24:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mro-industrial-enamel-20-oz-safety-orange-6-cans-case-620-1451>: HTTP status code is not handled or not allowed 2026-01-25 02:24:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/executive-swivel-chair-mid-back-leather-black>: HTTP status code is not handled or not allowed 2026-01-25 02:24:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buyers-stainless-steel-underbody-box-w-rotary-paddle-black-14x16x24>: HTTP status code is not handled or not allowed 2026-01-25 02:24:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pe-buchner-table-top-funnel-with-medium-porosity-removable-plate-24-id-13-height>: HTTP status code is not handled or not allowed 2026-01-25 02:24:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tool-crib-penetrating-oil-20-oz-6-cans-case-620-1543>: HTTP status code is not handled or not allowed 2026-01-25 02:24:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tool-crib-insulating-varnish-20-oz-red-6-cans-case-620-1525>: HTTP status code is not handled or not allowed 2026-01-25 02:24:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:24:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/29d-worktop-freezer-food-prep-48w-wtf48a>: HTTP status code is not handled or not allowed 2026-01-25 02:24:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-x-8-2-mil-reclosable-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 02:24:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/school-milk-coolers-dual-access-cold-wall-st-34w-st34n>: HTTP status code is not handled or not allowed 2026-01-25 02:24:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-x-5-4-mil-reclosable-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 02:24:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-48-l-x-2-w-x-1-thick-black>: HTTP status code is not handled or not allowed 2026-01-25 02:25:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/32d-undercounter-refrigerator-freezer-food-prep-67w-ucf67a>: HTTP status code is not handled or not allowed 2026-01-25 02:25:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers landed on page that is not a product page. 2026-01-25 02:25:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bk-steel-ball-knobs-tapped-254mm-diameter-mm-length-51618>: HTTP status code is not handled or not allowed 2026-01-25 02:25:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-48-l-x-6-w-x-2-thick-white>: HTTP status code is not handled or not allowed 2026-01-25 02:25:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reach-in-refrigerator-horizon-solid-half-solid-doors-52w-hr2-1hs>: HTTP status code is not handled or not allowed 2026-01-25 02:25:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plus-precision-liquid-in-glass-thermometer-30-to-124f-76mm-immersion>: HTTP status code is not handled or not allowed 2026-01-25 02:25:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 52583 pages (at 95 pages/min), scraped 26479 items (at 41 items/min) 2026-01-25 02:25:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-x-40-50-basis-weight-kra-paper-sheets>: HTTP status code is not handled or not allowed 2026-01-25 02:25:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:25:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:25:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/seat-belt-cutter>: HTTP status code is not handled or not allowed 2026-01-25 02:25:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plus-astm-like-liquid-in-glass-thermometer-99c-weathering-test-35mm-immersion>: HTTP status code is not handled or not allowed 2026-01-25 02:25:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-sheet-48-l-x-24-w-x-1-thick-black>: HTTP status code is not handled or not allowed 2026-01-25 02:25:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/painting_supplies/spray_paint_primer already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:25:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-ptfe-sheet-12-l-x-12-w-x-3-8-thick-black>: HTTP status code is not handled or not allowed 2026-01-25 02:25:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:25:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:25:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-54w-x-24d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 02:25:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 02:25:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-25 02:25:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/food_beverages/emergency_food_drink already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:25:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-5-tier-30w-x-21d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 02:25:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/01-142-72-hour-emergency-food-supply-34-servings>: HTTP status code is not handled or not allowed 2026-01-25 02:25:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-vented-labeled-2-color-ethanol-wm-wash-bottles-500ml-16oz-pe-w-nat-pp-cap-3pk>: HTTP status code is not handled or not allowed 2026-01-25 02:25:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-3-point-hitch-mover-tractor-attachment-category-1-2>: HTTP status code is not handled or not allowed 2026-01-25 02:25:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-5-tier-24w-x-18d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 02:25:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/painting_supplies/spray_paint_primer landed on page that is not a product page. 2026-01-25 02:25:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-25 02:25:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gezm-108es-2rs-spherical-plain-bearing-inch-extended-inner-ring-sealed>: HTTP status code is not handled or not allowed 2026-01-25 02:25:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-25 02:25:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commercial-electric-baseboard-convection-heater-60-l-240-208v>: HTTP status code is not handled or not allowed 2026-01-25 02:25:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/food_beverages/emergency_food_drink>: HTTP status code is not handled or not allowed 2026-01-25 02:25:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/radiant-ceiling-panels-500w-208v>: HTTP status code is not handled or not allowed 2026-01-25 02:25:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flexzilla-pressure-washer-hose-w-14-x-50-m22-ftgs>: HTTP status code is not handled or not allowed 2026-01-25 02:25:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-32-gallon-square-recycled-plastic-receptacle-w-liner-green>: HTTP status code is not handled or not allowed 2026-01-25 02:25:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5120-2560-btu-fan-forced-wall-heater-without-summer-fan-switch-120v-3>: HTTP status code is not handled or not allowed 2026-01-25 02:25:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/lockers/locker_accessories/locker_shelves_rods already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:25:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-2-piece-stopcock-for-1-2-tubing-polyethylene>: HTTP status code is not handled or not allowed 2026-01-25 02:25:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stor-file-letter-box-12-7-8w-x-25-3-8d-x-10-1-4h-white-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:25:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gac-110f-spherical-plain-thrust-bearing-angular-contact-metric>: HTTP status code is not handled or not allowed 2026-01-25 02:25:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/four-section-locker>: HTTP status code is not handled or not allowed 2026-01-25 02:25:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corn-fill-angle-broom-lacquered-pine-handle-brown-fg637300brn>: HTTP status code is not handled or not allowed 2026-01-25 02:25:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/lockers/locker_accessories/locker_shelves_rods>: HTTP status code is not handled or not allowed 2026-01-25 02:25:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss-coffee-brewer-3-plates-gb260>: HTTP status code is not handled or not allowed 2026-01-25 02:25:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-labeled-4-color-isopropanol-wm-wash-bottles-1000ml-32oz-pe-w-ylw-pp-cap-4pk>: HTTP status code is not handled or not allowed 2026-01-25 02:25:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/box-sealing-tape-373-clear-2-inch-x-110-yards-36-per-case-bulk>: HTTP status code is not handled or not allowed 2026-01-25 02:25:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-5-tier-42w-x-18d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 02:25:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:25:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cast-iron-pr-pump-1-6-hp-single-phase>: HTTP status code is not handled or not allowed 2026-01-25 02:25:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oreck-12-inch-polish-pad-white>: HTTP status code is not handled or not allowed 2026-01-25 02:25:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/fans_blowers/blowers/confined_space_blowers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:25:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cylinder-truck-6-cylinders-1500-lb-capacity-986870>: HTTP status code is not handled or not allowed 2026-01-25 02:25:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9513-8-inch-axial-ac-metal-blower>: HTTP status code is not handled or not allowed 2026-01-25 02:26:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/fans_blowers/blowers/confined_space_blowers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:26:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/label_tapes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:26:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9510-60-12-inch-manhole-ventilation-passthru-mvp>: HTTP status code is not handled or not allowed 2026-01-25 02:26:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/splash-shield-2-24-articulating-arms-15-x-24-plexiglass>: HTTP status code is not handled or not allowed 2026-01-25 02:26:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls>: HTTP status code is not handled or not allowed 2026-01-25 02:26:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spinbar-teflon-octagon-magnetic-stirring-bar-508-x-8mm-red>: HTTP status code is not handled or not allowed 2026-01-25 02:26:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/fans_blowers/blowers/confined_space_blowers>: HTTP status code is not handled or not allowed 2026-01-25 02:26:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:26:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-304-stainless-steel-locker-18w-x-18d-x-78h-single-tier-3-wide-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 02:26:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 52666 pages (at 83 pages/min), scraped 26520 items (at 41 items/min) 2026-01-25 02:26:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/fans_blowers/blowers/confined_space_blowers>: HTTP status code is not handled or not allowed 2026-01-25 02:26:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coat-rod-12-wide-for-hallowell-304-stainless-steel-lockers>: HTTP status code is not handled or not allowed 2026-01-25 02:26:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trimmer-line-spool-bracket-lt40>: HTTP status code is not handled or not allowed 2026-01-25 02:26:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/twin-pocket-leatherette-grained-portfolios-royal-blue-25-box>: HTTP status code is not handled or not allowed 2026-01-25 02:26:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-rubber-mudflaps-1-4-inch-thick-18x24-b2418lsp>: HTTP status code is not handled or not allowed 2026-01-25 02:26:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-wide-vented-steel-locker-double-tier-24x24x74-2-door-putty-1>: HTTP status code is not handled or not allowed 2026-01-25 02:26:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lk8-chrome-basket-strainer-and-tailpiece-for-bar-sink>: HTTP status code is not handled or not allowed 2026-01-25 02:26:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:26:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 02:26:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-x-54-4-gauge-vinyl-w-center-zipper-and-side-gusset-crystal-clear>: HTTP status code is not handled or not allowed 2026-01-25 02:26:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/label_tapes landed on page that is not a product page. 2026-01-25 02:26:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/application-tape-labels-66-roll>: HTTP status code is not handled or not allowed 2026-01-25 02:26:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:26:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/desk-highlighter-chisel-tip-pocket-clip-fluorescent-pink-dozen>: HTTP status code is not handled or not allowed 2026-01-25 02:26:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:26:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/recyclockout-lockout-tagout-hasp-1-inch-steel-with-tabs-7106>: HTTP status code is not handled or not allowed 2026-01-25 02:26:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/140111-hfh-actkp-km-klp-kl3-ss-k-ss-8l-x-4w-x-3h>: HTTP status code is not handled or not allowed 2026-01-25 02:26:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-sterileware-double-bagged-sterile-scoop-sampling-system-125ml-4oz>: HTTP status code is not handled or not allowed 2026-01-25 02:26:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/140101-female-qc-lead-8way-5m-black>: HTTP status code is not handled or not allowed 2026-01-25 02:26:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 02:26:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/405003-rfid-coded-non-contact-switch-sp-rfid-u-10m>: HTTP status code is not handled or not allowed 2026-01-25 02:26:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-buna-n-roll-7200-l-x-36-w-x-1-32-thick-black-high-strength-90a>: HTTP status code is not handled or not allowed 2026-01-25 02:26:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/203008hf-km-tongue-interlock-switch-hf-act-3nc-1no-12npt-die-cast>: HTTP status code is not handled or not allowed 2026-01-25 02:26:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-25 02:26:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-double-tier-locker-starter-1-wide-12w-x-12d-x-36h-unassembled-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:26:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock landed on page that is not a product page. 2026-01-25 02:26:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-double-tier-locker-starter-1-wide-12w-x-18d-x-36h-unassembled-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:26:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-double-tier-locker-adder-1-wide-18w-x-18d-x-36h-unassembled-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:26:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-silicone-roll-120-l-x-36-w-x-1-16-thick-red-food-70a-high-temp-adhesive>: HTTP status code is not handled or not allowed 2026-01-25 02:26:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-silicone-roll-240-l-x-36-w-x-1-16-thick-red-food-70a>: HTTP status code is not handled or not allowed 2026-01-25 02:26:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96-x-36-square-edge-work-bench-shop-top-tan>: HTTP status code is not handled or not allowed 2026-01-25 02:26:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/seq-dual-speed-electric-chain-hoist-1-4-ton-15-lift-33-5-5-ft-min-115v>: HTTP status code is not handled or not allowed 2026-01-25 02:26:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/competitor-ventilated-three-tier-locker-2-wide-12w-x-12d-x-24h-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:26:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-neoprene-sheet-36-l-x-12-w-x-1-32-thick-black-high-strength-80a-acrylic-adhesive>: HTTP status code is not handled or not allowed 2026-01-25 02:26:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-polypropylene-rimless-90ml-centrifuge-tubes-162mm-12pk>: HTTP status code is not handled or not allowed 2026-01-25 02:26:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-30-esd-square-edge-mobile-production-bench-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:26:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/142002-gls-rope-pull-switch-2nc-1no-12npt-die-cast>: HTTP status code is not handled or not allowed 2026-01-25 02:26:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-double-tier-locker-3-wide-15w-x-18d-x-36h-assembled-almond>: HTTP status code is not handled or not allowed 2026-01-25 02:26:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/142059a-gls-rope-pull-switch-wled-std-act-2nc-2no-die-cast>: HTTP status code is not handled or not allowed 2026-01-25 02:26:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/142059b-gls-rope-pull-switch-wled-2nc-2no-110120v-ac-die-cast>: HTTP status code is not handled or not allowed 2026-01-25 02:26:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/SVT-1848-48-W-x-18-D-18-ga-T-430-Stainless-Steel-Workbench>: HTTP status code is not handled or not allowed 2026-01-25 02:26:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/SVTR-1860-18-W-x-60-D-T-430-18-ga-Stainless-Steel-Workbench-with-a-15-backsplash>: HTTP status code is not handled or not allowed 2026-01-25 02:26:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/seq-dual-speed-electric-chain-hoist-1-2-ton-15-lift-25-4-2-ft-min-115v>: HTTP status code is not handled or not allowed 2026-01-25 02:26:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20l-x-20w-x-38h-corner-block-white>: HTTP status code is not handled or not allowed 2026-01-25 02:26:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-buna-n-strip-120-l-x-4-w-x-1-16-thick-black-high-strength-80a>: HTTP status code is not handled or not allowed 2026-01-25 02:26:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precision-brass-thumb-screw-1-4-20-3-4-thread-1-2-head-dia-3-16-head-h-pkg-of-10>: HTTP status code is not handled or not allowed 2026-01-25 02:26:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-five-tier-locker-3-wide-12w-x-15d-x-12h-assembled-almond>: HTTP status code is not handled or not allowed 2026-01-25 02:26:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-1-2-mini-bellows-plunger>: HTTP status code is not handled or not allowed 2026-01-25 02:26:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precision-thumb-screw-w-shoulder-m5-x-0-8-9mm-thread-11-1mm-head-dia-stainless-pkg-of-5>: HTTP status code is not handled or not allowed 2026-01-25 02:26:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/statfree-t2-plus-8482-66460-dissipative-dual-layer-rubber-cut-mat-36-d-x-72-w-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:26:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-step-calibration-block-steel>: HTTP status code is not handled or not allowed 2026-01-25 02:27:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-front-action-sealer-for-5-8w-x-5-8thickness>: HTTP status code is not handled or not allowed 2026-01-25 02:27:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-head-t-knob-thumb-screw-m6-x-1-0-30mm-thread-45mm-head-dia-24mm-head-h-pkg-of-5>: HTTP status code is not handled or not allowed 2026-01-25 02:27:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cast-acrylic-sheet-48-l-x-48-w-x-1-8-thick-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:27:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-single-tier-locker-2-wide-12w-x-12d-x-60h-assembled-almond>: HTTP status code is not handled or not allowed 2026-01-25 02:27:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-rack-netting-three-bay-8>: HTTP status code is not handled or not allowed 2026-01-25 02:27:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brazos-series-gray-outdoor-stack-chair-with-flex-comfort-material-4-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:27:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_alarms_signals/strobe_flashing_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:27:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u-shaped-round-tubing-hangrail-semi-gloss-white>: HTTP status code is not handled or not allowed 2026-01-25 02:27:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:27:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-248ledmy120a-48-mm-led-stacklight-module-yellow-120v-ac>: HTTP status code is not handled or not allowed 2026-01-25 02:27:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/55131-power-cord-1-pkg-qty>: HTTP status code is not handled or not allowed 2026-01-25 02:27:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 52762 pages (at 96 pages/min), scraped 26559 items (at 39 items/min) 2026-01-25 02:27:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-rack-netting-two-bay-23>: HTTP status code is not handled or not allowed 2026-01-25 02:27:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ecp4410t-4-125-hp-1785-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:27:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pensacola-chaise-lounge-green>: HTTP status code is not handled or not allowed 2026-01-25 02:27:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30441-vacuum-hose-adapter-1id-to-1-14id-1-pkg-qty>: HTTP status code is not handled or not allowed 2026-01-25 02:27:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-chm264a-33airover-hp-1140-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:27:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/die_cut_labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:27:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_alarms_signals/strobe_flashing_lights>: HTTP status code is not handled or not allowed 2026-01-25 02:27:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:27:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/die_cut_labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:27:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-buna-n-sheet-12-l-x-12-w-x-3-16-thick-black-high-strength-90a-acrylic-adhesive-bulk-rs-h90-57>: HTTP status code is not handled or not allowed 2026-01-25 02:27:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a0062-internal-muffler-1-pkg-qty>: HTTP status code is not handled or not allowed 2026-01-25 02:27:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents landed on page that is not a product page. 2026-01-25 02:27:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:27:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/basic-pack-half-mask-respirator-kit-m-1-each-h05-8721m>: HTTP status code is not handled or not allowed 2026-01-25 02:27:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/indigo-nxt-half-cube-ice-machine-30-wide-310-lbs-24-hrs-prod-water-cooled>: HTTP status code is not handled or not allowed 2026-01-25 02:27:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b0018-spindle-516-24-int-1-pkg-qty>: HTTP status code is not handled or not allowed 2026-01-25 02:27:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/die_cut_labels>: HTTP status code is not handled or not allowed 2026-01-25 02:27:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/label_tapes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:27:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs>: HTTP status code is not handled or not allowed 2026-01-25 02:27:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/adhesives_sealants_tape/caulk_sealants/adhesive_caulk_sealants already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:27:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/die_cut_labels>: HTTP status code is not handled or not allowed 2026-01-25 02:27:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-270fa1248d-incandescentled-bulb-module-amber-12-48v-dc>: HTTP status code is not handled or not allowed 2026-01-25 02:27:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cl3515t-2-hp-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:27:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs landed on page that is not a product page. 2026-01-25 02:27:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30442-vacuum-hose-swivel-adapter-1id-x-1id-thread-1-pkg-qty>: HTTP status code is not handled or not allowed 2026-01-25 02:27:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-purpose-pipe-dope-1-fl-oz-tube>: HTTP status code is not handled or not allowed 2026-01-25 02:27:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/die_cut_labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:27:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/draft-inducer-blower-3300-rpm-115v-oao-2-4-fl-amps>: HTTP status code is not handled or not allowed 2026-01-25 02:27:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/174-150455-prop-65-regulatory-sign-restaurant-14-x-10-polyester-black-on-white>: HTTP status code is not handled or not allowed 2026-01-25 02:27:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/label_tapes landed on page that is not a product page. 2026-01-25 02:27:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/condenser-fans-motor-1-6-hp-825-rpm-208-230v-teao>: HTTP status code is not handled or not allowed 2026-01-25 02:27:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-hinged-top-basket-truck-6-bu-orange-vinyl-wood-base-all-swivel>: HTTP status code is not handled or not allowed 2026-01-25 02:27:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pipe-lube-1-gal>: HTTP status code is not handled or not allowed 2026-01-25 02:27:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/draft-inducer-blower-3000-rpm-115v-oao-1-5-fl-amps-ball-bearings>: HTTP status code is not handled or not allowed 2026-01-25 02:27:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fan-and-blower-1-4-hp-1725-rpm-115-230v-odp>: HTTP status code is not handled or not allowed 2026-01-25 02:27:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/adhesives_sealants_tape/caulk_sealants/adhesive_caulk_sealants landed on page that is not a product page. 2026-01-25 02:27:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/die_cut_labels>: HTTP status code is not handled or not allowed 2026-01-25 02:27:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/indestructable-too-folding-table-96-inch-x-30-inch-charcoal>: HTTP status code is not handled or not allowed 2026-01-25 02:27:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/banner-there-is-no-i-teamwork-3-x-5>: HTTP status code is not handled or not allowed 2026-01-25 02:27:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-270ledb240v-led-bulb-blue-240v>: HTTP status code is not handled or not allowed 2026-01-25 02:27:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-inch-straight-jaw-dipped-handle-tongue-groove-pliers>: HTTP status code is not handled or not allowed 2026-01-25 02:27:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/condenser-fans-motor-1-2-hp-825-rpm-208-230v-teao-48-frame>: HTTP status code is not handled or not allowed 2026-01-25 02:27:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-crowd-control-76-magnetic-wall-mount-with-yellow-cleaning-belt>: HTTP status code is not handled or not allowed 2026-01-25 02:27:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-boltless-shelving-48-inch-w-x-18-inch-d-x-84-inch-h-3-levels-starter-w-wood-decking-black>: HTTP status code is not handled or not allowed 2026-01-25 02:27:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unit-heater-motor-1-3-hp-1100-rpm-115v-teao>: HTTP status code is not handled or not allowed 2026-01-25 02:27:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-aom3617t-1-hp-860-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:27:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tablet-cell-phone-locker-9w-x-12d-x-30-1-2h-5-door-1-wide-padlock-hasp-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:27:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cecp4103t-4-25-hp-1780-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:27:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/counter-height-industrial-cabinet-36-w-x-24-d-x-45-h-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:27:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/counter-height-industrial-cabinet-36-w-x-24-d-x-45-h-brown>: HTTP status code is not handled or not allowed 2026-01-25 02:27:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-cabinet-30-w-x-24-d-x-75-h-brown>: HTTP status code is not handled or not allowed 2026-01-25 02:27:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inkjet-brochure-paper-matte-white-8-1-2in-x-11in-150-sheets-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:27:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cfm3146a-75-hp-1140-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:27:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hydraulic-drum-dumper-stationary-3>: HTTP status code is not handled or not allowed 2026-01-25 02:27:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/l100-series-manual-chain-hoist-w-overload-protection-1-4-ton15ft-lift>: HTTP status code is not handled or not allowed 2026-01-25 02:27:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/revolution-nevertear-paper-white-8-mil-8-1-2in-x-11in-500-sheets-ream>: HTTP status code is not handled or not allowed 2026-01-25 02:27:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-flow-control-valve-flow-screw-adjustment-1-2-tube-x-3-8-universal-thread-1-2-7-l>: HTTP status code is not handled or not allowed 2026-01-25 02:27:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ssc-series-electric-chain-hoist-2-ton-20-ft-lift-115v-230v>: HTTP status code is not handled or not allowed 2026-01-25 02:27:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vestil-steel-wheel-chock-with-handle-safety-chain-fab-10-ergo>: HTTP status code is not handled or not allowed 2026-01-25 02:27:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sparco-4-digit-tally-counter-w-finger-ring>: HTTP status code is not handled or not allowed 2026-01-25 02:27:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-matte-presentation-paper-bright-white-11in-x-14in-50-sheets-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:28:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/photo_presentation_paper already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:28:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/papers-multipurpose20-paper-white-8-1-2in-x-11in-20-lb-1-500-sheets-carton>: HTTP status code is not handled or not allowed 2026-01-25 02:28:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clearview-industrial-cabinet-60-w-x-24-d-x-75-h-brown>: HTTP status code is not handled or not allowed 2026-01-25 02:28:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/contractor-knit-roller-cover-smooth-3-8-nap>: HTTP status code is not handled or not allowed 2026-01-25 02:28:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-three-phase-odp-motor-1-hp-1760-rpm-230-460v-odp-143t-frame>: HTTP status code is not handled or not allowed 2026-01-25 02:28:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/peakworks-fbh10020b-contractor-series-harness-310-lb-capacity-universal-fit-class-ap>: HTTP status code is not handled or not allowed 2026-01-25 02:28:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pre-configured-6x10-partitioned-office-60-h-starter-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:28:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 52855 pages (at 93 pages/min), scraped 26592 items (at 33 items/min) 2026-01-25 02:28:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s30120-301-series-ratcheting-faceshield-polycarbonate-window>: HTTP status code is not handled or not allowed 2026-01-25 02:28:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/paper_stationery/photo_presentation_paper landed on page that is not a product page. 2026-01-25 02:28:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/farm-duty-single-phase-motor-2-hp-1725-rpm-230v-tefc>: HTTP status code is not handled or not allowed 2026-01-25 02:28:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oem-replacement-motor-1-7-hp-1050-rpm-115v-oao>: HTTP status code is not handled or not allowed 2026-01-25 02:28:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-boltless-shelving-48-inch-w-x-24-inch-d-1-level-w-wood-decking-black>: HTTP status code is not handled or not allowed 2026-01-25 02:28:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-uniform-cabinet-with-full-width-rod-60-w-x-24-d-x-75-h-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:28:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-straight-male-1-8-tube-x-1-4-swift-fit-universal-thread-3-4-l>: HTTP status code is not handled or not allowed 2026-01-25 02:28:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/definite_purpose_electric_motors/agricultural_motors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:28:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c-oem-replacement-motor-1-8-hp-1650-rpm-200-230v-teao>: HTTP status code is not handled or not allowed 2026-01-25 02:28:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/toolbox-z300-white-interfold-135-sheets-box-8-boxes-case-2020002>: HTTP status code is not handled or not allowed 2026-01-25 02:28:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-uniform-cabinet-with-4-shelves-60-w-x-24-d-x-75-h-red>: HTTP status code is not handled or not allowed 2026-01-25 02:28:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-5-tier-add-on-unit-30-w-x-18-d-x-74-h>: HTTP status code is not handled or not allowed 2026-01-25 02:28:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-straight-male-14mm-tube-x-1-2-swift-fit-universal-thread-1-1-4-l>: HTTP status code is not handled or not allowed 2026-01-25 02:28:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/definite_purpose_electric_motors/oem_replacement_ac_motors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:28:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c-oem-replacement-motor-1-10-hp-1100-rpm-208-230v-teao>: HTTP status code is not handled or not allowed 2026-01-25 02:28:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pool-pump-motor-5-hp-3450-rpm-208-230v-odp>: HTTP status code is not handled or not allowed 2026-01-25 02:28:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/definite_purpose_electric_motors/agricultural_motors landed on page that is not a product page. 2026-01-25 02:28:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-cabinet-with-6-drawers-60-w-x-24-d-x-75-h-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:28:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-stackable-shelf-ledge-back-30-w-x-2-h>: HTTP status code is not handled or not allowed 2026-01-25 02:28:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-boltless-shelving-48-inch-w-x-24-inch-d-x-84-inch-h-3-levels-add-on-w-wire-decking-black>: HTTP status code is not handled or not allowed 2026-01-25 02:28:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wood-end-table-with-steel-frame-24-x-24-black>: HTTP status code is not handled or not allowed 2026-01-25 02:28:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-straight-male-6mm-tube-x-1-8-bspp-1-l>: HTTP status code is not handled or not allowed 2026-01-25 02:28:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/recycled-two-pocket-paper-portfolios-with-prongs-assorted>: HTTP status code is not handled or not allowed 2026-01-25 02:28:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/apex-digital-clinical-scale-with-mechanical-height-rod-600-lb-x-0-2-lb>: HTTP status code is not handled or not allowed 2026-01-25 02:28:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/definite_purpose_electric_motors/oem_replacement_ac_motors>: HTTP status code is not handled or not allowed 2026-01-25 02:28:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tailpipe-storage-shelving-48-inch-w-x-18-inch-d-x-120-inch-h-3-levels-individual-tan>: HTTP status code is not handled or not allowed 2026-01-25 02:28:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-xtra-4-shelf-utility-cart-black-with-all-sides-open>: HTTP status code is not handled or not allowed 2026-01-25 02:28:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-swivel-run-tee-6mm-tube-x-1-8-swift-fit-universal-thread-5-6-l>: HTTP status code is not handled or not allowed 2026-01-25 02:28:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cabinet-with-electronic-lock-and-card-reader-48-w-x-24-d-x-75-h-white>: HTTP status code is not handled or not allowed 2026-01-25 02:28:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-swivel-branch-tee-4mm-tube-x-1-8-bspp-1-1-3-l>: HTTP status code is not handled or not allowed 2026-01-25 02:28:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-boltless-shelving-60-inch-w-x-30-inch-d-x-84-inch-h-3-levels-add-on-w-o-decking-black>: HTTP status code is not handled or not allowed 2026-01-25 02:28:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-idvsm3665t-5-hp-6200-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:28:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-boltless-shelving-96-inch-w-x-48-inch-d-1-level-w-wood-decking-black>: HTTP status code is not handled or not allowed 2026-01-25 02:28:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-profile-condensate-pump-24-foot-lift-115v>: HTTP status code is not handled or not allowed 2026-01-25 02:28:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rivetwell-particle-board-decking-60-inch-w-x-30-inch-d-x-5-8-inch>: HTTP status code is not handled or not allowed 2026-01-25 02:28:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/professional-lysol-disinfectant-foam-cleaner>: HTTP status code is not handled or not allowed 2026-01-25 02:28:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jack-plate-with-non-skid-surface-18-18-2-14469>: HTTP status code is not handled or not allowed 2026-01-25 02:28:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/maxirough-janitor-cart-96980>: HTTP status code is not handled or not allowed 2026-01-25 02:28:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:28:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-x-8-butyl-pads-for-adhesion-to-road-surface>: HTTP status code is not handled or not allowed 2026-01-25 02:28:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/recycled-poly-sheet-protector-standard-weight-reduced-glare-11-x-8-1-2-100-bx>: HTTP status code is not handled or not allowed 2026-01-25 02:28:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:28:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-tear-drop-pallet-rack-add-on-96-w-x-48-d-x-96-h-b2282523n>: HTTP status code is not handled or not allowed 2026-01-25 02:28:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-floor-pads-black-14>: HTTP status code is not handled or not allowed 2026-01-25 02:28:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metal-inserts-anchor-core-48-l-15-pc-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 02:28:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-l3513-50-1-5-hp-2850-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:29:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-1-4-multisocket-x-1-2-female-nptf-2-1-3-l>: HTTP status code is not handled or not allowed 2026-01-25 02:29:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/padded-mri-safe-exam-stool-15-diameter-x-21-h-w-2-vinyl-cushion-w-crutch-tips>: HTTP status code is not handled or not allowed 2026-01-25 02:29:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/knee-boot-size-6-15-h-composite-toe-chevron-plus-outsole-blue-w-red-sole>: HTTP status code is not handled or not allowed 2026-01-25 02:29:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits>: HTTP status code is not handled or not allowed 2026-01-25 02:29:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acrylic-cashier-shield-with-pass-thru-24L-x-24H-clear-with-racetrack-base>: HTTP status code is not handled or not allowed 2026-01-25 02:29:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/teak-outdoor-table-36-l-36-w-30-h>: HTTP status code is not handled or not allowed 2026-01-25 02:29:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-flow-control-12mm-tube-x-3-8-swift-fit-universal-thread-1-1-4-l>: HTTP status code is not handled or not allowed 2026-01-25 02:29:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:29:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/atrix-omega-brush-tool-black>: HTTP status code is not handled or not allowed 2026-01-25 02:29:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/atrix-high-capacity-hepa-filter-black>: HTTP status code is not handled or not allowed 2026-01-25 02:29:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-bottle-wine-security-unit-8-wine-shelves-chrome-14-w-x-36-l-x-66-h>: HTTP status code is not handled or not allowed 2026-01-25 02:29:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/71572-24-inch-commercial-wall-fan-non-oscillating-1-4-hp-4400-cfm>: HTTP status code is not handled or not allowed 2026-01-25 02:29:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 52951 pages (at 96 pages/min), scraped 26634 items (at 42 items/min) 2026-01-25 02:29:09 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits landed on page that is not a product page. 2026-01-25 02:29:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/71586-30-inch-commercial-pedestal-fan-non-oscillating-1-4-hp-6100-cfm>: HTTP status code is not handled or not allowed 2026-01-25 02:29:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-slide-valve-3-8-nptf-1-3-8-l>: HTTP status code is not handled or not allowed 2026-01-25 02:29:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-thermostat-at023-snap-action-90-130-f>: HTTP status code is not handled or not allowed 2026-01-25 02:29:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bmp71-series-indoor-outdoor-industrial-vinyl-labels-2-w-blue-m71c-2000-595-bl>: HTTP status code is not handled or not allowed 2026-01-25 02:29:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deep-freeze-rubber-antifatigue-mat-4-wide-up-to-60-3-8-thick-black>: HTTP status code is not handled or not allowed 2026-01-25 02:29:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interlake-mecalux-pallet-rack-tear-drop-welded-upright-frame-42-d-x-216-h-green>: HTTP status code is not handled or not allowed 2026-01-25 02:29:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/heat_shrink_wire_labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:29:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs landed on page that is not a product page. 2026-01-25 02:29:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bmp21-series-self-laminating-vinyl-wire-cable-labels-1-1-2-blk-wht-m21-1500-427>: HTTP status code is not handled or not allowed 2026-01-25 02:29:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quick-jump-floor-jack-35-ton-t83012>: HTTP status code is not handled or not allowed 2026-01-25 02:29:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/burn-gel-packets-25-box>: HTTP status code is not handled or not allowed 2026-01-25 02:29:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-boot-men-s-size-4-15-h-steel-pr-midsole-cleated-sole-black>: HTTP status code is not handled or not allowed 2026-01-25 02:29:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-welded-sc-1d-2424-nc-compact-storage-locker-24-w-x-24-d-x-52-h>: HTTP status code is not handled or not allowed 2026-01-25 02:29:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ecp844206tr-4-200-hp-1190-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:29:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultradrain-markers>: HTTP status code is not handled or not allowed 2026-01-25 02:29:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-diamond-series-laboratory-medical-glass-door-refrigerator-72-cu-ft-nswdr723wwg-0>: HTTP status code is not handled or not allowed 2026-01-25 02:29:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/heat_shrink_wire_labels>: HTTP status code is not handled or not allowed 2026-01-25 02:29:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bookcase-5-shelf-29-7-8-w-x-12-5-8-d-x-67-11-16-h-white>: HTTP status code is not handled or not allowed 2026-01-25 02:29:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-laminating-tags-6-1-4-x-3-1-8-green-100-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:29:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-polyurethane-cleanroom-stool-with-casters-medium-bench-height-black>: HTTP status code is not handled or not allowed 2026-01-25 02:29:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96in-144-led-strip-light-with-3m-adhesive-back-clear-and-cool-56297145>: HTTP status code is not handled or not allowed 2026-01-25 02:29:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/propane-heater-infrared-vacuum-straight-tube-100000-40l>: HTTP status code is not handled or not allowed 2026-01-25 02:29:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/big-d-enzym-d-mountain-air-quart>: HTTP status code is not handled or not allowed 2026-01-25 02:29:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-buckle-ice-traction-12-overshoe-mens-sz-16-waterproof-vulcanized-rubber-blk>: HTTP status code is not handled or not allowed 2026-01-25 02:29:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/espresso-36-oz-ground-coffee>: HTTP status code is not handled or not allowed 2026-01-25 02:29:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-33in-red-rectangular-stop-turn-tail-light-kit-with-15-led-5625115>: HTTP status code is not handled or not allowed 2026-01-25 02:29:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-25-x-3-42in-clear-rectangular-flood-light-with-6-led-1492191>: HTTP status code is not handled or not allowed 2026-01-25 02:29:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-x-1-5in-rectangular-clear-flood-light-with-3-led-1492139>: HTTP status code is not handled or not allowed 2026-01-25 02:29:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oil-safe-stumpy-pour-spout-lid-red>: HTTP status code is not handled or not allowed 2026-01-25 02:29:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-powered-orbit-cart-with-40ah-battery-black>: HTTP status code is not handled or not allowed 2026-01-25 02:29:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-powered-audio-visual-cart-lockable-cabinet-100ah-battery-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:29:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pressure-sensitive-pipe-marker-chilled-water-supply-2>: HTTP status code is not handled or not allowed 2026-01-25 02:29:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ejmm3709t-7-5-hp-3470-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:29:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/plumbing_parts_accessories/pipe_hose_tubing_accessories/pipe_markers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:29:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brake-drum-dolly-otc5017a>: HTTP status code is not handled or not allowed 2026-01-25 02:29:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/door-floor-stop-brushed-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 02:29:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/food_beverages/coffee_beverages/tea already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:29:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pads-pressure-replacement-non-asbestos-2pk-amm9183>: HTTP status code is not handled or not allowed 2026-01-25 02:29:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precoiled-strap-on-pipe-marker-condensate-drain-3>: HTTP status code is not handled or not allowed 2026-01-25 02:29:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/400-coverall-front-zipper-serged-seams-open-wrist-ankles-white-3x>: HTTP status code is not handled or not allowed 2026-01-25 02:29:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2000-coverall-front-zipper-serged-seams-yellow-xl>: HTTP status code is not handled or not allowed 2026-01-25 02:29:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tops-plain-paper-easel-pad-50-sheet-16-lb-unruled-27-x-34-2-carton-white-paper>: HTTP status code is not handled or not allowed 2026-01-25 02:29:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ejmm2534t-30-hp-3510-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:29:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precoiled-strap-on-pipe-marker-acid-4>: HTTP status code is not handled or not allowed 2026-01-25 02:29:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/plumbing/plumbing_parts_accessories/pipe_hose_tubing_accessories/pipe_markers>: HTTP status code is not handled or not allowed 2026-01-25 02:29:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-2-premium-pullover-hi-visibilty-t-shirt-lime-polyester-2xl>: HTTP status code is not handled or not allowed 2026-01-25 02:29:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/food_beverages/coffee_beverages/tea>: HTTP status code is not handled or not allowed 2026-01-25 02:29:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qsd-w-universal-queen-size-divider-2w-x-5h-plastic-wh-pkg-qty-100>: HTTP status code is not handled or not allowed 2026-01-25 02:29:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/23-gauge-1-3-8-pin-nailer>: HTTP status code is not handled or not allowed 2026-01-25 02:29:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-wide-welded-steel-lockers-double-tier-18x18x72-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:29:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/nails_pins_brads/staples_1 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:30:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-16-mag-nutsetter-x-1-3-4-s2-modified-steel-gunmetal-grey>: HTTP status code is not handled or not allowed 2026-01-25 02:30:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v201-safety-vest-polyester-mesh-hook-closure-orange-one-size>: HTTP status code is not handled or not allowed 2026-01-25 02:30:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mlrs1mc-countertop-breast-milk-refrigerator-1-cu-ft>: HTTP status code is not handled or not allowed 2026-01-25 02:30:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/44481-a40-liquidparticle-protection-apron-white-28w-x-40l-100case>: HTTP status code is not handled or not allowed 2026-01-25 02:30:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6pc-nutsetter-and-power-bit-set-gunmetal-grey-s2-modified-steel>: HTTP status code is not handled or not allowed 2026-01-25 02:30:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manual-label-dispenser-for-up-to-4w-labels>: HTTP status code is not handled or not allowed 2026-01-25 02:30:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-8482-enclosed-bulletin-board-cork-60-w-x-36-h-2-doors>: HTTP status code is not handled or not allowed 2026-01-25 02:30:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-pick-shelving-5-level-double-tilt-straight-78-h-x-50-w-x-48-d-starter>: HTTP status code is not handled or not allowed 2026-01-25 02:30:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-weather-faux-rattan-sofa-light-gray-with-light-gray-cushions>: HTTP status code is not handled or not allowed 2026-01-25 02:30:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/square-2-power-bit-x-3-1-2-s2-modified-steel-gunmetal-grey>: HTTP status code is not handled or not allowed 2026-01-25 02:30:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 53045 pages (at 94 pages/min), scraped 26671 items (at 37 items/min) 2026-01-25 02:30:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-vl3606t-3-hp-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:30:09 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/nails_pins_brads/staples_1 landed on page that is not a product page. 2026-01-25 02:30:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/btkct011-bleeding-trauma-kit-molle-bag-1-cat-tourniquet-chest-seal>: HTTP status code is not handled or not allowed 2026-01-25 02:30:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-plastic-sign-caution-chock-wheels-before-loading-or-unloading>: HTTP status code is not handled or not allowed 2026-01-25 02:30:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-x-4-seamless-pipe-nipple-schedule-80-304-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 02:30:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nitriledipped-gloves-activgrip-nitrile-wmicrofinish-grip-12-l-open-xl>: HTTP status code is not handled or not allowed 2026-01-25 02:30:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shop-desk-w-pigeonhole-storage-pegboard-w-shelf-34-1-2-w-x-30-d-x-38-to-42-1-2-h-slope-top-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:30:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lever-hoist-with-overload-protection-3-ton-capacity-10-lift>: HTTP status code is not handled or not allowed 2026-01-25 02:30:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-vem3661t-5-3-hp-1760-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:30:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-economy-manual-chain-hoist-1-ton-capacity-20-lift>: HTTP status code is not handled or not allowed 2026-01-25 02:30:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-plastic-sign-danger-this-machine-starts-automatically-2>: HTTP status code is not handled or not allowed 2026-01-25 02:30:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-economy-manual-chain-hoist-2-ton-capacity-10-lift>: HTTP status code is not handled or not allowed 2026-01-25 02:30:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torx-174-tamper-27-insert-bit-x-1-s2-modified-steel-gunmetal-grey>: HTTP status code is not handled or not allowed 2026-01-25 02:30:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/furniture_parts_accessories/partition_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:30:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torx-174-tamper-45-insert-bit-x-1-bull-nose-s2-modified-steel-gunmetal-grey>: HTTP status code is not handled or not allowed 2026-01-25 02:30:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transit-bike-shelter-7-30-43-6-l-x-7-5-w-30-bike-capacity-barrel-roof>: HTTP status code is not handled or not allowed 2026-01-25 02:30:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/cubicle_partition_panels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:30:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-m3455-25-hp-1140-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:30:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/cubicle_partition_panels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:30:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roady-polywoven-airbag-24-x-48>: HTTP status code is not handled or not allowed 2026-01-25 02:30:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-aluminum-sign-caution-floor-load-limit-not-to-exceed>: HTTP status code is not handled or not allowed 2026-01-25 02:30:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-eco2s-wall-heater-240v-1750w-white-dove>: HTTP status code is not handled or not allowed 2026-01-25 02:30:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/daredevilstubby-spade-bits-1-18dia>: HTTP status code is not handled or not allowed 2026-01-25 02:30:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/furniture_parts_accessories/partition_accessories>: HTTP status code is not handled or not allowed 2026-01-25 02:30:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/top-grain-cowhide-drivers-straight-thumb-regular-grade-l>: HTTP status code is not handled or not allowed 2026-01-25 02:30:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:30:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/screw-pitch-gauge-14-34>: HTTP status code is not handled or not allowed 2026-01-25 02:30:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/partitions_dividers/cubicle_partition_panels>: HTTP status code is not handled or not allowed 2026-01-25 02:30:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/partitions_dividers/cubicle_partition_panels>: HTTP status code is not handled or not allowed 2026-01-25 02:30:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-vinyl-sign-notice-restricted-area-authorized-personnel-only>: HTTP status code is not handled or not allowed 2026-01-25 02:30:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-zdvsm3665t-5-hp-6200-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:30:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/little-giant-wwc3060-hd6phfl-mobile-heavy-duty-cabinet-workbench-steel-top>: HTTP status code is not handled or not allowed 2026-01-25 02:30:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs>: HTTP status code is not handled or not allowed 2026-01-25 02:30:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torx-174-10-insert-bit-x-1-gunmetal-bronze-s2-modified-steel>: HTTP status code is not handled or not allowed 2026-01-25 02:30:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-vswdm3538-5-hp-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:30:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-foot-patio-umbrella-navy-white-cabana-stripe-hardwood-pole-grove>: HTTP status code is not handled or not allowed 2026-01-25 02:30:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wsl2-3684-ah-welded-steel-workbenches-w-back-and-end-stops-36-x-84-adj-height>: HTTP status code is not handled or not allowed 2026-01-25 02:30:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mj-2d-2448-fl-48-w-mobile-bench-cabinets-butcher-block-top-2-doors>: HTTP status code is not handled or not allowed 2026-01-25 02:30:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-foot-patio-umbrella-olefin-hunter-green-hardwood-pole-grove>: HTTP status code is not handled or not allowed 2026-01-25 02:30:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ip67-blue-smart-waterproof-battery-charger-with-bluetooth-24v-12a-1-120v-nema-5-15>: HTTP status code is not handled or not allowed 2026-01-25 02:30:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yoke-roller-sealed-crowned-od-15-roller-dia>: HTTP status code is not handled or not allowed 2026-01-25 02:30:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ww3072-heavy-duty-workbench-30-x-72-10000-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-25 02:30:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bactizyme-drain-cleaner-maintainer-fresh-and-clean-five-gallon>: HTTP status code is not handled or not allowed 2026-01-25 02:30:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-battery-cable-with-5-16-ring-terminals-4-awg-2-ft-black>: HTTP status code is not handled or not allowed 2026-01-25 02:30:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machine-length-stretch-wrap-film-cast-30-x-5000-80-gauge-clear>: HTTP status code is not handled or not allowed 2026-01-25 02:30:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/battery-cable-set-with-3-8-ring-terminals-1-0-awg-4-ft-black-red>: HTTP status code is not handled or not allowed 2026-01-25 02:30:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-vinyl-sign-notice-employees-only-3>: HTTP status code is not handled or not allowed 2026-01-25 02:30:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:30:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cam-follower-sealed-hex-head-crowned-od-2-l-stud-125-w-roller-225-roller-dia>: HTTP status code is not handled or not allowed 2026-01-25 02:31:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-esd-14w-x-54l-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 02:31:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-battery-cable-with-5-16-ring-terminals-1-0-awg-6-ft-red>: HTTP status code is not handled or not allowed 2026-01-25 02:31:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-tier-hanging-chair-table-truck-holds-42-chairs-10-tables>: HTTP status code is not handled or not allowed 2026-01-25 02:31:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hazardous-waste-vinyl-labels-notice-this-container-has-been-triple-rinsed>: HTTP status code is not handled or not allowed 2026-01-25 02:31:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-compartment-box-24-compartments>: HTTP status code is not handled or not allowed 2026-01-25 02:31:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-step-heavy-duty-mobile-work-platform-24-x-72-platform>: HTTP status code is not handled or not allowed 2026-01-25 02:31:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 53138 pages (at 93 pages/min), scraped 26714 items (at 43 items/min) 2026-01-25 02:31:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caution-sign-14x10-aluminum-forkli-traffic-area>: HTTP status code is not handled or not allowed 2026-01-25 02:31:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-battery-cable-with-3-8-ring-terminals-2-0-awg-6-ft-red>: HTTP status code is not handled or not allowed 2026-01-25 02:31:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/over-the-panel-single-hook-qty-6-black>: HTTP status code is not handled or not allowed 2026-01-25 02:31:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelf-truck-esd-18w-x-24l-x-80h-polyurethane-4-swivel-2-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 02:31:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs landed on page that is not a product page. 2026-01-25 02:31:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-battery-cable-with-3-8-ring-terminals-1-0-awg-5-ft-black>: HTTP status code is not handled or not allowed 2026-01-25 02:31:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-4-tier-esd-14w-x-36l-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 02:31:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-trash-container-garbage-can-lid-55-gallon-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:31:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:31:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-4-tier-esd-14w-x-72l-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 02:31:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/danger-sign-14x10-rigid-plastic-no-smoking-matches-or-open-flames>: HTTP status code is not handled or not allowed 2026-01-25 02:31:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mppt-wirebox-m-100-30-50-150-35-black-abs-plastic>: HTTP status code is not handled or not allowed 2026-01-25 02:31:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hc-40404-lube-oil-with-extenda-spout-4-oz>: HTTP status code is not handled or not allowed 2026-01-25 02:31:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honeywell-small-universal-thermostat-guard-clear-cover-clear-base-opaque-wallplate>: HTTP status code is not handled or not allowed 2026-01-25 02:31:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vital-cluster-box-unit-w-vogue-classic-accessories-13-unit-1-parcel-locker-sandstone>: HTTP status code is not handled or not allowed 2026-01-25 02:31:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hc-45025-sandcloth-waterproof-10-yd-plumber-s-grit-rool>: HTTP status code is not handled or not allowed 2026-01-25 02:31:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/notice-sign-14x10-rigid-plastic-keep-door-closed>: HTTP status code is not handled or not allowed 2026-01-25 02:31:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10555b-conditioning-qc-disc-type-r-2-aluminum-oxide-very-fine>: HTTP status code is not handled or not allowed 2026-01-25 02:31:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-shelf-truck-esd-2-shelf-24w-x-36l-x-40h-polyurethane-4-swivel-2-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 02:31:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs landed on page that is not a product page. 2026-01-25 02:31:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oty-30924-abs-special-milky-clear-cement-1-gallon>: HTTP status code is not handled or not allowed 2026-01-25 02:31:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/protective-wear-isolation-bundle-22-7-16-w-x-3-7-8-d-x-10-h-transparent>: HTTP status code is not handled or not allowed 2026-01-25 02:31:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-stage-air-compressor-w-silencer-dryer-5-hp-80-gal-1-phase-230v>: HTTP status code is not handled or not allowed 2026-01-25 02:31:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-esd-24w-x-54l-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 02:31:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-inch-x-6-inch-x-4-inch-long-corrugated-cartons>: HTTP status code is not handled or not allowed 2026-01-25 02:31:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-inch-x-6-inch-x-6-inch-long-corrugated-boxes>: HTTP status code is not handled or not allowed 2026-01-25 02:31:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=tankless-rotary-screw-compressor-w-dryer-7-5-hp-1-phase-230v already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:31:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11460-coated-psa-disc-stick-on-6-aluminum-oxide-medium>: HTTP status code is not handled or not allowed 2026-01-25 02:31:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-1-2-inch-x-12-1-2-inch-x-15-inch-doublewall-boxes>: HTTP status code is not handled or not allowed 2026-01-25 02:31:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plant-marking-stencil-20x20-fire-extinguisher>: HTTP status code is not handled or not allowed 2026-01-25 02:31:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-grade-disposable-nitrile-gloves-powder-free-small-blue-100-box-gnpr-sm-1m>: HTTP status code is not handled or not allowed 2026-01-25 02:31:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-electric-office-partition-panel-48-14-w-x-65-12-h-tan>: HTTP status code is not handled or not allowed 2026-01-25 02:31:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-dual-heat-electric-garage-heater-w-bluetooth-25-589-btu-7500w>: HTTP status code is not handled or not allowed 2026-01-25 02:31:42 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=tankless-rotary-screw-compressor-w-dryer-7-5-hp-1-phase-230v landed on page that is not a product page. 2026-01-25 02:31:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/martin-yale-400-tabletop-paper-jogger>: HTTP status code is not handled or not allowed 2026-01-25 02:31:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-plastic-garbage-can-with-lid-20-gal-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:31:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rust-ole-com-cv740-sys-100-voc-dtm-alkyd-ena-rust-prev-mnt-pnt-sr-gy>: HTTP status code is not handled or not allowed 2026-01-25 02:31:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-4-tier-esd-21w-x-30l-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 02:31:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/individual-character-stencil-24-letter-n>: HTTP status code is not handled or not allowed 2026-01-25 02:31:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8x1-5-8-bugle-head-drywall-screws-0826CQG>: HTTP status code is not handled or not allowed 2026-01-25 02:31:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electric-office-partition-panel-36-14-w-x-46-h-tan>: HTTP status code is not handled or not allowed 2026-01-25 02:31:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-office-partition-panel-wpartial-window-raceway-60-14-w-x-65-12-h-tan>: HTTP status code is not handled or not allowed 2026-01-25 02:31:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rust-oleum-m1600-sys-solv-bas-precis-lin-invert-mark-pnt-aero-safe-grn>: HTTP status code is not handled or not allowed 2026-01-25 02:31:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-esd-24w-x-48l-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 02:31:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-24x3-4-flat-undercut-head-thread-cutting-screws-type-f-1012FPUB>: HTTP status code is not handled or not allowed 2026-01-25 02:31:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rust-oleum-m1600-sys-slvnt-based-prec-lin-invrt-mark-pnt-aero-fluor-or>: HTTP status code is not handled or not allowed 2026-01-25 02:31:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-16x4-indent-hex-washer-head-thread-cutting-screws-type-f-3764FW>: HTTP status code is not handled or not allowed 2026-01-25 02:32:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-deluxe-office-partition-panel-48-14-w-x-46-12-h-tan>: HTTP status code is not handled or not allowed 2026-01-25 02:32:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rust-oleum-v7500-series-450-voc-dtm-alkyd-enamel-semi-gloss-white>: HTTP status code is not handled or not allowed 2026-01-25 02:32:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8x1-2-modified-truss-head-self-drilling-screws-0808KPM>: HTTP status code is not handled or not allowed 2026-01-25 02:32:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-safety-sign-fire-equipment-keep-clear-plastic>: HTTP status code is not handled or not allowed 2026-01-25 02:32:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roxbury-king-tufted-upholstered-platform-bed-black-with-memory-foam-mattress>: HTTP status code is not handled or not allowed 2026-01-25 02:32:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 53235 pages (at 97 pages/min), scraped 26761 items (at 47 items/min) 2026-01-25 02:32:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14x3-4-pan-head-self-tapping-screws-type-a-thread-1412APPB>: HTTP status code is not handled or not allowed 2026-01-25 02:32:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roxbury-king-size-tufted-upholstered-platform-bed-in-beige>: HTTP status code is not handled or not allowed 2026-01-25 02:32:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8x3-4-pan-head-self-tapping-screws-type-a-b-thread-0812ABPPB>: HTTP status code is not handled or not allowed 2026-01-25 02:32:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brown-viton-o-ring-dash-014-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 02:32:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultimate-productivity-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:32:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/press-brake-tooling-36-press-brake>: HTTP status code is not handled or not allowed 2026-01-25 02:32:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/100-ton-ironworker-3-phase-208-volt-10-brake>: HTTP status code is not handled or not allowed 2026-01-25 02:32:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-punch-die-set-with-storage-case-12-piece>: HTTP status code is not handled or not allowed 2026-01-25 02:32:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rust-oleum-industrial-choice-1600-system-gen-purp-enamel-aero-safe-gr>: HTTP status code is not handled or not allowed 2026-01-25 02:32:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/viton-o-ring-5mm-wide-155mm-id-pack-of-1>: HTTP status code is not handled or not allowed 2026-01-25 02:32:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-rubber-overboots-mens-black-size-1112-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 02:32:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-32x1-5-8-pan-head-machine-screws-0826MPPBZ>: HTTP status code is not handled or not allowed 2026-01-25 02:32:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-dry-floor-brush-for-cat-c16v-wet-dry-vacuum-641759>: HTTP status code is not handled or not allowed 2026-01-25 02:32:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-toe-shoe-covers-mens-black-with-yellow-toe-size-8595-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 02:32:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spaghettiloop-strapon-traction-soles-mens-red-one-size-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 02:32:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silicone-rubber-cord-0-187-cross-section-100-ft-length>: HTTP status code is not handled or not allowed 2026-01-25 02:32:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ingersoll-rand-air-compressor-2545e10v-value-plus-460v-3ph-60hz>: HTTP status code is not handled or not allowed 2026-01-25 02:32:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8x2-flat-self-drilling-screws-0832KQF>: HTTP status code is not handled or not allowed 2026-01-25 02:32:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4x3-16-3hd-pan-head-high-low-screws-0403HPP>: HTTP status code is not handled or not allowed 2026-01-25 02:32:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tri-temp-non-filtered-water-dispenser-black-with-stainless>: HTTP status code is not handled or not allowed 2026-01-25 02:32:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/riverdale-king-size-tufted-upholstered-platform-bed-in-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:32:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reinforced-high-pressure-fda-silicone-tubing-1id-x-1-38od-x-25-ft>: HTTP status code is not handled or not allowed 2026-01-25 02:32:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ingersoll-rand-air-compressor-2545e10-premium-200v-3ph-60hz>: HTTP status code is not handled or not allowed 2026-01-25 02:32:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6x1-1-4-bugle-head-drywall-screws-0620YPG>: HTTP status code is not handled or not allowed 2026-01-25 02:32:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buna-n-o-ring-4mm-wide-142mm-id-pack-of-5>: HTTP status code is not handled or not allowed 2026-01-25 02:32:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-led-dock-light-40w-4900-lumens-5000k-on-off-switch-with-25-l-arm>: HTTP status code is not handled or not allowed 2026-01-25 02:32:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/printed-barricade-tape-danger-lead-removal-authorized-personnel-only-2>: HTTP status code is not handled or not allowed 2026-01-25 02:32:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-25-11112-60-psikpa-14-cbm-plastic>: HTTP status code is not handled or not allowed 2026-01-25 02:32:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-4-11110-30-inhg-vac-14-lm-plastic>: HTTP status code is not handled or not allowed 2026-01-25 02:32:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brown-viton-o-ring-dash-369-pack-of-1>: HTTP status code is not handled or not allowed 2026-01-25 02:32:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-shrink-bags-48x46x72-4-mil-25-rl>: HTTP status code is not handled or not allowed 2026-01-25 02:32:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-desk-gray-3>: HTTP status code is not handled or not allowed 2026-01-25 02:32:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evac-chair-6254000000>: HTTP status code is not handled or not allowed 2026-01-25 02:32:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:32:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kold-locker-outdoor-freezer-minus-10-degrees-f-with-floor-1-1-2hp-lh-door-96-inches-w-x-96-inches-d-x-91-inches-h>: HTTP status code is not handled or not allowed 2026-01-25 02:32:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-shirt-bag-inthank-youin-with-handles-11-1-2in-x-6-1-2in-x-21in-12-5-micron-900-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:32:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sl-gpw-f2-gen-eye-pod-wi-fi-pipe-inspection-location-system-w-gen-eye-hot-spot-locator>: HTTP status code is not handled or not allowed 2026-01-25 02:32:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glow-nyc-sign-down>: HTTP status code is not handled or not allowed 2026-01-25 02:32:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glow-nyc-sign-down-right>: HTTP status code is not handled or not allowed 2026-01-25 02:32:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/highlighters_markers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:32:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/p2r-work-rechargeable-led-penlight>: HTTP status code is not handled or not allowed 2026-01-25 02:32:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/self_luminous_emergency_exit_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:32:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4x1-4-round-head-type-u-drive-screws-0404U>: HTTP status code is not handled or not allowed 2026-01-25 02:33:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/address_shipping_labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:33:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges>: HTTP status code is not handled or not allowed 2026-01-25 02:33:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-16x12-carriage-bolt-37192C>: HTTP status code is not handled or not allowed 2026-01-25 02:33:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stick-porous-point-pen-medium-0-7mm-assorted-ink-barrel-8-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:33:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/pens already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:33:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-32-weld-nuts-with-tab-base-11NWS1>: HTTP status code is not handled or not allowed 2026-01-25 02:33:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/highlighters_markers landed on page that is not a product page. 2026-01-25 02:33:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/self_luminous_emergency_exit_signs>: HTTP status code is not handled or not allowed 2026-01-25 02:33:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/pens already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:33:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 53331 pages (at 96 pages/min), scraped 26811 items (at 50 items/min) 2026-01-25 02:33:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/address_shipping_labels>: HTTP status code is not handled or not allowed 2026-01-25 02:33:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interion-174-office-partition-panel-48-1-4-w-x-96-h-tan>: HTTP status code is not handled or not allowed 2026-01-25 02:33:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/comfort-grip-retractable-ballpoint-pen-1mm-black-ink-clear-barrel-dozen>: HTTP status code is not handled or not allowed 2026-01-25 02:33:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6206nr-open-wsnap-ring-30mm-bore-62mm-od>: HTTP status code is not handled or not allowed 2026-01-25 02:33:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:33:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-8x4-hex-tap-bolts-grade-8-10064BHT8>: HTTP status code is not handled or not allowed 2026-01-25 02:33:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6204zz-double-shielded-20mm-bore-47mm-od>: HTTP status code is not handled or not allowed 2026-01-25 02:33:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-fixed-spreader-beam-yellow-80000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 02:33:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/self_luminous_emergency_exit_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:33:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/pens>: HTTP status code is not handled or not allowed 2026-01-25 02:33:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/self_luminous_emergency_exit_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:33:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/pens landed on page that is not a product page. 2026-01-25 02:33:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6003zznr-double-shielded-wsnap-ring-17mm-bore-35mm-od>: HTTP status code is not handled or not allowed 2026-01-25 02:33:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cable-puller-jcp-series-with-6-lift-4000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 02:33:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/84-120-economy-h-beam-adjustable-length-yellow-126-l-x-40-w-x-36-h-655lbs-4000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 02:33:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6901zz-double-shielded-12mm-bore-24mm-od>: HTTP status code is not handled or not allowed 2026-01-25 02:33:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs landed on page that is not a product page. 2026-01-25 02:33:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-16-x-8-working-length-j-hook-style-b>: HTTP status code is not handled or not allowed 2026-01-25 02:33:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/self_luminous_emergency_exit_signs>: HTTP status code is not handled or not allowed 2026-01-25 02:33:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-lift-beam-yellow-16-l-x-16-w-x-6-h-40lbs-10000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 02:33:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/self_luminous_emergency_exit_signs landed on page that is not a product page. 2026-01-25 02:33:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mini-m-series-five-drawer-anesthesia-cart-w-standard-key-lock-sand>: HTTP status code is not handled or not allowed 2026-01-25 02:33:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-waterproof-reversible-raincoat-ansi-class-3-300d-oxfordpu-coating-limeblack-4xl>: HTTP status code is not handled or not allowed 2026-01-25 02:33:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-deep-parts-bin-32-openings>: HTTP status code is not handled or not allowed 2026-01-25 02:33:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-tray-pan-tray-racks-24x30>: HTTP status code is not handled or not allowed 2026-01-25 02:33:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dyson-airblade-153-v-back-panel-964691-01>: HTTP status code is not handled or not allowed 2026-01-25 02:33:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-spread-basket-lifting-beam-yellow-10000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 02:33:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-29-hydraulic-ergonomic-workbench-steel-top>: HTTP status code is not handled or not allowed 2026-01-25 02:33:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/19290-americana-full-brim-hard-hat-w-accy-slots-mega-ratchet-4-pt-suspension-hi-viz-lime>: HTTP status code is not handled or not allowed 2026-01-25 02:33:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-voltage-370-440-oval-run-capacitor-455-mfd-45-5dv>: HTTP status code is not handled or not allowed 2026-01-25 02:33:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medium-weight-inspection-gloves-unhemmed-mens-97-520>: HTTP status code is not handled or not allowed 2026-01-25 02:33:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/depository-safe-mb2014k-dual-key-lock-16-1-2w-x-14d-x-20h-black>: HTTP status code is not handled or not allowed 2026-01-25 02:33:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dot-placard-poison-6-2>: HTTP status code is not handled or not allowed 2026-01-25 02:33:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dot-placard-toxic-6-2>: HTTP status code is not handled or not allowed 2026-01-25 02:33:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/eye_protection/safety_glasses/frameless_safety_glasses already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:33:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/saf-t-flag-no-diagonal>: HTTP status code is not handled or not allowed 2026-01-25 02:33:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s371-ansi-class-3-raincoat-hi-vis-lime-3x-61484>: HTTP status code is not handled or not allowed 2026-01-25 02:33:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-tall-anesthesia-cart-6-drawers-e-lock-47-5-8-w-x-22-l-x-66-3-4-h-taupe>: HTTP status code is not handled or not allowed 2026-01-25 02:33:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/62396-s252c-class-2-mesh-solid-surveyor-vest-hi-vis-orange-2xl>: HTTP status code is not handled or not allowed 2026-01-25 02:33:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/eye_protection/safety_glasses/frameless_safety_glasses already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:33:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-voltage-370-440-oval-run-capacitor-10-mfd-10dv>: HTTP status code is not handled or not allowed 2026-01-25 02:33:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-voltage-370-440-oval-run-capacitor-153-mfd-15-3dv>: HTTP status code is not handled or not allowed 2026-01-25 02:33:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-feet-x-14-feet-sf-9-93-oz-water-resist-canvas-tarp-tan>: HTTP status code is not handled or not allowed 2026-01-25 02:33:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/eye_protection/safety_glasses/frameless_safety_glasses landed on page that is not a product page. 2026-01-25 02:33:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-series-aluminium-isolation-cart-6-drawers-30-w-x-22-l-x-43-3-4-h-white>: HTTP status code is not handled or not allowed 2026-01-25 02:33:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-socks-6-x-3-20-gallon-capacity-18pack>: HTTP status code is not handled or not allowed 2026-01-25 02:34:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/eye_protection/safety_glasses/frameless_safety_glasses>: HTTP status code is not handled or not allowed 2026-01-25 02:34:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-series-lightweight-emergency-cart-6-drawers-key-lock-36-3-4-wx22-lx40-1-2-h-taupe>: HTTP status code is not handled or not allowed 2026-01-25 02:34:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6213zznr-double-shielded-wsnap-ring-65mm-bore-120mm-od>: HTTP status code is not handled or not allowed 2026-01-25 02:34:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-series-lightweight-emergency-cart-6-drawers-key-lock-36-3-4-wx22-lx40-1-2-h-mauve>: HTTP status code is not handled or not allowed 2026-01-25 02:34:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/fans_blowers/ceiling_fans/residential_ceiling_fans already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:34:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/retractable-belt-stanchion-40-high-14-base-13-belt-6-social-distancing>: HTTP status code is not handled or not allowed 2026-01-25 02:34:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/do-not-double-stack-edge-protectors-3-x-3-x-48-160-thick-white-1600-skid>: HTTP status code is not handled or not allowed 2026-01-25 02:34:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tall-savary-dilator-drying-cart-hepa-filter-23-89-100-wx22-lx51-89-100-h-yellow>: HTTP status code is not handled or not allowed 2026-01-25 02:34:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 53422 pages (at 91 pages/min), scraped 26854 items (at 43 items/min) 2026-01-25 02:34:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-tall-emergency-crash-cart-6-drawers-36-3-4-w-x-22-l-x-43-h-burgundy>: HTTP status code is not handled or not allowed 2026-01-25 02:34:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-hr-time-switched-duty-cycle-dial-plastic-enclosure>: HTTP status code is not handled or not allowed 2026-01-25 02:34:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slow-moving-vehicle-triangle-galvanized-steel-red>: HTTP status code is not handled or not allowed 2026-01-25 02:34:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-epoxy-wire-shelf-platform-p2124b-24w-x-21d>: HTTP status code is not handled or not allowed 2026-01-25 02:34:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-spread-basket-lifting-beam-yellow-1000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 02:34:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/fans_blowers/ceiling_fans/residential_ceiling_fans>: HTTP status code is not handled or not allowed 2026-01-25 02:34:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-limit-thermostat-spst-open-140f-close-120f>: HTTP status code is not handled or not allowed 2026-01-25 02:34:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rotating-light-120vac-red-371l-120r>: HTTP status code is not handled or not allowed 2026-01-25 02:34:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/litestak-light-module-24vdc-green-lsl-024g>: HTTP status code is not handled or not allowed 2026-01-25 02:34:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flammable-gas-label-worded-pvc-free-film-500-roll>: HTTP status code is not handled or not allowed 2026-01-25 02:34:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1500mm-drylin-t-hard-anodized-aluminum-rail-size-15>: HTTP status code is not handled or not allowed 2026-01-25 02:34:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vynckier-mp1614s-vj-16-inch-x-14-inch-steel-mounting-plate>: HTTP status code is not handled or not allowed 2026-01-25 02:34:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sunlite-88460-su-t8-led-4-15w-is-pps-dlc-40k-t8-led-tube-15w-2200-lum-4000k-type-a-plug-play>: HTTP status code is not handled or not allowed 2026-01-25 02:34:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-column-medical-storage-cart-glass-door-22-1-2-w-x-27-d-x-75-3-4-h-pink>: HTTP status code is not handled or not allowed 2026-01-25 02:34:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/strobe-light-edison-base-120vac-blue-lp3e-120b>: HTTP status code is not handled or not allowed 2026-01-25 02:34:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/boss-hutch-with-2-doors-48x-12-x36-mahogany>: HTTP status code is not handled or not allowed 2026-01-25 02:34:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/strobe-sounder-18-30vdc-red-lp7-18-30r>: HTTP status code is not handled or not allowed 2026-01-25 02:34:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reducer-414815>: HTTP status code is not handled or not allowed 2026-01-25 02:34:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-shipper-ata-case-919-wheeled-case-38l-x-25w-x-15h-silver>: HTTP status code is not handled or not allowed 2026-01-25 02:34:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6487-cooling-multi-band-hi-vis-lime>: HTTP status code is not handled or not allowed 2026-01-25 02:34:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/boss-66-credenza-mahogany>: HTTP status code is not handled or not allowed 2026-01-25 02:34:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/drum_barrel_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:34:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wash-your-hands-before-leaving-this-room-sticker-14x10-adhesive-backed-vinyl>: HTTP status code is not handled or not allowed 2026-01-25 02:34:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fabric-task-chair-high-back-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:34:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/suredry-14-scope-drying-cabinet-w-e-lock-36-w-x-24-d-x-93-h-beige>: HTTP status code is not handled or not allowed 2026-01-25 02:34:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyjohn-comfort-inn-ada-portable-restroom-tan>: HTTP status code is not handled or not allowed 2026-01-25 02:34:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-short-emergency-crash-cart-md30-emg1-std-width-4-drawers-breakaway-lock-white>: HTTP status code is not handled or not allowed 2026-01-25 02:34:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/social_distancing_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:34:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-series-48-wide-door-electric-heated-208360-12-kw-obsidian-black>: HTTP status code is not handled or not allowed 2026-01-25 02:34:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-rack-fork-clearance-bar-48-l>: HTTP status code is not handled or not allowed 2026-01-25 02:34:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m20-25-carbon-steel-insert-for-metal-450-20>: HTTP status code is not handled or not allowed 2026-01-25 02:34:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/utility_racks/drum_barrel_racks>: HTTP status code is not handled or not allowed 2026-01-25 02:34:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1161301-construction-style-harness-pass-through-tongue-buckle-m-l>: HTTP status code is not handled or not allowed 2026-01-25 02:34:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m14-125-carbon-steel-insert-for-metal-750-14>: HTTP status code is not handled or not allowed 2026-01-25 02:34:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/engraved-sign-lunch-room-brown>: HTTP status code is not handled or not allowed 2026-01-25 02:34:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-column-medical-storage-cabinet-key-lock-glass-doors-42-wx29-3-5-dx76-h-red>: HTTP status code is not handled or not allowed 2026-01-25 02:34:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tall-isolation-cart-3-drawers-standard-key-lock-37-1-2-wx22-lx42-63-100-h-burgundy>: HTTP status code is not handled or not allowed 2026-01-25 02:34:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tall-isolation-cart-3-drawers-standard-key-lock-37-1-2-wx22-lx42-63-100-h-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:34:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/278-expo-ii-telescoping-shipping-case-trade-show-case-33l-x-26w-x-6h-black>: HTTP status code is not handled or not allowed 2026-01-25 02:34:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carrying-case-foam-filled-707-series-20l-x-16w-x-8h-black>: HTTP status code is not handled or not allowed 2026-01-25 02:34:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/social_distancing_signs landed on page that is not a product page. 2026-01-25 02:34:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/500ml-pp-bottle-top-filter-nylon-filter-material-0-45-181-m-90mm-non-sterile-12-pk>: HTTP status code is not handled or not allowed 2026-01-25 02:34:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1161509-vest-style-positioning-climbing-harness-pass-through-tongue-buckle-2xl>: HTTP status code is not handled or not allowed 2026-01-25 02:34:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/floor-sign-caution-men-working-above-below>: HTTP status code is not handled or not allowed 2026-01-25 02:34:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd3p2e-general-purpose-3-hp-1800-rpm-230-460v-tefc>: HTTP status code is not handled or not allowed 2026-01-25 02:34:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/d1055-oem-motor-for-iec-psc-1-12-hp-1375-rpm-115-127v-open-ventilated>: HTTP status code is not handled or not allowed 2026-01-25 02:34:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/d1045-oem-motor-for-iec-psc-1-30-hp-1100-rpm-115-127v-open-ventilated>: HTTP status code is not handled or not allowed 2026-01-25 02:34:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/syringe-filter-mce-0-45-181-m-13mm-sterile-yellow-polypropylene-75-pk>: HTTP status code is not handled or not allowed 2026-01-25 02:34:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-32-single-in-finsert-170-006-br>: HTTP status code is not handled or not allowed 2026-01-25 02:34:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/half-ht-double-col-med-storage-cabinet-keylock-40-1-10-wx29-1-4-dx40-3-4-h-hammertone-green>: HTTP status code is not handled or not allowed 2026-01-25 02:34:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quad-column-medical-storage-cabinet-glass-doors-82-1-8-wx30-1-4-dx77-3-4-h-beige>: HTTP status code is not handled or not allowed 2026-01-25 02:34:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grunt-work-glove-hi-vis-yellow-synthetic-leather-palm-xl-1-pair-ggt-17hvhv1l>: HTTP status code is not handled or not allowed 2026-01-25 02:34:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-trumpet-stainless-steel-marine-horn-power-coated-white-finish-12-volt-1125>: HTTP status code is not handled or not allowed 2026-01-25 02:34:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48327-4hp-straight-line-die-grinder-30000-rpm-gearless-rear-exhaust-1-4-6mm-collets>: HTTP status code is not handled or not allowed 2026-01-25 02:35:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/light-up-dial-portable-lock-box-set-your-own-combination-5424d>: HTTP status code is not handled or not allowed 2026-01-25 02:35:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cutting-edge-90-x-60-x-375>: HTTP status code is not handled or not allowed 2026-01-25 02:35:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-18-ga-bin-cabinet-with-68-storage-bins-dual-locking-48-w-x-18-d-x-72-h>: HTTP status code is not handled or not allowed 2026-01-25 02:35:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-20-straight-ultrasonic-insert-flanged-fl-420-hi>: HTTP status code is not handled or not allowed 2026-01-25 02:35:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/patient_support_immobilization/splints_supports already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:35:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-ga-extra-heavy-duty-triple-tier-locker-3-compartments-18-w-x-18-d-x-75-h-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:35:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/graphic-machine-labels-danger-disconnect-before-servicing>: HTTP status code is not handled or not allowed 2026-01-25 02:35:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-32-flush-press-insert-stainless-240-008-cr>: HTTP status code is not handled or not allowed 2026-01-25 02:35:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a10106phc-polyproencl-1004x1004x638in>: HTTP status code is not handled or not allowed 2026-01-25 02:35:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ws161608ss-watershedwall-mt-encl-type-4x-1600x1600x800in>: HTTP status code is not handled or not allowed 2026-01-25 02:35:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ns-antibacterial-splinting-material-24-x-36-x-1-8-macro-perforated>: HTTP status code is not handled or not allowed 2026-01-25 02:35:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/patient_support_immobilization/splints_supports already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:35:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/security-notice-plastic-use-of-cellular-phone-is-prohibited-this-facility>: HTTP status code is not handled or not allowed 2026-01-25 02:35:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 53513 pages (at 91 pages/min), scraped 26882 items (at 28 items/min) 2026-01-25 02:35:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-acting-pneumatic-actuator-1336-in-lbs-80psi>: HTTP status code is not handled or not allowed 2026-01-25 02:35:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lmk25-din-rail-or-panel-mtg-brackets-fits-250mm>: HTTP status code is not handled or not allowed 2026-01-25 02:35:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/patient_support_immobilization/splints_supports>: HTTP status code is not handled or not allowed 2026-01-25 02:35:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-8482-standard-vehicle-first-aid-kit>: HTTP status code is not handled or not allowed 2026-01-25 02:35:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cp253512-concepthmi-encl-wblack-extrusions-type-4-250x350x120mm>: HTTP status code is not handled or not allowed 2026-01-25 02:35:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-condenser-fan-motor-for-nexel-models-243005-243006>: HTTP status code is not handled or not allowed 2026-01-25 02:35:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/patient_support_immobilization/splints_supports>: HTTP status code is not handled or not allowed 2026-01-25 02:35:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/siesta-sunshine-resin-dining-arm-chair-green>: HTTP status code is not handled or not allowed 2026-01-25 02:35:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/siesta-gio-resin-outdoor-bar-stool-orange>: HTTP status code is not handled or not allowed 2026-01-25 02:35:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machine-labels-warning-keep-hands-away>: HTTP status code is not handled or not allowed 2026-01-25 02:35:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/grills/grills_smokers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:35:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:35:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hoist-ring-180-pivot-35-270-lb-load-capacity>: HTTP status code is not handled or not allowed 2026-01-25 02:35:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-machine-labels-caution-keep-hands-clear>: HTTP status code is not handled or not allowed 2026-01-25 02:35:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vertical-lifting-90-degree-turning-plate-clamp-orange-steel-4000-lbs-cap-1-opening>: HTTP status code is not handled or not allowed 2026-01-25 02:35:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pur-fill-nf12-professional-straw-foam-nf12>: HTTP status code is not handled or not allowed 2026-01-25 02:35:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-part-ez-flo-kit-spray-foam-2pez01>: HTTP status code is not handled or not allowed 2026-01-25 02:35:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pageris-long-barrel-24-stainless-steel-foam-gun-dispenser-gu24>: HTTP status code is not handled or not allowed 2026-01-25 02:35:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-35-ton-m12-pivot-swivel-load-ring>: HTTP status code is not handled or not allowed 2026-01-25 02:35:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:35:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nickel-plated-brass-non-drip-coupling-1-body-size-blue-color-ring-nbr-seals>: HTTP status code is not handled or not allowed 2026-01-25 02:35:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/american-lock-solid-aluminum-rectangular-padlock-black-no-a1105blk>: HTTP status code is not handled or not allowed 2026-01-25 02:35:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/outdoor_furniture/grills/grills_smokers landed on page that is not a product page. 2026-01-25 02:35:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 02:35:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-5-16in-one-piece-cast-coupler-0091558>: HTTP status code is not handled or not allowed 2026-01-25 02:35:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-3-way-l-port-ss-150-flanged-ball-valve-w-da-pneumatic>: HTTP status code is not handled or not allowed 2026-01-25 02:35:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-button-front-lab-coat-white-polycombed-cotton-s>: HTTP status code is not handled or not allowed 2026-01-25 02:35:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs>: HTTP status code is not handled or not allowed 2026-01-25 02:35:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-5-16in-chrome-hitch-ball-w-2-3-4in-shank-10-000-lb-capacity-1802027>: HTTP status code is not handled or not allowed 2026-01-25 02:35:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-button-closure-lab-coat-white-polycotton-2xl>: HTTP status code is not handled or not allowed 2026-01-25 02:35:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-lab-coat-light-gray-polycombed-cotton-50>: HTTP status code is not handled or not allowed 2026-01-25 02:35:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g-9-garolite-sheet-132-thick-x-36-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 02:35:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-non-drip-coupling-3-8-body-size-red-color-ring-nbr-seals>: HTTP status code is not handled or not allowed 2026-01-25 02:35:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-3-4-screw-pin-anchor-shackle>: HTTP status code is not handled or not allowed 2026-01-25 02:35:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vertical-reel-lifter-3000-lbs-capacity-yellow-steel>: HTTP status code is not handled or not allowed 2026-01-25 02:35:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-shelving-72x24x72-2>: HTTP status code is not handled or not allowed 2026-01-25 02:35:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-lab-coat-navy-polycombed-cotton-regular-42>: HTTP status code is not handled or not allowed 2026-01-25 02:35:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-sheet-38-thick-x-8-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 02:35:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-x-19-pegboard-panel-blue-2>: HTTP status code is not handled or not allowed 2026-01-25 02:35:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/odp-w-rigid-base-1hp-1800rpm-f2-mount>: HTTP status code is not handled or not allowed 2026-01-25 02:35:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/frost-surface-mounted-sanitary-napkin-disposal-stainless>: HTTP status code is not handled or not allowed 2026-01-25 02:35:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/odp-w-rigid-base-365ts-frame-75-hp-1800-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:35:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/decorative_lighting/flush_mount_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:35:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/frost-toilet-seat-cover-dispenser-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 02:35:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x42x74-chrome-wire-shelving-with-8-hopper-bins-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:35:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/odp-w-rigid-base-2hp-3600rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:35:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-industrial-long-sleeve-work-shirt-gray-polycotton-regular-4xl>: HTTP status code is not handled or not allowed 2026-01-25 02:35:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-assembly-for-b60zhd-pallet-trucks-hy-4616703-xl-a>: HTTP status code is not handled or not allowed 2026-01-25 02:35:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/odp-w-rigid-base-40-hp-3600-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:35:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:35:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unisex-concealed-snap-front-lab-coat-royal-blue-nomexaramid-2xl>: HTTP status code is not handled or not allowed 2026-01-25 02:35:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12x36x74-chrome-wire-shelving-with-24-giant-stacking-bins-black>: HTTP status code is not handled or not allowed 2026-01-25 02:35:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12x36x74-chrome-wire-shelving-with-32-6-h-shelf-bins-yellow>: HTTP status code is not handled or not allowed 2026-01-25 02:35:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-high-security-combo-padlock-with-key-control-short-shackle>: HTTP status code is not handled or not allowed 2026-01-25 02:36:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-built-in-combination-lock-with-long-bolt-green-dial-left-hinged>: HTTP status code is not handled or not allowed 2026-01-25 02:36:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/office-konnect-153-stackable-pencil-cup-2-pack-black-6-case>: HTTP status code is not handled or not allowed 2026-01-25 02:36:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pin-for-mp-mpb-040-ac-pallet-trucks-yl-518360601>: HTTP status code is not handled or not allowed 2026-01-25 02:36:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x36x74-chrome-wire-shelving-with-24-6-h-shelf-bins-yellow>: HTTP status code is not handled or not allowed 2026-01-25 02:36:04 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/decorative_lighting/flush_mount_lights landed on page that is not a product page. 2026-01-25 02:36:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-25 02:36:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/connector-for-m-series-stacker-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 02:36:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bushing-for-gpw-series-pallet-trucks-cr-042053-009>: HTTP status code is not handled or not allowed 2026-01-25 02:36:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/composites/composite_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:36:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washable-school-glue-1-gal-liquid>: HTTP status code is not handled or not allowed 2026-01-25 02:36:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 53606 pages (at 93 pages/min), scraped 26917 items (at 35 items/min) 2026-01-25 02:36:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spring-powered-handheld-compact-stapler-assorted-20-case>: HTTP status code is not handled or not allowed 2026-01-25 02:36:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:36:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-premium-scissors-gray-36-case>: HTTP status code is not handled or not allowed 2026-01-25 02:36:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/office-flat-clinch-stapler-40-sheets-metallic-purple-12-case>: HTTP status code is not handled or not allowed 2026-01-25 02:36:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b8-174-powercrown-153-premium-staples-1-4-leg-6mm-1-2-crown-5000-pack-100-case>: HTTP status code is not handled or not allowed 2026-01-25 02:36:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/cart_truck_accessories/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:36:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-containment-tray-without-grating-yellow>: HTTP status code is not handled or not allowed 2026-01-25 02:36:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disappearing-purple-all-purpose-glue-sticks-purple-clear-60-box>: HTTP status code is not handled or not allowed 2026-01-25 02:36:18 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/composites/composite_sheet_stock landed on page that is not a product page. 2026-01-25 02:36:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12x36x75-steel-shelving-with-14-giant-stacking-bins-ivory>: HTTP status code is not handled or not allowed 2026-01-25 02:36:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x36x75-steel-shelving-with-21-giant-stacking-bins-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:36:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/adhesives_sealants_tape/adhesives_glue/general_purpose_glue already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:36:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-25 02:36:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-for-ewr-series-pallet-trucks-ba-146158>: HTTP status code is not handled or not allowed 2026-01-25 02:36:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:36:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-twin-ibc-spill-pallet-no-drain-w1-left-side-bucket-shelf>: HTTP status code is not handled or not allowed 2026-01-25 02:36:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/carts-trucks/cart_truck_accessories/pallet_truck_parts_accessories/pallet_truck_wheel_kits>: HTTP status code is not handled or not allowed 2026-01-25 02:36:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-ner-dual-speed-elec-hoist-motor-trolley-1-15-2845>: HTTP status code is not handled or not allowed 2026-01-25 02:36:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-ner-electric-chain-hoist-hook-suspension-1-20-28>: HTTP status code is not handled or not allowed 2026-01-25 02:36:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-acetal-plastic-bar-14-thick-x-34-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 02:36:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xx-garolite-bar-18-thick-x-2-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 02:36:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/adhesives_sealants_tape/adhesives_glue/general_purpose_glue>: HTTP status code is not handled or not allowed 2026-01-25 02:36:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-ner-electric-chain-hoist-push-trolley-1-12-10-18>: HTTP status code is not handled or not allowed 2026-01-25 02:36:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-sorter-with-5-compartments-black>: HTTP status code is not handled or not allowed 2026-01-25 02:36:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 02:36:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3m-no-touch-earplugs-corded>: HTTP status code is not handled or not allowed 2026-01-25 02:36:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-3-hole-punch-11-32-inch-punch-size-with-40-sheet-capacity>: HTTP status code is not handled or not allowed 2026-01-25 02:36:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/cart_truck_accessories/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:36:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camcover-versa-6-1-2-granite-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:36:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-assembly-for-pe-3000-pallet-trucks-cr-100462-a-sg>: HTTP status code is not handled or not allowed 2026-01-25 02:36:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/koch-filter-merv-8-high-capacity-xl8-pleated-extended-surface-25-inch-w-x-25-inch-h-x-1-inch-d>: HTTP status code is not handled or not allowed 2026-01-25 02:36:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/koch-filter-merv-8-standard-capacity-xl8-pleated-panel-ext-surface-20-inch-w-x-30-inch-h-x-1-inch-d>: HTTP status code is not handled or not allowed 2026-01-25 02:36:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/550s-top-chest-w-10-drawers-50-w-x-25-1-4-d-x-27-h-black>: HTTP status code is not handled or not allowed 2026-01-25 02:36:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:36:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-bank-top-chest-w-8-drawers-55-w-x-22-3-8-d-x-27-2-3-h-red-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:36:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kl-lever-hoists-6600-lb-capacity-20-ft-lift>: HTTP status code is not handled or not allowed 2026-01-25 02:36:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kl-lever-hoists-6600-lb-capacity-15-ft-lift>: HTTP status code is not handled or not allowed 2026-01-25 02:36:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fork-riser-for-6hbe30-pallet-trucks-to-00590-42109-71>: HTTP status code is not handled or not allowed 2026-01-25 02:36:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-ner-dual-speed-electric-chain-hoist-1-12-10-183>: HTTP status code is not handled or not allowed 2026-01-25 02:36:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/koch-filter-merv-8-standard-capacity-xl8-pleated-panel-ext-surface-16-inch-w-x-24-inch-h-x-1-inch-d>: HTTP status code is not handled or not allowed 2026-01-25 02:36:44 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/cart_truck_accessories/pallet_truck_parts_accessories/pallet_truck_wheel_kits landed on page that is not a product page. 2026-01-25 02:36:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-cf-hand-chain-hoist-1-15-lift>: HTTP status code is not handled or not allowed 2026-01-25 02:36:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:36:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x36x74-chrome-wire-shelving-with-18-quickpick-double-open-bins-yellow>: HTTP status code is not handled or not allowed 2026-01-25 02:36:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shaft-for-pe-4000-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 02:36:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 02:36:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/koch-filter-merv-8-standard-capacity-xl8-pleated-panel-ext-surface-16-inch-w-x-30-inch-h-x-1-inch-d>: HTTP status code is not handled or not allowed 2026-01-25 02:36:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/composites/composite_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:36:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/37u-linier-server-cabinet-solid-solid-doors-36-depth>: HTTP status code is not handled or not allowed 2026-01-25 02:36:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:36:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roller-cabinet-w-8-drawers-34-w-x-25-1-4-d-x-41-h-red>: HTTP status code is not handled or not allowed 2026-01-25 02:36:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 02:36:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-salvage-drum-30-gallon>: HTTP status code is not handled or not allowed 2026-01-25 02:36:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/toolbox-no-tray-15-3-8-l-x-7-3-4-w-x-5-1-4-h-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:37:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/cable_management/cable_protectors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:37:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/composites/composite_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-25 02:37:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/composites/composite_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:37:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 02:37:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bushing-for-mpw060e-a897-pallet-trucks-yl-524140878>: HTTP status code is not handled or not allowed 2026-01-25 02:37:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-venturi-mounted-direct-drive-exhaust-fan>: HTTP status code is not handled or not allowed 2026-01-25 02:37:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/antistatic-hexagonal-weighing-dish-polystyrene-50ml-case-of-500>: HTTP status code is not handled or not allowed 2026-01-25 02:37:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/material-handling-basket-21l-x-13-1-4w-x-5-7-16h-0-5-wire-plain-steel>: HTTP status code is not handled or not allowed 2026-01-25 02:37:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 53694 pages (at 88 pages/min), scraped 26954 items (at 37 items/min) 2026-01-25 02:37:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/cable_management/cable_protectors>: HTTP status code is not handled or not allowed 2026-01-25 02:37:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/composites/composite_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-25 02:37:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-8482-gas-spring-led-lcd-monitor-wall-mount-arm-with-vesa-plate-black>: HTTP status code is not handled or not allowed 2026-01-25 02:37:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/full-cut-butcher-coat-white-polyestercotton-xl>: HTTP status code is not handled or not allowed 2026-01-25 02:37:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crc-dry-moly-lubricants-16-oz-aerosol-can>: HTTP status code is not handled or not allowed 2026-01-25 02:37:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cook-pants-white-polyestercotton-36-x-36>: HTTP status code is not handled or not allowed 2026-01-25 02:37:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xx-garolite-bar-1-thick-x-4-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 02:37:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/atv-v-bar-tire-chains-4-link-spacing-pair-1064355>: HTTP status code is not handled or not allowed 2026-01-25 02:37:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/basic-packing-bench-plastic-laminate-top-t-mold-edge-72-x-32>: HTTP status code is not handled or not allowed 2026-01-25 02:37:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/optional-type-wl-grooved-toe-plate-for-liftkar-sal-stair-climbing-trucks>: HTTP status code is not handled or not allowed 2026-01-25 02:37:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:37:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cook-pants-black-polyestercotton-44-x-36>: HTTP status code is not handled or not allowed 2026-01-25 02:37:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rcl-2200ir>: HTTP status code is not handled or not allowed 2026-01-25 02:37:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-116-thick-x-4-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 02:37:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/bins_totes_containers/storage_totes_tubs/storage_totes_with_lids already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:37:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ballymore-14-step-perforated-cantilever-ladder-42-overhang>: HTTP status code is not handled or not allowed 2026-01-25 02:37:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-retardant-neoprene-foam-strip-with-acrylic-adhesive-18-thick-x-6-wide-x-10-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 02:37:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/royal-basket-trucks-174-linen-cart-19-5-cu-ft-29-l-x-29-w-x-47-h-white>: HTTP status code is not handled or not allowed 2026-01-25 02:37:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/lighted_exit_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:37:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40-magazine-rotary-floor-display-mahogany>: HTTP status code is not handled or not allowed 2026-01-25 02:37:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-14-thick-x-5-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 02:37:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xx-garolite-bar-14-thick-x-4-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 02:37:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proline-ergoline-electric-height-adjustable-workbench-ele6030ppf-l14-60-w-x-30-d-light-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:37:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wheelbarrow-ladder-6-steps-aluminum-350-lb-capacity-14-d-top-step-24-w-step>: HTTP status code is not handled or not allowed 2026-01-25 02:37:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-sheet-2-12-thick-x-6-wide-x-6-long>: HTTP status code is not handled or not allowed 2026-01-25 02:37:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/bins_totes_containers/storage_totes_tubs/storage_totes_with_lids>: HTTP status code is not handled or not allowed 2026-01-25 02:37:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/battery-connector-for-102xm-walkie-pallet-truck>: HTTP status code is not handled or not allowed 2026-01-25 02:37:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-men-s-10-viking-black-overshoe-w-4-way-cleated-outsole-pvc-size-small>: HTTP status code is not handled or not allowed 2026-01-25 02:37:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 02:37:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bariatric-sled-base-chair-light-oak-blue-vinyl-3>: HTTP status code is not handled or not allowed 2026-01-25 02:37:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/office_furniture/literature_mail_sorters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:37:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-jack-toe-guard-protector>: HTTP status code is not handled or not allowed 2026-01-25 02:37:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/lighted_exit_signs landed on page that is not a product page. 2026-01-25 02:37:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:37:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-stage-snow-blower-shear-pin-set-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:37:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-tuftex-yellow-bib-overall-plain-front-pvc-size-2x>: HTTP status code is not handled or not allowed 2026-01-25 02:37:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-button-front-chef-coat-pearl-buttons-black-polyestercotton-2xl>: HTTP status code is not handled or not allowed 2026-01-25 02:37:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/rainwear already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:37:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/long-cook-shirt-white-plain-weave-spun-polyester-2xl>: HTTP status code is not handled or not allowed 2026-01-25 02:37:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x48x74-chrome-wire-shelving-with-91-shelf-bins-yellow>: HTTP status code is not handled or not allowed 2026-01-25 02:37:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/female-mannequin-sitting-legs-to-side-gloss-finish-white>: HTTP status code is not handled or not allowed 2026-01-25 02:37:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/office_furniture/literature_mail_sorters landed on page that is not a product page. 2026-01-25 02:37:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x48x74-chrome-wire-shelving-with-66-shelf-bins-red>: HTTP status code is not handled or not allowed 2026-01-25 02:37:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-x-8-x-5-euro-fix-solid-container-red>: HTTP status code is not handled or not allowed 2026-01-25 02:37:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cap-snap-13mm-pe-for-vacuum-and-test-tubes-clear-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:37:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-webtex-yellow-jacket-w-hood-snaps-pvc-size-3x>: HTTP status code is not handled or not allowed 2026-01-25 02:37:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x60x74-chrome-wire-shelving-with-118-shelf-bins-red>: HTTP status code is not handled or not allowed 2026-01-25 02:37:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transfer-pipet-58ml-fine-tip-157mm-sterile-400-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:37:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-sheet-132-thick-x-24-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 02:37:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/rainwear>: HTTP status code is not handled or not allowed 2026-01-25 02:37:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beaker-three-corner-clarified-polypropylene-graduated-50ml-100-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:37:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 02:37:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-tuftex-yellow-jacket-w-attached-hood-pvc-size-small>: HTTP status code is not handled or not allowed 2026-01-25 02:37:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slatwall-countertop-spinner-2-side-18w-x-12d-x-2112h-maple>: HTTP status code is not handled or not allowed 2026-01-25 02:37:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-stacking-bin-16-1-2x14-3-4x7-red>: HTTP status code is not handled or not allowed 2026-01-25 02:37:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cap-plug-16mm-santoprene-for-vacuum-and-test-tubes-red-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:37:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powr-flite-30-wide-area-sweeper-vacuum>: HTTP status code is not handled or not allowed 2026-01-25 02:37:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/four-shelf-mesh-truck-24-x-48>: HTTP status code is not handled or not allowed 2026-01-25 02:38:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extruded-aluminum-hose-cable-bridge-silver-60-x-2118-x-3916>: HTTP status code is not handled or not allowed 2026-01-25 02:38:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-men-s-boot-16-goliath-black-plain-toe-w-power-lug-outsole-pvc-size-11>: HTTP status code is not handled or not allowed 2026-01-25 02:38:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sample-tube-5ml-external-threads-polypropylene-round-bottom-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:38:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-men-s-35-hip-wader-black-plain-toe-w-cleated-outsole-pvc-size-9>: HTTP status code is not handled or not allowed 2026-01-25 02:38:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2hp-3-phase-total-enclosed-canister-dust-collector-ufo-dc1023>: HTTP status code is not handled or not allowed 2026-01-25 02:38:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-men-s-56-6-chest-wader-black-steel-toe-w-cleated-outsole-pvc-size-10>: HTTP status code is not handled or not allowed 2026-01-25 02:38:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-office-partition-panel-24-1-4w-x-42h-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:38:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-men-s-15-yellow-shoe-cover-pvc-size-medium>: HTTP status code is not handled or not allowed 2026-01-25 02:38:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 53787 pages (at 93 pages/min), scraped 26987 items (at 33 items/min) 2026-01-25 02:38:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-head-cap-screw-m8-x-1-25-x-22mm-steel-zinc-clear-class-8-8-din-933-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 02:38:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-men-s-boot-16-goliath-black-steel-toe-w-power-lug-outsole-pvc-size-10>: HTTP status code is not handled or not allowed 2026-01-25 02:38:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elephant-bark-rubber-floor-mat-316-thick-4-x-10-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:38:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-men-s-35-chest-wader-black-steel-toe-steel-midsole-pvc-size-13>: HTTP status code is not handled or not allowed 2026-01-25 02:38:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-men-s-boot-16-hazmax-ez-fit-green-yellow-steel-toe-mid-sole-pvc-size-10>: HTTP status code is not handled or not allowed 2026-01-25 02:38:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-back-draft-damper>: HTTP status code is not handled or not allowed 2026-01-25 02:38:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-sheet-3-thick-x-8-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 02:38:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/flooring/rubber_flooring already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:38:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-allsteel-triple-brace-double-hinge-folding-chair-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:38:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/science-lab-table-36h-with-chemres-top-dual-book-compartment-30-x-60>: HTTP status code is not handled or not allowed 2026-01-25 02:38:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-sheet-4-thick-x-8-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 02:38:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:38:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-office-partition-panel-with-window-48-1-4w-x-60h-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:38:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/flooring/rubber_flooring already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:38:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/americana-cap-vented-with-accessory-slots-and-4-point-slide-lock-suspension-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:38:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tube-holder-use-with-gtr-hd-series-tube-rotators-24-place-for-50ml-microcentrifuge-tubes>: HTTP status code is not handled or not allowed 2026-01-25 02:38:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/flooring/rubber_flooring>: HTTP status code is not handled or not allowed 2026-01-25 02:38:27 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-25 02:38:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/erb10-scratch-protection-rebar-guard-3-8>: HTTP status code is not handled or not allowed 2026-01-25 02:38:28 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:27 2026-01-25 02:38:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 02:38:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/flooring/rubber_flooring>: HTTP status code is not handled or not allowed 2026-01-25 02:38:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-heat-chin-strap-black>: HTTP status code is not handled or not allowed 2026-01-25 02:38:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fb-pic3588a-layflat-color-sewer-drain-camera-150-cable-w-sonde-transmitter-footage-counter>: HTTP status code is not handled or not allowed 2026-01-25 02:38:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:38:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tube-holder-use-with-gtr-hd-series-tube-rotators-48-places-for-15ml-microcentrifuge-tubes>: HTTP status code is not handled or not allowed 2026-01-25 02:38:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galvanized-steel-skatewheel-conveyor-24-oaw-90-degree-curve-20-wpf>: HTTP status code is not handled or not allowed 2026-01-25 02:38:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-rod-38-diameter-x-2-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 02:38:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-drawer-bench-high-24-w-cabinet-light-gray-6>: HTTP status code is not handled or not allowed 2026-01-25 02:38:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-steel-toe-waterproof-boots-16-height-black-size-11-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 02:38:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:38:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:38:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 02:38:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:38:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/rainwear already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:38:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microscope-slides-glass-25-x-75mm-90-ground-edges-safety-corners-blue-frosted-144-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:38:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transfer-pipet-150ml-narrow-stem-large-bulb-155mm-sterile-polypropylenes-200-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:38:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/100g-3-point-hitch-sprayer-pto-7560c-pump-50-12-hose>: HTTP status code is not handled or not allowed 2026-01-25 02:38:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-drawer-counter-high-36-w-cabinet-red-2>: HTTP status code is not handled or not allowed 2026-01-25 02:38:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=e-1440vs-with-acu-rite-203-dro already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:38:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 02:38:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 02:38:56 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/rainwear landed on page that is not a product page. 2026-01-25 02:38:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/149648-toughstripe-max-floor-marking-tape-vinyl-4-inch-wide-100-feet-long-red>: HTTP status code is not handled or not allowed 2026-01-25 02:38:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/freezing-box-3-cardboard-64-place-8x8-format-fits-3ml-4ml-and-5ml-vials-yellow-48-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:38:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-drawer-bench-high-24-w-cabinet-avalanche-blue-4>: HTTP status code is not handled or not allowed 2026-01-25 02:38:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 02:38:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jtm-1254rvs-with-acu-rite-203-3x-q-dro-x-powerfeed>: HTTP status code is not handled or not allowed 2026-01-25 02:38:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/undercounter-refrigerator-33-38f-27-58w-30-18d-29-34h-tuc-27>: HTTP status code is not handled or not allowed 2026-01-25 02:39:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polly-products-econo-mizer-6-ft-backed-bench-with-arms-green-bench-brown-frame>: HTTP status code is not handled or not allowed 2026-01-25 02:39:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=e-1440vs-with-acu-rite-203-dro>: HTTP status code is not handled or not allowed 2026-01-25 02:39:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gh-1440b-geared-head-bench-lathe-with-stand-foot-brake-with-newall-dp500-dro>: HTTP status code is not handled or not allowed 2026-01-25 02:39:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/refrigerator-reach-in-1-section-27w-29-34d-78-38h-t-23g-2>: HTTP status code is not handled or not allowed 2026-01-25 02:39:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/lathes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:39:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/idlertensioner-pulley-lt-duty-89013>: HTTP status code is not handled or not allowed 2026-01-25 02:39:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 53873 pages (at 86 pages/min), scraped 27030 items (at 43 items/min) 2026-01-25 02:39:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ghb-1236-geared-head-bench-lathe-with-stand-foot-brake-acu-rite-203-dro>: HTTP status code is not handled or not allowed 2026-01-25 02:39:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-aquamax-plastic-locker-single-tier-1-wide-12w-x-18d-x-72h-taupe-body-doors>: HTTP status code is not handled or not allowed 2026-01-25 02:39:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powershred-99ci-100-jam-proof-cross-cut-shredder>: HTTP status code is not handled or not allowed 2026-01-25 02:39:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jtm-1254rvs-with-newall-dp700-3x-k-dro-x-y-powerfeeds-air-power-drawbar>: HTTP status code is not handled or not allowed 2026-01-25 02:39:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-aquamax-plastic-locker-six-tier-1-wide-12w-x-18d-x-12h-taupe-body-doors>: HTTP status code is not handled or not allowed 2026-01-25 02:39:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-swivel-ring-dsr-m-18>: HTTP status code is not handled or not allowed 2026-01-25 02:39:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11-camwear-spoon-clear>: HTTP status code is not handled or not allowed 2026-01-25 02:39:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hvbs-712dv-7-inch-x-12-inch-variable-speed-deluxe-horizontal-vertical-bandsaw>: HTTP status code is not handled or not allowed 2026-01-25 02:39:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/sticky_notes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:39:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/machining/machinery/lathes>: HTTP status code is not handled or not allowed 2026-01-25 02:39:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-citrus-cleaner-degreaser-case-of-six-32-oz-trigger-spray-bottles>: HTTP status code is not handled or not allowed 2026-01-25 02:39:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forklift-maintenance-platform-40x48-cal-osha>: HTTP status code is not handled or not allowed 2026-01-25 02:39:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/training-table-60-x-24-walnut>: HTTP status code is not handled or not allowed 2026-01-25 02:39:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-quick-coupling-european-high-flow-profile-with-1-4-fnpt-connection>: HTTP status code is not handled or not allowed 2026-01-25 02:39:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-boxes-9-x-9-x-12->: HTTP status code is not handled or not allowed 2026-01-25 02:39:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3962-10-swing-spout-polished-chrome>: HTTP status code is not handled or not allowed 2026-01-25 02:39:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slide-mailer-cardboard-for-2-slides-50-box-20-boxes-unit>: HTTP status code is not handled or not allowed 2026-01-25 02:39:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-corrugated-boxes-8-x-6-x-4>: HTTP status code is not handled or not allowed 2026-01-25 02:39:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slide-mailer-polypropylene-flip-top-for-2-slides-natural-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:39:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/paper_stationery/sticky_notes>: HTTP status code is not handled or not allowed 2026-01-25 02:39:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/purpose-carry-pouch-with-belt-loop>: HTTP status code is not handled or not allowed 2026-01-25 02:39:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e-1440vs-with-newall-dp700-dro>: HTTP status code is not handled or not allowed 2026-01-25 02:39:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/etm-949-mill-with-2-axis-acu-rite-203-dro-and-x-y-axis-jet-powerfeeds>: HTTP status code is not handled or not allowed 2026-01-25 02:39:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-36-mobile-plastic-square-edge-work-bench-adjustable-height-1-5-8-top>: HTTP status code is not handled or not allowed 2026-01-25 02:39:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/summit-stainless-steel-outdoor-serving-cart>: HTTP status code is not handled or not allowed 2026-01-25 02:39:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/market-tray-pens-9x26x1-5-black>: HTTP status code is not handled or not allowed 2026-01-25 02:39:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-boxes-5-x-5-x-12->: HTTP status code is not handled or not allowed 2026-01-25 02:39:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-melamine-floor-pad-5-per-case>: HTTP status code is not handled or not allowed 2026-01-25 02:39:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stackbin-4000-series-maple-butcher-block-square-edge-96-w-x-36-d-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:39:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/market-tray-10x15-white>: HTTP status code is not handled or not allowed 2026-01-25 02:39:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/etm-949-mill-with-3-axis-acu-rite-300-knee-dro-and-servo-x-y-axis-powerfeeds>: HTTP status code is not handled or not allowed 2026-01-25 02:39:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rack-for-15ml-and-20ml-microcentrifuge-tubes-stackable-polycarbonate-pc-24-place-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:39:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rv-pette-dispenser-tip-for-repeat-volume-pipettors-certified-sterile-05ml-100-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:39:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camrack-base-rack-full-size-4-inch-inside-stack-height-beige-nsf>: HTTP status code is not handled or not allowed 2026-01-25 02:39:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-commercial-duty-exhaust-fan-1-phase-1-hp>: HTTP status code is not handled or not allowed 2026-01-25 02:39:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-commercial-duty-exhaust-fan-1-phase-3-4-hp>: HTTP status code is not handled or not allowed 2026-01-25 02:39:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-commercial-duty-exhaust-fan-1-phase-3-4-hp>: HTTP status code is not handled or not allowed 2026-01-25 02:39:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rv-pette-dispenser-tip-for-repeat-volume-pipettors-50ml-1-gray-adapter-included-25-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:39:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bottle-rectangular-hdpe-48mm-polypropylene-screw-cap-500ml-16oz-12-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:39:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pipette-diamond-jr-100ul-orange-autoclavable>: HTTP status code is not handled or not allowed 2026-01-25 02:39:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/justrite-smokers-cease-fire-cigarette-butt-receptacle-1-gallon-black>: HTTP status code is not handled or not allowed 2026-01-25 02:39:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bottle-wide-mouth-with-handle-square-polypropylene-4l>: HTTP status code is not handled or not allowed 2026-01-25 02:39:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-totally-enclosed-high-pressure-exhaust-fan-3-phase-5-hp>: HTTP status code is not handled or not allowed 2026-01-25 02:39:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-security-truck-poly-z-brite-24-w-x-36-l-x-69-h>: HTTP status code is not handled or not allowed 2026-01-25 02:39:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1848-sf-5-1-2-x-1-4-aluminum-cut-round-nose-tree-bur>: HTTP status code is not handled or not allowed 2026-01-25 02:39:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securall-12g-sliding-door-flammable-cabinet-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 02:39:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1846-sb-1-1-4-x-1-4-aluminum-cut-cylindrical-bur-w-end-cut>: HTTP status code is not handled or not allowed 2026-01-25 02:39:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/abrasives/points_burrs/burrs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:39:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-1-2-silver-base-anti-microbial-binder-side-open-mauve>: HTTP status code is not handled or not allowed 2026-01-25 02:40:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fantech-centrifugal-6-duct-fan-molded-housing-263>: HTTP status code is not handled or not allowed 2026-01-25 02:40:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-x-60-mat-ultimat-light-blue-um-2460lb>: HTTP status code is not handled or not allowed 2026-01-25 02:40:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/funnel-powder-polypropylene-100mm-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:40:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/esd_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:40:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-x-60-mat-ultimat-light-blue-um-3060lb>: HTTP status code is not handled or not allowed 2026-01-25 02:40:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/190xl025-standard-timing-belt-xl-1-4-x-19-t95-trapezoidal>: HTTP status code is not handled or not allowed 2026-01-25 02:40:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-black-stealthnet-standard-duty-bird-netting-50-x-75-n1-b230>: HTTP status code is not handled or not allowed 2026-01-25 02:40:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/abrasives/points_burrs/burrs>: HTTP status code is not handled or not allowed 2026-01-25 02:40:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unex-gravity-flow-roller-rack-with-wheel-bed-starter-96w-x-72d-x-84h-with-4-levels>: HTTP status code is not handled or not allowed 2026-01-25 02:40:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 53970 pages (at 97 pages/min), scraped 27072 items (at 42 items/min) 2026-01-25 02:40:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forged-steel-forks-1-5-thick-42-l>: HTTP status code is not handled or not allowed 2026-01-25 02:40:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/202xl075-standard-timing-belt-xl-3-4-x-20-3-16-t101-trapezoidal>: HTTP status code is not handled or not allowed 2026-01-25 02:40:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/95584057-bulb-ba15d-115v-5w>: HTTP status code is not handled or not allowed 2026-01-25 02:40:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/980l050-standard-timing-belt-l-1-2-x-98-t261-trapezoidal>: HTTP status code is not handled or not allowed 2026-01-25 02:40:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/floor_mats/esd_mats landed on page that is not a product page. 2026-01-25 02:40:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-1-4-x-77-1-2-h-deluxe-office-partition-panel-with-pass-thru-cable-gray-277560gy>: HTTP status code is not handled or not allowed 2026-01-25 02:40:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/statfree-uc-dissipative-dual-layer-rubber-mat-sky-blue-0080in-x-30in-x-40ft>: HTTP status code is not handled or not allowed 2026-01-25 02:40:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microcentrifuge-tube-05ml-polypropylene-assorted-stand-up-zipbag-mg-500-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:40:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/supco-88-oil-additive>: HTTP status code is not handled or not allowed 2026-01-25 02:40:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bottle-with-screwcap-wide-mouth-polypropylene-graduated-2000ml>: HTTP status code is not handled or not allowed 2026-01-25 02:40:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/panel-saw-laser-cutting-guide>: HTTP status code is not handled or not allowed 2026-01-25 02:40:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/internal-cap-fittings-12dia-furniture-grade-pvc-white>: HTTP status code is not handled or not allowed 2026-01-25 02:40:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-foot-plate-with-molded-base>: HTTP status code is not handled or not allowed 2026-01-25 02:40:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pipette-tip-1000-5000ul-1-5ml-natural-use-with-finnpipette-lab-systems-edp2-smi-200-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:40:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/furniture-pipe-fittings-5l-1dia-furniture-grade-pvc-white>: HTTP status code is not handled or not allowed 2026-01-25 02:40:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/structural-plastic-vented-shelving-36-x-36-x-75>: HTTP status code is not handled or not allowed 2026-01-25 02:40:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-frame-guard-right>: HTTP status code is not handled or not allowed 2026-01-25 02:40:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tork-universal-masterfold-large-dispenser-napkins-d786e>: HTTP status code is not handled or not allowed 2026-01-25 02:40:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microcentrifuge-tube-15ml-polypropylene-lot-certified-mg-natural-500-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:40:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-height-bath-chair-with-quick-release-back-arms-white-dc-hy3523l-wh-gg>: HTTP status code is not handled or not allowed 2026-01-25 02:40:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/snap-n-rack-tube-rack-for-15ml-and-20ml-microcentrifuge-tube-100-place-polypropylene-red>: HTTP status code is not handled or not allowed 2026-01-25 02:40:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ep-725-12-poly-tubing-dispenser-with-slide-cutter-12>: HTTP status code is not handled or not allowed 2026-01-25 02:40:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50btl42-taper-bushed-58-pitch-42-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:40:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camcruiser-vending-cart-4-full-size-pans-6-deep-black>: HTTP status code is not handled or not allowed 2026-01-25 02:40:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gray-fine-fiber-universal-medium-weight-pads-100-bale-18-x-15>: HTTP status code is not handled or not allowed 2026-01-25 02:40:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-25b29hx38mpb-14-pitch-38-plain-bore-29-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:40:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/extra_heavy_boltless_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:40:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/airvantage-ad-061302-6-non-vacuum-random-orbital-sander-3-16-orbit>: HTTP status code is not handled or not allowed 2026-01-25 02:40:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60bs23hx134-34-pitch-1-34-finished-bore-23-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:40:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-ratchet-3-8-inch-drive-76-ft-lb-torque>: HTTP status code is not handled or not allowed 2026-01-25 02:40:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-reversible-pistol-grip-air-drill-3-8-inch-chuck-2000-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:40:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ice-caddies-dark-brown-125-lbs-cap-4-swivel-1-with-brake>: HTTP status code is not handled or not allowed 2026-01-25 02:40:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60bs23hx1316-34-pitch-1-316-finished-bore-23-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:40:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securall-r-11-insulation-b2400>: HTTP status code is not handled or not allowed 2026-01-25 02:40:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/siesta-capri-resin-dining-arm-chair-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:40:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelf-mobile-computer-lanstation-workstation-keyboard-tray-40-hx24-wx60-l-black-3-shelf>: HTTP status code is not handled or not allowed 2026-01-25 02:40:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beaker-polypropylene-printed-graduations-250ml-16-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:40:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securall-primer-coat-saltwater-agb600>: HTTP status code is not handled or not allowed 2026-01-25 02:40:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-visibility-rain-jacket-ansi-class-3-5xl-yellowblack>: HTTP status code is not handled or not allowed 2026-01-25 02:40:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-o-smith-single-phase-jet-pump-motor-115-230-volts-3450-rpm-1-1-2hp-3>: HTTP status code is not handled or not allowed 2026-01-25 02:40:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/extra_heavy_boltless_shelves>: HTTP status code is not handled or not allowed 2026-01-25 02:40:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/structural-plastic-vented-shelf-66-x-16>: HTTP status code is not handled or not allowed 2026-01-25 02:40:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/extra_heavy_boltless_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:40:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9in-pneumatic-swivel-caster-for-little-giant-carts>: HTTP status code is not handled or not allowed 2026-01-25 02:40:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-plastic-dunnage-rack-36-x-36-1500-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-25 02:40:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-2-female-nps-pipe-thread-x-3-4-male-garden-hose-thread-adapter>: HTTP status code is not handled or not allowed 2026-01-25 02:40:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tube-urine-centrifuge-12ml-with-flared-top-polystyrene-graduated-1500-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:41:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-ft-aluminum-park-bench-without-backrest-black>: HTTP status code is not handled or not allowed 2026-01-25 02:41:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35a24hx12-a-plate-38-pitch-12-bore-24-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:41:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/armor-poly-vci-bags-4-mil-55-x-45-x-68-blue-50-roll>: HTTP status code is not handled or not allowed 2026-01-25 02:41:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/extra_heavy_boltless_shelves landed on page that is not a product page. 2026-01-25 02:41:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-bags-12-x-15-2-mil-white-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:41:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-anti-static-air-bubble-rolls-12-x-750-x-3-16-4-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:41:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/packard-c-frame-nutone-replacement-motor-120-volts-3000-rpm-3>: HTTP status code is not handled or not allowed 2026-01-25 02:41:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foam-mattress-5-inch-thick-full-size-fits-13-series-full-size-cribs>: HTTP status code is not handled or not allowed 2026-01-25 02:41:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/made-in-usa-1-1-white-red-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:41:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 54067 pages (at 97 pages/min), scraped 27115 items (at 43 items/min) 2026-01-25 02:41:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60bs25hx112-34-pitch-1-12-finished-bore-25-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:41:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/palletized-shipment-3-10-fluorescent-orange-black>: HTTP status code is not handled or not allowed 2026-01-25 02:41:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/memory-saver-kit>: HTTP status code is not handled or not allowed 2026-01-25 02:41:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-twist-ties-5-x-5-32-black-2000-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:41:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-tec-surge-suppressor>: HTTP status code is not handled or not allowed 2026-01-25 02:41:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microtube-05ml-polypropylene-no-cap-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:41:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mar-2-white-black>: HTTP status code is not handled or not allowed 2026-01-25 02:41:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/skid-of-3-5-fluorescent-green-black>: HTTP status code is not handled or not allowed 2026-01-25 02:41:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jul-2-3-bright-yellow-black>: HTTP status code is not handled or not allowed 2026-01-25 02:41:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-40btl45-taper-bushed-12-pitch-45-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:41:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/facial-recognition-time-clock-system>: HTTP status code is not handled or not allowed 2026-01-25 02:41:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-replacement-active-carbon-filter-293052-4-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:41:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-tier-6-door-premium-steel-locker-12-w-x-18-d-x-36-h-black-assembled>: HTTP status code is not handled or not allowed 2026-01-25 02:41:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/packing-list-enclosed-3-4-fluorescent-red-black>: HTTP status code is not handled or not allowed 2026-01-25 02:41:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hazard-class-3-flammable-liquid-4-4-white-red>: HTTP status code is not handled or not allowed 2026-01-25 02:41:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/to-go-rack-chrome-18w-x-72l-x-34h-3-wire-shelves-dividers-ledges>: HTTP status code is not handled or not allowed 2026-01-25 02:41:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/curbside-truck-chrome-24w-x-54l-x-69h-4-wire-shelves-polyurethane-2-rigid-casters>: HTTP status code is not handled or not allowed 2026-01-25 02:41:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-card-carrier-8-x-10-x-2>: HTTP status code is not handled or not allowed 2026-01-25 02:41:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/curbside-truck-chrome-24w-x-30l-x-69h-4-wire-shelves-polyurethane-2-rigid-casters>: HTTP status code is not handled or not allowed 2026-01-25 02:41:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-day-shurrelease-blue-painters-tape-multi-surface-blue-24mm-x-55m>: HTTP status code is not handled or not allowed 2026-01-25 02:41:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-12bs19hx35-metric-34-pitch-35mm-finished-bore-19-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:41:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/curbside-cart-silver-ep-18w-x-36l-x-40h-3-wire-shelves-polyurethane-2-rigid-casters>: HTTP status code is not handled or not allowed 2026-01-25 02:41:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-1-mr-clip-green-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-25 02:41:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/optional-50-l-3-8-diameter-hose>: HTTP status code is not handled or not allowed 2026-01-25 02:41:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bubble-lined-poly-mailers-000-4-x-8-white-500-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:41:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/superscrape-impressions-custom-logo-mat-4-x-6>: HTTP status code is not handled or not allowed 2026-01-25 02:41:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/use-no-hooks-3-4-white-red-black>: HTTP status code is not handled or not allowed 2026-01-25 02:41:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-bulk-storage-locker-single-tier-starter-36x36x90>: HTTP status code is not handled or not allowed 2026-01-25 02:41:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-20bs14hx40-metric-1-14-pitch-40mm-finished-bore-14-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:41:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3rd-quarter-2-dia-fluorescent-green-black>: HTTP status code is not handled or not allowed 2026-01-25 02:41:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-slotted-pallet-rack-starter-no-deck-120-inches-w-x-42-inches-d-x-96-inches-h>: HTTP status code is not handled or not allowed 2026-01-25 02:41:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-slotted-pallet-rack-starter-no-deck-96-inches-w-x-36-inches-d-x-120-inches-h>: HTTP status code is not handled or not allowed 2026-01-25 02:41:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/light-blue-discs-1-dia>: HTTP status code is not handled or not allowed 2026-01-25 02:41:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-hand-crank-mechanical-scissor-lift-table-1000-lb-capacity-39-x-29-platform>: HTTP status code is not handled or not allowed 2026-01-25 02:41:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mantis-2x2-white-220-volt-euro-plug>: HTTP status code is not handled or not allowed 2026-01-25 02:41:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-inch-slantwall-exhaust-fan-belt-drive-1-hp-21100-cfm-115-230v>: HTTP status code is not handled or not allowed 2026-01-25 02:41:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/doubled-sided-floor-stand-tip-out-bins-48-h>: HTTP status code is not handled or not allowed 2026-01-25 02:41:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1671-20-20-1-4-id-1-4-mnpt-polyurethane-rekoil-hose>: HTTP status code is not handled or not allowed 2026-01-25 02:41:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-drawer-mobile-cabinet-48-wx24-dx37-1-2-h-avalanche-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:41:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-16bs18hx25-metric-1-pitch-25mm-finished-bore-18-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:41:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/do-not-inventory-date-3-5-fluorescent-red-black>: HTTP status code is not handled or not allowed 2026-01-25 02:41:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/683-6-three-in-one-brass-3-8-fnpt-hose-manifold>: HTTP status code is not handled or not allowed 2026-01-25 02:41:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pw-csi-tweezer>: HTTP status code is not handled or not allowed 2026-01-25 02:41:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/683-4-three-in-one-brass-1-4-fnpt-hose-manifold>: HTTP status code is not handled or not allowed 2026-01-25 02:42:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80bs35x138-1-pitch-1-38-finished-bore-35-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:42:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/apr-2-3-fluorescent-orange-black>: HTTP status code is not handled or not allowed 2026-01-25 02:42:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-drawer-double-mobile-cabinet-60-wx27-dx37-1-2-h-black-1>: HTTP status code is not handled or not allowed 2026-01-25 02:42:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/melitta-coffee-filters-4-100-count-3-pack-gr>: HTTP status code is not handled or not allowed 2026-01-25 02:42:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nonni-s-biscotti-dark-chocolate-almond-25-count-gr>: HTTP status code is not handled or not allowed 2026-01-25 02:42:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/group-30-lithium-ion-powersports-battery-rechargeable-700a-12-8v>: HTTP status code is not handled or not allowed 2026-01-25 02:42:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 54168 pages (at 101 pages/min), scraped 27166 items (at 51 items/min) 2026-01-25 02:42:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-wds12s2acr-0-5-hp-3-phase-1755-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:42:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40-metal-frame-creeper-with-adjustable-headrest-kti-74961>: HTTP status code is not handled or not allowed 2026-01-25 02:42:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-6001-1-4-hp-1-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:42:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-drawer-mobile-multi-drawer-cabinet-48-wx27-dx37-1-2-h-red-1>: HTTP status code is not handled or not allowed 2026-01-25 02:42:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crystal-light-drink-mix-pitcher-packs-lemonade-16-count-gr>: HTTP status code is not handled or not allowed 2026-01-25 02:42:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-uj10c1k21m-10-hp-1-phase-3500-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:42:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ductless-mini-split-air-conditioner-with-heat-pump-12000-btu-20-seer-115v-white>: HTTP status code is not handled or not allowed 2026-01-25 02:42:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sunkist-wrapped-fruit-gems-soft-candy-2-lb-gr>: HTTP status code is not handled or not allowed 2026-01-25 02:42:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stock-picking-truck-18-48-63-back-to-back-30-cells-13-w>: HTTP status code is not handled or not allowed 2026-01-25 02:42:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-step-16-wx53-d-stainless-steel-rolling-safety-ladder-serrated-grating>: HTTP status code is not handled or not allowed 2026-01-25 02:42:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-chrome-linen-cart-with-solid-bottom-shelf-60-l-x-24-w-x-78-h>: HTTP status code is not handled or not allowed 2026-01-25 02:42:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sensible-portions-garden-veggie-straws-variety-pack-1-oz-30-count-gr>: HTTP status code is not handled or not allowed 2026-01-25 02:42:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-1769-1-hp-1-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:42:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=us-motor-wds12s2acr-0-5-hp-3-phase-1755-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:42:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-6215-1-hp-1-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:42:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-1875-1-3-hp-1-phase-825-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:42:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-t34cm2j-3-4-hp-1-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:42:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lateral-44-2-drawer-fire-and-water-resistant-file-cabinet-putty-lat2w44p>: HTTP status code is not handled or not allowed 2026-01-25 02:42:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-t12c2j4-1-2-hp-1-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:42:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keebler-original-club-crackers-snack-stacks-50-oz-gr>: HTTP status code is not handled or not allowed 2026-01-25 02:42:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-step-16-w-stainless-steel-fold-store-ladder-serrated-grating>: HTTP status code is not handled or not allowed 2026-01-25 02:42:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-step-16-w-stainless-steel-fold-store-ladder-perforated-tread>: HTTP status code is not handled or not allowed 2026-01-25 02:42:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cushion-max-anti-fatigue-mat-48-x-72-black>: HTTP status code is not handled or not allowed 2026-01-25 02:42:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hog-heaven-anti-fatigue-mat-7-8-thick-24-w-black-from-3-up-to-60>: HTTP status code is not handled or not allowed 2026-01-25 02:42:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/power_tools/impact_wrenches_drivers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:42:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ada-water-filtered-cooler-light-gray-granite-wall-hung-115v-60hz-5-amps-1>: HTTP status code is not handled or not allowed 2026-01-25 02:42:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3m-7000028857-polyester-film-tape-850-red-2x72-19-mil-24-rolls-per-case>: HTTP status code is not handled or not allowed 2026-01-25 02:42:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/linea-54-in-w-x-66-in-h-statuary-bronze-garment-rack-with-2-swivel-hangbars>: HTTP status code is not handled or not allowed 2026-01-25 02:42:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-x-25-gray-recirculating-ducting-for-all-foreman-w-clamp>: HTTP status code is not handled or not allowed 2026-01-25 02:42:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-36-wx24-dx87-h-closed-5-shelf-8-drawer-everest-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:42:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-72-vinyl-shower-curtain-blue-taffeta-embossed-6-gauge-12-per-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:42:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-d32cm2j14-1-1-2-hp-1-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:42:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hgx-led-security-flood-lights-2-round-heads-4100-lumens-white>: HTTP status code is not handled or not allowed 2026-01-25 02:42:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/closed-shelving-drawer-unit-36x18x39-54-drawers-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:42:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/power_tools/impact_wrenches_drivers>: HTTP status code is not handled or not allowed 2026-01-25 02:42:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-u7e3h-7-5-hp-3-phase-1185-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:42:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-u32p3dc-1-5-hp-3-phase-1175-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:42:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-ee446b-3-4-hp-3-phase-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:42:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wwe-khn37-25-1-h-143-5tc-cast-iron-helical-bevel-speed-reducer-143-5tc-input-flange-25-1-foot-mt>: HTTP status code is not handled or not allowed 2026-01-25 02:42:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-dj5c2k18m-5-hp-1-phase-1740-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:42:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-eu03-1-2-hp-1-phase-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:42:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-ready-built-ii-locker-triple-tier-1-wide-12x18x78-parchment>: HTTP status code is not handled or not allowed 2026-01-25 02:42:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-agl10fl1nb-1-hp-1-phase-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:42:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wwe-khn47-5-1-h-145tc-cast-iron-helical-bevel-speed-reducer-145tc-input-flange-5-1-foot-mt>: HTTP status code is not handled or not allowed 2026-01-25 02:42:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/floor-supported-portable-personal-safety-partition-4w-x-6h-clear>: HTTP status code is not handled or not allowed 2026-01-25 02:42:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-d10p1h-10-hp-3-phase-3505-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:42:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-uj5p1gm-5-hp-3-phase-3525-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:42:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vxmpdg-microplate-vortex-mixer-digital-120v>: HTTP status code is not handled or not allowed 2026-01-25 02:42:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vp30n8-vacuum-packer-30cm-sealing-bar>: HTTP status code is not handled or not allowed 2026-01-25 02:42:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/outdoor_tools_power_equipment/string_trimmers_edgers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:42:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-u15e2dc-15-hp-3-phase-1775-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:42:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-rivet-steel-shelving-with-wood-deck-48-w-x-30-d-x-84-h-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:42:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:42:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-inch-heavy-duty-plastic-chain-25-feet-black>: HTTP status code is not handled or not allowed 2026-01-25 02:43:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-galvanized-steel-blower-wheel-3-13-16-diameter-1-4-bore-1>: HTTP status code is not handled or not allowed 2026-01-25 02:43:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-yc1p2b-1-hp-3-phase-1760-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:43:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-un3t2gc-3-hp-3-phase-1770-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:43:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-inch-heavy-duty-plastic-chain-100-feet-yellow>: HTTP status code is not handled or not allowed 2026-01-25 02:43:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-stanchion-2-inch-links-c-hooks-2-inch-pole-14-inch-base-41-inch-height-blue->: HTTP status code is not handled or not allowed 2026-01-25 02:43:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/outdoor_tools_power_equipment/string_trimmers_edgers landed on page that is not a product page. 2026-01-25 02:43:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-bags-3x3-4-mil-1000-ctn>: HTTP status code is not handled or not allowed 2026-01-25 02:43:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rs15i-a145-230-3-60-base-mount-145-psi>: HTTP status code is not handled or not allowed 2026-01-25 02:43:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves>: HTTP status code is not handled or not allowed 2026-01-25 02:43:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 54263 pages (at 95 pages/min), scraped 27202 items (at 36 items/min) 2026-01-25 02:43:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rs15i-a138-tas-230-3-60-base-mount-138-psi>: HTTP status code is not handled or not allowed 2026-01-25 02:43:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-d1p2g-1-hp-3-phase-1760-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:43:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-drilling-and-tapping-screw-assortment-253-piece>: HTTP status code is not handled or not allowed 2026-01-25 02:43:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vented-wall-shelf-14x36>: HTTP status code is not handled or not allowed 2026-01-25 02:43:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-5470-3-4-hp-1-phase-1075-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:43:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camshelving-stationary-starter-5-vented-shelves-24x60x72>: HTTP status code is not handled or not allowed 2026-01-25 02:43:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-4740-1-12-hp-1-phase-850-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:43:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camshelving-add-on-unit-4-vented-shelves-18x48x72>: HTTP status code is not handled or not allowed 2026-01-25 02:43:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camshelving-stationary-starter-4-vented-shelves-24x48x64>: HTTP status code is not handled or not allowed 2026-01-25 02:43:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camshelving-stationary-starter-4-vented-shelves-21x36x64>: HTTP status code is not handled or not allowed 2026-01-25 02:43:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-1699-1-hp-1-phase-1625-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:43:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/252726-lipstick-holder>: HTTP status code is not handled or not allowed 2026-01-25 02:43:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camshelving-stationary-starter-5-vented-shelves-24x36x72>: HTTP status code is not handled or not allowed 2026-01-25 02:43:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-rod-2-3-4-90-degree-bend>: HTTP status code is not handled or not allowed 2026-01-25 02:43:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-rod-4-30-degree-bend-2>: HTTP status code is not handled or not allowed 2026-01-25 02:43:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wwe-kan87-30-1-h-184tc-cast-iron-helical-bevel-speed-reducer-184tc-input-flange-30-1>: HTTP status code is not handled or not allowed 2026-01-25 02:43:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coldfest-food-pan-cover-1-2-size-flat-polycarbonate-clear-nsf>: HTTP status code is not handled or not allowed 2026-01-25 02:43:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grip-ring-stem-so-tread-caster-w-brake-100mm-dia-7-16-x-1-7-16-stem-black>: HTTP status code is not handled or not allowed 2026-01-25 02:43:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/office-partition-panel-with-partial-window-gray-24-1-4-w-x-96-h>: HTTP status code is not handled or not allowed 2026-01-25 02:43:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-bike-wave-green-free-standing-1>: HTTP status code is not handled or not allowed 2026-01-25 02:43:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/see-thru-3-magazine-6-pamphlet-display>: HTTP status code is not handled or not allowed 2026-01-25 02:43:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/versa-food-bars-serving-buffet-cold-food-kentucky-green-60-x-29-low>: HTTP status code is not handled or not allowed 2026-01-25 02:43:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-compartment-steel-literature-organizer-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:43:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-pair-of-replacement-keys-262065ak>: HTTP status code is not handled or not allowed 2026-01-25 02:43:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-camcart-heated-food-pan-carrier-heated-top-door-only-front-loading-110v-black>: HTTP status code is not handled or not allowed 2026-01-25 02:43:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/end-panel-for-flat-top-locker-15-x-60-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:43:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-degree-angle-label-holder-shelf-bins-price-per-pkg-of-24>: HTTP status code is not handled or not allowed 2026-01-25 02:43:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cherry-counter-blue-panel-u-shaped-reception-station-4>: HTTP status code is not handled or not allowed 2026-01-25 02:43:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/end-panel-for-slope-top-locker-15-x-72-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:43:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-locker-single-tier-1-wide-12-x-12-x-60-beige>: HTTP status code is not handled or not allowed 2026-01-25 02:43:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fda-tra-tote-48x42x29-1500-lb-capacity-natural>: HTTP status code is not handled or not allowed 2026-01-25 02:43:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xl-waste-32-gallon-grey>: HTTP status code is not handled or not allowed 2026-01-25 02:43:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camshelving-elements-shelf-plate-kit-18-w-x-72-l-venteded-brushed-graphite>: HTTP status code is not handled or not allowed 2026-01-25 02:43:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/versa-food-bars-work-table-cold-food-kentucky-green-72-x-36-standard>: HTTP status code is not handled or not allowed 2026-01-25 02:43:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-camcart-food-pan-carrier-front-loading-approximately-cap-60-qt-6-casters-green>: HTTP status code is not handled or not allowed 2026-01-25 02:43:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/frameless-stainless-steel-mirror-24-wx36-h>: HTTP status code is not handled or not allowed 2026-01-25 02:43:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/union-tee-composite-12mm-tube-x-10mm-tube-x-12mm-tube>: HTTP status code is not handled or not allowed 2026-01-25 02:43:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/union-tee-composite-10mm-tube-x-8mm-tube-x-10mm-tube>: HTTP status code is not handled or not allowed 2026-01-25 02:43:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pizza-dough-box-white-polycarbonate>: HTTP status code is not handled or not allowed 2026-01-25 02:43:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dunnage-rack-slotted-top-speckled-gray-21x30>: HTTP status code is not handled or not allowed 2026-01-25 02:43:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/45degree-plug-in-elbow-3-8-tube-x-3-8-stud>: HTTP status code is not handled or not allowed 2026-01-25 02:43:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-female-12mm-tube-x-1-2-bspp>: HTTP status code is not handled or not allowed 2026-01-25 02:43:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camshelving-elements-post-kit-for-mobile-unit-21-w-x-78-h-brushed-graphite>: HTTP status code is not handled or not allowed 2026-01-25 02:43:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-workbench-4-backsplash-galv-120-w-x-30-d>: HTTP status code is not handled or not allowed 2026-01-25 02:43:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-workbench-2-3-4-backsplash-galv-60-w-x-30-d>: HTTP status code is not handled or not allowed 2026-01-25 02:43:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/male-adapter-6mm-straight-x-1-8-bspp>: HTTP status code is not handled or not allowed 2026-01-25 02:43:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/union-elbow-metal-release-collet-14mm-tube>: HTTP status code is not handled or not allowed 2026-01-25 02:43:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reducer-8mm-stud-x-6mm-tube>: HTTP status code is not handled or not allowed 2026-01-25 02:43:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inline-flow-control-1-4-male-x-1-4-tube>: HTTP status code is not handled or not allowed 2026-01-25 02:43:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forced-air-circulation-incubator-449-liters-115-volt>: HTTP status code is not handled or not allowed 2026-01-25 02:43:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-convection-incubator-161-liters-230-volt>: HTTP status code is not handled or not allowed 2026-01-25 02:43:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-heavy-duty-electronic-caliper-0-12-inch-0-300mm-range>: HTTP status code is not handled or not allowed 2026-01-25 02:43:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-spray-gun-for-battery-powered-portable-ac-coil-cleaner-604147>: HTTP status code is not handled or not allowed 2026-01-25 02:43:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-ply-looped-end-mop-w-5-red-band-large-red-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-25 02:43:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ecp-paper-crumpler-w-idler-system-36-inch-l>: HTTP status code is not handled or not allowed 2026-01-25 02:43:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ecp-premium-manual-cutter-48-inch-l>: HTTP status code is not handled or not allowed 2026-01-25 02:43:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/birdslide-small-end-caps-gray-pair-of-5>: HTTP status code is not handled or not allowed 2026-01-25 02:43:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-pigeon-trap-40l-x-22w-x-13h>: HTTP status code is not handled or not allowed 2026-01-25 02:43:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bird-shock-flex-track-lead-out-wire-50l-roll-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:43:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bird-flite-x-wide-bird-spike-50l>: HTTP status code is not handled or not allowed 2026-01-25 02:43:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/synthetic-winch-line-w-integrated-shackle-716-dia-x-175l>: HTTP status code is not handled or not allowed 2026-01-25 02:43:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/synthetic-winch-line-w-integrated-shackle-716-dia-x-25l>: HTTP status code is not handled or not allowed 2026-01-25 02:43:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbw-vinyl-bumper-truck-2-rigid-2-swivel-casters-18-bushel-capacity-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:44:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbw-poly-bumper-truck-20-bushel-capacity-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:44:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aquasense-single-hole-gooseneck-sensor-faucet-with-0-5-gpm-aerator-chrome>: HTTP status code is not handled or not allowed 2026-01-25 02:44:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pinehurst-chaise-lounge-whitewash>: HTTP status code is not handled or not allowed 2026-01-25 02:44:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_chair_table_sets/patio_bar_dining_sets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:44:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cape-coral-gold-lines-slab-door-wood-32w-x-80h-white>: HTTP status code is not handled or not allowed 2026-01-25 02:44:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/key-largo-silver-lines-slab-door-wood-30w-x-80h-ice-maple>: HTTP status code is not handled or not allowed 2026-01-25 02:44:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kissimmee-glazed-light-slab-door-wood--glass-24w-x-80h-white>: HTTP status code is not handled or not allowed 2026-01-25 02:44:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arcadia-glazed-light-slab-door-wood--glass-28w-x-80h-white>: HTTP status code is not handled or not allowed 2026-01-25 02:44:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/largo-silver-lines-slab-door-wood-28w-x-80h-ice-maple>: HTTP status code is not handled or not allowed 2026-01-25 02:44:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/key-largo-silver-lines-slab-door-wood-28w-x-80h-whiskey-oak>: HTTP status code is not handled or not allowed 2026-01-25 02:44:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/key-largo-night-lines-slab-door-wood-28w-x-80h-white>: HTTP status code is not handled or not allowed 2026-01-25 02:44:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:44:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/european-lockset-w-keyed-mechanism-key-key-chrome>: HTTP status code is not handled or not allowed 2026-01-25 02:44:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 54360 pages (at 97 pages/min), scraped 27225 items (at 23 items/min) 2026-01-25 02:44:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gainesville-night-lines-slab-door-wood-30w-x-80h-ice-maple>: HTTP status code is not handled or not allowed 2026-01-25 02:44:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flush-pull-handle-nickel>: HTTP status code is not handled or not allowed 2026-01-25 02:44:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/l-type-casing-extra-deep-white-enamel>: HTTP status code is not handled or not allowed 2026-01-25 02:44:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-shower-commode-chair-slideout-footrest-435-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 02:44:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hog-heavenanti-fatigue-mat-58-thick-10l-x-3w-black>: HTTP status code is not handled or not allowed 2026-01-25 02:44:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_chair_table_sets/patio_bar_dining_sets>: HTTP status code is not handled or not allowed 2026-01-25 02:44:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cushion-max-anti-fatigue-mat-58-thick-20l-x-2w-charcoal>: HTTP status code is not handled or not allowed 2026-01-25 02:44:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hog-heavenanti-fatigue-mat-58-thick-60l-x-5w-black>: HTTP status code is not handled or not allowed 2026-01-25 02:44:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:44:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hog-heavenanti-fatigue-mat-78-thick-5l-x-2w-black-striped-border>: HTTP status code is not handled or not allowed 2026-01-25 02:44:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hog-heavenanti-fatigue-mat-78-thick-30l-x-3w-black>: HTTP status code is not handled or not allowed 2026-01-25 02:44:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hog-heavenanti-fatigue-mat-78-thick-15l-x-2w-black-striped-border>: HTTP status code is not handled or not allowed 2026-01-25 02:44:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polywoven-dunnage-air-bags-4-ply-102l-x-46-12w-pack-of-180>: HTTP status code is not handled or not allowed 2026-01-25 02:44:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors>: HTTP status code is not handled or not allowed 2026-01-25 02:44:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bison-polywoven-dunnage-air-bags-2-ply-72l-x-36w-pack-of-875>: HTTP status code is not handled or not allowed 2026-01-25 02:44:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polywoven-dunnage-air-bags-10-ply-108l-x-46-12w-pack-of-120>: HTTP status code is not handled or not allowed 2026-01-25 02:44:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polywoven-dunnage-air-bags-8-ply-96l-x-36w-pack-of-165>: HTTP status code is not handled or not allowed 2026-01-25 02:44:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polywoven-dunnage-air-bags-8-ply-48l-x-46-12w-pack-of-270>: HTTP status code is not handled or not allowed 2026-01-25 02:44:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chillshield-mens-insulated-jacket-2xl-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:44:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavyweight-plastic-fork-white-pack-of-1000>: HTTP status code is not handled or not allowed 2026-01-25 02:44:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:44:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deerfield-printed-paper-plates-9-dia-pack-of-1000>: HTTP status code is not handled or not allowed 2026-01-25 02:44:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coffee-filters-w-flat-bottom-12-cups-pack-of-3000>: HTTP status code is not handled or not allowed 2026-01-25 02:44:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-cold-drink-cups-5-oz-polypropylene-translucent-pack-of-2500>: HTTP status code is not handled or not allowed 2026-01-25 02:44:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portion-cups-2-oz-clear-pack-of-2500>: HTTP status code is not handled or not allowed 2026-01-25 02:44:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paper-lids-for-bowls-kraft-3-5-8-dia-pack-of-500>: HTTP status code is not handled or not allowed 2026-01-25 02:44:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/foodservice_disposables/disposable_cups already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:44:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-cup-carrier-tray-pack-of-300>: HTTP status code is not handled or not allowed 2026-01-25 02:44:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/non-lifting-synthetic-shackle-5-16-dia-x-5-l-4-3-8-ton-wll>: HTTP status code is not handled or not allowed 2026-01-25 02:44:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/medical_devices_instruments/testing_kits_supplies already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:44:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rear-handle-circular-saw-bare-tool-only-7-14-dia-24v>: HTTP status code is not handled or not allowed 2026-01-25 02:44:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-tool-combo-kit-w-hammer-drill-turbo-mode-quick-eject-impact-driver-stacked-lithium>: HTTP status code is not handled or not allowed 2026-01-25 02:44:34 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/doors_windows/doors landed on page that is not a product page. 2026-01-25 02:44:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eyewash-station-wall-mount>: HTTP status code is not handled or not allowed 2026-01-25 02:44:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/colormaxx-paint-primer-12-oz-gloss-leather-brown-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 02:44:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/epo-1-electric-pizza-oven-with-bell-timer>: HTTP status code is not handled or not allowed 2026-01-25 02:44:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iscope-trinocular-multihead-microscope-w-2-extra-binocular-heads-plan-pli-410s40s100x>: HTTP status code is not handled or not allowed 2026-01-25 02:44:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premier-chromatography-refrigerator-49-cuft-capacity-glass-door>: HTTP status code is not handled or not allowed 2026-01-25 02:44:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/foodservice_disposables/disposable_cups>: HTTP status code is not handled or not allowed 2026-01-25 02:44:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-compartment-drawer-organizer-23w-x-16-516d-x-3-916h-black>: HTTP status code is not handled or not allowed 2026-01-25 02:44:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ductless-mini-split-air-conditioner-w-heat-pump-wifi-enabled-12000-btu-1-ton-12ft-lineset>: HTTP status code is not handled or not allowed 2026-01-25 02:44:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scotch-weld-low-odor-acrylic-adhesive-duo-pak-400-ml-capacity-tan>: HTTP status code is not handled or not allowed 2026-01-25 02:44:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/medical_devices_instruments/testing_kits_supplies>: HTTP status code is not handled or not allowed 2026-01-25 02:44:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7100148777-scotch-weld-epoxy-adhesive-duo-pak-485-ml-capacity-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:44:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/process_cooling_equipment/chillers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:44:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-cooled-chiller-w-closed-loop-water-application-3-ton-capacity-24352-btus-per-hour>: HTTP status code is not handled or not allowed 2026-01-25 02:44:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arsenal-bucket-truck-tool-board-w-bucket-hooks-kit-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:44:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/accu-therm-reusable-hot-cold-gel-pack-12l-x-10w-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-25 02:44:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manual-hydraulic-patient-lift-6-point-cradle>: HTTP status code is not handled or not allowed 2026-01-25 02:44:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/terry-adult-bib-w-hook-loop-strap-18l-x-30w-white-12-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:44:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-head-pediatric-stethoscopes-22l-tubing-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:44:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/exam_patient_room_supplies/exam_room_consumables/cotton_balls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:44:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/curad-compression-hosiery-w-20-30-mmhg-knee-high-small-size-a-regular-tan>: HTTP status code is not handled or not allowed 2026-01-25 02:44:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lorna-nonperforating-towel-forceps-stainless-steel-3-1516l>: HTTP status code is not handled or not allowed 2026-01-25 02:44:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/process_cooling_equipment/chillers>: HTTP status code is not handled or not allowed 2026-01-25 02:44:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/non-sterile-procto-swab-w-rayon-tip-16l-100-per-box5-boxes-per-case>: HTTP status code is not handled or not allowed 2026-01-25 02:44:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/duocare-oral-care-self-care-caddy-12-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:44:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/curad-sterile-alcohol-prep-pads-medium-2-ply-100-per-box30-boxes-per-case>: HTTP status code is not handled or not allowed 2026-01-25 02:44:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-main-brush-for-wide-area-carpet-vacuum-641830>: HTTP status code is not handled or not allowed 2026-01-25 02:44:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oem-washergasket-set-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 02:44:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/studio-dumbbell-rack-27w-x-31d-x-50h-black>: HTTP status code is not handled or not allowed 2026-01-25 02:44:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/covered-tube-medium-resistance-48l-red>: HTTP status code is not handled or not allowed 2026-01-25 02:44:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kettlebell-prime-100-lb>: HTTP status code is not handled or not allowed 2026-01-25 02:44:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/business-furniture-l-desk-w-3-drawers-cabinet-59-7-16w-x-71-5-16d-x-29-15-16h-storm-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:44:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/exam_patient_room_supplies/exam_room_consumables/cotton_balls>: HTTP status code is not handled or not allowed 2026-01-25 02:45:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/business-furniture-coventry-chaise-lounge-w-arms-41-1-2w-x-62-3-16d-x-35-3-4h-beige>: HTTP status code is not handled or not allowed 2026-01-25 02:45:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/business-furniture-coventry-chaise-lounge-w-arms-41-1-2w-x-62-3-16d-x-35-3-4h-cream>: HTTP status code is not handled or not allowed 2026-01-25 02:45:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/business-furniture-hybrid-bookcase-cabinet-w-hutch-36w-x-15-1-2d-x-71h-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:45:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/business-furniture-hybird-low-storage-cabinet-w-doors-shelves-60w-x-16d-x-21h-storm-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:45:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/western-global-251-gallon-transcube-gasoline-transportable-fuel-storage-tank-emergency-vent-kit>: HTTP status code is not handled or not allowed 2026-01-25 02:45:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/western-global-552-gallon-transcube-diesel-transportable-fuel-storage-tank-with-fuel-gauge>: HTTP status code is not handled or not allowed 2026-01-25 02:45:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-tier-3-door-digital-wood-locker-36-w-x-15-d-x-72-h-cherry-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 02:45:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-tier-3-door-wood-locker-12-w-x-15-d-x-72-h-gray-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 02:45:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/organizer-cabinet-w-fixed-shelves-assembled-48w-x-24d-x-78h-tropic-sand>: HTTP status code is not handled or not allowed 2026-01-25 02:45:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/organizer-mobile-cabinet-assembled-48w-x-24d-x-78h-white>: HTTP status code is not handled or not allowed 2026-01-25 02:45:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cabinet-w-fixed-shelves-assembled-48w-x-24d-x-72h-navy-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:45:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cabinet-w-fixed-shelves-assembled-48w-x-24d-x-72h-white>: HTTP status code is not handled or not allowed 2026-01-25 02:45:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bookcase-assembled-36w-x-18d-x-33h-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:45:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 54453 pages (at 93 pages/min), scraped 27246 items (at 21 items/min) 2026-01-25 02:45:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quartz-enhanced-visibility-sweatshirt-polyester-class-3-black-5xl>: HTTP status code is not handled or not allowed 2026-01-25 02:45:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-diamond-quilted-jacket-w-removable-hood-blue-2xl>: HTTP status code is not handled or not allowed 2026-01-25 02:45:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-premium-heavy-duty-vest-w-multi-pockets-4xl5xl-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:45:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-hype-lite-heavy-duty-vest-lgxl-purple>: HTTP status code is not handled or not allowed 2026-01-25 02:45:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-diamond-quilted-jacket-w-removable-hood-black-large>: HTTP status code is not handled or not allowed 2026-01-25 02:45:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-tone-short-sleeve-polo-shirt-polyester-black-small>: HTTP status code is not handled or not allowed 2026-01-25 02:45:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quartz-duck-sherpa-lined-heavy-weight-jacket-brown-5xl>: HTTP status code is not handled or not allowed 2026-01-25 02:45:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onyx-standard-safety-vest-w-black-contrasting-trim-class-2-2xl-lime>: HTTP status code is not handled or not allowed 2026-01-25 02:45:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fr-waterproof-flame-resistant-winter-quilted-jacket-class-3-green-3xl>: HTTP status code is not handled or not allowed 2026-01-25 02:45:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fr-waterproof-flame-resistant-winter-quilted-jacket-class-3-green-small>: HTTP status code is not handled or not allowed 2026-01-25 02:45:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/advantage-lift-lifting-magnet-w-locking-on--off-handle-1000-lb-flat-capacity>: HTTP status code is not handled or not allowed 2026-01-25 02:45:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/friedrich--inverter-smart-window-air-conditioner-e-star-1180-w-115v-12000-btu>: HTTP status code is not handled or not allowed 2026-01-25 02:45:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lineset-for-diy-24k--36k-indoor-16l>: HTTP status code is not handled or not allowed 2026-01-25 02:45:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forkliftable-truck-w-1-deck-3600-lb-cap-53-12l-x-24w-x-58-12h-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:45:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-bar--pipe-truck-3600-lb-capacity-48l-x-30w-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:45:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forkliftable-truck-w-2-shelves-3600-lb-cap-53-12l-x-30w-x-58-12h-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:45:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-shelf-truck-2-handle-3600-lb-capacity-59l-x-24w-x-34h-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:45:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chill-its-6615-high-performance-terry-cloth-bandana-polyester-tie-closure-black>: HTTP status code is not handled or not allowed 2026-01-25 02:45:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8384-hi-vis-quilted-parka-winter-jacket-5xl-lime>: HTTP status code is not handled or not allowed 2026-01-25 02:45:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8935-hi-vis-ranger-sun-hat-2xl3xl-lime>: HTTP status code is not handled or not allowed 2026-01-25 02:45:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8915bk-hi-vis-rain-pants-w-black-bottom-class-e-polyester-4xl-lime>: HTTP status code is not handled or not allowed 2026-01-25 02:45:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8373-hi-vis-hooded-sweatshirt-black-bottom-class-3-4xl-lime>: HTTP status code is not handled or not allowed 2026-01-25 02:45:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-435hl-comfort-hinge-gel-knee-pads-w-hook-loop-straps-black-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 02:45:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-860-heavy-lifting-utility-gloves-small-black>: HTTP status code is not handled or not allowed 2026-01-25 02:45:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/squids-3772-radio-holster-trap-large-black>: HTTP status code is not handled or not allowed 2026-01-25 02:45:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8373-hi-vis-hooded-sweatshirt-black-bottom-class-3-xl-lime>: HTTP status code is not handled or not allowed 2026-01-25 02:45:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-7072-cut-resistant-gloves-nitrile-coated-ansi-a4-2xl-green-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 02:45:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-7000-nitrile-coated-gloves-w-microfoam-palm-medium-gray-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 02:45:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8928-hi-vis-insulated-bib-pants-ansi-class-e-300d-polyester-3xl-orange>: HTTP status code is not handled or not allowed 2026-01-25 02:45:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8281bk-hi-vis-performance-long-sleeve-t-shirt-type-r-class-2-4xl-lime>: HTTP status code is not handled or not allowed 2026-01-25 02:45:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-7071-cut-resistant-gloves-polyurethane-coated-ansi-a7-m-gray-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 02:45:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-7024-cut-resistant-gloves-polyurethane-coated-ansi-a2-xl-gray-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 02:45:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chill-its-8935mf-hi-vis-ranger-sun-hat-2xl3xl-lime>: HTTP status code is not handled or not allowed 2026-01-25 02:45:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8379-hi-vis-3-in-1-winter-bomber-jacket-5xl-orange>: HTTP status code is not handled or not allowed 2026-01-25 02:45:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chill-its-6695-sun-protection-arm-sleeves-xl2xl-camo-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 02:45:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-bag-dispenser-10-18w-x-3-1516d-x-7-18h-transparent-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 02:45:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-earloop-face-mask-dispenser-7-1116w-x-4-18d-x-4-38h-transparent>: HTTP status code is not handled or not allowed 2026-01-25 02:45:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-air-impact-wrench-1600-torque-pistol-grip>: HTTP status code is not handled or not allowed 2026-01-25 02:45:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jack-hammer-1-x-4-14-hex-shank-263-stroke-length-25-bore>: HTTP status code is not handled or not allowed 2026-01-25 02:45:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/light-duty-spring-balancer-44-lb-capacity-52-lift-black>: HTTP status code is not handled or not allowed 2026-01-25 02:45:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chipping-hammer-58-hex-shank-2-stroke-2200-bpm>: HTTP status code is not handled or not allowed 2026-01-25 02:45:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20v-mid-torque-38-cordless-impact-wrench-kit-550-ft-lbs-nut-busting-torque-1-battery-and-charger>: HTTP status code is not handled or not allowed 2026-01-25 02:45:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-person-digital-locker-72in-w-x-18in-d-x-78in-h-tan-assembled>: HTTP status code is not handled or not allowed 2026-01-25 02:45:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-tier-1-door-digital-locker-12in-w-x-15in-d-x-66in-h-gray-assembled>: HTTP status code is not handled or not allowed 2026-01-25 02:45:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5000-utility-series-kit-w-4-ft-barrel-150-cfm>: HTTP status code is not handled or not allowed 2026-01-25 02:45:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/big-tall-heavy-duty-side-chair-fabric-black>: HTTP status code is not handled or not allowed 2026-01-25 02:45:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-kit-for-fs-458f>: HTTP status code is not handled or not allowed 2026-01-25 02:45:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/timer-for-kf-series-hand-sealers-w-cutter>: HTTP status code is not handled or not allowed 2026-01-25 02:45:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-kit-for-frm-1120c>: HTTP status code is not handled or not allowed 2026-01-25 02:45:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-convex-mirror-w-stainless-steel-backer-plastic-outdoor-30-dia>: HTTP status code is not handled or not allowed 2026-01-25 02:45:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roundtangular-convex-mirror-glass-outdoor-15l-x-24w>: HTTP status code is not handled or not allowed 2026-01-25 02:45:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-hot-cold-supply-hose-for-sensor-bathroom-sink-faucet-732416>: HTTP status code is not handled or not allowed 2026-01-25 02:45:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sds-plus-rebar-cutter-12l-x-58-dia>: HTTP status code is not handled or not allowed 2026-01-25 02:45:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corn-led-corn-bulb-27w-4100-lumens-medium-screw-base-5000k-super-white>: HTTP status code is not handled or not allowed 2026-01-25 02:45:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tough-coat-adv-spray-paint-w-rust-barrier-tech-20-oz-cap-gloss-dark-machinery-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:45:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-degree-cone-bur-double-cut-2l-x-316-shank-dia>: HTTP status code is not handled or not allowed 2026-01-25 02:45:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/radius-tree-shaped-bur-double-cut-7l-x-14-shank-dia>: HTTP status code is not handled or not allowed 2026-01-25 02:45:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-shaped-bur-aluma-cut-3l-x-14-shank-dia>: HTTP status code is not handled or not allowed 2026-01-25 02:45:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/radius-cylindrical-bur-single-cut-2-34l-x-14-shank-dia>: HTTP status code is not handled or not allowed 2026-01-25 02:45:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bur-box-double-cut-sa5-sc3-sc5-sd5-sf3-sf5-sg3-sl4-set-of-8>: HTTP status code is not handled or not allowed 2026-01-25 02:45:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bur-box-aluma-cut-sb5-sc3-sc5-sd5-sf3-sf5-sg3-sl4-set-of-8>: HTTP status code is not handled or not allowed 2026-01-25 02:45:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t8-led-tube-light-bulb-fa8-base-43w-5500-lumens-6500k-daylight>: HTTP status code is not handled or not allowed 2026-01-25 02:45:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-round-mini-panel-light-fixture-18w-1100-lumens-120v-9-size-white>: HTTP status code is not handled or not allowed 2026-01-25 02:45:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/incu-shaker-10lr-w-non-slip-rubber-mat-230v-5060-hz-30-300-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:45:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dx-analytical-balance-w-internal-calibration-120g-capacity-00001g-readability>: HTTP status code is not handled or not allowed 2026-01-25 02:45:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stir-bar-1-x-516>: HTTP status code is not handled or not allowed 2026-01-25 02:45:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stir-bar-1-x-516-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 02:45:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nextpette-carousel-stand-w-6-pipettes>: HTTP status code is not handled or not allowed 2026-01-25 02:45:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gmrs-frs-two-way-radio-50-channels-46255-4677125-mhz-blacksilver-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-25 02:45:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compact-balance-1200g-capacity-01g-readability-115v>: HTTP status code is not handled or not allowed 2026-01-25 02:46:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/block-for-multitherm-touch-4x50ml-tube-capacity>: HTTP status code is not handled or not allowed 2026-01-25 02:46:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f1-grade-calibration-weight-200g-capacity>: HTTP status code is not handled or not allowed 2026-01-25 02:46:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roto-mini-plus-rotator-w-tube-holders-230v-5060-hz-5-70-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:46:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precision-balance-320g-capacity-0001g-readability-115v>: HTTP status code is not handled or not allowed 2026-01-25 02:46:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/analytical-balance-210g-capacity-00001g-readability-115v>: HTTP status code is not handled or not allowed 2026-01-25 02:46:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-king-sk2000-pallet-load-stop-beam-3-or-4-setback-w-same-clip-108w-yellow>: HTTP status code is not handled or not allowed 2026-01-25 02:46:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-water-filter-3600-gallon-capacity-pack-of-two>: HTTP status code is not handled or not allowed 2026-01-25 02:46:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-heavy-duty-cabinet-w-4-shelves-96-red-bins-14-ga-36w-x-24d-x-76h>: HTTP status code is not handled or not allowed 2026-01-25 02:46:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bur-wallet-pouch-wo-logo-aluma-cut-powerz-sa-1-sc-3-sf-3-sc-5-sf-5>: HTTP status code is not handled or not allowed 2026-01-25 02:46:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bur-wallet-pouch-wo-logo-double-cut-powerz-sc-5-sc-3-sf-5-sf-3-set-of-4>: HTTP status code is not handled or not allowed 2026-01-25 02:46:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbi-sala-nano-lok-personal-self-retracting-lifeline-w-rebar-hook-overhead-anchor-point-72l>: HTTP status code is not handled or not allowed 2026-01-25 02:46:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ball-end-mill-2-flute-3l-x-12-dia-1-cut-length>: HTTP status code is not handled or not allowed 2026-01-25 02:46:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ball-end-mill-power-a-2-flute-3l-x-12-dia-1-cut-length>: HTTP status code is not handled or not allowed 2026-01-25 02:46:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cloth-belt-film-lok-single-flex-464w-600-grit-120l-x-9w-pack-of-200>: HTTP status code is not handled or not allowed 2026-01-25 02:46:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-starter-36l-x-24w-x-60h-antique-white>: HTTP status code is not handled or not allowed 2026-01-25 02:46:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 54553 pages (at 100 pages/min), scraped 27261 items (at 15 items/min) 2026-01-25 02:46:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-add-on-48l-x-36w-x-54h-black>: HTTP status code is not handled or not allowed 2026-01-25 02:46:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-add-on-48l-x-40w-x-84h-antique-white>: HTTP status code is not handled or not allowed 2026-01-25 02:46:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/end-unit-w-t-joiner-support-clip-48l-x-16w-x-96h-antique-white>: HTTP status code is not handled or not allowed 2026-01-25 02:46:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/end-unit-w-t-joiner-support-clip-36l-x-12w-x-60h-antique-white>: HTTP status code is not handled or not allowed 2026-01-25 02:46:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-air-conditioner-12000-btu-1330w-115v-white>: HTTP status code is not handled or not allowed 2026-01-25 02:46:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v-flex-full-body-safety-harness-quick-connect-buckle-leg-strap-sxl>: HTTP status code is not handled or not allowed 2026-01-25 02:46:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/squids-5542-phone-style-barcode-scanner-holster-w-belt-loop-large-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:46:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reelsafe-spring-retractable-hose-reel-300-psi-38-id-x-70l>: HTTP status code is not handled or not allowed 2026-01-25 02:46:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-flex-ii-vinyl-bulk-roll-smooth-24-x-160-x-30-clear>: HTTP status code is not handled or not allowed 2026-01-25 02:46:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/skullerz-vali-afas-frameless-safety-glasses-indooroutdoor-lens-matte-black-frame>: HTTP status code is not handled or not allowed 2026-01-25 02:46:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/skullerz-odin-afas-safety-glasses-clear-lens-matte-black-frame>: HTTP status code is not handled or not allowed 2026-01-25 02:46:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kwikboost-edgepower-portable-power-station-heavy-use-bundle>: HTTP status code is not handled or not allowed 2026-01-25 02:46:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spout-cap-orange-small-34405>: HTTP status code is not handled or not allowed 2026-01-25 02:46:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobilecare-drawer-divider-for-23w-drawers-23l-x-18w-x-3h-white>: HTTP status code is not handled or not allowed 2026-01-25 02:46:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-probe-covers-for-535-vital-signs-monitor-2000-case>: HTTP status code is not handled or not allowed 2026-01-25 02:46:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h1-2-bar-steel-ladder-rack-system-for-gmc-savana-1996-on-black>: HTTP status code is not handled or not allowed 2026-01-25 02:46:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/toggle-bolt-anchor-w-swivel-d-ring-11-16-dia-drill-hole-10l>: HTTP status code is not handled or not allowed 2026-01-25 02:46:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80b20hx1mpb-1-pitch-1-plain-bore-20-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:46:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-24bs12hx42-metric-1-12-pitch-42mm-finished-bore-12-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:46:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/groundbreaker-pull-on-boots-composite-toe-size-8m-11-inch-h-brown>: HTTP status code is not handled or not allowed 2026-01-25 02:46:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trushot-2-0-power-cleaner-clean-fresh-scent-10-oz-cartridge-4-pk>: HTTP status code is not handled or not allowed 2026-01-25 02:46:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-safety-crafted-waterproof-western-work-boots-steel-toe-size-10m-10-inch-h-tan>: HTTP status code is not handled or not allowed 2026-01-25 02:46:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trailgrip-work-athletic-hiker-boots-alloy-toe-size-15w-4-inch-h-coyote-black>: HTTP status code is not handled or not allowed 2026-01-25 02:46:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trailgrip-work-athletic-hiker-boots-alloy-toe-size-14w-4-inch-h-coyote-black>: HTTP status code is not handled or not allowed 2026-01-25 02:46:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:46:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-safety-crafted-waterproof-work-boots-steel-toe-size-9m-6-inch-h-dark-brown>: HTTP status code is not handled or not allowed 2026-01-25 02:46:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:46:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-41b20hx58mpb-12-pitch-58-plain-bore-20-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:46:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-safety-crafted-waterproof-logger-work-boots-steel-toe-size-14w-8-inch-h-black>: HTTP status code is not handled or not allowed 2026-01-25 02:46:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/work-premier-casual-chukka-boots-steel-toe-size-10eee-4-inch-h-dark-cognac>: HTTP status code is not handled or not allowed 2026-01-25 02:46:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/outdoor-blank-out-led-direct-view-banking-signs-open-cosed-18-w-x-7-h-x-2-1-2-d-red-green>: HTTP status code is not handled or not allowed 2026-01-25 02:46:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60bs40x11516-34-pitch-1-1516-finished-bore-40-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:46:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steri-drape-abdominal-perineal-drapes-6665-78-inch-x-106-inch-lithotomy-position-10-cs>: HTTP status code is not handled or not allowed 2026-01-25 02:46:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-08a20h-metric-a-plate-12-pitch-10mm-bore-20-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:46:42 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-25 02:46:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80bs30x11516-1-pitch-1-1516-finished-bore-30-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:46:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-25 02:46:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steri-drape-adhesive-towel-drape-9069-36-inch-x-30-inch-50-each-carton-4-carton-case>: HTTP status code is not handled or not allowed 2026-01-25 02:46:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:46:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-16bs24x30-metric-1-pitch-30mm-finished-bore-24-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:46:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-boltless-shelving-36-in-w-x-18-in-d-x-84-in-h-5-shelves-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:46:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60bs43x118-34-pitch-1-18-finished-bore-43-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:46:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-20bs20hx65-metric-1-14-pitch-65mm-finished-bore-20-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:46:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80bs32x134-1-pitch-1-34-finished-bore-32-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:46:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vitals-skate-inspired-work-shoes-composite-toe-size-15m-wintermoss-whitecap-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:46:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-microcentrifuge-tubes-with-cap-sterile-1-5-ml-assorted-500-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:46:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zig-elusion-heritage-work-women-s-low-cut-sneaker-composite-toe-size-11m-black>: HTTP status code is not handled or not allowed 2026-01-25 02:46:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80a09hx34-a-plate-1-pitch-34-bore-9-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:46:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-25 02:46:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-120f32-qd-bushed-1-12-pitch-32-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:46:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-screw-cap-macrotube-sterile-5-ml-opaque-black-500-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:46:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-pipette-tips-for-halo-pipettors-5-ml-100-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:46:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80bs36x178-1-pitch-1-78-finished-bore-36-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:46:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-10a19h-metric-a-plate-58-pitch-16mm-bore-19-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:46:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:46:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zig-elusion-heritage-work-men-s-low-cut-sneaker-composite-toe-size-8-5w-black>: HTTP status code is not handled or not allowed 2026-01-25 02:46:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vitals-skate-inspired-work-shoes-composite-toe-size-9-5m-navy-aged-indigo>: HTTP status code is not handled or not allowed 2026-01-25 02:47:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-fraction-collector-tube-rack-for-1-5-ml-2-ml-tubes-80-place-blue-5-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:47:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/true-skate-inspired-work-shoes-composite-toe-size-8w-black>: HTTP status code is not handled or not allowed 2026-01-25 02:47:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/true-skate-inspired-work-shoes-composite-toe-size-14m-black-gum>: HTTP status code is not handled or not allowed 2026-01-25 02:47:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-24bs18hx60-metric-1-12-pitch-60mm-finished-bore-18-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:47:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-24bs18hx70-metric-1-12-pitch-70mm-finished-bore-18-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:47:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:47:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/true-skate-inspired-work-shoes-composite-toe-size-11-5w-navy>: HTTP status code is not handled or not allowed 2026-01-25 02:47:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/true-skate-inspired-work-shoes-composite-toe-size-14w-black-gum>: HTTP status code is not handled or not allowed 2026-01-25 02:47:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-25 02:47:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-legend-600-597cc-snowmobile-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:47:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 54645 pages (at 92 pages/min), scraped 27290 items (at 29 items/min) 2026-01-25 02:47:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/true-skate-inspired-work-shoes-composite-toe-size-6-5w-black>: HTTP status code is not handled or not allowed 2026-01-25 02:47:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stone-skate-inspired-work-shoes-composite-toe-size-6-5w-blue-navy>: HTTP status code is not handled or not allowed 2026-01-25 02:47:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:47:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-25 02:47:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stone-skate-inspired-work-shoes-composite-toe-size-12w-gray-black>: HTTP status code is not handled or not allowed 2026-01-25 02:47:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galion-s4-6-industrial-replacement-battery-7ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 02:47:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-40a35hx58-a-plate-12-pitch-58-bore-35-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:47:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-crafted-western-work-boots-w-steel-toe-and-internal-met-guard-size-8w-brown>: HTTP status code is not handled or not allowed 2026-01-25 02:47:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-40bs27hx78-12-pitch-78-finished-bore-27-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:47:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/philips-pxl12090-medical-replacement-battery-9ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 02:47:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premier-work-casual-slip-on-sneaker-steel-toe-size-11-5eee-cognac>: HTTP status code is not handled or not allowed 2026-01-25 02:47:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-25 02:47:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anti-static-bubble-bags-18-x-23-1-2-100-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:47:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-10bs22hx19-metric-58-pitch-19mm-finished-bore-22-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:47:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-10a24h-metric-a-plate-58-pitch-15mm-bore-24-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:47:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premier-work-casual-oxford-shoes-steel-toe-14d-black>: HTTP status code is not handled or not allowed 2026-01-25 02:47:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/i-beam-trolley-anchor-for-beams-from-3-10-and-up-to-7-8-thick>: HTTP status code is not handled or not allowed 2026-01-25 02:47:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/severance-hss-6-flute-chatterless-countersink-1-2-diameter-90-degree>: HTTP status code is not handled or not allowed 2026-01-25 02:47:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/severance-chatter-free-stop-countersink-cutter-90-1-2-7-32-pilot-hole>: HTTP status code is not handled or not allowed 2026-01-25 02:47:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portalac-pe4-5-alarm-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:47:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-ingredient-bin-148bin-yw-14-5-8-l-x-9-1-4-w-x-23-1-4-h-yellow>: HTTP status code is not handled or not allowed 2026-01-25 02:47:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4036-conveyor-sheave-36>: HTTP status code is not handled or not allowed 2026-01-25 02:47:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ls50l11b-battery-powered-knockout-punch-driver-tool-kit>: HTTP status code is not handled or not allowed 2026-01-25 02:47:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-shelves-36-w-x-24-d-storage-cabinet-tan>: HTTP status code is not handled or not allowed 2026-01-25 02:47:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-40b13hx12mpb-12-pitch-12-plain-bore-13-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:47:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:47:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80bs23x2716-1-pitch-2-716-finished-bore-23-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:47:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/balder-junior-wheelchair-replacement-battery-55ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:47:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ctm-hs-685-wheelchair-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:47:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-loop-with-acrylic-adhesive-3-4-inch-x-75-feet>: HTTP status code is not handled or not allowed 2026-01-25 02:47:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mil-spec-hook-n-loop-tape-fasteners-coyote-1-inch-x-75-feet>: HTTP status code is not handled or not allowed 2026-01-25 02:47:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/altronix-smp3pmp16cb-alarm-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:47:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-25a19hx14-a-plate-14-pitch-14-bore-19-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:47:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-25b13hx14mpb-14-pitch-14-plain-bore-13-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:47:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evolve-skate-inspired-high-top-work-shoes-composite-toe-size-3w-triple-black>: HTTP status code is not handled or not allowed 2026-01-25 02:47:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:47:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-25 02:47:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/food_storage/food_storage_containers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:47:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yellow-reflective-strap-with-strap-lock-buckle-100l-x-2w-set-of-2>: HTTP status code is not handled or not allowed 2026-01-25 02:47:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/floatride-energy-3-adventure-men-s-athletic-work-shoes-composite-toe-size-14m-black>: HTTP status code is not handled or not allowed 2026-01-25 02:47:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/irontuff-jackoat-black-hivis-lime-50-comfort-rating-5xl-regular>: HTTP status code is not handled or not allowed 2026-01-25 02:47:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elsar-124-emergency-light-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:47:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-large-draft-shield-22l-x-22w-50240>: HTTP status code is not handled or not allowed 2026-01-25 02:47:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hybrid-skate-inspired-work-shoes-composite-toe-size-8m-navy-celestial-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:47:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extreme-softshell-high-bib-black-60f-comfort-rating-s>: HTTP status code is not handled or not allowed 2026-01-25 02:48:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:48:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/can-am-quest-opt-500cc-atv-replacement-battery-12v-i>: HTTP status code is not handled or not allowed 2026-01-25 02:48:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evolve-skate-inspired-work-shoes-composite-toe-size-10w-navy>: HTTP status code is not handled or not allowed 2026-01-25 02:48:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evolve-skate-inspired-work-shoes-composite-toe-size-10-5w-navy>: HTTP status code is not handled or not allowed 2026-01-25 02:48:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers>: HTTP status code is not handled or not allowed 2026-01-25 02:48:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adi-4180-5-emergency-light-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:48:04 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/food_storage/food_storage_containers landed on page that is not a product page. 2026-01-25 02:48:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-stage-12hp-pump-l800003>: HTTP status code is not handled or not allowed 2026-01-25 02:48:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mule-bkm2-emergency-light-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:48:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evolve-skate-inspired-work-shoes-composite-toe-size-7-5w-navy>: HTTP status code is not handled or not allowed 2026-01-25 02:48:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interstate-faytz12s-powersports-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:48:09 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-25 02:48:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 54739 pages (at 94 pages/min), scraped 27333 items (at 43 items/min) 2026-01-25 02:48:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/end-caps-for-2-channel-eg2125-36-cable-guard-orange>: HTTP status code is not handled or not allowed 2026-01-25 02:48:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/46-ada-round-expanded-metal-picnic-table-green>: HTTP status code is not handled or not allowed 2026-01-25 02:48:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-frame-motor-wwht60-18-364t-f2-gp-tefc-rigid-3-ph-f2-364t-705-fla>: HTTP status code is not handled or not allowed 2026-01-25 02:48:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:48:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bb4500-work-high-top-work-sneaker-composite-toe-size-8m-black-holographic-spectrum>: HTTP status code is not handled or not allowed 2026-01-25 02:48:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:48:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ezip-650-electric-scooter-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 02:48:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:48:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usb-2-0-hi-speed-to-ethernet-nic-network-adapter-10-100-mbps>: HTTP status code is not handled or not allowed 2026-01-25 02:48:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/healthflex-privacy-screen-3-panel-vinyl-sandalwood>: HTTP status code is not handled or not allowed 2026-01-25 02:48:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/general_purpose_electric_motors/three_phase_motors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:48:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hcf-pacelite-707-electric-scooter-replacement-battery-7ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 02:48:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/healthflex-privacy-screen-5-panel-vinyl-ice>: HTTP status code is not handled or not allowed 2026-01-25 02:48:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-25 02:48:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-mx-z-x-1200-1170cc-snowmobile-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:48:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/colored-carton-sealing-tape-2-x-55-yds-orange-pack-of-36>: HTTP status code is not handled or not allowed 2026-01-25 02:48:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-25 02:48:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/908x34629-kick-plate-364x34x8-polished-ss>: HTTP status code is not handled or not allowed 2026-01-25 02:48:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-25 02:48:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-mx-z-380-368cc-snowmobile-replacement-battery-2002-2006-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:48:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/general_purpose_electric_motors/three_phase_motors>: HTTP status code is not handled or not allowed 2026-01-25 02:48:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-frame-motor-wwht100-12-444tc-gp-tefc-rigid-c-3-ph-444tc-111-fla>: HTTP status code is not handled or not allowed 2026-01-25 02:48:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-frame-motor-wwht50-12-365tc-gp-tefc-rigid-c-3-ph-365tc-601-fla>: HTTP status code is not handled or not allowed 2026-01-25 02:48:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cc-pump-motor-wwe40-18-324jm-tefc-rigid-c-3-ph-324jm-40-hp-1800-rpm>: HTTP status code is not handled or not allowed 2026-01-25 02:48:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-16-ga-steel-foot-guards-steel-toe-clip-rubber-strap-5-12w-large-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 02:48:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-steel-toe-guards-leather-cuff-quick-fastener-web-strap-4w-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 02:48:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-100-1000-gigabit-multimode-fiber-media-converter-rj45-sc>: HTTP status code is not handled or not allowed 2026-01-25 02:48:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbi-sala-nano-lok-xl-twin-leg-self-retracting-lifeline-w-quick-connect-comfort-grip-hooks-9l>: HTTP status code is not handled or not allowed 2026-01-25 02:48:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dvi-or-dvi-d-to-vga-hd15-cable-adapter-connector-f-m>: HTTP status code is not handled or not allowed 2026-01-25 02:48:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-purpose-metal-access-panel-key-lock-12l-x-12w>: HTTP status code is not handled or not allowed 2026-01-25 02:48:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electronic-time-clock-blue-white-pix-200-030>: HTTP status code is not handled or not allowed 2026-01-25 02:48:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carton-sealing-tape-do-not-break-stretch-wrap-2-x-110-yds-redwhite-pack-of-36>: HTTP status code is not handled or not allowed 2026-01-25 02:48:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carton-sealing-tape-glass--handle-with-care-2-x-55-yds-redwhite-pack-of-36>: HTTP status code is not handled or not allowed 2026-01-25 02:48:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ajc-national-power-gs012p3-6v-5ah-emergency-light-battery>: HTTP status code is not handled or not allowed 2026-01-25 02:48:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exide-ep100-12-sealed-lead-acid-replacement-battery-100ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:48:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/r-d-56391-sealed-lead-acid-replacement-battery-18ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:48:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-yellow-wall-mount-hdmw>: HTTP status code is not handled or not allowed 2026-01-25 02:48:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/optional-wheel-kit-acm-wk>: HTTP status code is not handled or not allowed 2026-01-25 02:48:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/epdm-commercial-grade-60a-rubber-sheet-1-8-thick-x-6-x-6-length-black-5-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:48:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ready-built-ii-3-tier-3-doors-locker-12w-x-18d-x-84h-light-gray-assembled>: HTTP status code is not handled or not allowed 2026-01-25 02:48:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ready-built153-3-tier-9-doors-locker-36w-x-12d-x-78h-light-gray-assembled>: HTTP status code is not handled or not allowed 2026-01-25 02:48:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/products-1000-unassembled-direct-drive-cement-mixer>: HTTP status code is not handled or not allowed 2026-01-25 02:48:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/75-nonslip-coating-durable-antislip-paint-1-gal-orange-75-or-gl>: HTTP status code is not handled or not allowed 2026-01-25 02:48:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-blade-for-pocketboy-130mm-extra-fine-teeth>: HTTP status code is not handled or not allowed 2026-01-25 02:48:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ar8021-air-receivers-surge-tanks-30gal-175psi>: HTTP status code is not handled or not allowed 2026-01-25 02:48:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ready-built-ii153-3-tier-3-doors-locker-12w-x-12d-x-82h-black-assembled>: HTTP status code is not handled or not allowed 2026-01-25 02:48:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bb-hrl9-12-sealed-lead-acid-replacement-battery-9ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:48:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/denios-n-series-9-10w-x-5-10d-x-8-2h-non-combustible-outdoor-storage-building-for-6-drums>: HTTP status code is not handled or not allowed 2026-01-25 02:48:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50-ft-50-amp-generator-cord-with-nema-14-50-male-and-locking-female>: HTTP status code is not handled or not allowed 2026-01-25 02:49:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ready-built-ii-2-tier-2-doors-locker-12w-x-15d-x-83h-light-gray-assembled>: HTTP status code is not handled or not allowed 2026-01-25 02:49:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-waterfall-deck-shelving-5-shelves-48w-x-18d-x-84h-midnight-ebony>: HTTP status code is not handled or not allowed 2026-01-25 02:49:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simplex-20819296-industrial-replacement-battery-1-3ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:49:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ibc-low-level-tank-alarm-polypropylene-with-feed-pipe-and-flow-indicator-3-4-connection-tube>: HTTP status code is not handled or not allowed 2026-01-25 02:49:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-tech-closed-style-metal-shelving-8-shelves-500-lb-cap-36w-x-12d-x-87h-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:49:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vent-cap-male-thread-15-1>: HTTP status code is not handled or not allowed 2026-01-25 02:49:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caterpillar-d300-industrial-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:49:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-eye-level-digital-scale-w-digital-height-rod-automatic-bmi>: HTTP status code is not handled or not allowed 2026-01-25 02:49:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uniform-exchange-9-person-9-doors-locker-24w-x-15d-x-84h-dark-gray-assembled>: HTTP status code is not handled or not allowed 2026-01-25 02:49:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/health-o-meter-professional-portable-height-rod>: HTTP status code is not handled or not allowed 2026-01-25 02:49:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 54830 pages (at 91 pages/min), scraped 27369 items (at 36 items/min) 2026-01-25 02:49:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-end-panel-for-locker-18w-x-15d-x-72h-black>: HTTP status code is not handled or not allowed 2026-01-25 02:49:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/closed-front-base-for-locker-12w-x-18d-x-6h-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 02:49:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/toyo-6fmh18-sealed-lead-acid-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:49:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/calm75-201-145tc-aluminum-worm-gear-reducer-75mm-201-145tc-nema-frame>: HTTP status code is not handled or not allowed 2026-01-25 02:49:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caterpillar-e240-industrial-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:49:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/calm50-151-56c-aluminum-worm-gear-reducer-50mm-151-56c-nema-frame>: HTTP status code is not handled or not allowed 2026-01-25 02:49:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/calm50-1001-56c-aluminum-worm-gear-reducer-50mm-1001-56c-nema-frame>: HTTP status code is not handled or not allowed 2026-01-25 02:49:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-tech-closed-style-metal-shelving-8-shelves-800-lb-cap-36w-x-24d-x-87h-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:49:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-step-heavy-duty-double-entry-work-platform-36w-steps-dep3-3648>: HTTP status code is not handled or not allowed 2026-01-25 02:49:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-tech-closed-style-shelving-6-shelves-starter-unit-36w-x-12d-x-87h-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:49:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sealer-sales-12-meshed-seal-direct-heat-sealer-foot-operated>: HTTP status code is not handled or not allowed 2026-01-25 02:49:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-in-x-12-in-aluminum-door-louver>: HTTP status code is not handled or not allowed 2026-01-25 02:49:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-tech-open-style-metal-shelving-8-shelves-375-lb-cap-48w-x-12d-x-87h-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:49:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mighty-products-mt-st-162000-16-x-20-flatbed-truck-tarp-steel-tarp>: HTTP status code is not handled or not allowed 2026-01-25 02:49:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/28-in-backdraft-damper>: HTTP status code is not handled or not allowed 2026-01-25 02:49:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-in-x-18-in-steel-door-louver>: HTTP status code is not handled or not allowed 2026-01-25 02:49:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-tech-open-stand-alone-shelving-6-shelves-500-lb-cap-36w-x-24d-x-87h-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:49:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-tile-low-profile-circular-design-50cm-slate>: HTTP status code is not handled or not allowed 2026-01-25 02:49:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powertron-ps640-6v-45ah-ups-battery>: HTTP status code is not handled or not allowed 2026-01-25 02:49:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shovel-nose-hand-truck-tf-362-10p-pneumatic-with-foot-kick-dual-handle>: HTTP status code is not handled or not allowed 2026-01-25 02:49:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/29418ex-spherical-roller-thrust-bearing-extra-capacity-steel-cage>: HTTP status code is not handled or not allowed 2026-01-25 02:49:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ld3000-high-speed-electric-2-1-4w-x-3l-label-dispenser>: HTTP status code is not handled or not allowed 2026-01-25 02:49:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-in-2-1-2-in-galvanized-steel-pipe-nipple-150-psi-lead-free>: HTTP status code is not handled or not allowed 2026-01-25 02:49:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-in-4-in-galvanized-steel-pipe-nipple-150-psi-lead-free>: HTTP status code is not handled or not allowed 2026-01-25 02:49:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-manual-chain-hoist-14-ton-capacity-15-lift>: HTTP status code is not handled or not allowed 2026-01-25 02:49:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eveleen-indoor-outdoor-rocking-chair-graywhite>: HTTP status code is not handled or not allowed 2026-01-25 02:49:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/altronix-al802ul2ada-alarm-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 02:49:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-in-4-in-black-steel-pipe-nipple-150-psi-lead-free>: HTTP status code is not handled or not allowed 2026-01-25 02:49:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n187236-5-equipment-stands-5-shelf-shelving-unit-18w-x-72h-x-36l-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 02:49:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/everest---jennings-3p-wheelchair-replacement-battery-55ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:49:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dyno-lever-hoist-34-ton-capacity-20-lift>: HTTP status code is not handled or not allowed 2026-01-25 02:49:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oil-filled-radiator-heater-w-safety-protection-1500w-120v-1-phase>: HTTP status code is not handled or not allowed 2026-01-25 02:49:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/plumbing_fittings/pipe_clamps_fittings/black_malleable_pipe_fittings already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:49:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mounting-kit-w-wall-mount-template-for-obh230-wall>: HTTP status code is not handled or not allowed 2026-01-25 02:49:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-lambswool-monsoon-strip-window-washer-refill-mc18>: HTTP status code is not handled or not allowed 2026-01-25 02:49:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/teftec-alpha-trac-wheelchair-replacement-battery-75ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:49:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/plumbing_pumps/well_water_pumps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:49:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/condensate-removal-pump-vcma20ulst-automatic-115v-80-gph-at-1-20-lift>: HTTP status code is not handled or not allowed 2026-01-25 02:49:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shepard-meyra-967-wheelchair-replacement-battery-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:49:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnetic-drive-pump-2md-115v-510-gph-at-1>: HTTP status code is not handled or not allowed 2026-01-25 02:49:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/condensate-removal-pump-vcl14uls-automatic-115v-200-gph-at-1>: HTTP status code is not handled or not allowed 2026-01-25 02:49:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/type-i-steel-safety-can-with-funnel-2-gallon-7-5l-self-close-lid-red-7120110>: HTTP status code is not handled or not allowed 2026-01-25 02:49:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reagent-bottle-wide-mouth-hdpe-125ml-capacity-clear-pack-of-500>: HTTP status code is not handled or not allowed 2026-01-25 02:49:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reagent-bottle-wide-mouth-hdpe-1000ml-capacity-clear-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-25 02:49:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/plumbing/plumbing_fittings/pipe_clamps_fittings/black_malleable_pipe_fittings>: HTTP status code is not handled or not allowed 2026-01-25 02:49:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reagent-bottle-narrow-mouth-pp-250ml-capacity-clear-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-25 02:49:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reagent-bottle-narrow-mouth-pp-1000ml-capacity-clear-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 02:49:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-bec-scoota-bug-wheelchair-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:49:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/plumbing/plumbing_pumps/well_water_pumps>: HTTP status code is not handled or not allowed 2026-01-25 02:50:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-straight-leg-slotted-shim-5-b-125>: HTTP status code is not handled or not allowed 2026-01-25 02:50:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quantum-rehab-p210-wheelchair-replacement-battery-5ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 02:50:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6320-deep-groove-ball-bearing-open-100mm-bore-215mm-od>: HTTP status code is not handled or not allowed 2026-01-25 02:50:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/column-protector-9-column-opening-lime-w-round-insert-436758>: HTTP status code is not handled or not allowed 2026-01-25 02:50:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/test-tube-rack-for-16-20mm-tubes-36-places-wire-epoxy-coated-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:50:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wash-bottle-unitary-ldpe-125ml-capacity-white-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-25 02:50:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 54928 pages (at 98 pages/min), scraped 27414 items (at 45 items/min) 2026-01-25 02:50:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cyclone-5023-180-grit-brown-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-25 02:50:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/production-booster-463t72gy-72w-x-36h-3-shelves-office-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:50:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60202rs-deep-groove-ball-bearing-double-sealed-100mm-bore-150mm-od>: HTTP status code is not handled or not allowed 2026-01-25 02:50:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-electric-vibrator-mve-11504-t6-1800rpm-3-phase-60hz-575v-4pole>: HTTP status code is not handled or not allowed 2026-01-25 02:50:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exhaust-flange-evaporator-for-global-industrial-1-2-ton-portable-outdoor-ac-604151>: HTTP status code is not handled or not allowed 2026-01-25 02:50:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exhaust-flange-condenser-for-global-industrial-1-2-ton-portable-outdoor-ac-604151>: HTTP status code is not handled or not allowed 2026-01-25 02:50:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/condenser-motor-capacitor-for-global-industrial-1-2-ton-portable-outdoor-ac>: HTTP status code is not handled or not allowed 2026-01-25 02:50:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6216zz-deep-groove-ball-bearing-double-shielded-80mm-bore-140mm-od>: HTTP status code is not handled or not allowed 2026-01-25 02:50:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-tier-12-door-wood-locker-36-w-x-15-d-x-72-h-gray-assembled>: HTTP status code is not handled or not allowed 2026-01-25 02:50:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-tier-6-door-digital-wood-locker-36-w-x-15-d-x-72-h-cherry-assembled>: HTTP status code is not handled or not allowed 2026-01-25 02:50:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-tier-4-door-digital-wood-locker-12-w-x-15-d-x-72-h-cherry-assembled>: HTTP status code is not handled or not allowed 2026-01-25 02:50:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/type-1a-aluminum-step-ladder-4-ft-dxl201004>: HTTP status code is not handled or not allowed 2026-01-25 02:50:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ticonderoga-my-first-triwrite-beginner-hb-2-pencil-with-eraser-36box>: HTTP status code is not handled or not allowed 2026-01-25 02:50:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelf-truck-54l-x-24w-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 02:50:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/32006-tapered-roller-bearing-metric-30mm-bore-55mm-od>: HTTP status code is not handled or not allowed 2026-01-25 02:50:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelf-truck-24l-x-24w-x-60h-polyurethane-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 02:50:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/homelite-ut13122-lawn-and-garden-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:50:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60440258915-cloth-belt-577f-37w-x-75l-60-grit>: HTTP status code is not handled or not allowed 2026-01-25 02:50:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60440196933-cloth-belt-341d-2w-x-60l-p120-grit>: HTTP status code is not handled or not allowed 2026-01-25 02:50:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60440197618-cloth-belt-341d-1-1-2w-x-60l-40-grit>: HTTP status code is not handled or not allowed 2026-01-25 02:50:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:50:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mechanical-pencil-07mm-assorted-barrels-10pack>: HTTP status code is not handled or not allowed 2026-01-25 02:50:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zebra-zgrip-mechanical-pencil-rubber-grip-refillable-07mm-clearblack-dozen>: HTTP status code is not handled or not allowed 2026-01-25 02:50:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelf-truck-42l-x-24w-x-69h-polyurethane-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 02:50:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60440200511-341d-3-4dia-x-1-2w-p320-grit-100-case>: HTTP status code is not handled or not allowed 2026-01-25 02:50:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-cvt3gw5i-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:50:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cylindrical-bearing-double-row-bore-75mm-0010-to-0020-radial-clearance-nn3015m2kc9nap4>: HTTP status code is not handled or not allowed 2026-01-25 02:50:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/professional-5-drawer-service-cart-locking-top-black>: HTTP status code is not handled or not allowed 2026-01-25 02:50:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers>: HTTP status code is not handled or not allowed 2026-01-25 02:50:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/accent-pen-style-liquid-highlighter-chisel-tip-fluorescent-yellow-ink-dozen>: HTTP status code is not handled or not allowed 2026-01-25 02:50:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergi-lite-ilsm36-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:50:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sunworks-construction-paper-12x18-blue-50-sheets>: HTTP status code is not handled or not allowed 2026-01-25 02:50:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergi-lite-lse182-emergency-light-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:50:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8020-25-series-1-1516-x-3-1516-twelve-t-slotted-extrusion-profile-238-316l-stock-bar>: HTTP status code is not handled or not allowed 2026-01-25 02:50:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uni-paint-marker-oilbased-fine-red-ink>: HTTP status code is not handled or not allowed 2026-01-25 02:50:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bisley-womens-short-sleeve-t-shirt-class-2-m-hi-vis-orange>: HTTP status code is not handled or not allowed 2026-01-25 02:50:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/energel-retractable-pen-needle-fine-tip-05mm-black-ink-dozen>: HTTP status code is not handled or not allowed 2026-01-25 02:50:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/energel-retractable-pen-refill-07mm-black-ink>: HTTP status code is not handled or not allowed 2026-01-25 02:50:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:50:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/46-round-in-ground-mount-picnic-table-black>: HTTP status code is not handled or not allowed 2026-01-25 02:50:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/liquid-mixer-0-11-quick-set-cmx1>: HTTP status code is not handled or not allowed 2026-01-25 02:50:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/highlighters_markers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:50:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/skate-wheel-conveyor-flexible-extendable-52-to-207l-x-28w-hdpe>: HTTP status code is not handled or not allowed 2026-01-25 02:50:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dyna-ray-2200-emergency-light-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:50:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/122-in3-hydraulic-hand-pump-p2001-2speed>: HTTP status code is not handled or not allowed 2026-01-25 02:51:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-acting-hollow-hole-hydraulic-cylinder-hc2002t-20-ton-2-stroke>: HTTP status code is not handled or not allowed 2026-01-25 02:51:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-25 02:51:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crossover-ladder-48l-x-28-12w-x-94-716h>: HTTP status code is not handled or not allowed 2026-01-25 02:51:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-ess-i-emergency-light-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:51:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wg4wf-svmc-outdoor-speaker-strobe-white-fire>: HTTP status code is not handled or not allowed 2026-01-25 02:51:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/highlighters_markers landed on page that is not a product page. 2026-01-25 02:51:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/king-kombo-ladder-w-grip-n-go-release-hinge-5-type-iaa-4-step-375-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 02:51:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/johnlite-2920-emergency-light-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:51:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eg1t-genesis-trim-plate-for-two-gang-or-4-square-boxes-white>: HTTP status code is not handled or not allowed 2026-01-25 02:51:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 55022 pages (at 94 pages/min), scraped 27457 items (at 43 items/min) 2026-01-25 02:51:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hyperlite-extension-ladder-fiberglass-24-type-ia-300-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 02:51:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sonnenschein-5-emergency-light-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:51:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/full_height_storage_cabinets/stationary_storage_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:51:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergi-lite-e11-emergency-light-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:51:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-tech-ut1280-12v-9ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-25 02:51:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unikor-vt1207-12v-7ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-25 02:51:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interlake-bulk-storage-rack-galvanized-metal-shelf-12-inch-l-x-42-inch-w>: HTTP status code is not handled or not allowed 2026-01-25 02:51:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/k1800-15-drawer-roller-cabinet-39-38w-x-18d-x-39h-black>: HTTP status code is not handled or not allowed 2026-01-25 02:51:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kung-long-wp46c-6v-45ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-25 02:51:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/cabinets/full_height_storage_cabinets/stationary_storage_cabinets>: HTTP status code is not handled or not allowed 2026-01-25 02:51:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/run-post-10-6h>: HTTP status code is not handled or not allowed 2026-01-25 02:51:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-panel-4w-x-8h>: HTTP status code is not handled or not allowed 2026-01-25 02:51:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-stage-electric-air-compressor-75-hp-80-gal-cap-1-phase-230v-489-lb-wt>: HTTP status code is not handled or not allowed 2026-01-25 02:51:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zeus-pc1312-12v-12ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-25 02:51:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/firstpower-lfp1250-sealed-lead-acid-agm-vrla-battery>: HTTP status code is not handled or not allowed 2026-01-25 02:51:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interstate-12v-80ah-12v-75ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-25 02:51:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qt-max-two-stage-air-compressor-vertical-5-hp-80-gallon-capacity-1-phase-230v>: HTTP status code is not handled or not allowed 2026-01-25 02:51:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/concrete-mount-kit-for-galaxy-2205-lb-weight-black>: HTTP status code is not handled or not allowed 2026-01-25 02:51:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/parasystems-12v-4ah-12v-5ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-25 02:51:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/umbrellas_awnings_shades/outdoor_umbrellas_bases/umbrella_bases already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:51:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/embassy-12ce75-t1-12v-7ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-25 02:51:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tabloid-heavy-duty-view-binder-slant-ring-1-1-2inch-capacity-11-x-17-white>: HTTP status code is not handled or not allowed 2026-01-25 02:51:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6760-single-unibearing-bearing>: HTTP status code is not handled or not allowed 2026-01-25 02:51:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/technacell-zgid-emergency-light-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:51:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/combination-puller-with-2-3-swiveling-jaws-35-ton-cap-4-34-opening-3-78-adj-reach>: HTTP status code is not handled or not allowed 2026-01-25 02:51:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/omnis-controller-w-enclosure-90-amp>: HTTP status code is not handled or not allowed 2026-01-25 02:51:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-12-255-emergency-light-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:51:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/outdoor_furniture/umbrellas_awnings_shades/outdoor_umbrellas_bases/umbrella_bases>: HTTP status code is not handled or not allowed 2026-01-25 02:51:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aura-carbon-infrared-heater-w-remote-control-3000w-208240v-stainless>: HTTP status code is not handled or not allowed 2026-01-25 02:51:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aura-carbon-infrared-heater-w-remote-control-6000w-208240v-black>: HTTP status code is not handled or not allowed 2026-01-25 02:51:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergi-lite-m17-emergency-light-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:51:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/separating-puller-with-side-clamp-2-jaws-4-ton-capacity-4-opening-3-78-reach>: HTTP status code is not handled or not allowed 2026-01-25 02:51:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sure-lites-2xr410-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:51:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5340-quick-cross-clamp>: HTTP status code is not handled or not allowed 2026-01-25 02:51:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5475-single-shaft-predrilled-mounting-plate-1>: HTTP status code is not handled or not allowed 2026-01-25 02:51:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-w-x-36-d-production-workbench-with-drawer-upright-shelf-esd-laminate-square-edge-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:51:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2580-45-degree-support-18l>: HTTP status code is not handled or not allowed 2026-01-25 02:51:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lithonia-llpbe-6v-45ah-alarm-battery>: HTTP status code is not handled or not allowed 2026-01-25 02:52:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2015-end-cap-1l>: HTTP status code is not handled or not allowed 2026-01-25 02:52:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3455-51618-x-34-shcs-and-slidein-economy-tnut>: HTTP status code is not handled or not allowed 2026-01-25 02:52:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4332-2-hole-inside-gusset-corner-bracket-15w>: HTTP status code is not handled or not allowed 2026-01-25 02:52:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ge-caddx-networx-nx-8e-12v-75ah-alarm-battery>: HTTP status code is not handled or not allowed 2026-01-25 02:52:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2334-deluxe-flange-mount-caster-swivel-wbrake>: HTTP status code is not handled or not allowed 2026-01-25 02:52:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nerp005s-15-ner-electric-hoist-push-trolley-12-ton-15-lift-29-ftmin-230v>: HTTP status code is not handled or not allowed 2026-01-25 02:52:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3782-rollin-tnut-wflex-handle>: HTTP status code is not handled or not allowed 2026-01-25 02:52:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 55117 pages (at 95 pages/min), scraped 27509 items (at 52 items/min) 2026-01-25 02:52:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2531-45-degree-support-12l>: HTTP status code is not handled or not allowed 2026-01-25 02:52:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6870-single-unibearing-bearing-assembly-short>: HTTP status code is not handled or not allowed 2026-01-25 02:52:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nerp010sd-10-ner-electric-hoist-push-trolley-1-ton-10-lift-2845-ftmin-208v>: HTTP status code is not handled or not allowed 2026-01-25 02:52:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6534-highcycle-double-flange-linear-bearing-brake-kit-ready>: HTTP status code is not handled or not allowed 2026-01-25 02:52:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gas-gas-enduro-4t-400cc-motorcycle-replacement-battery-2002-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:52:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gas-gas-enduro-4t-450cc-motorcycle-replacement-battery-2002-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:52:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-contoured-full-zip-hooded-sweatshirt-class-3-m-hi-vis-yellow>: HTTP status code is not handled or not allowed 2026-01-25 02:52:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ion-audio-job-rocker-max-speaker-replacement-battery-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:52:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-contoured-full-zip-hooded-sweatshirt-class-3-3xl-hi-vis-yellow>: HTTP status code is not handled or not allowed 2026-01-25 02:52:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chamberlain-hd930ev-garage-door-replacement-battery-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:52:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/handy-cabinet-w-lower-handle-placement-36w-x-13d-x-27h-textured-dove-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:52:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vexilar-underwater-fish-scout-fs800-fish-finder-battery-7ah-12v>: HTTP status code is not handled or not allowed 2026-01-25 02:52:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ner020ld-15-ner-dual-speed-electric-chain-hoist-2-ton-15-lift-1425-ftmin-230v>: HTTP status code is not handled or not allowed 2026-01-25 02:52:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-contoured-softshell-jacket-ansi-4xl-hi-vis-yellow>: HTTP status code is not handled or not allowed 2026-01-25 02:52:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/squids-5547-arm--wrist-scanner-mount-holder-one-size-black>: HTTP status code is not handled or not allowed 2026-01-25 02:52:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ner030c-20-ner-electric-hoist-hook-suspension-3-ton-20-lift-17-ftmin-230v>: HTTP status code is not handled or not allowed 2026-01-25 02:52:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/contoured-reversible-puffer-vest-class-2-2xl-hi-vis-yellow>: HTTP status code is not handled or not allowed 2026-01-25 02:52:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honda-ii-sp-50cc-motorcycle-replacement-battery-2002-2010-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:52:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ner030cd-15-ner-dual-speed-electric-chain-hoist-3-ton-15-lift-173-ftmin-208v>: HTTP status code is not handled or not allowed 2026-01-25 02:52:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motron-thunder-50cc-motorcycle-replacement-battery-1995-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:52:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/atk-406-cc-406cc-motorcycle-replacement-battery-1991-1996-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:52:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cr3-edge-personal-srl-single-leg-galvanized-cable-aluminium-snap-hook-8l>: HTTP status code is not handled or not allowed 2026-01-25 02:52:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=7215bmuc3-angular-contact-ball-bearing-flush-ground-75mm-bore-x-130mm-od-x-25mm-w already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:52:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mbk-target-50cc-motorcycle-replacement-battery-1984-2008-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:52:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nerm005l-l-10-ner-electric-hoist-motor-trolley-12-ton-10-lift-15-ftmin-230v>: HTTP status code is not handled or not allowed 2026-01-25 02:52:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/freedom-961-electric-scooter-replacement-battery-14ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 02:52:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/import-5-16-inch-extra-long-drill-bit-8-inch-long>: HTTP status code is not handled or not allowed 2026-01-25 02:52:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thompson-t350cs-electric-scooter-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 02:52:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wus-tech-m4ahd-mobility-scooter-replacement-battery-55ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:52:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=7215bmuc3-angular-contact-ball-bearing-flush-ground-75mm-bore-x-130mm-od-x-25mm-w landed on page that is not a product page. 2026-01-25 02:52:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:52:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/recycling-bin-label-w-sign-adhesive-vinyl-4-x-12-pack-of-5>: HTTP status code is not handled or not allowed 2026-01-25 02:52:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:52:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/snerm030c-l-10-sner-electric-hoist-motor-trolley-3-ton-10-lift-35-ftmin-230v>: HTTP status code is not handled or not allowed 2026-01-25 02:52:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/centrifugal-stirrer-18900078-use-w-os20-ptfe-coated>: HTTP status code is not handled or not allowed 2026-01-25 02:52:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60090002100-roll-211k-2w-x-50-yd-180>: HTTP status code is not handled or not allowed 2026-01-25 02:52:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/linearorbital-shaker-fixing-clip-18900031-for-round-flasks-use-w-sk-o180-sk-o330>: HTTP status code is not handled or not allowed 2026-01-25 02:52:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/full-safety-knee-boots-steel-toe-size-9-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:52:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs>: HTTP status code is not handled or not allowed 2026-01-25 02:52:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs>: HTTP status code is not handled or not allowed 2026-01-25 02:52:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dispensmate-plus-bottletop-dispensers-73110002-45mm-thread-1-10ml>: HTTP status code is not handled or not allowed 2026-01-25 02:52:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cast-in-place-rigid-post-anchor-point-5000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 02:52:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g-tek-polykor-seamless-knit-blended-cr-gloves-silicone-coated-ansi-a7-xxl-black-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 02:52:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96-x-48-slatwall-panel-birch-laminate-without-inserts-pkg-qty-4>: HTTP status code is not handled or not allowed 2026-01-25 02:52:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-x-48-slatwall-panel-maple-veneer-without-inserts-pkg-qty-4>: HTTP status code is not handled or not allowed 2026-01-25 02:52:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bombardier-formula-500-deluxe-500cc-snowmobile-replacement-battery-1997-1998-12v>: HTTP status code is not handled or not allowed 2026-01-25 02:52:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-side-printable-business-card-2-inch-x-3-1-2-inch-ivory-200-cards-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:53:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modesto-32-commercial-planter-neptune-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:53:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hook-tape-individual-strips-with-adhesive-75l-x-12w-black>: HTTP status code is not handled or not allowed 2026-01-25 02:53:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thermal-register-cash-roll-3-inch-x-225-ft-white-24-rolls-carton>: HTTP status code is not handled or not allowed 2026-01-25 02:53:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/Foam-Cushion-20-x-16-x-4-72003>: HTTP status code is not handled or not allowed 2026-01-25 02:53:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durapro-knee-boots-safety-loc-outsole-steel-toe-size-5-14h-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:53:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/preprinted-monthly-tab-divider-printed-jan-to-dec-55x85-12-tabs-white-clear>: HTTP status code is not handled or not allowed 2026-01-25 02:53:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 55212 pages (at 95 pages/min), scraped 27554 items (at 45 items/min) 2026-01-25 02:53:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cotton-tip-applicator-3-in-10-bag>: HTTP status code is not handled or not allowed 2026-01-25 02:53:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electric-edge-o-dock-leveler-36-58l-x-72w-ramp-steel-25000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 02:53:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stackable-shelf-bin-8-1-4w-x-12d-x-7h-orange>: HTTP status code is not handled or not allowed 2026-01-25 02:53:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stackable-shelf-bin-6w-x-18d-x-7h-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:53:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medi-man-rehab-products-patient-lift-5000-medical-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:53:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wheel-chock-urethane-11l-x-8w-x-8h>: HTTP status code is not handled or not allowed 2026-01-25 02:53:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sky-dancer-blower-w-led-lights-2-speeds-1-hp-br-450l>: HTTP status code is not handled or not allowed 2026-01-25 02:53:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultratech-ut-12180-ut12180-sealed-lead-acid-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:53:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dispensing-bin-w-1-compartment-medium-12w-x-9d-x-7-1-4h>: HTTP status code is not handled or not allowed 2026-01-25 02:53:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bubble-sof-tred-12-thick-safety-anti-fatigue-floor-mat-2x6-blackyellow>: HTTP status code is not handled or not allowed 2026-01-25 02:53:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anti-short-plastic-bushing-polyethylene-38-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 02:53:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-clear-quartz-lamp-630-36489-for-462-463-and-mm-electric-infrared-heaters-3650w-480v>: HTTP status code is not handled or not allowed 2026-01-25 02:53:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-pvc-liquid-tight-connector-1-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 02:53:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-person-stage-12-inch-w-x-32-ft-l-2332>: HTTP status code is not handled or not allowed 2026-01-25 02:53:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/19-inch-w-x-8-ft-l-plywood-decked-aluma-plank-5308-19>: HTTP status code is not handled or not allowed 2026-01-25 02:53:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triple-grip-plastic-anchor-10-size-blue-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 02:53:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lr-threaded-conduit-body-w-cover-and-gasket-2-12-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 02:53:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conical-plastic-anchor-4-6-and-8-sizes-red-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 02:53:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ef-flame-resistant-classic-coverall-cec2-navy-size-50-long>: HTTP status code is not handled or not allowed 2026-01-25 02:53:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ef-flame-resistant-classic-coverall-cec2-navy-size-48-regular>: HTTP status code is not handled or not allowed 2026-01-25 02:53:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tuflex-spartus-recycled-rubber-tile-rpsplr913-interlock-27l-x-27w-x-38-thickness-charcoal>: HTTP status code is not handled or not allowed 2026-01-25 02:53:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chauffeur-mobility-viva-power-645-wheelchair-replacement-battery-55ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:53:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rod-coupling-nut-carbon-steel-34-10-size-25-pk>: HTTP status code is not handled or not allowed 2026-01-25 02:53:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-recycling-container-with-bottle-can-opening36-gallon-capacity-red-261622RD>: HTTP status code is not handled or not allowed 2026-01-25 02:53:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pride-mobility-ultra-sc40u-sc44u-wheelchair-replacement-battery-9ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 02:53:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/no-thread-connector-2-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 02:53:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ll-threaded-conduit-body-w-cover-and-gasket-2-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 02:53:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ground-clamp-w-12-hub-bronze-2-12-4-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 02:53:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swireaw15240-replacement-emitter-for-15kw-240v-saw-xl-series-scombi-hos>: HTTP status code is not handled or not allowed 2026-01-25 02:53:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kpex-71-02-200-sba-high-leverage-cobolt-bolt-cutters-comfort-grip-8-oal>: HTTP status code is not handled or not allowed 2026-01-25 02:53:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/fall_protection/harnesses already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:53:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hook-on-strap-carbon-steel-14-gauge-26-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 02:53:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-precision-bearing-7908cyuglp4-universal-ground-single-40mm-bore-62mm-od>: HTTP status code is not handled or not allowed 2026-01-25 02:53:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pull-rope-polypropylene-316-x-600-size>: HTTP status code is not handled or not allowed 2026-01-25 02:53:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triple-grip-anchor-kit-w-combo-pan-head-screw-8-size-201-pieces>: HTTP status code is not handled or not allowed 2026-01-25 02:53:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wheelchairs-of-kansas-prairie-cruiser-wheelchair-replacement-battery-75ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:53:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibration-mount-2-tapped-holes-200-inch-dia-163-inch-h-3-8-16-thread>: HTTP status code is not handled or not allowed 2026-01-25 02:53:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibration-shock-absorption-mount-threaded-stud-157-inch-dia-20mm-h-m8-x-125-thread>: HTTP status code is not handled or not allowed 2026-01-25 02:53:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ibt-technologies-bt24-12-emergency-light-replacement-battery-26ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:53:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/fall_protection/harnesses>: HTTP status code is not handled or not allowed 2026-01-25 02:53:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hook-on-strap-carbon-steel-14-gauge-18-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 02:53:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-12edx100bvc-retrofit-emergency-light-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:54:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/spill_control_supplies/chemical_sorbents_spill_kits/chemical_sorbents already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:54:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leveling-mount-w-lag-bolt-holes-5-8-11-x-6-inches-stud-315-inches-base-dia-7375-inchesh-5000-lbs-load-cap>: HTTP status code is not handled or not allowed 2026-01-25 02:54:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8002-class-3-waterproof-quiltlined-bomber-jacket-orangeblack-xl>: HTTP status code is not handled or not allowed 2026-01-25 02:54:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slotted-flat-head-machine-screw-6-32-x-1-14-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 02:54:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grip-16-w-2-step-steel-roll-ladder-10-d-top-stp-orange-218go>: HTTP status code is not handled or not allowed 2026-01-25 02:54:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perf-16-w-5-step-steel-roll-ladder-21-d-top-hndrl-orange-fsh51821po>: HTTP status code is not handled or not allowed 2026-01-25 02:54:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perf-24-w-7-step-steel-roll-ladder-14-d-top-hndrl-lck-orang-fs073014po>: HTTP status code is not handled or not allowed 2026-01-25 02:54:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grip-24-w-12-step-stl-roll-lad-21-d-top-stp-hndrl-lck-orng-fs123221go>: HTTP status code is not handled or not allowed 2026-01-25 02:54:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/69000-series-rectangular-desk-shell-66-inchw-x-30-inchd-x-29-1-2-inchh-cherry>: HTTP status code is not handled or not allowed 2026-01-25 02:54:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-captain-side-chair-24-inchw-x-25-inchd-x-30-3-4-inchh-burgundy-vinyl>: HTTP status code is not handled or not allowed 2026-01-25 02:54:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wallpro-400-orange-wall-mount-retracting-barrier-15-yellowblack-authorized-access-only-belt>: HTTP status code is not handled or not allowed 2026-01-25 02:54:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 55308 pages (at 96 pages/min), scraped 27599 items (at 45 items/min) 2026-01-25 02:54:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/wall_mount_belts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:54:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/spill_control_supplies/chemical_sorbents_spill_kits/chemical_sorbents landed on page that is not a product page. 2026-01-25 02:54:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-head-cap-screw-grade-5-carbon-steel-14-20-x-1-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 02:54:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/can-am-outlander-650-efi-xt-p-650cc-atv-replacement-battery-2010-2011-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:54:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inventory-cone-grey-3-sided-with-dry-erase-decal>: HTTP status code is not handled or not allowed 2026-01-25 02:54:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96h-x-48w-slatwall-panelmaple-laminate-with-aluminum-inserts-968345MI>: HTTP status code is not handled or not allowed 2026-01-25 02:54:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48h-x-48w-slatwall-panelcherry-laminate-with-aluminum-inserts-968343CI>: HTTP status code is not handled or not allowed 2026-01-25 02:54:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sheet-metal-screw-combo-pan-head-carbon-steel-10-x-1-size-tuff-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 02:54:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96h-x-48w-slatwall-panelrecoatable-white-without-inserts-968340R>: HTTP status code is not handled or not allowed 2026-01-25 02:54:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sheet-metal-screw-combo-pan-head-carbon-steel-8-x-34-size-tuff-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 02:54:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-electric-stapler-70-sheet-5000-staple-capacity-black>: HTTP status code is not handled or not allowed 2026-01-25 02:54:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/wall_mount_belts landed on page that is not a product page. 2026-01-25 02:54:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrf175401de56c-cast-iron-right-angle-worm-gear-reducer-401-ratio-56c-frame>: HTTP status code is not handled or not allowed 2026-01-25 02:54:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/saunders-recycled-plastic-clipboard-with-antimicrobial-protection-8-1-2-x-12-green>: HTTP status code is not handled or not allowed 2026-01-25 02:54:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrs262201r-cast-iron-right-angle-worm-gear-reducer-201-ratio>: HTTP status code is not handled or not allowed 2026-01-25 02:54:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrs206301l-cast-iron-right-angle-worm-gear-reducer-301-ratio>: HTTP status code is not handled or not allowed 2026-01-25 02:54:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-no-1-non-skid-paper-clips-silver-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:54:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-heavy-duty-staples-1-4-leg-length-100-per-strip-5000-box>: HTTP status code is not handled or not allowed 2026-01-25 02:54:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-staple-cartridge-3-8-leg-length-5000-per-cartridge-1-cartridge-box>: HTTP status code is not handled or not allowed 2026-01-25 02:54:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrf26251l1824tc-cast-iron-right-angle-worm-gear-reducer-51-ratio-1824t-frame>: HTTP status code is not handled or not allowed 2026-01-25 02:54:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-staples-1-4-leg-length-210-per-strip-3750-box>: HTTP status code is not handled or not allowed 2026-01-25 02:54:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrf262101de1824tc-cast-iron-right-angle-worm-gear-reducer-101-ratio-1824t-frame>: HTTP status code is not handled or not allowed 2026-01-25 02:54:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classic-stapler-20-sheet-210-staple-capacity-black>: HTTP status code is not handled or not allowed 2026-01-25 02:54:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/light-duty-stapler-20-sheet-210-staple-capacity-black>: HTTP status code is not handled or not allowed 2026-01-25 02:54:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hercules-sachs-limbo-ml-50cc-motorcycle-replacement-battery-1995-1997-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:54:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lock-washer-18-8-stainless-steel-6-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 02:54:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-container-placard-label-holder-estb6-8-x-10-white>: HTTP status code is not handled or not allowed 2026-01-25 02:54:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simson-albartros-electric-start-50cc-motorcycle-replacement-battery-2001-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:54:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/closed-starter-csb85-1836-8-36-inch-w-x-18-inch-d-x-85-inch-h-8-shelves-20-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:54:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f71gy00020001002-victor-wall-mnt-control-station-yellow-3-hole-3-position-selector2-funct>: HTTP status code is not handled or not allowed 2026-01-25 02:54:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bx1236l-36-inch-w-x-12-inch-d-22-gauge-600-lb-capacity-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:54:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-head-tap-bolt-silicon-bronze-516-18-x-12-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 02:54:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/phillipsslotted-mushroom-head-toggle-bolt-14-x-3-size-50-pk>: HTTP status code is not handled or not allowed 2026-01-25 02:54:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-head-lag-screw-carbon-steel-38-x-5-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 02:54:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harley-davidson-screamin-eagle-softail-deuce-fxstdse-1550cc-motorcycle-battery-2003-12v>: HTTP status code is not handled or not allowed 2026-01-25 02:54:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lock-washer-spring-steel-6-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 02:54:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wal-8-4-flute-extractor>: HTTP status code is not handled or not allowed 2026-01-25 02:54:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kawasaki-jet-ski-jt1200-c-1200cc-personal-watercraft-battery-2002-2007-12v>: HTTP status code is not handled or not allowed 2026-01-25 02:55:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/208v-normally-closed-steam-actuator-without-end-switch-ag14d020>: HTTP status code is not handled or not allowed 2026-01-25 02:55:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-ucfl207-23-flange-mount-bearing-2-bolt-flange-set-screw-locking-bore-36-5125-mm>: HTTP status code is not handled or not allowed 2026-01-25 02:55:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-row-spherical-roller-bearing-22324exw33c3-120mm-bore-260mm-od>: HTTP status code is not handled or not allowed 2026-01-25 02:55:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/piaggio-vespa-ntt-quartz-50cc-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:55:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-hcfu207-20-flange-mount-bearing-4-bolt-flange-eccentric-locking-collar-bore-31-75-mm>: HTTP status code is not handled or not allowed 2026-01-25 02:55:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/linebacker-1-channel-splittop-cable-protector-34000-lb-cap-38-12-x-14-14-x-3-yellow>: HTTP status code is not handled or not allowed 2026-01-25 02:55:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-row-spherical-roller-bearing-22313exw33kc3-65mm-bore-140mm-od>: HTTP status code is not handled or not allowed 2026-01-25 02:55:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-row-spherical-roller-bearing-23120ex1w33c3-100mm-bore-165mm-od>: HTTP status code is not handled or not allowed 2026-01-25 02:55:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 55403 pages (at 95 pages/min), scraped 27648 items (at 49 items/min) 2026-01-25 02:55:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mediline-suiter-gemini-1-mobility-scooter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:55:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-row-spherical-roller-bearing-23124ex1w33c3-120mm-bore-200mm-od>: HTTP status code is not handled or not allowed 2026-01-25 02:55:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/longbo-lb150t-2-voyager-150cc-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:55:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-ucp206-19-pillow-block-bearing-set-screw-locking-bore-30-1625-mm>: HTTP status code is not handled or not allowed 2026-01-25 02:55:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/currie-womense-cruiser-electric-scooter-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 02:55:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quantum-label-holder-for-stacking--post-baskets-44l-clear>: HTTP status code is not handled or not allowed 2026-01-25 02:55:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quantum-hanging-basket-label-holder-16-12l-clear>: HTTP status code is not handled or not allowed 2026-01-25 02:55:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-row-cylindrical-roller-bearing-nj220myc3-100mm-bore-180mm-od>: HTTP status code is not handled or not allowed 2026-01-25 02:55:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/optiway-technology-1700-sp-mobility-scooter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:55:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-log-book-for-material-handling-equipment>: HTTP status code is not handled or not allowed 2026-01-25 02:55:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/everest-jennings-everest-mobility-scooter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:55:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drive-medical-sunfireplus-mobility-scooter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:55:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-row-cylindrical-roller-bearing-nu212-60mm-bore-110mm-od>: HTTP status code is not handled or not allowed 2026-01-25 02:55:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reverse-alarm-for-global-industrial-utility-vehicle-615162>: HTTP status code is not handled or not allowed 2026-01-25 02:55:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/cart_truck_accessories/vehicle_tugger_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:55:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-in-t316-stainless-steel-full-port-ball-valve-3-piece-1000-psi>: HTTP status code is not handled or not allowed 2026-01-25 02:55:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/cart_truck_accessories/vehicle_tugger_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:55:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/klip-it-closed-starter-csk73-1236-8-36-inch-w-x-12-inch-d-x-73-inch-h-8-shelves-20-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:55:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-in-x-3-1-2-in-lead-free-seamless-red-brass-pipe-nipple-140-psi-sch-40-import>: HTTP status code is not handled or not allowed 2026-01-25 02:55:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-chrome-esd-wire-shelving-unit-starter-60w-x-30d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 02:55:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-chrome-esd-wire-shelving-unit-starter-72w-x-12d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 02:55:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-chrome-esd-wire-shelving-unit-starter-72w-x-14d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 02:55:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-bolt-open-add-on-oat75-3636-6-36-inch-w-x-36-inch-d-x-75-inch-h-6-shelves-20-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:55:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-2-in-lead-free-brass-coupling-fnpt-125-psi-import>: HTTP status code is not handled or not allowed 2026-01-25 02:55:34 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/cart_truck_accessories/vehicle_tugger_accessories landed on page that is not a product page. 2026-01-25 02:55:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-shelf-stainless-steel-wire-shelving-unit-starter-72w-x-21d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 02:55:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pride-mobility-jazzy-1103ultra-mobility-scooter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:55:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/cart_truck_accessories/vehicle_tugger_accessories landed on page that is not a product page. 2026-01-25 02:55:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bombardier-gtx-sport-500s-597cc-snowmobile-replacement-battery-2009-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:55:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-mx-z-blizzard-4-tec-1200-1200cc-snowmobile-battery-2016-2018-12v>: HTTP status code is not handled or not allowed 2026-01-25 02:55:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-in-x-6-in-lead-free-seamless-red-brass-pipe-nipple-140-psi-sch-40-import>: HTTP status code is not handled or not allowed 2026-01-25 02:55:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-stainless-steel-wire-shelving-unit-add-on-24w-x-21d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 02:55:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-bolt-closed-add-on-cat75-1236-5-36-inch-w-x-12-inch-d-x-75-inch-h-5-shelves-20-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:55:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baja-motorsports-sc150-on-road-147cc-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:55:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-chrome-wire-shelving-unit-starter-30w-x-14d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 02:55:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-bolt-open-add-on-oat87-1236-8x-36-inch-w-x-12-inch-d-x-87-inch-h-8-shelves-18-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:55:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-shelf-stainless-steel-wire-shelving-unit-add-on-42w-x-14d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 02:55:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-pad-for-m30-m40-m50-40150-40220-40310-40311-40312-black>: HTTP status code is not handled or not allowed 2026-01-25 02:55:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-shelf-stainless-steel-wire-shelving-unit-add-on-54w-x-18d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 02:55:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-chrome-wire-shelving-unit-add-on-72w-x-18d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 02:55:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-inking-message-stamp-12-phrases-3-8-x-1-1-4-red>: HTTP status code is not handled or not allowed 2026-01-25 02:55:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yamaha-rs-vector-mountain-973cc-snowmobile-replacement-battery-2005-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:55:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wtta-12-5-gallon-expansion-tank-asme-3-4-in-fnpt>: HTTP status code is not handled or not allowed 2026-01-25 02:55:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-chrome-wire-shelving-unit-add-on-36w-x-12d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 02:55:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/colored-poly-mailers-12-x-15-1-2-2-5-mil-red-100-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:55:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-ski-doo-mx-z-670-669cc-snowmobile-replacement-battery-1996-1998-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:55:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-shelf-stainless-steel-wire-shelving-unit-add-on-72w-x-36d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 02:55:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/stamps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:55:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-chrome-wire-shelving-unit-starter-60w-x-12d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 02:55:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-mx-z-renegade-800r-800cc-snowmobile-replacement-battery-2004-2013-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:55:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bombardier-legend-550f-550cc-snowmobile-replacement-battery-2003-2004-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:55:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-chrome-wire-shelving-unit-starter-60w-x-36d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 02:55:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bombardier-skandic-swt-e-tec-600-ho-600cc-snowmobile-battery-2012-2018-12v>: HTTP status code is not handled or not allowed 2026-01-25 02:55:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-stainless-steel-wire-shelving-unit-starter-42w-x-18d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 02:56:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-certified-line-regulator-w-vent-limiter-325-3l-v-1-2-up-to-140000-btu>: HTTP status code is not handled or not allowed 2026-01-25 02:56:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-poly-bags-10-x-14-8-mil-clear-500-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:56:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-mx-z-800-r-adrenaline-snowmobile-replacement-battery-2008-2010-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:56:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-renegade-x-1200-1170cc-snowmobile-replacement-battery-2014-2016-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:56:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/basic_supplies/stamps>: HTTP status code is not handled or not allowed 2026-01-25 02:56:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-poly-green-wire-shelving-unit-add-on-60w-x-12d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 02:56:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-corrugated-boxes-10-x-10-x-2-200lb-test-ect-32-25-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:56:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-corrugated-boxes-11-1-4-x-8-3-4-x-8-200lb-test-ect-32-25-pack>: HTTP status code is not handled or not allowed 2026-01-25 02:56:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-poly-green-wire-shelving-unit-starter-60w-x-12d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 02:56:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-ski-doo-renegade-adernaline-ace-900-899cc-snowmobile-battery-2014-12v>: HTTP status code is not handled or not allowed 2026-01-25 02:56:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/66-in-concrete-round-picnic-table-brown>: HTTP status code is not handled or not allowed 2026-01-25 02:56:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drawer-divider-for-350712-medication-drawer>: HTTP status code is not handled or not allowed 2026-01-25 02:56:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-ski-doo-gsx-se-600-ho-594cc-snowmobile-replacement-battery-2014-2015-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:56:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bombardier-renegade-adrenaline-600-ho-600cc-snowmobile-battery-2014-2016-12v>: HTTP status code is not handled or not allowed 2026-01-25 02:56:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 55504 pages (at 101 pages/min), scraped 27685 items (at 37 items/min) 2026-01-25 02:56:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-8590-containment-berm-foam-wall-15-x-50copolymer-2000>: HTTP status code is not handled or not allowed 2026-01-25 02:56:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-poly-green-wire-shelving-unit-starter-36w-x-12d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 02:56:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-fuse-for-20-evaporative-cooler-model-600580-292219>: HTTP status code is not handled or not allowed 2026-01-25 02:56:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-8477-containment-berm-foam-wall-12-x-30-22-oz-pvc>: HTTP status code is not handled or not allowed 2026-01-25 02:56:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/door-frame-with-masonry-stud-steelcraft-hinge-location-left-hand-36w-x-80h>: HTTP status code is not handled or not allowed 2026-01-25 02:56:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/door-frame-with-masonry-stud-hinge-location-right-hand-36w-x-84h>: HTTP status code is not handled or not allowed 2026-01-25 02:56:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-power-system-52ah-1000w-1331wh>: HTTP status code is not handled or not allowed 2026-01-25 02:56:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/queuepro-250-retractable-belt-barrier-40-black-post-11-red-belt-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-25 02:56:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safetypro-335-caution-do-not-enter-belt-barrier-40-post-35-yellowblack-pk-of-2>: HTTP status code is not handled or not allowed 2026-01-25 02:56:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hollow-steel-flush-door-cylindrical-prep-steelcraft-hinge-16-ga-36w-x-84h>: HTTP status code is not handled or not allowed 2026-01-25 02:56:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-mx-z-sport-ace-600-600cc-snowmobile-replacement-battery-2014-2015-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:56:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bluff-mobilepro43-yard-ramp-with-hinged-lip--adj-legs-25000-lb-cap-36l-x-96w-x-48h>: HTTP status code is not handled or not allowed 2026-01-25 02:56:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/-heavy-duty-quick-attach-hopper-2-cu-yd-4000-lb-capacity-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:56:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25dtp-4-two-channel-pole-2-duplex-receptacles-8-removable-communications-105l>: HTTP status code is not handled or not allowed 2026-01-25 02:56:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-plus-phenolic-locker-11-821212601-double-tier-12-x-12-x-30-dove-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:56:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-plus-phenolic-locker-11-8z1818721-z-style-18-x-18-x-72-silver-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:56:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-ski-doo-skandic-wt-600-ho-594cc-snowmobile-replacement-battery-2014-2016-12v>: HTTP status code is not handled or not allowed 2026-01-25 02:56:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bombardier-grand-touring-1200-1200cc-snowmobile-replacement-battery-2010-2013-12v>: HTTP status code is not handled or not allowed 2026-01-25 02:56:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-plus-phenolic-locker-11-8z1218601-z-style-12-x-18-x-60-graphite-grafix>: HTTP status code is not handled or not allowed 2026-01-25 02:56:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:56:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fmmusl-9-14830-bnp-m4-led-mushroom-165w-3000-cct-brushed-nickel>: HTTP status code is not handled or not allowed 2026-01-25 02:56:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-renegade-adernaline-ace-900-899cc-snowmobile-replacement-battery-2014-12v>: HTTP status code is not handled or not allowed 2026-01-25 02:56:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-1-2-inch-x-3-8-inch-white-dove-mini-roller-2-pk-140605062>: HTTP status code is not handled or not allowed 2026-01-25 02:56:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smart-nfc-smartcards--key-fobs-whiteblack-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-25 02:56:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yamaha-rs90lt-rs-vector-ltx-900cc-snowmobile-replacement-battery-2008-2009-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:56:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-summit-highmark-800-800cc-snowmobile-replacement-battery-2000-2004-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:56:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yamaha-rs90xt-rs-vector-xtx-900cc-snowmobile-replacement-battery-2016-2017-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:56:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vented-plastic-shelving-30x21x74-nexelon-finish-331450N>: HTTP status code is not handled or not allowed 2026-01-25 02:56:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-mx-z-blizzard-600-ho-sdi-594cc-snowmobile-replacement-battery-2007-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:56:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions>: HTTP status code is not handled or not allowed 2026-01-25 02:56:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plunger-can-1-quart-nonmetalic-red-14018>: HTTP status code is not handled or not allowed 2026-01-25 02:56:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lucy-3-piece-outdoor-bistro-set-31-1-2-w-x-29-1-2-h-table-white>: HTTP status code is not handled or not allowed 2026-01-25 02:56:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4932-double-sided-vhb-acrylic-foam-tape-1-2-x-5-yds-25-mil-white>: HTTP status code is not handled or not allowed 2026-01-25 02:56:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/building_construction_tape/gaffer_tape already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:56:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-367ab-butterfly-valve-with-buna-seals--10-position-handle-6-lug>: HTTP status code is not handled or not allowed 2026-01-25 02:56:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4016-double-sided-foam-tape-1-x-5-yds-1-16-thick-natural->: HTTP status code is not handled or not allowed 2026-01-25 02:56:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-545vbnl-lf-commercial-multi-turn-sillcock-valve-with-vacuum-breaker-12-fnpt>: HTTP status code is not handled or not allowed 2026-01-25 02:56:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-550pnl-lf-brass-14-turn-frostfree-sillcock-valve-12-x-10-f1807-pex>: HTTP status code is not handled or not allowed 2026-01-25 02:56:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-367di-butterfly-valve-with-buna-seals--10-position-handle-2-lug>: HTTP status code is not handled or not allowed 2026-01-25 02:56:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/r-673nl-lf-brass-14-turn-sillcock-valve-12-fnpt>: HTTP status code is not handled or not allowed 2026-01-25 02:56:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/argoscabinet-gun-storage-cabinet-awc50h12r-1-holds-12-rifles-w-2-half-shelves-42w-x-15d-x-50h-battleship-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:56:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bently-laboratories-sm-0200-medical-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:56:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/building_construction_tape/gaffer_tape>: HTTP status code is not handled or not allowed 2026-01-25 02:56:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inverter-duty-motor-y1999-182thfw7729-3hp-230-460v-1800rpm-3ph-182tc-tefc>: HTTP status code is not handled or not allowed 2026-01-25 02:56:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cell-phone-locker-with-access-panel-19158-20asc-surface-mounted-combo-locks-20-a-doors-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 02:56:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/severe-duty-motor-e618-213ttfs6526-75hp-460v-1800rpm-3ph-213t-fr-tefc>: HTTP status code is not handled or not allowed 2026-01-25 02:56:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lmabfdwssk-filtered-bottle-filling-station-w-single-ada-cooler-stainless>: HTTP status code is not handled or not allowed 2026-01-25 02:57:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ezsdwslk-ezh20-bottle-filling-station-w-single-ada-ez-cooler-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:57:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/118590-large-look-out-sensor-ceiling-hung-1-box-3-sensors-3-lights-50-cord>: HTTP status code is not handled or not allowed 2026-01-25 02:57:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jump-n-carry-jnc4000pro-jump-starter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:57:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wx-explosion-proof-infrared-gas-catalytic-heater-10000-btu-30a>: HTTP status code is not handled or not allowed 2026-01-25 02:57:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/solar-jumper1000--j1000-jump-starter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:57:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/tables/laboratory_tables already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:57:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s012-056c17d2098-1-2hp-1800rpm-115-230v-1ph-56-fr-dp>: HTTP status code is not handled or not allowed 2026-01-25 02:57:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-311-cast-iron-swing-check-valve-8-flanged>: HTTP status code is not handled or not allowed 2026-01-25 02:57:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/900-098-bu-cable-tie-hook-tape-8l-x-12w-blue-25pk>: HTTP status code is not handled or not allowed 2026-01-25 02:57:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 55596 pages (at 92 pages/min), scraped 27724 items (at 39 items/min) 2026-01-25 02:57:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/300-175-crimpro-crimper-for-mc3-solar-panel-contacts>: HTTP status code is not handled or not allowed 2026-01-25 02:57:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mounting-kit-for-fx-series-heaters-fx5fx6-16-frame-size>: HTTP status code is not handled or not allowed 2026-01-25 02:57:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50g-1g-screw-knob-stainless-steel-precision-weight-with-case-astm-e617-class-4>: HTTP status code is not handled or not allowed 2026-01-25 02:57:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hyperpure-pe-rt-tubing-1-12id-x-1-58od-x-100-coil-white>: HTTP status code is not handled or not allowed 2026-01-25 02:57:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cortina-plastx-type-i-fold-flat-barricade-engineer-grade-sheeting-24l-x-12w-panel-orangewhite>: HTTP status code is not handled or not allowed 2026-01-25 02:57:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/tables/laboratory_tables>: HTTP status code is not handled or not allowed 2026-01-25 02:57:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cortina-plastx-type-ii-fold-flat-barricade-engineer-grade-sheeting-24l-x-12w-panel-orangewhite>: HTTP status code is not handled or not allowed 2026-01-25 02:57:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-pipe-fitting-90-degree-two-socket-tee-1-dia>: HTTP status code is not handled or not allowed 2026-01-25 02:57:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/condenser-fan-motor-p195-1hp-1075-rpm-200-230-460-v-1-ph-48z-fr-open>: HTTP status code is not handled or not allowed 2026-01-25 02:57:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/amstron-ap-12120f2-sealed-lead-acid-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 02:57:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-shelf-nexelon-blue-wire-shelving-unit-add-on-72w-x-21d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 02:57:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/long-way-lw-6fm20a-sealed-lead-acid-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:57:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rockford-rlsw-30-2-lint-separators-17-l-x-17-w-x-16-h-steel>: HTTP status code is not handled or not allowed 2026-01-25 02:57:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/origin-or-12220hr-nb-sealed-lead-acid-replacement-battery-5ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 02:57:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wh20gb506-plugmold-125v-15a-5l-10-outlets>: HTTP status code is not handled or not allowed 2026-01-25 02:57:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-cal-treadmill-mat-fitness-equipment-mat-black-3-16-x-4-x-7-5>: HTTP status code is not handled or not allowed 2026-01-25 02:57:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4810ulbd20r-tl-cabinetmate-surge-strip-125v-20a-48l-10-outlets>: HTTP status code is not handled or not allowed 2026-01-25 02:57:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-nexelon-blue-wire-shelving-unit-add-on-24w-x-18d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 02:57:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-cal-coin-grip-anti-slip-rubber-mat-2mm-x-4-x-13-rolled-rubber-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:57:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bk2010a2-entrance-end-fitting-125v-15a-3l>: HTTP status code is not handled or not allowed 2026-01-25 02:57:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bk20gb512-plugmold-125v-15a-5l>: HTTP status code is not handled or not allowed 2026-01-25 02:57:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wst1-3048-36-pb-48w-x-30d-fixed-height-welded-steel-workbench-pegboard-panel>: HTTP status code is not handled or not allowed 2026-01-25 02:57:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/j60b0b-90-rackmount-125v-15a-20l-6-outlets>: HTTP status code is not handled or not allowed 2026-01-25 02:57:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mercury-topaz-84456-sealed-lead-acid-replacement-battery-26ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:57:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-shelf-nexelon-blue-wire-shelving-unit-starter-36w-x-14d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 02:57:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-nexelon-blue-wire-shelving-unit-starter-48w-x-18d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 02:57:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-tee-23000-use-for-1516w-grid-white>: HTTP status code is not handled or not allowed 2026-01-25 02:57:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/expertpower-exp12180-sealed-lead-acid-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:57:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vestil-aluminum-treadplate-deck-platform-truck-atp-c-2448-2hdl-2-handles-3600-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 02:57:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/panasonic-lc-rc1217pg-sealed-lead-acid-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:57:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ajc-interstate-dcm0026-12v-26ah-wheelchair-battery>: HTTP status code is not handled or not allowed 2026-01-25 02:57:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unison-union-mx-12012-sealed-lead-acid-replacement-battery-9ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 02:57:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gogreen-power-gg-24712-12-household-extension-cord-white>: HTTP status code is not handled or not allowed 2026-01-25 02:57:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crunchy-granola-bar-peanut-butter-15-oz-18box>: HTTP status code is not handled or not allowed 2026-01-25 02:57:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pp-heavyweight-15-gal-cylindrical-tank-w-cover-13dia-x-27h-3-16wall-white>: HTTP status code is not handled or not allowed 2026-01-25 02:57:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/apex-battery-apx12220-sealed-lead-acid-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:57:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merits-mini-coupe-s539-wheelchair-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:57:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/19-gallon-2-door-manual-fume-hood-acid-cabinet-30x21-5-8x35-3-4-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:57:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fortress-scientific-760v-gp24-wheelchair-replacement-battery-7ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 02:57:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-shelf-nexelate-silver-epoxy-wire-shelving-unit-starter-30w-x-24d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 02:57:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-nexelate-silver-epoxy-wire-shelving-unit-starter-54w-x-21d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 02:57:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/forklifts_attachments/forklift_platforms already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:57:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electric-mobility-rascal-320-wheelchair-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:57:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-nexelate-silver-epoxy-wire-shelving-unit-starter-54w-x-21d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 02:57:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fallte-8246-heavyweight-6-shock-absorbing-lanyard-single-leg-with-2-snap-hooks>: HTTP status code is not handled or not allowed 2026-01-25 02:57:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/permobil-street-electric-wheelchair-replacement-battery-8ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 02:57:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/golden-technologies-alante-wheelchair-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:58:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/amigo-mobility-rt-junior-wheelchair-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:58:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/werner-6-type-1aa-fiberglass-podium-ladder-w-casters-375-lb-cap-pd7306-4c>: HTTP status code is not handled or not allowed 2026-01-25 02:58:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/carts-trucks/forklifts_attachments/forklift_platforms>: HTTP status code is not handled or not allowed 2026-01-25 02:58:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fortress-winner-4-wheel-wheelchair-replacement-battery-55ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:58:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/four-wing-condenser-fan-blade-interchangeable-hub-aluminum-blade-ccw-12-inch-dia-33-pitch>: HTTP status code is not handled or not allowed 2026-01-25 02:58:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnetic-whiteboard-desk-divider-screen-with-universal-clamp>: HTTP status code is not handled or not allowed 2026-01-25 02:58:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/outdoor_structures_storage/canopies_carports_utility_tents/storage_tents already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:58:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-poly-z-brite-wire-shelving-unit-add-on-24w-x-24d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 02:58:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/w-04092-1-1-2-in-x-1-1-4-in-x-1-in-wrot-copper-reducing-tee-copper>: HTTP status code is not handled or not allowed 2026-01-25 02:58:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 55691 pages (at 95 pages/min), scraped 27765 items (at 41 items/min) 2026-01-25 02:58:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-poly-z-brite-wire-shelving-unit-add-on-60w-x-24d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 02:58:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-poly-z-brite-wire-shelving-unit-add-on-36w-x-14d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 02:58:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-1-8-personal-rolling-paper-trimmer>: HTTP status code is not handled or not allowed 2026-01-25 02:58:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pride-mobility-jazzy-600-xl-wheelchair-replacement-battery-55ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:58:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sealed-flange-ball-bearings-with-insulator-and-thurst-collar-7-8-inch-w>: HTTP status code is not handled or not allowed 2026-01-25 02:58:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/cutting_tools/scissors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:58:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/john-deere-wsgatpl-lawn-and-garden-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 02:58:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-poly-z-brite-wire-shelving-unit-starter-60w-x-14d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 02:58:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/solid-shafts-3-4-inch-dia-36-inch-long-flatted-on-pulley-end>: HTTP status code is not handled or not allowed 2026-01-25 02:58:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acl-f-30x36-combination-louver-damper-with-flange-30wx36h>: HTTP status code is not handled or not allowed 2026-01-25 02:58:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brundage-replacement-bearings-oiler-1-inch-shaft-dia>: HTTP status code is not handled or not allowed 2026-01-25 02:58:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brundage-replacement-bearings-sealed-3-4-inch-shaft-dia>: HTTP status code is not handled or not allowed 2026-01-25 02:58:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/outdoor_structures_storage/canopies_carports_utility_tents/storage_tents landed on page that is not a product page. 2026-01-25 02:58:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-galvanized-steel-solid-shelving-unit-add-on-72w-x-24d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 02:58:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/small-hubless-fan-blade-9-inch-dia-31-pitch-cw-1-5-8-inch-blade-depth-5-blade>: HTTP status code is not handled or not allowed 2026-01-25 02:58:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/small-plastic-push-on-fan-blade-5-1-2-inch-dia-ccw-3-16-inch-bore-1-1-8-inch-blade-depth-4-blade>: HTTP status code is not handled or not allowed 2026-01-25 02:58:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/astralite-eu-hd-180-emergency-light-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:58:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vinyl-numbers-adhesive-21-numbers-6-black>: HTTP status code is not handled or not allowed 2026-01-25 02:58:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-inlet-blower-wheel-4-3-4-inch-dia-ccw-3450-rpm-1-2-inch-bore-2-15-16-inch-w-galvanized>: HTTP status code is not handled or not allowed 2026-01-25 02:58:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/basic_supplies/cutting_tools/scissors>: HTTP status code is not handled or not allowed 2026-01-25 02:58:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelite-3-shelf-vented-plastic-mat-shelving-unit-starter-42w-x-21d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 02:58:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergi-lite-lsm182cp-emergency-light-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:58:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/902-113-utility-component-storage-box>: HTTP status code is not handled or not allowed 2026-01-25 02:58:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelite-3-shelf-vented-plastic-mat-shelving-unit-starter-48w-x-21d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 02:58:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/902-379-11-pc-jumbo-pack-assorted-star-tip-1-15-16-power-bit-set>: HTTP status code is not handled or not allowed 2026-01-25 02:58:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cp-301g-precision-wire-stripper-awg-30-20>: HTTP status code is not handled or not allowed 2026-01-25 02:58:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sure-lites-l026117-emergency-light-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:58:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f-clamp-2-throat-depth-8-opening>: HTTP status code is not handled or not allowed 2026-01-25 02:58:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-slider-casement-window-a-c-10000-btu-115v-r32>: HTTP status code is not handled or not allowed 2026-01-25 02:58:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-wide-manual-multi-roll-tape-dispenser>: HTTP status code is not handled or not allowed 2026-01-25 02:58:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/label_tapes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:58:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/york-wide-light-ar2e1-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:58:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/label_tapes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:58:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chloride-power-clblc-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:58:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-extra-hd-boltless-shelving-add-on-48-in-w-x-12-in-d-x-84-in-h-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 02:58:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/filter-glare-24w-bk>: HTTP status code is not handled or not allowed 2026-01-25 02:58:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-mlw-2e-ny-i-emergency-light-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:58:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/siltron-multi-pak1000-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:58:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/label_tapes landed on page that is not a product page. 2026-01-25 02:58:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sonnenschein-a512-24g-emergency-light-replacement-battery-26ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:58:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/label_tapes landed on page that is not a product page. 2026-01-25 02:58:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chloride-power-cmfre100-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:58:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/palmgren-9629308-0-8-standard-duty-c-clamp-w-black-oxide-spindles>: HTTP status code is not handled or not allowed 2026-01-25 02:58:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hl10000-series-right-hand-pedestal-steel-desk-30-x-66-black-walnut>: HTTP status code is not handled or not allowed 2026-01-25 02:58:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:58:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fbs-standard-bellman-cart-2499-bk-bk-stainless-black-carpet-black-bumper-8-rubber>: HTTP status code is not handled or not allowed 2026-01-25 02:58:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elite-all-welded-combination-storage-cabinet-solid-door-36w-x-18d-x-72h-black>: HTTP status code is not handled or not allowed 2026-01-25 02:58:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/national-power-gs012p2-emergency-light-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:58:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ps-80-f-evergreen-privacy-screen>: HTTP status code is not handled or not allowed 2026-01-25 02:58:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carpenter-watchman-713524-emergency-light-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 02:59:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/four-shelf-extra-hd-boltless-shelving-add-on-36inw-x-24ind-x-60inh-wood-deck-usa>: HTTP status code is not handled or not allowed 2026-01-25 02:59:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks landed on page that is not a product page. 2026-01-25 02:59:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/three-shelf-wide-boltless-shelving-add-on-3300-lb-96inwx48indx96inh-wood-deck-usa>: HTTP status code is not handled or not allowed 2026-01-25 02:59:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/room_dividers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:59:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/five-shelf-extra-hd-boltless-shelving-add-on-48inw-x-24ind-x-84inh-wood-deck-usa>: HTTP status code is not handled or not allowed 2026-01-25 02:59:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/room_dividers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:59:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/four-shelf-heavy-duty-boltless-shelving-add-on-36inw-x-18ind-x-60inh-wood-deck-usa>: HTTP status code is not handled or not allowed 2026-01-25 02:59:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ps-80-pc-blue-privacy-screen>: HTTP status code is not handled or not allowed 2026-01-25 02:59:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yamaha-yfm400-big-bear-2x4-400cc-atv-replacement-battery-2000-2004-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:59:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 55783 pages (at 92 pages/min), scraped 27804 items (at 39 items/min) 2026-01-25 02:59:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/awrd-partition-88x84-f-charcoal-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:59:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/five-shelf-extra-hd-boltless-shelving-add-on-48in-w-x-18in-d-x-60in-h-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 02:59:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/awrd-partition-88x84-f-powder-blue>: HTTP status code is not handled or not allowed 2026-01-25 02:59:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smoking-shelter-62wsfdkb-4sided-wl-r-open-front-15l-x-5w-flat-roof-dk-bronze>: HTTP status code is not handled or not allowed 2026-01-25 02:59:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/versifoot-partition-leg>: HTTP status code is not handled or not allowed 2026-01-25 02:59:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/partitions_dividers/room_dividers landed on page that is not a product page. 2026-01-25 02:59:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yamaha-ytm225-tri-moto-225cc-atv-replacement-battery-1983-1986-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:59:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qvtrk-84-individual-v-track-length-84>: HTTP status code is not handled or not allowed 2026-01-25 02:59:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qvtrk-120-individual-v-track-length-120>: HTTP status code is not handled or not allowed 2026-01-25 02:59:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/three-shelf-wide-boltless-shelving-add-on-60-in-w-x-48-in-d-x-96-in-h-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 02:59:18 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/partitions_dividers/room_dividers landed on page that is not a product page. 2026-01-25 02:59:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/peg-perego-apache-hp253-ride-on-toys-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 02:59:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-ski-doo-outlander-660-efi-660cc-atv-replacement-battery-2006-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:59:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/room_dividers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:59:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bombardier-traxter-650-4x4-auto-cvt-650cc-atv-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:59:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/three-shelf-wide-boltless-shelving-add-on-48-in-w-x-48-in-d-x-96-in-h-laminate-deck-usa>: HTTP status code is not handled or not allowed 2026-01-25 02:59:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/heaters/stoves_fireplaces_log_sets_fire_pits/fireplaces_log_sets_fireboxes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:59:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/heaters/stoves_fireplaces_log_sets_fire_pits/fireplace_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:59:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mt244836-3k295-machine-table-with-2-shelves-48w-x-24d-x-36h>: HTTP status code is not handled or not allowed 2026-01-25 02:59:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/k2022-unf-carbon-eyebolt-shoulder-pattern-5-16-24-1-1-8-shank>: HTTP status code is not handled or not allowed 2026-01-25 02:59:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/imported-slotted-angle-plates-open-end-ground-finish-8-inch-x-6-inch-x-5-inch>: HTTP status code is not handled or not allowed 2026-01-25 02:59:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ironhorse-lsc-1820cc-motorcycle-replacement-battery-2005-2007-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:59:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/partitions_dividers/room_dividers>: HTTP status code is not handled or not allowed 2026-01-25 02:59:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pam-esd-anti-fatigue--anti-static-insole-mens-1415>: HTTP status code is not handled or not allowed 2026-01-25 02:59:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/heaters/stoves_fireplaces_log_sets_fire_pits/fireplace_accessories>: HTTP status code is not handled or not allowed 2026-01-25 02:59:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/k2001-carbon-eyebolt-plain-pattern-1-4-20-1-shank>: HTTP status code is not handled or not allowed 2026-01-25 02:59:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/905-case-w-foam-12-1-2l-x-10w-x-6h-orange>: HTTP status code is not handled or not allowed 2026-01-25 02:59:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-gauge-mobile-bench-cabinet-2210a-lu-95>: HTTP status code is not handled or not allowed 2026-01-25 02:59:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/husaberg-fs450e-450cc-motorcycle-replacement-battery-2004-2008-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:59:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honda-cb600-sf-hornet-600cc-motorcycle-replacement-battery-2001-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:59:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-vis-cooling-safety-t-shirt-long-sleeve-ansi-class-3-xl-yellow>: HTTP status code is not handled or not allowed 2026-01-25 02:59:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/heaters/stoves_fireplaces_log_sets_fire_pits/fireplaces_log_sets_fireboxes landed on page that is not a product page. 2026-01-25 02:59:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/national-standard-school-bus-kit-weatherproof-steel-50-person-991>: HTTP status code is not handled or not allowed 2026-01-25 02:59:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honda-vtx1800t-1800cc-motorcycle-replacement-battery-2007-2011-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:59:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/heaters/electric_heaters/electric_baseboard_heaters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 02:59:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-welded-workbench-60-x-36-louvered-panel-steel-square-edge>: HTTP status code is not handled or not allowed 2026-01-25 02:59:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-welded-workbench-48-x-24-open-base-steel-square-edge>: HTTP status code is not handled or not allowed 2026-01-25 02:59:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/w-series-replacement-grille-wg-white>: HTTP status code is not handled or not allowed 2026-01-25 02:59:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd-welded-workbench-72-x-36-lower-shelf--louvered-panel-steel-square-edge>: HTTP status code is not handled or not allowed 2026-01-25 02:59:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electric-wall-heater-whf2420i-2000w-240v-forced-air-interior-only-white>: HTTP status code is not handled or not allowed 2026-01-25 02:59:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-toilet-tissue-dispenser-dual-roll-horizontal-ss-5224-000000>: HTTP status code is not handled or not allowed 2026-01-25 02:59:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/we-wvhs30-18-286tp-12-vhs-motor-30hp-1800rpm-286tp-230-460v-wpi>: HTTP status code is not handled or not allowed 2026-01-25 02:59:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd-welded-workbench-60-x-24-lower-shelf--louvered-panel-steel-square-edge>: HTTP status code is not handled or not allowed 2026-01-25 02:59:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bsm-inc-ada-ss-flat-sink-1-station-body-only-wall-mount-18l-x-18w-x-5d-sink>: HTTP status code is not handled or not allowed 2026-01-25 02:59:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/malaguti-evolution-50cc-motorcycle-replacement-battery-1963-2005-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:59:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd-welded-workbench-60-x-36-adj-height-louvered-panel-steel-square-edge>: HTTP status code is not handled or not allowed 2026-01-25 02:59:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harley-davidson-low-rider-fxdl-1690cc-motorcycle-battery-2014-2017-12v>: HTTP status code is not handled or not allowed 2026-01-25 02:59:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securio-autofeed-150c-150-sheet-stack-5-7-sheet-micro-cut-9-gal-capacity>: HTTP status code is not handled or not allowed 2026-01-25 02:59:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/heaters/electric_heaters/electric_baseboard_heaters>: HTTP status code is not handled or not allowed 2026-01-25 02:59:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-cabinet-flush-door-with-137-yellow-bins-16-ga-all-welded-cabinet-48-w-x-24-d-x-78-h-gray>: HTTP status code is not handled or not allowed 2026-01-25 02:59:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/168074102-magic-touch-2-ice-pan-w-lid-40l-black>: HTTP status code is not handled or not allowed 2026-01-25 02:59:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd-welded-workbench-60-x-24-adj-height-lower-shelf--3-backstop-steel-square-edge>: HTTP status code is not handled or not allowed 2026-01-25 02:59:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-quality-e-15-electric-scooter-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 02:59:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gssafe-1702-class-2-heavy-duty-safety-vest-orange-m>: HTTP status code is not handled or not allowed 2026-01-25 02:59:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/voyager-gt-overboots-with-heels-cleated-outsole-4xl-12h-black>: HTTP status code is not handled or not allowed 2026-01-25 02:59:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gssafe-5113-class-3-microfiber-birdseye-short-sleeve-t-shrit-w-black-bottom-lime-2xl-tall>: HTTP status code is not handled or not allowed 2026-01-25 02:59:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lance-cali-classic-150-150cc-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:59:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/albrecht-0-3-8-inch-high-precsion-keyless-drill-chuck-2jt>: HTTP status code is not handled or not allowed 2026-01-25 02:59:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jenoptik-80-sfera-80cc-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 02:59:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/permobil-chairman-mobility-scooter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 03:00:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/navigator-5153-gt-insulated-overboots-cleated-outsole-xl-15h-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:00:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/peugeot-vivacity-100cc-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 03:00:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-bags-2-mil-5-6-1000-ctn>: HTTP status code is not handled or not allowed 2026-01-25 03:00:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-r-turning-tool-holder>: HTTP status code is not handled or not allowed 2026-01-25 03:00:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carport-patio-cover-kit-hg9100-arcadia-series-16-6l-x-11-1w>: HTTP status code is not handled or not allowed 2026-01-25 03:00:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-stp-stl-all-direct-safet-roll-lad-wt-actu-24-expan-stp-waad123214x>: HTTP status code is not handled or not allowed 2026-01-25 03:00:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-welded-workbench-60-x-24-lower-shelf--1-drawer-steel-square-edge>: HTTP status code is not handled or not allowed 2026-01-25 03:00:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-heated-softshell-jacket-m-black>: HTTP status code is not handled or not allowed 2026-01-25 03:00:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-stp-stl-safet-stair-slp-roll-lad-wt-actu-24-perf-stp-wa-sw53214p>: HTTP status code is not handled or not allowed 2026-01-25 03:00:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zap-3-pro-flex-48v-electric-scooter-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 03:00:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-stp-stl-safet-stair-slp-roll-lad-wt-actu-24-serr-stp-wa-sw73214g>: HTTP status code is not handled or not allowed 2026-01-25 03:00:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drive-medical-hawk-mobility-scooter-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 03:00:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 55879 pages (at 96 pages/min), scraped 27832 items (at 28 items/min) 2026-01-25 03:00:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/air_conditioners_chillers/window_air_conditioners already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:00:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/board-container-placard-label-holder-ipx-1-4-holes-blank-5-1-2-x-7-7-8>: HTTP status code is not handled or not allowed 2026-01-25 03:00:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/invacare-pronto-m50-mobility-scooter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 03:00:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mini-soil-locker-llmini-gosv-silvervein-16-1-2-x-16-1-4-x-39-1-2>: HTTP status code is not handled or not allowed 2026-01-25 03:00:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slope-top-for-16-1-2-wide-lockers-1p17011-gosv-silvervein-16-1-2-x-16-1-4-x-5-1-2>: HTTP status code is not handled or not allowed 2026-01-25 03:00:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dalton-4-wheel-scooter-sc-s145hd20ar-mobility-scooter-battery-35ah-12v>: HTTP status code is not handled or not allowed 2026-01-25 03:00:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelate-42w-x-14d-x-80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 03:00:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/air_conditioners_chillers/window_air_conditioners>: HTTP status code is not handled or not allowed 2026-01-25 03:00:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/econo-strip-heavy-duty-1-12-x-16l>: HTTP status code is not handled or not allowed 2026-01-25 03:00:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xdu1266-cantilever-rack-double-sided-upright-66d-x-12h>: HTTP status code is not handled or not allowed 2026-01-25 03:00:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelate-60w-x-36d-x-60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 03:00:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelate-30w-x-21d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 03:00:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-nexelate-48w-x-21d-x-60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 03:00:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beaded-chain-3-ball-6l>: HTTP status code is not handled or not allowed 2026-01-25 03:00:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelate-24w-x-18d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 03:00:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-legend-700-697cc-snowmobile-replacement-battery-2002-2003-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 03:00:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bombardier-mx-z-x-600-595cc-snowmobile-replacement-battery-2009-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 03:00:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vets-3630-37-x-30-18-ga-equipment-stand-galvanized-base>: HTTP status code is not handled or not allowed 2026-01-25 03:00:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-mx-z-550f-550cc-snowmobile-replacement-battery-2003-2008-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 03:00:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mm12-154-db-retractable-awning-manual-12-w-x-10-d-x-10-h-dusty-blue>: HTTP status code is not handled or not allowed 2026-01-25 03:00:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-wheel-adjustable-sheet-and-panel-truck-60-x-36-deck-size-3600-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-25 03:00:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-black-epoxy-24w-x-21d-x-60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 03:00:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-black-epoxy-36w-x-21d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 03:00:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vy-sled-base-black>: HTTP status code is not handled or not allowed 2026-01-25 03:00:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-a45-4l470-v-belt-1-2-x-47>: HTTP status code is not handled or not allowed 2026-01-25 03:00:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-footrest>: HTTP status code is not handled or not allowed 2026-01-25 03:00:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-b37-5l400-v-belt-5-8-x-40>: HTTP status code is not handled or not allowed 2026-01-25 03:00:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8263frhl-hi-vis-fr-safety-vest-class-2-mesh-hook-loop-economy-2xl-3xl-lime>: HTTP status code is not handled or not allowed 2026-01-25 03:00:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-poly-green-42w-x-14d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 03:00:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-b89-5l920-v-belt-5-8-x-92>: HTTP status code is not handled or not allowed 2026-01-25 03:00:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3000-air-evacuation-tool-lt3000-330-scfm-55-barrel>: HTTP status code is not handled or not allowed 2026-01-25 03:01:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machine-vac-replacement-gasket-mv2000g>: HTTP status code is not handled or not allowed 2026-01-25 03:01:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/supply-126-2ft-mixed-lamp-recycling-box>: HTTP status code is not handled or not allowed 2026-01-25 03:01:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-lockout-padlock-keyed-different-1-1-2-plastic-nylon-green-6-pk>: HTTP status code is not handled or not allowed 2026-01-25 03:01:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-lockout-padlock-keyed-alike-1-1-2-plastic-nylon-orange-6-pk>: HTTP status code is not handled or not allowed 2026-01-25 03:01:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pump-motor-jl1307a-1-phase-075-hp-115-230-volts-1725-rpm-60-hz-open-56j>: HTTP status code is not handled or not allowed 2026-01-25 03:01:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rackguard-4-w-x-2-11-16-d-x-23-1-2-h-yellow>: HTTP status code is not handled or not allowed 2026-01-25 03:01:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/as-40x-1000x-monocular-microscope-1-3mp-digital-camera-6-pc-tweezers-set-100-pc-slide-set>: HTTP status code is not handled or not allowed 2026-01-25 03:01:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/as-40x-1000x-monocular-microscope-25-pc-slides-set-tweezers-plus-usb-digital-microscope>: HTTP status code is not handled or not allowed 2026-01-25 03:01:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelite-72w-x-18d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 03:01:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 55972 pages (at 93 pages/min), scraped 27886 items (at 54 items/min) 2026-01-25 03:01:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-motor-230-460-v-15-hp-1755-rpm-3-ph-145tc-opsb>: HTTP status code is not handled or not allowed 2026-01-25 03:01:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-12-outlet-power-strip-9-ft-cord-14-3-lighted-switch-15a-1875-watts-etl>: HTTP status code is not handled or not allowed 2026-01-25 03:01:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/222-glass-vina-stemless-red-wine-1675-oz-12-pack>: HTTP status code is not handled or not allowed 2026-01-25 03:01:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_microscopes/stereo_microscopes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:01:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hvac-motor-ehm3714t-3-ph-10-hp-208-230-460-v-1770-rpm-tefc-215t-frame>: HTTP status code is not handled or not allowed 2026-01-25 03:01:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/severe-duty-motor-vecp2333t-3-ph-15-hp-230-460-v-1765-rpm-tefc-254tc-frame>: HTTP status code is not handled or not allowed 2026-01-25 03:01:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-motor-230-460-v-20-hp-1765-rpm-3-ph-256t-opsb>: HTTP status code is not handled or not allowed 2026-01-25 03:01:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-phase-motor-cem4104t-5-30-hp-1765-rpm-286tc-frame-c-face-mount-tefc-575-volts>: HTTP status code is not handled or not allowed 2026-01-25 03:01:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-black-epoxy-72w-x-24d-x-69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 03:01:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelite-48w-x-24d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 03:01:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-beam-flashlight-275-lumens-black-wr>: HTTP status code is not handled or not allowed 2026-01-25 03:01:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-phase-motor-em4406t-5-1785-hp-150-rpm-445t-frame-foot-mount-tefc-575-volts>: HTTP status code is not handled or not allowed 2026-01-25 03:01:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-phase-motor-em4410t-5-1785-hp-125-rpm-444t-frame-foot-mount-tefc-575-volts>: HTTP status code is not handled or not allowed 2026-01-25 03:01:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/whistler-cap-style-hard-hat-hdpe-shell-vented-4-pt-textile-suspension-ratchet-adjustment-white>: HTTP status code is not handled or not allowed 2026-01-25 03:01:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_microscopes/stereo_microscopes>: HTTP status code is not handled or not allowed 2026-01-25 03:01:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kilimanjaro-full-brim-hard-hat-hdpe-shell-4-pt-nylon-webbing-cradle-ratchet-adjustment-white>: HTTP status code is not handled or not allowed 2026-01-25 03:01:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pip-squeaks-telescoping-marker-tower-assorted-colors-50-set>: HTTP status code is not handled or not allowed 2026-01-25 03:01:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washdown-motor-cswdm3539-3-phase-05-hp-1165-rpm-208-230-460-volts-tenv-56c-fr>: HTTP status code is not handled or not allowed 2026-01-25 03:01:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washdown-motor-idcswdm3558t-3-phase-2-hp-1750-rpm-230-460-volts-tenv-145tc-fr>: HTTP status code is not handled or not allowed 2026-01-25 03:01:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/light-duty-steel-wheel-875-r-8x175-2-1-2-centered-hub-1-2bb-rib-tread>: HTTP status code is not handled or not allowed 2026-01-25 03:01:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/log-splitter-wheel-hs408b-1ia-480-8-1-bearing-lrb>: HTTP status code is not handled or not allowed 2026-01-25 03:01:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-motor-230-460-v-100-hp-1800-rpm-3-ph-404t-dp>: HTTP status code is not handled or not allowed 2026-01-25 03:01:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foam-sheet-for-benchmixer-xl-xlq-no-holes>: HTTP status code is not handled or not allowed 2026-01-25 03:01:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1002412-00u0019-gondola-shelving-24w-x-12d-almond>: HTTP status code is not handled or not allowed 2026-01-25 03:01:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anodized-aluminum-resettable-3-dial-combination-lock-14530-c-yellow-price-per-pack>: HTTP status code is not handled or not allowed 2026-01-25 03:01:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quick-flip-block-24-x-1-5ml-or-32-x-0-2ml-14-x-0-5ml-tubes>: HTTP status code is not handled or not allowed 2026-01-25 03:01:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2002w-6-personal-fan-2-speed-110v-white>: HTTP status code is not handled or not allowed 2026-01-25 03:01:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adapter-for-z366-series-50ml-round-2-pk>: HTTP status code is not handled or not allowed 2026-01-25 03:01:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hvac-motor-em3611t-g-3-ph-3-hp-208-230-460-v-1760-rpm-tefc-182t-frame>: HTTP status code is not handled or not allowed 2026-01-25 03:01:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylon-brush-for-7-16-inch-id-tubes-15-pack>: HTTP status code is not handled or not allowed 2026-01-25 03:01:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelon-48w-x-24d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 03:01:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machined-chrome-steel-needle-roller-bearing-w-inner-ring-40-x-62-x-40mm>: HTTP status code is not handled or not allowed 2026-01-25 03:01:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-nexelon-24w-x-14d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 03:01:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machined-chrome-steel-needle-roller-bearing-w-inner-ring-17-x-30-x-13mm>: HTTP status code is not handled or not allowed 2026-01-25 03:01:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hvac-motor-em4106t-g-3-ph-20-hp-230-460-v-3600-rpm-tefc-256t-frame>: HTTP status code is not handled or not allowed 2026-01-25 03:01:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2300sz-direct-drive-zig-zag-sewing-machine>: HTTP status code is not handled or not allowed 2026-01-25 03:01:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machined-chrome-steel-needle-roller-bearing-w-inner-ring-90-x-125-x-35mm>: HTTP status code is not handled or not allowed 2026-01-25 03:01:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-poly-green-24wx21dx60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 03:01:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-poly-green-30wx18dx80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 03:01:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-nexelon-30w-x-18d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 03:01:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dt-dx024rd-life-ring-w-tape-uscg-tc-orange-24>: HTTP status code is not handled or not allowed 2026-01-25 03:01:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/solid-deck-pallet-dolly-w-fork-pockets-pds-40-6phfp2fl-3600-lb-cap-48x40>: HTTP status code is not handled or not allowed 2026-01-25 03:02:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ninja-bnf-gloves-15-gauge-nylon-nitrile-coated-palm-fingertips-w-dots-gray-black-l>: HTTP status code is not handled or not allowed 2026-01-25 03:02:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/gloves_sleeves/cut_resistant_gloves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:02:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelon-54w-x-24d-x-80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 03:02:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ed-401-electric-knife-sharpener-removable-guides-115v>: HTTP status code is not handled or not allowed 2026-01-25 03:02:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/awn-cn44-5g-window-entry-awning-5-4-1-2-w-x-4d-x-4-8-h-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:02:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/d-flange-kit-12ep1501a03sp-tefc-324-326-ut-nema-frame>: HTTP status code is not handled or not allowed 2026-01-25 03:02:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 56066 pages (at 94 pages/min), scraped 27933 items (at 47 items/min) 2026-01-25 03:02:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1210-x-1-1-x-19-1210-series-tapered-locking-steel-bushing-1-bore>: HTTP status code is not handled or not allowed 2026-01-25 03:02:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelon-42w-x-24d-x-60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 03:02:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/gloves_sleeves/cut_resistant_gloves>: HTTP status code is not handled or not allowed 2026-01-25 03:02:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rpm-ac-mounting-conversion-kit-417077-186-tenv-tebc-tefc-for-fl210>: HTTP status code is not handled or not allowed 2026-01-25 03:02:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e-x-1-9-16-1-9-16-x-6-e-series-quick-detach-steel-bushing-1-9-16-bore>: HTTP status code is not handled or not allowed 2026-01-25 03:02:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelon-54w-x-18d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 03:02:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelon-54w-x-18d-x-69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 03:02:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vortex-tube-3298-large-100-scrm-6800-btuhr>: HTTP status code is not handled or not allowed 2026-01-25 03:02:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/peanut-chute-w-cap-plastic>: HTTP status code is not handled or not allowed 2026-01-25 03:02:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelon-36w-x-21d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 03:02:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelon-24w-x-14d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 03:02:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/celltreat-scientific-products-15l-pipette-mla-precision-silver->: HTTP status code is not handled or not allowed 2026-01-25 03:02:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/celltreat-scientific-products-5l-pipette-ovation-f1-fixed-volume-one-stroke>: HTTP status code is not handled or not allowed 2026-01-25 03:02:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/knives-heavy-medium-weight-polystyrene-100-box-white>: HTTP status code is not handled or not allowed 2026-01-25 03:02:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/we-mc400-11875-vhs-alternate-coupling-bore-size-11875-frame-404tp-or-405tp>: HTTP status code is not handled or not allowed 2026-01-25 03:02:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glass-board-white-96-x-48>: HTTP status code is not handled or not allowed 2026-01-25 03:02:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/checkers-guard-dog-5-channel-ada-cable-protectors-gd5x125-ada>: HTTP status code is not handled or not allowed 2026-01-25 03:02:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rocacrylic-3800-acrylic-enamel-paint-5-gallon-pail-gloss-white>: HTTP status code is not handled or not allowed 2026-01-25 03:02:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/celltreat-scientific-products-220l-pipette-ovation-quickset-qs-yellow>: HTTP status code is not handled or not allowed 2026-01-25 03:02:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cordless-compact-pistol-grip-drill-driver-tool-12-keyless-chuck-1800-rpm>: HTTP status code is not handled or not allowed 2026-01-25 03:02:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manway-gasket-spiral-wound-11-x-15-x-1516-0175-thick-elliptical-pk-3>: HTTP status code is not handled or not allowed 2026-01-25 03:02:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manway-gasket-spiral-wound-11-x-15-x-114-0175-thick-elliptical-pk-3>: HTTP status code is not handled or not allowed 2026-01-25 03:02:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/squids-3386-arm-band-idbadge-holder-one-size-red>: HTTP status code is not handled or not allowed 2026-01-25 03:02:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/controls-merz-z756-6-ac-change-over-switch-no-zero-pos-3-pole-100a-4-hole-front-mount>: HTTP status code is not handled or not allowed 2026-01-25 03:02:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/topoge-series-180-handhole-gasket-234-x-334-x-12-black-rubber-obround-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 03:02:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jurni-multi-purpose-table-with-post-legs--glides-72l-x-24w-x-29h-columbian-walnut>: HTTP status code is not handled or not allowed 2026-01-25 03:02:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mm110w-one-gang-16-in-1-weatherproof-white>: HTTP status code is not handled or not allowed 2026-01-25 03:02:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/topoge-series-180-manway-gasket-10-x-16-x-58-black-rubber-elliptical-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 03:02:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cep-6506g-50a-temporary-power-box-twist-lock>: HTTP status code is not handled or not allowed 2026-01-25 03:02:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cep-6220pdc45-200-amp-3-ph-cart-w-45kva-transformer>: HTTP status code is not handled or not allowed 2026-01-25 03:02:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/princeton-tec-pistol-grip-spotlight-handheld-light-1600-lumens-210m-beam-distance-neon-yellow>: HTTP status code is not handled or not allowed 2026-01-25 03:02:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:02:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tx4000-esd-cleanroom-apparel-soft-sole-boot-cover-s-navy-blue>: HTTP status code is not handled or not allowed 2026-01-25 03:02:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-bolt-clamp-hc260-3-1-4-inch-nominal-dia-303-inch-min-dia-334-inch-max-dia>: HTTP status code is not handled or not allowed 2026-01-25 03:02:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/bins_totes_containers/organizers_caddies/drawer_organizers_dividers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:02:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3m153-quiet-space-headphones-bluetooth-black>: HTTP status code is not handled or not allowed 2026-01-25 03:02:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ma517-auxiliary-lens-05x-working-distance-150mm>: HTTP status code is not handled or not allowed 2026-01-25 03:02:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3wtbk-11116-tapered-bushing-kit-1-1116-fits-reducer-styles-smr3wsmr3>: HTTP status code is not handled or not allowed 2026-01-25 03:02:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats landed on page that is not a product page. 2026-01-25 03:02:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ediusa-mich-ach-style-ballistic-helmet-level-iiia-ballistic-resistance-medium-black>: HTTP status code is not handled or not allowed 2026-01-25 03:03:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ediusa-highcut-ballistic-helmet-with-side-rails-front-mount-xlarge-green>: HTTP status code is not handled or not allowed 2026-01-25 03:03:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/bins_totes_containers/organizers_caddies/drawer_organizers_dividers>: HTTP status code is not handled or not allowed 2026-01-25 03:03:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/techstar-next-generation-starcarts-bulk-trucks-31-1-2wx60dx35-1-2h-blue>: HTTP status code is not handled or not allowed 2026-01-25 03:03:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durham-dividers-for-compartment-boxes-fits-box-5223000>: HTTP status code is not handled or not allowed 2026-01-25 03:03:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/male-branch-tee-h3609x4-1-4-inch-x-1-4-inch-male-to-female>: HTTP status code is not handled or not allowed 2026-01-25 03:03:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 56162 pages (at 96 pages/min), scraped 27987 items (at 54 items/min) 2026-01-25 03:03:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wesco-work-positioning-post-lift-table-foot-control-48x32-4000-lb-cap-987726>: HTTP status code is not handled or not allowed 2026-01-25 03:03:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-crema-espresso-coffee-regular-22-lb-bag-vacuum-packed-with-one-way-valve>: HTTP status code is not handled or not allowed 2026-01-25 03:03:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:03:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/worldwide-electric-winl37-15-1-145tc-helical-inline-speed-reducer-145tc-input-flange-15-1-ratio>: HTTP status code is not handled or not allowed 2026-01-25 03:03:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/worldwide-electric-winl37-30-1-145tc-helical-inline-speed-reducer-145tc-input-flange-30-1-ratio>: HTTP status code is not handled or not allowed 2026-01-25 03:03:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4000-4-ball-valve-flanged-4-cast-iron>: HTTP status code is not handled or not allowed 2026-01-25 03:03:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/worldwide-electric-winl47-5-1-145tc-helical-inline-speed-reducer-145tc-input-flange-5-1-ratio>: HTTP status code is not handled or not allowed 2026-01-25 03:03:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hooded-fleece-sweater-14-25-oz-flame-resistant-l-navy>: HTTP status code is not handled or not allowed 2026-01-25 03:03:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pneumatic-air-valve-bav015-1-4-inch-5-port-3-position-detented-4-way-air-valve>: HTTP status code is not handled or not allowed 2026-01-25 03:03:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:03:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hydraulic-spreader-valve-hv715sae-valve-only-7-15-gpm-83-1-4-lpm-2000-psi-140-bar>: HTTP status code is not handled or not allowed 2026-01-25 03:03:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:03:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/snap-button-down-long-sleve-work-shirt-5xl-tall-black>: HTTP status code is not handled or not allowed 2026-01-25 03:03:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:03:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-racksolid-steel-deck-52-w-x-24-d-797214>: HTTP status code is not handled or not allowed 2026-01-25 03:03:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:03:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-visibility-athletic-performance-t-shirt-flame-resistant-6-25-oz-5xl-orange>: HTTP status code is not handled or not allowed 2026-01-25 03:03:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/foam/foam_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:03:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts landed on page that is not a product page. 2026-01-25 03:03:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clark-foam-products-1001022-foam-sheet-anti-static-poly-1-x-12x12-in>: HTTP status code is not handled or not allowed 2026-01-25 03:03:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 03:03:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ds-177031w-shrink-wrap-17w-x-31l-7mil-white>: HTTP status code is not handled or not allowed 2026-01-25 03:03:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:03:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts>: HTTP status code is not handled or not allowed 2026-01-25 03:03:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:03:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts>: HTTP status code is not handled or not allowed 2026-01-25 03:03:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5451800-one-piece-handle-24>: HTTP status code is not handled or not allowed 2026-01-25 03:03:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:03:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts landed on page that is not a product page. 2026-01-25 03:03:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-vis-full-zip-hooded-sweater-reflective-xl-tall-yellow>: HTTP status code is not handled or not allowed 2026-01-25 03:03:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/monitor_risers_stands already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:03:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:03:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-visibility-athletic-performance-t-shirt-flame-resistant-6-oz-s-yellow>: HTTP status code is not handled or not allowed 2026-01-25 03:03:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:03:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-locker-accessory-individual-slope-top-12w-x-12d-x-4h-725-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:03:34 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts landed on page that is not a product page. 2026-01-25 03:03:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/truck_boxes_utility_boxes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:03:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts>: HTTP status code is not handled or not allowed 2026-01-25 03:03:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/truck_boxes_utility_boxes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:03:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/underbed-truck-bx-black-steel-standard-4-5-cu-ft-524-5-02>: HTTP status code is not handled or not allowed 2026-01-25 03:03:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pack-rat-4-compart-drawer-white-48-l-x-39-34-w-x-12-12-h-338-3>: HTTP status code is not handled or not allowed 2026-01-25 03:03:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/monitor_risers_stands>: HTTP status code is not handled or not allowed 2026-01-25 03:03:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts>: HTTP status code is not handled or not allowed 2026-01-25 03:03:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/underbed-truck-bx-black-steel-compact-2-3-cu-ft-525-5-02>: HTTP status code is not handled or not allowed 2026-01-25 03:03:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pack-rat-1-compart-drawer-white-48-l-x-10-14-w-x-12-12-h-335-3>: HTTP status code is not handled or not allowed 2026-01-25 03:03:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts>: HTTP status code is not handled or not allowed 2026-01-25 03:03:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-28w-x-34l-tan>: HTTP status code is not handled or not allowed 2026-01-25 03:03:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats landed on page that is not a product page. 2026-01-25 03:03:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/truck_boxes_utility_boxes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:03:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e11935000-34hp-1800rpm-56c-odp-115230v-1ph-60hz-cont-40c-125sf-c-face-rigid>: HTTP status code is not handled or not allowed 2026-01-25 03:03:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/truck_boxes_utility_boxes landed on page that is not a product page. 2026-01-25 03:03:44 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/foam/foam_sheets landed on page that is not a product page. 2026-01-25 03:03:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/truck_boxes_utility_boxes>: HTTP status code is not handled or not allowed 2026-01-25 03:03:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:03:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-42w-x-31l-navy>: HTTP status code is not handled or not allowed 2026-01-25 03:03:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-36w-x-34l-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:03:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:03:49 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-25 03:03:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-38w-x-30l-black>: HTTP status code is not handled or not allowed 2026-01-25 03:03:50 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:28 2026-01-25 03:03:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:03:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5449200-36x18-utility-cart>: HTTP status code is not handled or not allowed 2026-01-25 03:03:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/truck_boxes_utility_boxes landed on page that is not a product page. 2026-01-25 03:03:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/activity-tables-60-l-x-30-w-standard-height-rectangular-walnut>: HTTP status code is not handled or not allowed 2026-01-25 03:03:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:03:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:03:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:03:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 03:03:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 03:03:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:03:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5652100-sase-shlvg-48x24x63-add-on>: HTTP status code is not handled or not allowed 2026-01-25 03:04:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 03:04:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5652000-sase-shlvg-48x24x63-starter>: HTTP status code is not handled or not allowed 2026-01-25 03:04:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-29w-x-29l-black>: HTTP status code is not handled or not allowed 2026-01-25 03:04:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:04:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/le9025123qb2-25ft-123-sjow-cable-cord-reel-w-20a-quad-box>: HTTP status code is not handled or not allowed 2026-01-25 03:04:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 03:04:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 03:04:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classic-pro-ctl-bandsaw-blade-14-10-long-x-1-14-wide-34-tpi-x-0042-thick>: HTTP status code is not handled or not allowed 2026-01-25 03:04:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/le9050123be-50ft-123-sjow-cable-cord-reel-w-20-a-bare-ends>: HTTP status code is not handled or not allowed 2026-01-25 03:04:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wesco-telescopic-folding-handle-steel-platform-truck-29x19-4-rubber-casters-987775>: HTTP status code is not handled or not allowed 2026-01-25 03:04:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 03:04:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lewiston-mailbox-post-ornate-base-pineapple-finial-support-brace-no-address-plates-bronze>: HTTP status code is not handled or not allowed 2026-01-25 03:04:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 03:04:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bayview-outdoor-solar-lamp>: HTTP status code is not handled or not allowed 2026-01-25 03:04:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 56239 pages (at 77 pages/min), scraped 28004 items (at 17 items/min) 2026-01-25 03:04:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lewiston-mailbox-post-fluted-base-ball-finial-w-3-address-plates-support-brace-black>: HTTP status code is not handled or not allowed 2026-01-25 03:04:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-inch-drive-dial-torque-wrench-400-2000-ft-lbs-56-280-mkg>: HTTP status code is not handled or not allowed 2026-01-25 03:04:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 03:04:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-42w-x-31l-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:04:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-inch-drive-socket-7-8-inch-12-point-1-9-16-inch-long>: HTTP status code is not handled or not allowed 2026-01-25 03:04:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-46w-x-33l-navy>: HTTP status code is not handled or not allowed 2026-01-25 03:04:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-piece-magnetic-screwdriver-bit-setlong-reach>: HTTP status code is not handled or not allowed 2026-01-25 03:04:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:04:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/450hs-34-inch-roller-cabinet-8-drawer-blue-34-inch-l-x-41-inch-h-x-25-inchd>: HTTP status code is not handled or not allowed 2026-01-25 03:04:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/shelf_trucks/job_site_security_trucks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:04:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/copper-mechanical-split-bolt-connection-for-2-conductors-range-of-equal-run-and-tap-16-str-8-str>: HTTP status code is not handled or not allowed 2026-01-25 03:04:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:04:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-set-3>: HTTP status code is not handled or not allowed 2026-01-25 03:04:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-42w-x-34l-black>: HTTP status code is not handled or not allowed 2026-01-25 03:04:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:04:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 03:04:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:04:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:04:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/carts-trucks/shelf_trucks/job_site_security_trucks>: HTTP status code is not handled or not allowed 2026-01-25 03:04:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:04:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hk1412-needle-bearing-drawn-cup-caged-metric-bore-13995mm>: HTTP status code is not handled or not allowed 2026-01-25 03:04:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 03:04:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-52w-x-33l-navy>: HTTP status code is not handled or not allowed 2026-01-25 03:04:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 03:04:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 03:04:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 03:04:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/em31155-2hp-3600rpm-56-frame-3ph-230-460v-odp-rigid-premium-efficiency>: HTTP status code is not handled or not allowed 2026-01-25 03:04:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/math-subtraction-0-12-flash-cards-3-inch-x-6-inch-91-cards-box>: HTTP status code is not handled or not allowed 2026-01-25 03:04:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-32w-x-32l-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:04:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-52w-x-33l-navy>: HTTP status code is not handled or not allowed 2026-01-25 03:04:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:04:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mounted-stainless-steel-waste-receptacle-11-gallon-326>: HTTP status code is not handled or not allowed 2026-01-25 03:04:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:04:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 03:04:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powr-flite-premium-comfort-pro-backpack-vacuum10-quart-261716>: HTTP status code is not handled or not allowed 2026-01-25 03:04:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/relaxed-fit-jeans-36w-x-30l-blue>: HTTP status code is not handled or not allowed 2026-01-25 03:04:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-30w-x-30l-navy>: HTTP status code is not handled or not allowed 2026-01-25 03:04:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:04:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stocktank-feedbunk-w-ser-75-g-blue>: HTTP status code is not handled or not allowed 2026-01-25 03:04:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:04:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 03:04:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/desktop-tape-dispenser-c60-bk-black-two-tone-1-pack>: HTTP status code is not handled or not allowed 2026-01-25 03:04:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:04:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 03:04:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ctl-bandsaw-blade-11-foot-6-inch-long-x-3-4-inch-wide-14-tpi-x-0035-thick>: HTTP status code is not handled or not allowed 2026-01-25 03:04:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ctl-bandsaw-blade-14-foot-6-inch-long-x-3-4-inch-wide-6-8-tpi-x-0035-thick>: HTTP status code is not handled or not allowed 2026-01-25 03:04:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commercial-grade-5-drawer-letter-vertical-file-llr48499-15-w-x-26-1-2-d-x-61-h-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:04:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:04:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/69000-series-rectangular-desk-shell-llr69903-48-w-x-30-d-x-29-1-2-cherry>: HTTP status code is not handled or not allowed 2026-01-25 03:04:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 03:04:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:04:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-back-executive-chair-llr62620-bonded-leather-black>: HTTP status code is not handled or not allowed 2026-01-25 03:04:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dgi-dwl3500xy-usb-2-axis-high-precision-digital-machinist-level>: HTTP status code is not handled or not allowed 2026-01-25 03:04:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:04:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-work-coveralls-54-tall-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:04:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:05:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-28w-x-34l-navy>: HTTP status code is not handled or not allowed 2026-01-25 03:05:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 03:05:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-jeans-indura-denim-14-oz-38w-x-30l-flame-resistant-blue>: HTTP status code is not handled or not allowed 2026-01-25 03:05:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 03:05:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 03:05:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-westex-30w-x-32l-black>: HTTP status code is not handled or not allowed 2026-01-25 03:05:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 03:05:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-42w-x-32l-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:05:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:05:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-50w-x-34l-dark-navy>: HTTP status code is not handled or not allowed 2026-01-25 03:05:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 03:05:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-work-coveralls-42-regular-red>: HTTP status code is not handled or not allowed 2026-01-25 03:05:09 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 03:05:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 56325 pages (at 86 pages/min), scraped 28038 items (at 34 items/min) 2026-01-25 03:05:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:05:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/automotive-black-rtv-silicone-12270-103-oz-cartridge>: HTTP status code is not handled or not allowed 2026-01-25 03:05:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-jeans-indura-denim-14-oz-31w-x-32l-flame-resistant-blue>: HTTP status code is not handled or not allowed 2026-01-25 03:05:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-visibility-ripstop-cargo-pants-32w-x-34l-navy>: HTTP status code is not handled or not allowed 2026-01-25 03:05:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-jeans-indura-denim-14-oz-31w-x-34l-flame-resistant-blue>: HTTP status code is not handled or not allowed 2026-01-25 03:05:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 03:05:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-vinyl-door-bottom-insert-06304-36-inch-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:05:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vmt-500-leveling-compression-mount-1-2-13x6-bolt-500-lbs>: HTTP status code is not handled or not allowed 2026-01-25 03:05:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 03:05:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-work-pants-reflective-material-flame-resistant-38w-x-unhemmed-red>: HTTP status code is not handled or not allowed 2026-01-25 03:05:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-profile-door-jamb-weatherstrip-kit-69923-white-36-inch-x-84-inch>: HTTP status code is not handled or not allowed 2026-01-25 03:05:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shrink-and-seal-indoor-window-kit-04184-crystal-clear-42-inch-x-62-inch>: HTTP status code is not handled or not allowed 2026-01-25 03:05:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galvanized-steel-sheet-56020-24-inchl-x-12-inchw-x-28-ga>: HTTP status code is not handled or not allowed 2026-01-25 03:05:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-westex-42w-x-34l-tan>: HTTP status code is not handled or not allowed 2026-01-25 03:05:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_repair_maintenance/fuel_exhaust_systems/tank_repair_sealant already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:05:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cargo-pants-with-double-reinforced-knees-flame-resistant-42w-x-unhemmed-navy>: HTTP status code is not handled or not allowed 2026-01-25 03:05:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wallbase-dry-back-73896-60-footl-x-4-inchw-black>: HTTP status code is not handled or not allowed 2026-01-25 03:05:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-westex-44w-x-32l-black>: HTTP status code is not handled or not allowed 2026-01-25 03:05:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-wire-shelving-54-inch-w-x-24-inch-d-x-63-inch-h>: HTTP status code is not handled or not allowed 2026-01-25 03:05:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/adhesives_sealants_tape/adhesives_glue/construction_adhesives already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:05:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-wire-shelving-add-on-42-inch-w-x-24-inch-d-x-63-inch-h>: HTTP status code is not handled or not allowed 2026-01-25 03:05:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/plumbing_valves/ball_valves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:05:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:05:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pl4082-plus-barrier-set-15-yellow-caution-banner>: HTTP status code is not handled or not allowed 2026-01-25 03:05:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:05:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:05:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_repair_maintenance/fuel_exhaust_systems/tank_repair_sealant>: HTTP status code is not handled or not allowed 2026-01-25 03:05:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:05:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-30w-x-14d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 03:05:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-decor-siliconized-hi-heat-enamel-flat-finish-black-quart-597776>: HTTP status code is not handled or not allowed 2026-01-25 03:05:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/adhesives_sealants_tape/adhesives_glue/construction_adhesives landed on page that is not a product page. 2026-01-25 03:05:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:05:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/plumbing/plumbing_valves/ball_valves>: HTTP status code is not handled or not allowed 2026-01-25 03:05:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 03:05:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-9a-3ph-remote-start-reset-button-24-60v-3-1-4-2a>: HTTP status code is not handled or not allowed 2026-01-25 03:05:42 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 03:05:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 03:05:44 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 03:05:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pc-horizontal-knob-65504-marked-with-degrees-f-for-pc-t155-and-t167-series-thermostats>: HTTP status code is not handled or not allowed 2026-01-25 03:05:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 03:05:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-12a-3ph-remote-start-start-stop-100-250v-10-13a>: HTTP status code is not handled or not allowed 2026-01-25 03:05:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/office_machines/calculators/display_calculators already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:05:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nestable-plastic-shelf-bin-4-3-8w-x-23-5-8d-x-8h-red>: HTTP status code is not handled or not allowed 2026-01-25 03:05:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_cabinets_drawers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:05:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-12a-3ph-remote-start-reset-button-24-60v-5-7-7-6a>: HTTP status code is not handled or not allowed 2026-01-25 03:05:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hurricane-fabric-dbl-slide-patio-door-hf88156-resincoated-ballistic-nyl-88l-x-156w>: HTTP status code is not handled or not allowed 2026-01-25 03:05:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-16a-3ph-start-stop-24-60v-16-20a>: HTTP status code is not handled or not allowed 2026-01-25 03:05:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-round-table-ladder-back-bar-ht-set-figured-mahogany-laminate-table-black-vinyl-chair>: HTTP status code is not handled or not allowed 2026-01-25 03:05:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-locker-polycarbonate-door-keypad-lcrl-1w3t-pc1s-02-31-a-3-tier-1-wide-12x18x82-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:05:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8945-0-deg-flex-coiljet-flexible-wand>: HTTP status code is not handled or not allowed 2026-01-25 03:05:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/office_machines/calculators/display_calculators landed on page that is not a product page. 2026-01-25 03:05:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/r24blhda-basket-truck>: HTTP status code is not handled or not allowed 2026-01-25 03:06:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_cabinets_drawers>: HTTP status code is not handled or not allowed 2026-01-25 03:06:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/512fsl-6-person-floor-mount-wash-station-with-double-foot-pedal-valves>: HTTP status code is not handled or not allowed 2026-01-25 03:06:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/457010-hex-cap-screw-1-4-20-x-1-carbon-steel-zinc-grade-5-ft-unc-usa-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 03:06:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deep-groove-ball-bearings-metric-6308-open-heavy-duty-40mm-bore-90mm-od>: HTTP status code is not handled or not allowed 2026-01-25 03:06:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deep-groove-ball-bearings-metric-6311-zz-2-metal-shields-heavy-duty-55mm-bore-120mm-od>: HTTP status code is not handled or not allowed 2026-01-25 03:06:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 56413 pages (at 88 pages/min), scraped 28080 items (at 42 items/min) 2026-01-25 03:06:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/screws/socket_screws/socket_head_cap_screws already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:06:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rhino-1-white-permanent-poly-labels>: HTTP status code is not handled or not allowed 2026-01-25 03:06:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/nuts/lock_nuts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:06:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/mailboxes_drop_boxes/residential_mailboxes_components already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:06:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/d648pb-osha-sign-danger-no-smoking-or-open-flame-10-x-14-whiteredblack>: HTTP status code is not handled or not allowed 2026-01-25 03:06:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:06:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/329410-hex-jam-nut-3-8-24-low-carbon-steel-unf-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 03:06:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-38a-1ph-remote-start-start-stop-24-60v-20-24a>: HTTP status code is not handled or not allowed 2026-01-25 03:06:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-52a-3ph-start-stop-24-60v-30-40a>: HTTP status code is not handled or not allowed 2026-01-25 03:06:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/nuts/lock_nuts landed on page that is not a product page. 2026-01-25 03:06:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c391rb-osha-sign-caution-hard-hats-required-10-x-14-yellowblack>: HTTP status code is not handled or not allowed 2026-01-25 03:06:21 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/mailboxes_drop_boxes/residential_mailboxes_components landed on page that is not a product page. 2026-01-25 03:06:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e30c-ss-extension-spring-375-od-x-0348-wire-dia-175-l-163-lbs-in-302-ss>: HTTP status code is not handled or not allowed 2026-01-25 03:06:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs>: HTTP status code is not handled or not allowed 2026-01-25 03:06:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tm5g-traffic-sign-reserved-parking-18-x-12-whiteblack>: HTTP status code is not handled or not allowed 2026-01-25 03:06:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tma1h-traffic-sign-van-accessible-6-x-12-whiteblue>: HTTP status code is not handled or not allowed 2026-01-25 03:06:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-65a-3ph-start-stop-24-60v-36-47a>: HTTP status code is not handled or not allowed 2026-01-25 03:06:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/travira-chat-chair-natural-sling-tekwood-vintage-armcaps-2pack>: HTTP status code is not handled or not allowed 2026-01-25 03:06:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n206rb-osha-sign-notice-hard-hat-safety-glasses-required-10-x-14-whiteblueblack>: HTTP status code is not handled or not allowed 2026-01-25 03:06:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-oscillating-pedestal-fan-24-diameter-1-2hp>: HTTP status code is not handled or not allowed 2026-01-25 03:06:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/screws/socket_screws/socket_head_cap_screws landed on page that is not a product page. 2026-01-25 03:06:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bluff-174-20cr8460-forkli-container-ramp-84-x-60-20-000-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 03:06:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-65a-1ph-direct-online-reset-button-100-250v-50-60a>: HTTP status code is not handled or not allowed 2026-01-25 03:06:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/areca-palm-with-bamboo-vase-silk-plant>: HTTP status code is not handled or not allowed 2026-01-25 03:06:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vsm-abrasive-belt-1807-aluminum-oxide-2-x-60-100-grit>: HTTP status code is not handled or not allowed 2026-01-25 03:06:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dieffenbachia-with-wood-vase-silk-plant-set-of-2-variegated>: HTTP status code is not handled or not allowed 2026-01-25 03:06:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelate-wire-shelving-36-w-x-18-d-x-86-h>: HTTP status code is not handled or not allowed 2026-01-25 03:06:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelate-wire-shelving-add-on-48-w-x-24-d-x-86-h>: HTTP status code is not handled or not allowed 2026-01-25 03:06:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/front-discharge-window-evaporative-cooler-6700-cfm-115v>: HTTP status code is not handled or not allowed 2026-01-25 03:06:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lx-stackable-classroom-stool-w-wheels-storage-269-lbs-capacity-white-powder-coat-paint-finish>: HTTP status code is not handled or not allowed 2026-01-25 03:06:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steam-basic-steam-cleaner-with-handpiece-4-5-bar-steam-pressure-115-v>: HTTP status code is not handled or not allowed 2026-01-25 03:06:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/razor-lightweight-headset-motorola-blackbox-hyt-relm-tekk-radios>: HTTP status code is not handled or not allowed 2026-01-25 03:06:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/intermediate-ul-recognized-nylon-cable-tie-8-l-x-1-8-w-blue-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 03:06:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-profile-iv-42-indoor-ceiling-fan-51059-white>: HTTP status code is not handled or not allowed 2026-01-25 03:07:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cw-10-hand-held-weapons-detector>: HTTP status code is not handled or not allowed 2026-01-25 03:07:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pioneer-motegi-3mt160-bg-single-lever-bathroom-faucet-with-brass-pop-up-pvd-brushed-gold>: HTTP status code is not handled or not allowed 2026-01-25 03:07:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nord-lock-1103-wedge-locking-washer-316-stainless-steel-m12-pkg-of-200>: HTTP status code is not handled or not allowed 2026-01-25 03:07:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nord-lock-2779-wedge-locking-washer-carbon-steel-delta-protekt-coated-3-4-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 03:07:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-21-1-4-jumbo-crowfoot-wrench-97740-forged-alloy-steelpolished>: HTTP status code is not handled or not allowed 2026-01-25 03:07:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/53-osha-stairway-stair36o-6-53>: HTTP status code is not handled or not allowed 2026-01-25 03:07:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-rack-guard-double-end-black>: HTTP status code is not handled or not allowed 2026-01-25 03:07:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/21-299a-surform-10-inch-half-round-blade-standard-cut-bulk>: HTTP status code is not handled or not allowed 2026-01-25 03:07:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 56506 pages (at 93 pages/min), scraped 28135 items (at 55 items/min) 2026-01-25 03:07:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aqua-shield-on-demand-stainless-union-circulator-for-single-pipe-systems>: HTTP status code is not handled or not allowed 2026-01-25 03:07:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-72-inch-w-x-24-inch-d-x-74-inch-h-2000-lbs-cap-4-open-grid-shelves-dunnage-stand>: HTTP status code is not handled or not allowed 2026-01-25 03:07:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-42-inch-w-x-24-inch-d-x-74-inch-h-2000-lbs-cap-4-open-and-1-solid-grid-shelves>: HTTP status code is not handled or not allowed 2026-01-25 03:07:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keyless-rigid-coupling-c600e325-steel-325d-x-6693d-314l-shaft>: HTTP status code is not handled or not allowed 2026-01-25 03:07:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compressor-l800051-model-lpv6546a-single-stage-compressor-pump-2-cylinder-15-4-hp>: HTTP status code is not handled or not allowed 2026-01-25 03:07:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pillow-block-bronze-bearing-pbpsbr062-press-steel-housing-selfalign-58-bore>: HTTP status code is not handled or not allowed 2026-01-25 03:07:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-sided-self-supporting-tank-hs-1800-135-dia-x-46h-1800-gallon-cap-green>: HTTP status code is not handled or not allowed 2026-01-25 03:07:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-frame-folding-tank-alf-1000-103l-x-103w-x-29h-1000-gallon-cap-yellow>: HTTP status code is not handled or not allowed 2026-01-25 03:07:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-30-inch-w-x-24-inch-d-x-86-inch-h-2000-lbs-cap-4-open-and-1-solid-grid-shelves>: HTTP status code is not handled or not allowed 2026-01-25 03:07:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-42-inch-w-x-21-inch-d-x-74-inch-h-2000-lbs-cap-4-solid-shelves>: HTTP status code is not handled or not allowed 2026-01-25 03:07:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-sided-self-supporting-tank-hs-5000-198-dia-x-60h-5000-gallon-cap-blue>: HTTP status code is not handled or not allowed 2026-01-25 03:07:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-36-inch-w-x-18-inch-d-x-74-inch-h-2000-lbs-cap-4-open-and-1-solid-grid-shelves>: HTTP status code is not handled or not allowed 2026-01-25 03:07:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/winding-arbor-wa5642-steel-0032w-slot-332od>: HTTP status code is not handled or not allowed 2026-01-25 03:07:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-36-inch-w-x-18-inch-d-x-74-inch-h-2000-lbs-cap-4-open-grid-shelves>: HTTP status code is not handled or not allowed 2026-01-25 03:07:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f5e072aa-72-aluminum-extension-and-nozzle-for-force-5>: HTTP status code is not handled or not allowed 2026-01-25 03:07:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-30-inch-w-x-24-inch-d-x-62-inch-h-2000-lbs-cap-4-solid-shelves>: HTTP status code is not handled or not allowed 2026-01-25 03:07:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kni-cobra-water-pump-plier-w-polished-head-plastic-coated-handle-22-inch-l>: HTTP status code is not handled or not allowed 2026-01-25 03:07:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/wire_shelving/poly_zbrite_wire_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:07:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gun-safe-1-hr-fire-2-story-impact-rating-digital-lock-22w-x-20d-x-59h>: HTTP status code is not handled or not allowed 2026-01-25 03:07:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/basic-stanchion-385h-wrinkle-black-post-7l-blue-retractable-belt>: HTTP status code is not handled or not allowed 2026-01-25 03:07:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/38-modular-frl-wo-end-ports-d33453-gauge-poly-bowl-manual>: HTTP status code is not handled or not allowed 2026-01-25 03:07:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/weekly-pill-dispenser>: HTTP status code is not handled or not allowed 2026-01-25 03:07:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/wire_shelving/poly_zbrite_wire_shelves landed on page that is not a product page. 2026-01-25 03:07:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/direct-fired-gas-heater-3050-3500000-btu-208240480v-ductable>: HTTP status code is not handled or not allowed 2026-01-25 03:07:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sae-metric-combination-wrench-set-1200fc-6-12-pt-1-4-3-4-8-15mm-opening-sizes-16-pcs>: HTTP status code is not handled or not allowed 2026-01-25 03:07:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-deck-48-d-x-34-w>: HTTP status code is not handled or not allowed 2026-01-25 03:07:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/hand_tools_1/chisels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:07:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rb4005-9-w-sublite-athletic-work-shoe-steel-toe-men-s-size-9>: HTTP status code is not handled or not allowed 2026-01-25 03:07:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flex-combination-wrench-set-1270hf-3-8-3-4-7-piece-set>: HTTP status code is not handled or not allowed 2026-01-25 03:07:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/open-shelf-unit-36w-x-15d-x-29h-birch-plywood>: HTTP status code is not handled or not allowed 2026-01-25 03:07:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/biopsy-punches-20-mm-ivory-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-25 03:07:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-fold-n-lock-mat-storage-18w-x-58d-x-48h-birch-plywood>: HTTP status code is not handled or not allowed 2026-01-25 03:07:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kid-coat-locker-4-wide-39w-x-15d-x-50-1-2h-gray-laminate-black-edge>: HTTP status code is not handled or not allowed 2026-01-25 03:07:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/hand_tools_1/chisels>: HTTP status code is not handled or not allowed 2026-01-25 03:07:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tm511k-traffic-sign-one-way-arrow-right-24-x-18-white>: HTTP status code is not handled or not allowed 2026-01-25 03:07:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-crook-handle-cane-w-vinyl-grip-6-pairs>: HTTP status code is not handled or not allowed 2026-01-25 03:08:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elongated-folding-bedside-elongated-commode-seat-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-25 03:08:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-work-height-platform-truck-30x48-mold-on-rubber-wheels-lip-edge>: HTTP status code is not handled or not allowed 2026-01-25 03:08:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-welded-service-cart-2-lip-shelves-24-x-48-2000-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-25 03:08:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/work-height-platform-truck-w-hardboard-top-24-x-60>: HTTP status code is not handled or not allowed 2026-01-25 03:08:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tm249k-traffic-sign-large-arrow-one-direction-sign-12-x-18-yellow>: HTTP status code is not handled or not allowed 2026-01-25 03:08:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dynaride-s2-wheelchair-detachable-desk-arm-elevating-legrest-18w-seat>: HTTP status code is not handled or not allowed 2026-01-25 03:08:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/work-height-platform-truck-30x48-mold-on-rubber-wheels-lip-edge>: HTTP status code is not handled or not allowed 2026-01-25 03:08:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-inch-lip-edge-platform-truck-24-x-36>: HTTP status code is not handled or not allowed 2026-01-25 03:08:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/energy-saver-garage-attic-fan-ga-es-1500-120v-1500-cfm>: HTTP status code is not handled or not allowed 2026-01-25 03:08:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dynarest-airfloat-100f-air-mattress-w-foam-base-pump-80l-x-30w-x-8h>: HTTP status code is not handled or not allowed 2026-01-25 03:08:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 56602 pages (at 96 pages/min), scraped 28187 items (at 52 items/min) 2026-01-25 03:08:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-low-temperature-freezer-115v-abt-120v-2186-21-cu-ft>: HTTP status code is not handled or not allowed 2026-01-25 03:08:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-containment-berm-foam-wall-12-feet-x-30-feet-24-mil-pvc-black-green>: HTTP status code is not handled or not allowed 2026-01-25 03:08:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-low-profile-beveled-slimline-vision-lite-vsl2432b-24-x-32-bronze>: HTTP status code is not handled or not allowed 2026-01-25 03:08:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/massage-ball-8-cm-3-2-yellow-1-each>: HTTP status code is not handled or not allowed 2026-01-25 03:08:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-oz-gel-warmer-3-bottle-capacity-conduction-heat-for-e-1-e-2>: HTTP status code is not handled or not allowed 2026-01-25 03:08:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/small-in-ground-floor-safe-with-combo-lock-if-1212c-9-12-x-9-12-x-12-14-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:08:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/amscope-stand-shop-measuring-microscope-50x-magnification-w-pen-light>: HTTP status code is not handled or not allowed 2026-01-25 03:08:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/table-72w-x-36d-x-31-34h-with-4-chairs-natural-table-w-black-poly-chairs>: HTTP status code is not handled or not allowed 2026-01-25 03:08:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/shelves_cpu_holders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:08:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gel-mouse-pad-w-wrist-rest-ccs23718-11-x-9-black>: HTTP status code is not handled or not allowed 2026-01-25 03:08:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transfix-4-feetl-x-4-feetw-portable-stage-with-adjustable-24-32-inchh-and-red-carpet>: HTTP status code is not handled or not allowed 2026-01-25 03:08:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/tool_storage/chests_roller_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:08:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dvd-recordable-media-ver96542-dvdr-dl-media-8x-speed-850-gb-capcity>: HTTP status code is not handled or not allowed 2026-01-25 03:08:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10000-currency-band-pmc55010-yellow-1000-box>: HTTP status code is not handled or not allowed 2026-01-25 03:08:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:08:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fl-slip-on-boat-shoe-smooth-leather-cognac-11d>: HTTP status code is not handled or not allowed 2026-01-25 03:08:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/shelves_cpu_holders landed on page that is not a product page. 2026-01-25 03:08:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbk-athletic-work-shoe-dual-layer-stretch-mesh-upper-black-dark-grey-14m>: HTTP status code is not handled or not allowed 2026-01-25 03:08:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_viewers_sight_glasses already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:08:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/75549-depressed-center-grinding-wheel-metal-grinding-6-x-1-4-arbor-7-8>: HTTP status code is not handled or not allowed 2026-01-25 03:08:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/tool_storage/chests_roller_cabinets landed on page that is not a product page. 2026-01-25 03:08:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jaw-safety-clip-4211s-78l-x-1332w-x-58h-for-10-ton-puller>: HTTP status code is not handled or not allowed 2026-01-25 03:08:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbk-athletic-work-shoe-ultraknit-flexweave-grey-white-11m>: HTTP status code is not handled or not allowed 2026-01-25 03:08:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26220-cobalt-drill-bit-135-5-16-x-4-1-2>: HTTP status code is not handled or not allowed 2026-01-25 03:08:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss316-66030x20-3x2-class-150-hex-bushing-stainless-steel-316>: HTTP status code is not handled or not allowed 2026-01-25 03:08:42 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-25 03:08:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_viewers_sight_glasses landed on page that is not a product page. 2026-01-25 03:08:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbk-athletic-work-shoe-ultraknit-flexweave-grey-white-15m>: HTTP status code is not handled or not allowed 2026-01-25 03:08:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbk-athletic-work-shoe-ultraknit-flexweave-black-10-5m>: HTTP status code is not handled or not allowed 2026-01-25 03:08:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-electric-hi-low-upholstered-mat-platform-table-96l-x-72w-x-19-27h>: HTTP status code is not handled or not allowed 2026-01-25 03:08:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:08:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pneumatic-mobile-stool-without-back-18-22h-black>: HTTP status code is not handled or not allowed 2026-01-25 03:08:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glassless-mirror-stationary-with-stand-vertical-36w-x-72h>: HTTP status code is not handled or not allowed 2026-01-25 03:08:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:08:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ia-11-inch-pull-on-workboot-leather-brown-10m>: HTTP status code is not handled or not allowed 2026-01-25 03:08:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ia-11-inch-pull-on-workboot-leather-brown-7-5w>: HTTP status code is not handled or not allowed 2026-01-25 03:08:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/freestanding-office-partition-panel-24-1-4-w-x-72-h-blue>: HTTP status code is not handled or not allowed 2026-01-25 03:08:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ml-a1106-anodized-aluminum-safety-padlock-1-1-2-inchw-1-1-2-inchtall-shackle-keyed-alike-blue>: HTTP status code is not handled or not allowed 2026-01-25 03:08:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-25 03:08:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ml-a1106-anodized-aluminum-safety-padlock-1-1-2-inchw-1-1-2-inchtall-shackle-keyed-alike-purple>: HTTP status code is not handled or not allowed 2026-01-25 03:08:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c1300-hot-water-dispenser-w-lever-handle>: HTTP status code is not handled or not allowed 2026-01-25 03:08:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-25 03:08:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commercial-products-pad-holder-qc-frame-hall-dusting-35-89cm-aluminum-yellow>: HTTP status code is not handled or not allowed 2026-01-25 03:09:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u1286-5a-premium-locker-five-tier-12x18x12-5-doors-assembled-parchment>: HTTP status code is not handled or not allowed 2026-01-25 03:09:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ironcat-30-irontex-flame-retardant-cotton-jacket-green-2xl-all-cotton>: HTTP status code is not handled or not allowed 2026-01-25 03:09:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-chrome-2-tier-wire-shelving-add-on-unit-36w-x-18d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 03:09:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/800-series-solid-door-wardrobe-cabinet-48wx18dx78h-midnight-ebony-assembled>: HTTP status code is not handled or not allowed 2026-01-25 03:09:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/400-series-solid-door-wardrobe-cabinet-48wx24dx72h-parchment-assembled>: HTTP status code is not handled or not allowed 2026-01-25 03:09:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u3282-1me-black-tie-locker-single-tier-12x18x72-3-doors-unassembled-black>: HTTP status code is not handled or not allowed 2026-01-25 03:09:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 56697 pages (at 95 pages/min), scraped 28239 items (at 52 items/min) 2026-01-25 03:09:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-chrome-2-tier-wire-shelving-add-on-unit-42w-x-24d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 03:09:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enclosed-power-relay-rib24p-20a-dpdt-24vac-dc>: HTTP status code is not handled or not allowed 2026-01-25 03:09:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compact-double-spring-sign-stand-for-roll-up-signs-36-inch-and-48-inch-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 03:09:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/panel-i-o-expander-ribmn24q2c-275-15a-2-spdt-24vac-dc-0-5vdc-control-w-mt212-4>: HTTP status code is not handled or not allowed 2026-01-25 03:09:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbwp-r-b-wire-front-load-wire-laundry-cart-double-pole-rack-2-25-bushel-chrome>: HTTP status code is not handled or not allowed 2026-01-25 03:09:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-gauge-straight-finish-nail-2-length-304-stainless-steel-pkg-of-25000-made-in-usa>: HTTP status code is not handled or not allowed 2026-01-25 03:09:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbwp-antimicrobial-large-capacity-laundry-cart-w-single-pole-rack>: HTTP status code is not handled or not allowed 2026-01-25 03:09:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbwp-8-bushel-antimicrobial-vinyl-basket-truck-all-swivel-casters-red>: HTTP status code is not handled or not allowed 2026-01-25 03:09:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reflective-marking-tape-white-1w-x-150l-roll>: HTTP status code is not handled or not allowed 2026-01-25 03:09:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tt-heel-grounder-125-cup-stretch-velcro-1-meg-blue>: HTTP status code is not handled or not allowed 2026-01-25 03:09:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/bulk_container_trucks_tilt_trucks/basket_trucks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:09:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbwp-16-bushel-antimicrobial-basket-all-swivel-casters-yellow>: HTTP status code is not handled or not allowed 2026-01-25 03:09:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-hydraulic-truck-axle-jack-w-3pc-extension-kit-22-ton-capacity>: HTTP status code is not handled or not allowed 2026-01-25 03:09:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/little-giant-174-aerial-safety-cage-8-14-w-click-casters-18515-240>: HTTP status code is not handled or not allowed 2026-01-25 03:09:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exterior-wall-mount-8-centrifugal-duct-fan-rvf8xl-115v-435-cfm>: HTTP status code is not handled or not allowed 2026-01-25 03:09:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ajc-expertpower-exp645-6v-4-5ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-25 03:09:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/carts-trucks/bulk_container_trucks_tilt_trucks/basket_trucks>: HTTP status code is not handled or not allowed 2026-01-25 03:09:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pf-pulsar-gamma-heated-carpet-extractor-15-gallon-220-psi>: HTTP status code is not handled or not allowed 2026-01-25 03:09:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ce-8-inch-high-level-cleaning-pipe-brush-w-safety-locking-black>: HTTP status code is not handled or not allowed 2026-01-25 03:09:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cha-anesthesia-cart-w-compact-height-electronic-lock-green>: HTTP status code is not handled or not allowed 2026-01-25 03:09:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/run-capacitor-55-plus-5-mfd-440v-round>: HTTP status code is not handled or not allowed 2026-01-25 03:09:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/run-capacitor-70-plus-5-mfd-440v-round>: HTTP status code is not handled or not allowed 2026-01-25 03:09:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-kensington-bench-wo-back-vertical-slat-black>: HTTP status code is not handled or not allowed 2026-01-25 03:09:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ajc-belkin-550va-f6h550-usb-12v-5ah-ups-battery>: HTTP status code is not handled or not allowed 2026-01-25 03:09:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ajc-power-patrol-sec1050-12v-4-5ah-ups-battery>: HTTP status code is not handled or not allowed 2026-01-25 03:09:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-x-2-pinnacle-thermoplastic-wheel-with-roller-bearing-for-1-2-axle-1000-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-25 03:09:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ajc-apc-backups-es-be650g1-12v-9ah-ups-battery>: HTTP status code is not handled or not allowed 2026-01-25 03:09:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ansi-grade-2-mechanical-lockset-ez-code-change-code-free-passage-silver-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:09:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hybrid-waterfree-urinal-hyb-4000>: HTTP status code is not handled or not allowed 2026-01-25 03:09:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-full-cantilever-jib-crane-4000-lb-capacity-14-span-wide-flange-beam>: HTTP status code is not handled or not allowed 2026-01-25 03:09:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/51-7-nd-31-b-retaining-magnet-assembly-w-key-ring-w-rubber-jacket-1-22-dia-steel>: HTTP status code is not handled or not allowed 2026-01-25 03:09:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/23in-value-plus-plunger-yellow-handle-gjo85130>: HTTP status code is not handled or not allowed 2026-01-25 03:09:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/liquid-hand-soap-with-skin-conditioner-1-gallon-4-case-gjo02105ct>: HTTP status code is not handled or not allowed 2026-01-25 03:09:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/501l-05-model-501l-with-flow-rate-of-05-gpm>: HTTP status code is not handled or not allowed 2026-01-25 03:09:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wern-proform-f3-climbing-positioning-harness-w-quick-connect-legs-xl>: HTTP status code is not handled or not allowed 2026-01-25 03:09:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/56f1-05-model-56f1-with-flow-rate-of-05-gpm>: HTTP status code is not handled or not allowed 2026-01-25 03:09:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/225584-24-slot-mascara-cosmetic-tray-for-pegboard-slatwall-12-x-1-875-acrylic>: HTTP status code is not handled or not allowed 2026-01-25 03:10:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wern-max-patrol-11-feet-web-thermoplastic-housing-self-retracting-lifeline-w-steel-form-hook>: HTTP status code is not handled or not allowed 2026-01-25 03:10:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700220-gre-12-pegboard-revolving-countertop-display-4-sided-green>: HTTP status code is not handled or not allowed 2026-01-25 03:10:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/laboratory_heating_cooling/laboratory_hotplates_stirrers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:10:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700451-yel-three-sided-spinning-pegboard-w-wheels-16-x-60-yellow-opaque>: HTTP status code is not handled or not allowed 2026-01-25 03:10:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/430-piece-wood-screw-assortment-4-to-10-slotted-flat-round-head-zinc>: HTTP status code is not handled or not allowed 2026-01-25 03:10:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 56791 pages (at 94 pages/min), scraped 28293 items (at 54 items/min) 2026-01-25 03:10:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lyn-1-tier-3-doors-locker-45-inch-w-x-18-inch-d-x-78-inch-h-gray-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 03:10:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700509-wht-two-sided-non-revolving-pegboard-countertop-display-8-x-20-white>: HTTP status code is not handled or not allowed 2026-01-25 03:10:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keystock-1-2-x-5-8-x-1-ft-high-carbon-steel-plain-bilateral-astm-a29>: HTTP status code is not handled or not allowed 2026-01-25 03:10:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700350-clr-pegboard-powerwing-13-75-x-44-clear>: HTTP status code is not handled or not allowed 2026-01-25 03:10:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700350-gre-pegboard-powerwing-13-75-x-44-green>: HTTP status code is not handled or not allowed 2026-01-25 03:10:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mrp-rigid-drum-insert-smooth-55-gallon-18-mil>: HTTP status code is not handled or not allowed 2026-01-25 03:10:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inplant-office-vinyl-cov-hdbd-8x10-3-wall-2-lgts-2-dplx-1-door-white>: HTTP status code is not handled or not allowed 2026-01-25 03:10:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/laboratory_heating_cooling/laboratory_hotplates_stirrers landed on page that is not a product page. 2026-01-25 03:10:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/703388-wht-4-sided-interlocking-pegboard-display-w-wheels-8-x-40-white-solid>: HTTP status code is not handled or not allowed 2026-01-25 03:10:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/igg-gage-block-set-shop-grade-nist-certified-81-pieces>: HTTP status code is not handled or not allowed 2026-01-25 03:10:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inplant-office-vinyl-cov-hdbd-8x10-4-wall-2-lgts-2-dplx-1-door-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:10:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/770820-pnk-pegboard-wall-panel-8-x-20-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 03:10:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hygrade-heavy-duty-super-cover-poly-tarp-sth810-10-mil-silverblack-8l-x-10w>: HTTP status code is not handled or not allowed 2026-01-25 03:10:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inplant-office-pntd-steel-drywall-16x16-3-wall-5-lgt-4-dplx-1-dr-wht>: HTTP status code is not handled or not allowed 2026-01-25 03:10:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/highperformance-bench-grinder-brush-dw4904-6-diameter-5-8-1-2-arbor-6000-rpm>: HTTP status code is not handled or not allowed 2026-01-25 03:10:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aware-wear-s721-non-ansi-female-vest-61913-pink-2x-large>: HTTP status code is not handled or not allowed 2026-01-25 03:10:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electric-office-partition-panel-with-partial-window-60-1-4w-x-76h-blue>: HTTP status code is not handled or not allowed 2026-01-25 03:10:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-series-swivel-plate-caster-663739-advanced-elastomer-2-1-2-dia-400-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 03:10:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-plastic-barrier-chain-2-inch-x-50-feet-l-slate-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:10:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-plastic-barrier-chain-in-a-pail-1-1-2-inch-x-300-feet-l-turquoise>: HTTP status code is not handled or not allowed 2026-01-25 03:10:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esd-open-bin-box-6w-x-18d-x-4-1-2h>: HTTP status code is not handled or not allowed 2026-01-25 03:10:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-step-aluminum-rolling-ladder-16inw-ribbed-tread-21ind-top-step-32in-handrails>: HTTP status code is not handled or not allowed 2026-01-25 03:10:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-2-1-2-inch-medium-duty-reflective-stanchion-kit-w-2-inch-x-50-feet-l-chain-yellow-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 03:10:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-step-aluminum-rolling-ladder-24inw-grip-tread-21ind-top-step-w-o-handrails>: HTTP status code is not handled or not allowed 2026-01-25 03:10:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-28-inch-reflective-traffic-cone-kit-w-2-inch-x-50-feet-l-chain-traffic-blue-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 03:10:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carboy-w-handle-screw-cap-107950000-polyethylene-20-l-1-pk-white>: HTTP status code is not handled or not allowed 2026-01-25 03:10:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-series-rigid-plate-caster-405108839-solid-polyolefin-5dia-650-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 03:10:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pip-latex-coated-cotton-gloves-x-small-12-pairs-pack>: HTTP status code is not handled or not allowed 2026-01-25 03:10:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/nuts/hex_nuts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:10:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-24-x-2-1-2-12-point-flange-screw-170m-psi-steel-plain-partial-thread-unf-pkg-of-10>: HTTP status code is not handled or not allowed 2026-01-25 03:10:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-exercise-putty-red-soft-1-pound>: HTTP status code is not handled or not allowed 2026-01-25 03:10:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/bolts/flange_bolts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:10:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/antimicrobial-exercise-putty-yellow-4-ounce-x-soft>: HTTP status code is not handled or not allowed 2026-01-25 03:10:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-wide-designer-wood-locker-23364-triple-tier-3-wide-15x24x24-maple-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 03:10:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ehs18x-nsf-14x10-ehs18x-nsf-14x10>: HTTP status code is not handled or not allowed 2026-01-25 03:10:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/fasteners/nuts/hex_nuts>: HTTP status code is not handled or not allowed 2026-01-25 03:11:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-eva-foam-roller-round-6-dia-x-12l>: HTTP status code is not handled or not allowed 2026-01-25 03:11:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/physical_therapy/exercise_fitness_equipment/resistance_devices already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:11:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/bolts/flange_bolts landed on page that is not a product page. 2026-01-25 03:11:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hand-exerciser-small-yellow-x-light>: HTTP status code is not handled or not allowed 2026-01-25 03:11:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/explosion-proof-blower-9515-01-16-dia-1-2hp-2900-cfm>: HTTP status code is not handled or not allowed 2026-01-25 03:11:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/loaded-gear-vx-200-tactical-right-hand-vest-bi12332-od-green>: HTTP status code is not handled or not allowed 2026-01-25 03:11:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 56886 pages (at 95 pages/min), scraped 28349 items (at 56 items/min) 2026-01-25 03:11:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-ton-cap-aluminum-gantry-crane-8-ft-span-adj-height-6-1-min-8-7-max>: HTTP status code is not handled or not allowed 2026-01-25 03:11:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-174-commercial-microwave-oven-0-9-cu-1000-watts-touch-control-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 03:11:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/physical_therapy/exercise_fitness_equipment/resistance_devices landed on page that is not a product page. 2026-01-25 03:11:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/plumbing_drains_traps/drains_traps/floor_drains_cleanouts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:11:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6in-x-1-2in-dralon-woven-mini-roller-covers-2-pack-12-case-6wv050d>: HTTP status code is not handled or not allowed 2026-01-25 03:11:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-part-reeved-cross-parallel-mount-assembly-model-33337902>: HTTP status code is not handled or not allowed 2026-01-25 03:11:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3350e-v-blk-versa-upholstered-esd-vinyl-stool-aluminum-base-black>: HTTP status code is not handled or not allowed 2026-01-25 03:11:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/shop_presses already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:11:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/907003-75h-75-ton-hand-hydraulic-press>: HTTP status code is not handled or not allowed 2026-01-25 03:11:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hydronic-expansion-tank-pfext2h-2-gallons>: HTTP status code is not handled or not allowed 2026-01-25 03:11:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/plumbing/plumbing_drains_traps/drains_traps/floor_drains_cleanouts>: HTTP status code is not handled or not allowed 2026-01-25 03:11:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/17in-x-30in-pear-shaped-landing-net-metallic-green-sw66680>: HTTP status code is not handled or not allowed 2026-01-25 03:11:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lawn-garden-tire-410-350-6-2-ply-stud>: HTTP status code is not handled or not allowed 2026-01-25 03:11:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sewn-liquid-bag-filter-nylon-mesh-4-1-8-dia-x-14-l-150-micron-plastic-flange-pkg-qty-50>: HTTP status code is not handled or not allowed 2026-01-25 03:11:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8ab-x-1-security-self-drilling-screw-pan-torx-head-410-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 03:11:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/welded-liquid-bag-filter-polyester-felt-7-1-8-x-32-100-micron-plastic-sure-seal-ring-pkg-50>: HTTP status code is not handled or not allowed 2026-01-25 03:11:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/liquid-bag-filter-polyester-felt-4-1-8-dia-x-8-l-25-micron-steel-ring-pkg-qty-50>: HTTP status code is not handled or not allowed 2026-01-25 03:11:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/shop_presses landed on page that is not a product page. 2026-01-25 03:11:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/liquid-bag-filter-nylon-mesh-8-3-8-dia-x-36-l-800-micron-steel-ring-pkg-qty-50>: HTTP status code is not handled or not allowed 2026-01-25 03:11:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/liquid-bag-filter-poly-p-felt-7-3-50-dia-x-32-l-25-micron-plastic-sure-seal-ring-pkg-qty-50>: HTTP status code is not handled or not allowed 2026-01-25 03:11:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sewn-liquid-bag-filter-polyester-multifilament-6-7-8-x-34-1500-micron-steel-ring-pkg-50>: HTTP status code is not handled or not allowed 2026-01-25 03:11:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22-1-2-x-17-1-2-x-3-clear-view-dividable-grid-containers>: HTTP status code is not handled or not allowed 2026-01-25 03:11:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-freestanding-office-partition-panel-with-partial-window-48-1-4-w-x-43-1-2-h-blue>: HTTP status code is not handled or not allowed 2026-01-25 03:11:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/office-partition-panel-with-partial-window-and-raceway-60-1-4w-x-46h-blue>: HTTP status code is not handled or not allowed 2026-01-25 03:11:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-tamperproof-phillips-pin-head-bit>: HTTP status code is not handled or not allowed 2026-01-25 03:11:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-elbow-8mm-tube-x-1-8-female>: HTTP status code is not handled or not allowed 2026-01-25 03:11:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-pc-short-paddle-with-out-holes-red>: HTTP status code is not handled or not allowed 2026-01-25 03:11:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pt100-external-probe-0-400c-stainless-steel-1-pack>: HTTP status code is not handled or not allowed 2026-01-25 03:11:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-epoxy-wire-shelving-add-on-189345gz-54w-x-24d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 03:11:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fancy-filled-candy-jar-assortment-5-lb-gr>: HTTP status code is not handled or not allowed 2026-01-25 03:11:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-wall-reclosable-seal-top-specimen-trans-bag-blue-tint-12w-15l>: HTTP status code is not handled or not allowed 2026-01-25 03:11:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cel-gro-tissue-culture-rotator-drum-holds-60-30mm-tubes>: HTTP status code is not handled or not allowed 2026-01-25 03:11:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-48-w-x-24-d-x-84-h-starter-3-shelf-levels-wood-deck-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:11:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:12:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-96-w-x-36-d-x-84-h-starter-3-shelf-levels-steel-deck-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:12:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:12:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/300ml-erlenmeyer-flask-clamp-30155-for-use-with-maxq-shaker-platforms>: HTTP status code is not handled or not allowed 2026-01-25 03:12:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:12:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-48-w-x-36-d-x-120-h-starter-3-shelf-levels-wood-deck-sand>: HTTP status code is not handled or not allowed 2026-01-25 03:12:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compact-digital-dry-bath-block-heater-single-block-capacity-100-240v>: HTTP status code is not handled or not allowed 2026-01-25 03:12:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-white-plastic-set-of-letters-numbers-symbols-car0902>: HTTP status code is not handled or not allowed 2026-01-25 03:12:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-48-w-x-48-d-x-72-h-add-on-3-shelf-levels-steel-deck-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:12:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks landed on page that is not a product page. 2026-01-25 03:12:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 56980 pages (at 94 pages/min), scraped 28405 items (at 56 items/min) 2026-01-25 03:12:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-48-w-x-48-d-x-72-h-add-on-3-shelf-levels-steel-deck-sand>: HTTP status code is not handled or not allowed 2026-01-25 03:12:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks landed on page that is not a product page. 2026-01-25 03:12:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-96-w-x-48-d-x-120-h-starter-3-shelf-levels-wood-deck-sand>: HTTP status code is not handled or not allowed 2026-01-25 03:12:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks landed on page that is not a product page. 2026-01-25 03:12:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-series-wheel-500004339-ws-4-x-2-phenolic-1-2-straight-roller-bearing-black>: HTTP status code is not handled or not allowed 2026-01-25 03:12:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-series-wheel-200004544-4-x-1-1-4-polyolefin-3-8-delrin-bushing-black>: HTTP status code is not handled or not allowed 2026-01-25 03:12:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:12:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/decor_linens/classroom_decorations already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:12:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s1203sgaf-amber-1000-series-amber-anti-fog-lens-20ea-cs>: HTTP status code is not handled or not allowed 2026-01-25 03:12:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:12:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sirenlock-battery-operated-alarmed-panic-lock-18-push-paddle>: HTTP status code is not handled or not allowed 2026-01-25 03:12:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:12:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sunx-ct91664-spf30-sunscreen-single-dose-pouch-100-per-pack>: HTTP status code is not handled or not allowed 2026-01-25 03:12:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/homak-between-the-studs-wall-safe-storage-cabinet-60h>: HTTP status code is not handled or not allowed 2026-01-25 03:12:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:12:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks>: HTTP status code is not handled or not allowed 2026-01-25 03:12:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/decor_linens/classroom_decorations landed on page that is not a product page. 2026-01-25 03:12:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t5-high-output-fluorescent-lamp-f24t5830hoalto-24w-3000k>: HTTP status code is not handled or not allowed 2026-01-25 03:12:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks>: HTTP status code is not handled or not allowed 2026-01-25 03:12:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/security_access_control/safes_lockboxes/gun_cabinets_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:12:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:12:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks>: HTTP status code is not handled or not allowed 2026-01-25 03:12:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:12:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22614020p002-flexible-conveyor-poly-skate-wheels-steel-ball-bearings-226-lb-ft>: HTTP status code is not handled or not allowed 2026-01-25 03:12:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bookcases_hutches already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:12:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-72-w-x-36-d-x-72-h-starter-3-shelf-levels-wire-deck-medium-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:12:34 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks landed on page that is not a product page. 2026-01-25 03:12:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alera-aleva256624mc-valencia-series-credenza-shells-65w-x-23-35d-medium-cherry>: HTTP status code is not handled or not allowed 2026-01-25 03:12:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/security_access_control/safes_lockboxes/gun_cabinets_racks>: HTTP status code is not handled or not allowed 2026-01-25 03:12:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks landed on page that is not a product page. 2026-01-25 03:12:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-inch-x-100-foot-pvc-lay-flat-discharge-hose-coupled-with-mxf-aluminum-short-shanks>: HTTP status code is not handled or not allowed 2026-01-25 03:12:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/champion-sports-pg10-playground-ball-10-red>: HTTP status code is not handled or not allowed 2026-01-25 03:12:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks landed on page that is not a product page. 2026-01-25 03:12:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-inch-pvc-lay-flat-discharge-bulk-hose>: HTTP status code is not handled or not allowed 2026-01-25 03:12:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bookcases_hutches landed on page that is not a product page. 2026-01-25 03:12:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-inch-x-25-foot-mill-wash-down-discharge-hose-coupled-with-cxe-aluminum-cam-and-groove-fittings>: HTTP status code is not handled or not allowed 2026-01-25 03:12:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/credenzas already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:12:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-40-14w-mobile-cabinet-5-drawers-63-compart-bright-blueindividual-lock>: HTTP status code is not handled or not allowed 2026-01-25 03:12:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-40-14w-mobile-cabinet-5-drawers-63-compart-light-gray-keyed-alike>: HTTP status code is not handled or not allowed 2026-01-25 03:12:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-40-14w-cabinet-12-drawer-174-compart-light-gray-no-lock>: HTTP status code is not handled or not allowed 2026-01-25 03:12:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/digital_readout_equipment already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:12:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1171310-faucet-service-sink-for-zurn-industries-llc>: HTTP status code is not handled or not allowed 2026-01-25 03:12:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plain-steel-chrome-plated-display-basket-20-inch-l-x-14-inch-w-x-5-inch-h-qty-1-4>: HTTP status code is not handled or not allowed 2026-01-25 03:12:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/credenzas>: HTTP status code is not handled or not allowed 2026-01-25 03:12:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-40-14w-mobile-cabinet-6-drawers-84-compart-classic-blue-no-lock>: HTTP status code is not handled or not allowed 2026-01-25 03:12:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/machining/machinery/digital_readout_equipment>: HTTP status code is not handled or not allowed 2026-01-25 03:13:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-ah21s-stainless-steel-utility-cart-handle-21w>: HTTP status code is not handled or not allowed 2026-01-25 03:13:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interion-174-restaurant-lunchroom-bar-height-table-72-x-30-x-42-h-charcoal>: HTTP status code is not handled or not allowed 2026-01-25 03:13:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unisex-ciel-blue-cotton-polyester-fashion-blend-reversible-scrub-pants-size-xs>: HTTP status code is not handled or not allowed 2026-01-25 03:13:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ironwood-solid-door-wood-storage-cabinet-natural-oak-color-finish>: HTTP status code is not handled or not allowed 2026-01-25 03:13:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/cable_management/cable_tie_tools already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:13:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-replacement-cutter-head-536142-seal-bolt-cutter>: HTTP status code is not handled or not allowed 2026-01-25 03:13:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/cable_management/cable_ties_wraps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:13:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-powder-pre-cut-exercise-band-xx-heavy-silver-48-l-strip-box-of-40>: HTTP status code is not handled or not allowed 2026-01-25 03:13:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 57066 pages (at 86 pages/min), scraped 28450 items (at 45 items/min) 2026-01-25 03:13:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/wiring_devices/wire_connectors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:13:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/wire_cable/electrical_wire_cable already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:13:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/29373-59-aluminum-handle-blue>: HTTP status code is not handled or not allowed 2026-01-25 03:13:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/wiring_devices/wire_connectors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:13:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/31046-10-upright-broom-soft-stiff-yellow>: HTTP status code is not handled or not allowed 2026-01-25 03:13:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/cable_management/cable_tie_tools>: HTTP status code is not handled or not allowed 2026-01-25 03:13:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/321547-o-ring-3-8-inch-id-x-1-16-inch-width>: HTTP status code is not handled or not allowed 2026-01-25 03:13:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/cable_management/cable_ties_wraps landed on page that is not a product page. 2026-01-25 03:13:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/wiring_devices/wire_connectors>: HTTP status code is not handled or not allowed 2026-01-25 03:13:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/wire_cable/electrical_wire_cable landed on page that is not a product page. 2026-01-25 03:13:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-piece-heavy-duty-rainsuit-yellow-polyester-2xl>: HTTP status code is not handled or not allowed 2026-01-25 03:13:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/wiring_devices/wire_connectors landed on page that is not a product page. 2026-01-25 03:13:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vs-2954-versatile-ada-compliant-handheld-shower-system>: HTTP status code is not handled or not allowed 2026-01-25 03:13:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-1781-brass-burner-orifice-56-natural-gas-7-16-inch-27-thread>: HTTP status code is not handled or not allowed 2026-01-25 03:13:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/56886-1-5-gallon-bucket-yellow>: HTTP status code is not handled or not allowed 2026-01-25 03:13:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-resistant-wall-panel-marinite-i-fire-block-in-steel-shell-28-x-75>: HTTP status code is not handled or not allowed 2026-01-25 03:13:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-surface-mounted-single-tissue-dispenser-satin>: HTTP status code is not handled or not allowed 2026-01-25 03:13:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-cubicle-collection-surface-mounted-single-tissue-dispenser>: HTTP status code is not handled or not allowed 2026-01-25 03:13:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tote-box-heat-resistant-40-to-300-degrees-23>: HTTP status code is not handled or not allowed 2026-01-25 03:13:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/checkerboard-hazard-tape-yellow-black>: HTTP status code is not handled or not allowed 2026-01-25 03:13:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eathm10s1p-ind-composite-vibration-dampening-extended-air-hammer2100-bpm-16-cfm-14-inlet>: HTTP status code is not handled or not allowed 2026-01-25 03:13:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reflective-delineator-post-with-square-base-49-in-h-yellow>: HTTP status code is not handled or not allowed 2026-01-25 03:13:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eatdr05s1p-industrial-12-reversible-air-drill-6-cfm-14-inlet>: HTTP status code is not handled or not allowed 2026-01-25 03:13:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/es07v080v1-75hp-two-stage-piston-compressor-80-gallon-vertical-175-psi-1-phase-208-230v>: HTTP status code is not handled or not allowed 2026-01-25 03:13:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/741284-door-gasket-10-3-8-inch-x-21-7-8-inch-for-randell>: HTTP status code is not handled or not allowed 2026-01-25 03:13:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-w-x-36-d-production-workbench-shop-top-square-edge-with-drawers-shelf-blue>: HTTP status code is not handled or not allowed 2026-01-25 03:13:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-shelf-liner-42w-x-24d-white>: HTTP status code is not handled or not allowed 2026-01-25 03:13:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-welded-frame-mirror-w-shelf-18-w-x-36-h>: HTTP status code is not handled or not allowed 2026-01-25 03:13:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/folding-bulk-container-lid-3>: HTTP status code is not handled or not allowed 2026-01-25 03:13:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powered-24-w-x-10-l-belt-conveyor-with-6-h-side-rails>: HTTP status code is not handled or not allowed 2026-01-25 03:13:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powered-24-w-x-50-l-belt-conveyor-without-side-rails>: HTTP status code is not handled or not allowed 2026-01-25 03:13:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-anti-static-bags-6w-x-8l-4-mil-pink-pack-of-1000>: HTTP status code is not handled or not allowed 2026-01-25 03:13:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-poly-bags-12w-x-15l-15-mil-clear-pack-of-1000>: HTTP status code is not handled or not allowed 2026-01-25 03:13:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/70355-216-5-0-pipe-brush-medium-white>: HTTP status code is not handled or not allowed 2026-01-25 03:13:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-classicseries-liquidmate-wall-mounted-soap-dispenser-beige>: HTTP status code is not handled or not allowed 2026-01-25 03:13:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cubbie-storage-organizer-12-sections-charcoal>: HTTP status code is not handled or not allowed 2026-01-25 03:13:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/bollards/safety_bollards_posts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:13:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/262265-leak-limiting-device>: HTTP status code is not handled or not allowed 2026-01-25 03:13:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/120-clearance-bar-red-bar-white-tapes>: HTTP status code is not handled or not allowed 2026-01-25 03:13:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/overhead_clearance_bars_door_guards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:13:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/urinal-screen-with-non-para-block-green-apple-12-case-01902>: HTTP status code is not handled or not allowed 2026-01-25 03:14:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-shelf-liner-24w-x-24d-blue>: HTTP status code is not handled or not allowed 2026-01-25 03:14:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/bollards/safety_bollards_posts>: HTTP status code is not handled or not allowed 2026-01-25 03:14:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stronghold-ventilated-all-around-cabinet-36-x-24-x-78>: HTTP status code is not handled or not allowed 2026-01-25 03:14:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/ventilated_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:14:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-fiberglass-filter-12-w-x-25-h-x-1-d>: HTTP status code is not handled or not allowed 2026-01-25 03:14:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/overhead_clearance_bars_door_guards>: HTTP status code is not handled or not allowed 2026-01-25 03:14:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/wire_cable/electrical_wire_cable already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:14:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/100-rectorseal-duct-78l-x-4w-x-2-34h-ivory>: HTTP status code is not handled or not allowed 2026-01-25 03:14:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 57158 pages (at 92 pages/min), scraped 28500 items (at 50 items/min) 2026-01-25 03:14:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8755-replacement-liner-for-13-by-55-modular-model-berm-textured-tan>: HTTP status code is not handled or not allowed 2026-01-25 03:14:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/recessed-valve-box-stainless-8-x-8-x-4>: HTTP status code is not handled or not allowed 2026-01-25 03:14:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/cabinets/ventilated_cabinets>: HTTP status code is not handled or not allowed 2026-01-25 03:14:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stronghold-extra-wide-metal-bin-compartment-cabinet-77-x-36-x-72>: HTTP status code is not handled or not allowed 2026-01-25 03:14:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/wire_cable/electrical_wire_cable>: HTTP status code is not handled or not allowed 2026-01-25 03:14:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fir-plywood-premium-grade-crate-premium42-34-59-collapsible-inside-42l-x-34w-x-59>: HTTP status code is not handled or not allowed 2026-01-25 03:14:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fir-plywood-premium-grade-crate-premium94-46-95-collapsible-inside-94l-x-46w-x-95>: HTTP status code is not handled or not allowed 2026-01-25 03:14:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stronghold-heavy-duty-adjustable-metal-bin-compartment-cabinet-48-x-29-x-72>: HTTP status code is not handled or not allowed 2026-01-25 03:14:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/56897-1-5-gallon-bucket-lid-orange>: HTTP status code is not handled or not allowed 2026-01-25 03:14:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dinex-tulip-cupswirl-5-oz-96-cs-black-dxswc503>: HTTP status code is not handled or not allowed 2026-01-25 03:14:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/compartment_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:14:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/-replacement-media-set-for-apex153-700-8d>: HTTP status code is not handled or not allowed 2026-01-25 03:14:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:14:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-30-shop-top-work-bench-adjustable-height-1-1-2-top>: HTTP status code is not handled or not allowed 2026-01-25 03:14:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beverage-table-72-w-x-30-d>: HTTP status code is not handled or not allowed 2026-01-25 03:14:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/263783-leveler-swivel-base>: HTTP status code is not handled or not allowed 2026-01-25 03:14:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-rack-108x36x96-starter-unit-green-orange-5480-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-25 03:14:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:14:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medium-cube-ice-machine>: HTTP status code is not handled or not allowed 2026-01-25 03:14:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits>: HTTP status code is not handled or not allowed 2026-01-25 03:14:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-weight-bar-10-piece-set>: HTTP status code is not handled or not allowed 2026-01-25 03:14:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/29147-12-angle-broom-extra-stiff-orange>: HTTP status code is not handled or not allowed 2026-01-25 03:14:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-wrist-and-ankle-weight-3-lb-gold->: HTTP status code is not handled or not allowed 2026-01-25 03:14:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/29387-59-fiberglass-handle-orange>: HTTP status code is not handled or not allowed 2026-01-25 03:14:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/murphys-tire-and-tube-mounting-compound-25lbs>: HTTP status code is not handled or not allowed 2026-01-25 03:14:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/diamond-vi-wind-front-brim-safety-helmet-adj-one-d-rotor-clamping-system-carbon-black>: HTTP status code is not handled or not allowed 2026-01-25 03:14:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits>: HTTP status code is not handled or not allowed 2026-01-25 03:14:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/262736-adapter1-8-inch-mpt-x-1-4-inch-fpt>: HTTP status code is not handled or not allowed 2026-01-25 03:14:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hand-weight-1-lb-yellow>: HTTP status code is not handled or not allowed 2026-01-25 03:14:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-sectional-stackbin-unit-37-w-x-12-d-x-4-1-2-h-blue-6-compartments>: HTTP status code is not handled or not allowed 2026-01-25 03:14:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hw-th9320wf5003-wifi-9000-color-touchscreen-thermostat>: HTTP status code is not handled or not allowed 2026-01-25 03:14:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/264122-flex-tube-kit-1-4-inch-x-30-inch>: HTTP status code is not handled or not allowed 2026-01-25 03:14:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-compartment-sectional-unit-37-w-x-20-d-x-53-1-2-h-black>: HTTP status code is not handled or not allowed 2026-01-25 03:14:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/262623-flat-roller>: HTTP status code is not handled or not allowed 2026-01-25 03:14:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cdc04040-socket-cap-screw>: HTTP status code is not handled or not allowed 2026-01-25 03:14:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/cabinets/compartment_cabinets landed on page that is not a product page. 2026-01-25 03:14:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industries-7-foot-rail-kit-2-tube-satin-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 03:14:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/custom_logo_mats/logo_rugs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:14:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/cookware_bakeware/pots_pans/saute_pots_pans already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:14:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/74-h-nexel-chrome-wire-shelving-36-w-x-30-d>: HTTP status code is not handled or not allowed 2026-01-25 03:14:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cmc05018-machine-screw>: HTTP status code is not handled or not allowed 2026-01-25 03:14:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a323-portable-dissolved-oxygen-meter-stara3230>: HTTP status code is not handled or not allowed 2026-01-25 03:14:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a221-portable-ph-meter-stara2210>: HTTP status code is not handled or not allowed 2026-01-25 03:14:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/floor_mats/custom_logo_mats/logo_rugs>: HTTP status code is not handled or not allowed 2026-01-25 03:15:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/cookware_bakeware/pots_pans/saute_pots_pans landed on page that is not a product page. 2026-01-25 03:15:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cdc05060-socket-cap-screw>: HTTP status code is not handled or not allowed 2026-01-25 03:15:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-cube-ice-maker-approx523-lb-production-full-size-cube>: HTTP status code is not handled or not allowed 2026-01-25 03:15:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelite-vented-plastic-mat-shelf-truck-30l-x-21w-x-60h-polyurethane-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 03:15:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-5-inch-1-piece-utility-tong-3>: HTTP status code is not handled or not allowed 2026-01-25 03:15:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/safety_railings_fittings/architectural_railing_components already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:15:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/anchors_hangers/wedge_anchors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:15:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 57250 pages (at 92 pages/min), scraped 28548 items (at 48 items/min) 2026-01-25 03:15:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industries-ball-tee-side-outlet-for-15-tubing-satin-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 03:15:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/relaxed-fit-flex-twill-cargo-pants-30l-inseam-x-38-waist-black>: HTTP status code is not handled or not allowed 2026-01-25 03:15:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/263834-cap-end-round-tubing>: HTTP status code is not handled or not allowed 2026-01-25 03:15:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-3-hp-motor-single-phase-three-speed>: HTTP status code is not handled or not allowed 2026-01-25 03:15:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insulated-flex-safety-bib-overall-xs-black>: HTTP status code is not handled or not allowed 2026-01-25 03:15:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/anchors_hangers/wedge_anchors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:15:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industries-glass-clip-for-1-2-glass-flat-back-satin-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 03:15:18 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/safety_railings_fittings/architectural_railing_components landed on page that is not a product page. 2026-01-25 03:15:18 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/anchors_hangers/wedge_anchors landed on page that is not a product page. 2026-01-25 03:15:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelite-vented-plastic-mat-shelf-truck-54l-x-18w-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 03:15:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industries-ball-cross-for-2-tubing-polished-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 03:15:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10910-20-halo-cable-edge-series-self-retracting-lifeline>: HTTP status code is not handled or not allowed 2026-01-25 03:15:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-adjustable-one-piece-seal-med-blue-100-pack>: HTTP status code is not handled or not allowed 2026-01-25 03:15:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/file_folders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:15:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/fasteners/anchors_hangers/wedge_anchors>: HTTP status code is not handled or not allowed 2026-01-25 03:15:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/75lb-weighing-scale>: HTTP status code is not handled or not allowed 2026-01-25 03:15:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-1791-brass-burner-jet-77-9-16-inch-diameter-liquid-propane>: HTTP status code is not handled or not allowed 2026-01-25 03:15:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/54-h-nexel-chrome-wire-shelving-add-on-36-w-x-18-d>: HTTP status code is not handled or not allowed 2026-01-25 03:15:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/54-h-nexel-chrome-wire-shelving-add-on-54-w-x-18-d>: HTTP status code is not handled or not allowed 2026-01-25 03:15:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/painting_supplies/paint_brushes_rollers_tools/paint_sprayers_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:15:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jointers-powermatic1791241>: HTTP status code is not handled or not allowed 2026-01-25 03:15:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hubbel-industrial-incandescent-50>: HTTP status code is not handled or not allowed 2026-01-25 03:15:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010401-gasket-proofer-for-star-manufacturing>: HTTP status code is not handled or not allowed 2026-01-25 03:15:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/passive-filter-427-replacement-lens-polycarbonate-shade-12-black-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-25 03:15:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hubbel-ratchet-locking-7>: HTTP status code is not handled or not allowed 2026-01-25 03:15:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/262117-junction-box>: HTTP status code is not handled or not allowed 2026-01-25 03:15:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/painting_supplies/paint_brushes_rollers_tools/paint_sprayers_accessories>: HTTP status code is not handled or not allowed 2026-01-25 03:15:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/414471-hbs-1321w-13-semi-auto-horizontal-bandsaw-230-460v-3-phase>: HTTP status code is not handled or not allowed 2026-01-25 03:15:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1031038-disconnect-hose-fem-w-knob-for-ultrafryer>: HTTP status code is not handled or not allowed 2026-01-25 03:15:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/hi_vis_jackets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:15:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/life-jacket-adult-20-001-adult>: HTTP status code is not handled or not allowed 2026-01-25 03:15:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-poly-oxford-3-in-1-waterproof-parka-jacket-m-navy>: HTTP status code is not handled or not allowed 2026-01-25 03:15:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/file_folders landed on page that is not a product page. 2026-01-25 03:15:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/relaxed-fit-twill-safety-cargo-utility-pants-30l-inseam-x-44-waist-black>: HTTP status code is not handled or not allowed 2026-01-25 03:15:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smead-1-34-accordion-expansion-colored-file-pocket-straight-tab-letter-blue>: HTTP status code is not handled or not allowed 2026-01-25 03:15:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clamps-wilton14728>: HTTP status code is not handled or not allowed 2026-01-25 03:15:51 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/hi_vis_jackets landed on page that is not a product page. 2026-01-25 03:15:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-stage-hardboard-96l-x-36w-x-8h>: HTTP status code is not handled or not allowed 2026-01-25 03:15:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/file_folders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:15:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-w-x-18-d-x-63-h-blue-nylon-cover>: HTTP status code is not handled or not allowed 2026-01-25 03:15:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=jet-690160 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:15:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1971154-kit-orifice-high-altitude-for-duke-manufacturing>: HTTP status code is not handled or not allowed 2026-01-25 03:15:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-100mm-bore-x-15mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 03:16:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/welding/welding_blankets_curtains already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:16:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-ripstop-fleece-lined-safety-jacket-5xl-fluorescent-orange>: HTTP status code is not handled or not allowed 2026-01-25 03:16:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/file_folders landed on page that is not a product page. 2026-01-25 03:16:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/hanging_file_folders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:16:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-63mm-bore-x-700mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 03:16:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=jet-690160 landed on page that is not a product page. 2026-01-25 03:16:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stage-pie-set-carpet-36w-stage-units-8h-16h-24h-red>: HTTP status code is not handled or not allowed 2026-01-25 03:16:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-100mm-bore-x-125mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 03:16:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 57338 pages (at 88 pages/min), scraped 28593 items (at 45 items/min) 2026-01-25 03:16:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/welding/welding_blankets_curtains landed on page that is not a product page. 2026-01-25 03:16:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-125mm-bore-x-90mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 03:16:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/folders_filing/folders/hanging_file_folders>: HTTP status code is not handled or not allowed 2026-01-25 03:16:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/561523-cold-faucet-assembly-for-ts-brass-bronze-works>: HTTP status code is not handled or not allowed 2026-01-25 03:16:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/697277-3-stainless-steel-scraper-pink>: HTTP status code is not handled or not allowed 2026-01-25 03:16:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-polyester-filter-8-w-x-30-h-x-1-d>: HTTP status code is not handled or not allowed 2026-01-25 03:16:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-wire-shelf-truck-48x18x70-1600-pound-capacity-1>: HTTP status code is not handled or not allowed 2026-01-25 03:16:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_tools/brushes_sponges_squeegees/cleaning_brushes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:16:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stronghold-stainless-steel-clearview-model-48-x-24-x-66>: HTTP status code is not handled or not allowed 2026-01-25 03:16:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/shelving_accessories/shelf_liners already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:16:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/janitorial_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:16:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dinex-the-heritage-collection-translucent-lid-2000-cs-translucent-dx40008714>: HTTP status code is not handled or not allowed 2026-01-25 03:16:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/77-wall-inlet-8l-x-4-38w-x-3h-white>: HTTP status code is not handled or not allowed 2026-01-25 03:16:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/breakfast-blend-coffee-96-pods>: HTTP status code is not handled or not allowed 2026-01-25 03:16:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/-protective-cover-for-apex153-1200--jetstream-240-portable-evaporative-coolers>: HTTP status code is not handled or not allowed 2026-01-25 03:16:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_tools/brushes_sponges_squeegees/cleaning_brushes>: HTTP status code is not handled or not allowed 2026-01-25 03:16:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-x-30-maple-square-edge-work-bench-adjustable-height-1-3-4-top>: HTTP status code is not handled or not allowed 2026-01-25 03:16:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-x-30-steel-top-workbench-adjustable-height>: HTTP status code is not handled or not allowed 2026-01-25 03:16:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/shelving/shelving_accessories/shelf_liners>: HTTP status code is not handled or not allowed 2026-01-25 03:16:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/cabinets/janitorial_cabinets>: HTTP status code is not handled or not allowed 2026-01-25 03:16:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dxcmpa1982054-19rhp-single-stage-piston-compressor-20-galhoriz155-psi1-phase-120-240v>: HTTP status code is not handled or not allowed 2026-01-25 03:16:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/29006-food-hoe-yellow>: HTTP status code is not handled or not allowed 2026-01-25 03:16:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/321068-rubber-disc-for-cleveland-range>: HTTP status code is not handled or not allowed 2026-01-25 03:16:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:16:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/storage_panels/tool_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:16:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-30-plastic-safety-edge-work-bench-fixed-height-1-5-8-top>: HTTP status code is not handled or not allowed 2026-01-25 03:16:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-1155-stainless-steel-flat-washer-18-8-size-10-100-box>: HTTP status code is not handled or not allowed 2026-01-25 03:16:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-unit-blk-yellow-15-black-yellow-w-clamp-belt-end>: HTTP status code is not handled or not allowed 2026-01-25 03:16:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wireway-husky-tear-drop-pallet-rack-120x42x144>: HTTP status code is not handled or not allowed 2026-01-25 03:16:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1901276-switch-dispenser-green-dpst-for-bunn-o-matic>: HTTP status code is not handled or not allowed 2026-01-25 03:16:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits>: HTTP status code is not handled or not allowed 2026-01-25 03:16:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/221622-indicator-knob>: HTTP status code is not handled or not allowed 2026-01-25 03:16:45 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/storage_panels/tool_storage_racks landed on page that is not a product page. 2026-01-25 03:16:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/opticlean-16-compartment-glass-rack-w-3-extenders-orange-blue-rg16-3c412>: HTTP status code is not handled or not allowed 2026-01-25 03:16:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inmate-property-storage-1-5-cu-ft-23-inch-l-x-17-1-2-inch-w-x-8-1-2-inch-h-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:16:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-sectional-stackbin-unit-37-w-x-18-3-4-d-x-7-1-2-h-blue-4-compartments>: HTTP status code is not handled or not allowed 2026-01-25 03:16:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hand-weight-0-5-lb-tan>: HTTP status code is not handled or not allowed 2026-01-25 03:16:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/260-10-1-bariatric-handrail-scale-800-lb-x-0-2-lb>: HTTP status code is not handled or not allowed 2026-01-25 03:16:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012403-dial-2-d-off-7-1-for-randell>: HTTP status code is not handled or not allowed 2026-01-25 03:16:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cdc03016-socket-cap-screw>: HTTP status code is not handled or not allowed 2026-01-25 03:16:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/original-wrist-and-ankle-weight-12-5-lb-olive>: HTTP status code is not handled or not allowed 2026-01-25 03:16:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-cu-yd-self-dumping-steel-hopper-with-bump-release-4000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 03:17:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/electrochemistry_meters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:17:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a324-portable-ph-ise-meter-stara3240>: HTTP status code is not handled or not allowed 2026-01-25 03:17:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ripstop-insulated-safety-bib-overall-4xl-fluorescent-orange>: HTTP status code is not handled or not allowed 2026-01-25 03:17:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industries-flange-floor-for-2-tubing-polished-brass>: HTTP status code is not handled or not allowed 2026-01-25 03:17:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-fixed-length-plastic-truck-seal-7-5-yellow-60lbs-pull-force-100-pack>: HTTP status code is not handled or not allowed 2026-01-25 03:17:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 57433 pages (at 95 pages/min), scraped 28647 items (at 54 items/min) 2026-01-25 03:17:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelite-vented-plastic-mat-shelf-truck-72l-x-18w-x-80h-polyurethane-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 03:17:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/anchors_hangers/wedge_anchors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:17:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/utility-pail-tapered-12-5-quart>: HTTP status code is not handled or not allowed 2026-01-25 03:17:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industries-tube-15-x-050-x-8-polished-brass>: HTTP status code is not handled or not allowed 2026-01-25 03:17:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/anchors_hangers/wedge_anchors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:17:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-in-cast-aluminum-propeller-5-8-in-bore>: HTTP status code is not handled or not allowed 2026-01-25 03:17:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1901287-faucet-black-assy-for-bunn-o-matic>: HTTP status code is not handled or not allowed 2026-01-25 03:17:18 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/electrochemistry_meters landed on page that is not a product page. 2026-01-25 03:17:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/741306-door-gasket-sponge-rubber-10ft>: HTTP status code is not handled or not allowed 2026-01-25 03:17:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/fasteners/anchors_hangers/wedge_anchors>: HTTP status code is not handled or not allowed 2026-01-25 03:17:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010437-tank-heater-kit-120v-1800w-for-bunn-o-matic>: HTTP status code is not handled or not allowed 2026-01-25 03:17:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/fasteners/anchors_hangers/wedge_anchors>: HTTP status code is not handled or not allowed 2026-01-25 03:17:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-security-weather-resistant-covered-padlocks-with-master-key-system-6127lj>: HTTP status code is not handled or not allowed 2026-01-25 03:17:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-solid-shelf-18x36>: HTTP status code is not handled or not allowed 2026-01-25 03:17:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-metal-cable-seal-1-16-wx24-l-green-50-pack>: HTTP status code is not handled or not allowed 2026-01-25 03:17:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=690197-jtm-4vs-1-milling-machine-w-x-y-axis-powerfeeds-2-hp already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:17:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pop-blind-rivet-3-16-x-6-6-large-flange-head-up-to-3-8-grip-aluminum-aluminum-pkg-of-250>: HTTP status code is not handled or not allowed 2026-01-25 03:17:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ripstop-safety-rain-jacket-csa-class-2-level-2-l-fluorescent-orange>: HTTP status code is not handled or not allowed 2026-01-25 03:17:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hubbel-industrial-incandescent-25>: HTTP status code is not handled or not allowed 2026-01-25 03:17:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-insulated-quilted-jacket-2xl-olive>: HTTP status code is not handled or not allowed 2026-01-25 03:17:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=690197-jtm-4vs-1-milling-machine-w-x-y-axis-powerfeeds-2-hp>: HTTP status code is not handled or not allowed 2026-01-25 03:17:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1421443-clamp-oetiker-5-8-inch-tube-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 03:17:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/74-h-nexel-chrome-wire-shelving-24-w-x-18-d>: HTTP status code is not handled or not allowed 2026-01-25 03:17:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/414478-j-7040-10-x-16-horizontal-mitering-bandsaw-230v-3-phase>: HTTP status code is not handled or not allowed 2026-01-25 03:17:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/86-h-nexel-chrome-wire-shelving-24-w-x-24-d>: HTTP status code is not handled or not allowed 2026-01-25 03:17:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clamps-wilton14798>: HTTP status code is not handled or not allowed 2026-01-25 03:17:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cpr-mask-with-02-inlet-10-502>: HTTP status code is not handled or not allowed 2026-01-25 03:17:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/414483-vbs-1408-14-vertical-bandsaw-115-230v1-phase>: HTTP status code is not handled or not allowed 2026-01-25 03:17:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48w-carpet-stage-configuration-9-stage-units-12-pie-units-guard-rails-grey>: HTTP status code is not handled or not allowed 2026-01-25 03:17:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smead-2-ply-top-file-jackets-2-accordion-expansion-letter-11-point-manila-50box>: HTTP status code is not handled or not allowed 2026-01-25 03:17:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-level-riser-carpet-96l-x-18w-8h-16h-24h-32h-blue>: HTTP status code is not handled or not allowed 2026-01-25 03:17:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bandsawaccessories-jet-708127>: HTTP status code is not handled or not allowed 2026-01-25 03:17:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-125mm-bore-x-320mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 03:17:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clamps-columbian-41423>: HTTP status code is not handled or not allowed 2026-01-25 03:17:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/type-r-class-3-heavy-duty-engineer-vest-orange-4xl-sv55-3zod-4x>: HTTP status code is not handled or not allowed 2026-01-25 03:17:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vises-wilton-63248>: HTTP status code is not handled or not allowed 2026-01-25 03:17:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/761023-safety-cover>: HTTP status code is not handled or not allowed 2026-01-25 03:17:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/74-h-nexel-chrome-wire-shelving-48-w-x-24-d>: HTTP status code is not handled or not allowed 2026-01-25 03:18:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-wire-shelving-48-w-x-18-d-x-86-h>: HTTP status code is not handled or not allowed 2026-01-25 03:18:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/86-h-nexel-chrome-wire-shelving-add-on-72-w-x-24-d>: HTTP status code is not handled or not allowed 2026-01-25 03:18:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/23awg-4-pr-utp-cat6e-550-mhz-cmr-1000-ft-box-green>: HTTP status code is not handled or not allowed 2026-01-25 03:18:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waist-apron-12x24-3-pockets-black>: HTTP status code is not handled or not allowed 2026-01-25 03:18:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 57530 pages (at 97 pages/min), scraped 28704 items (at 57 items/min) 2026-01-25 03:18:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-stage-hardboard-96l-x-36w-x-16h>: HTTP status code is not handled or not allowed 2026-01-25 03:18:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reducer-82280n-04-02-14-male-nptf-x-18-nptf-female-thread-nickel-plated-brass>: HTTP status code is not handled or not allowed 2026-01-25 03:18:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/front-of-the-house-waist-apron-12x24-3-comp-pocket-black>: HTTP status code is not handled or not allowed 2026-01-25 03:18:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blue-vinyl-reusable-cold-pack-neck-6-x-23>: HTTP status code is not handled or not allowed 2026-01-25 03:18:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-wide-span-rack-96x48x96-with-3-shelves-wood-deck-1100-lb-capacity-per-level>: HTTP status code is not handled or not allowed 2026-01-25 03:18:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-poly-oxford-lined-safety-parka-jacket-xxs-black>: HTTP status code is not handled or not allowed 2026-01-25 03:18:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnetic-card-holders-6-w-x-2-h-25-per-package>: HTTP status code is not handled or not allowed 2026-01-25 03:18:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-6432-single-acting-magnetic-cylinder-20mm-bore-x-50mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 03:18:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-quilted-safety-freezer-jacket-s-black>: HTTP status code is not handled or not allowed 2026-01-25 03:18:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-z-brite-wire-shelving-30-w-x-24-d-x-63-h>: HTTP status code is not handled or not allowed 2026-01-25 03:18:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1421599-tape-fiberglass-3-4-inch-x-54>: HTTP status code is not handled or not allowed 2026-01-25 03:18:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insulated-cold-gear-coverall-m-blackfluorescent-yellow-green>: HTTP status code is not handled or not allowed 2026-01-25 03:18:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-poly-oxford-lined-safety-parka-jacket-xl-black>: HTTP status code is not handled or not allowed 2026-01-25 03:18:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-poly-oxford-lined-safety-parka-jacket-xs-black>: HTTP status code is not handled or not allowed 2026-01-25 03:18:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-40mm-bore-x-170mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 03:18:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compression-sleeve-large>: HTTP status code is not handled or not allowed 2026-01-25 03:18:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1591066-brush-toilet-yellow>: HTTP status code is not handled or not allowed 2026-01-25 03:18:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-6432-single-acting-magnetic-cylinder-16mm-bore-x-10mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 03:18:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-50mm-bore-x-70mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 03:18:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-safety-parka-jacket-l-navy>: HTTP status code is not handled or not allowed 2026-01-25 03:18:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-z-brite-wire-shelving-add-on-30-w-x-24-d-x-74-h>: HTTP status code is not handled or not allowed 2026-01-25 03:18:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-23>: HTTP status code is not handled or not allowed 2026-01-25 03:18:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/bar_restaurant_equipment/bartending_supplies/bar_wine_tools already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:18:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-control-89968-04-04-knob-adj-14-tube-x-14-swift-fit-metallic-collet>: HTTP status code is not handled or not allowed 2026-01-25 03:18:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cold-pack-chilling-unit-with-12-standard-cold-packs-upright-4-3-cu-ft-capacity>: HTTP status code is not handled or not allowed 2026-01-25 03:18:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-insulated-ice-bag-with-tie-strings-7-x-13>: HTTP status code is not handled or not allowed 2026-01-25 03:18:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0020-inch-plastic-color-coded-shim-10-inch-x-20-inch-flat-sheet-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 03:18:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/bar_restaurant_equipment/bartending_supplies/bar_wine_tools>: HTTP status code is not handled or not allowed 2026-01-25 03:18:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adapter-82242n-02-02-18-male-g-thread-x-18-female-nptf-nickel-plated-brass>: HTTP status code is not handled or not allowed 2026-01-25 03:18:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-drum-li-tilt-72-high-dispensing-55-gal-drum-12v>: HTTP status code is not handled or not allowed 2026-01-25 03:18:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-40mm-bore-x-210mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 03:18:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eagle-disposal-can-galvanized-red-25-gallons>: HTTP status code is not handled or not allowed 2026-01-25 03:18:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eagle-screen-metal-disposal-can>: HTTP status code is not handled or not allowed 2026-01-25 03:18:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-heavy-duty-shelving-36x24x84-with-5-shelves-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 03:18:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-50mm-bore-x-230mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 03:18:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-50mm-bore-x-225mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 03:18:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clean-sanding-low-profile-finishing-disc-pad-20290-5-inch-x-11-16-inch-5-16-24-external-44-holes>: HTTP status code is not handled or not allowed 2026-01-25 03:18:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/00600-wall-anchor-6-x-6-powder-coated-steel-bolt-on-130-420-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-25 03:18:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/finish-thompson-centrifugal-pump-seal-kit>: HTTP status code is not handled or not allowed 2026-01-25 03:18:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-heavy-duty-shelving-36x12x84-with-5-shelves-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 03:18:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-high-capacity-wire-deck-shelf-96x24>: HTTP status code is not handled or not allowed 2026-01-25 03:18:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/00215-beamer-153-trolley-anchor-fits-3-10-beams-up-to-9-10-thick-130-420lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-25 03:18:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-heavy-duty-shelving-36x12x60-with-4-shelves-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 03:18:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1201191-brake-kit-f-5-inch-caster-w-hdwr>: HTTP status code is not handled or not allowed 2026-01-25 03:18:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-male-elbow-89110-04-06-14-tube-x-38-swift-fit-metallic-collet>: HTTP status code is not handled or not allowed 2026-01-25 03:19:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12mm-x-20mm-metric-keystock-plain-finish-1-meter-length>: HTTP status code is not handled or not allowed 2026-01-25 03:19:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-floor-curb-45-degree-corner-5-16-in-thick-yellow>: HTTP status code is not handled or not allowed 2026-01-25 03:19:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premier-undercounter-freestanding-refrigerator-solid-door-2-5-cu-ft-abt-hc-ucfs-0204>: HTTP status code is not handled or not allowed 2026-01-25 03:19:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011847-motor-gear-for-turbochef>: HTTP status code is not handled or not allowed 2026-01-25 03:19:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30109-plastic-barrier-chain-on-a-reel-hdpe-1-5-x200-6-38mm-gold>: HTTP status code is not handled or not allowed 2026-01-25 03:19:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 57629 pages (at 99 pages/min), scraped 28755 items (at 51 items/min) 2026-01-25 03:19:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/union-55040-6-6mm-tube-composite-body>: HTTP status code is not handled or not allowed 2026-01-25 03:19:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50104-plastic-barrier-chain-on-a-reel-hdpe-2-x125-8-51mm-green>: HTTP status code is not handled or not allowed 2026-01-25 03:19:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/51010-500-heavy-duty-plastic-barrier-chain-hdpe-2-x500-54mm-brown>: HTTP status code is not handled or not allowed 2026-01-25 03:19:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0010-inch-steel-shim-stock-12-inch-x-120-inch-roll>: HTTP status code is not handled or not allowed 2026-01-25 03:19:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/top-mount-above-ground-post-for-parcel-protector-vault-dvu0050-dvu0050pa-copper-vein>: HTTP status code is not handled or not allowed 2026-01-25 03:19:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/parcel-protector-vault-mailbox-and-parcel-drop-dvu0050-rear-access-copper-vein>: HTTP status code is not handled or not allowed 2026-01-25 03:19:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-branch-tee-89210-06-02-38-tube-x-18-swift-fit-metallic-collet>: HTTP status code is not handled or not allowed 2026-01-25 03:19:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lyon-locker-double-tier-12x18x36-2-door-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:19:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triplecut-adhesive-tape-for-smartcompliance-cabinet-2-x-5-yd-1-rollbox>: HTTP status code is not handled or not allowed 2026-01-25 03:19:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m6-x-1-0-x-40mm-phillips-oval-head-machine-screw-304-stainless-steel-din-966-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 03:19:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tote-box-dividable-8>: HTTP status code is not handled or not allowed 2026-01-25 03:19:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tote-box-accessory-long-divider-6>: HTTP status code is not handled or not allowed 2026-01-25 03:19:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/squids-3755-self-adhering-tool-tethering-tape-trap-144l-gray-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 03:19:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/union-elbow-60130-53-532-tube-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 03:19:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-tecgen-flame-resistant-work-shirt-4xl-short-tan>: HTTP status code is not handled or not allowed 2026-01-25 03:19:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30325-10-s-hook-acetal-copolymer-1-5-safety-pink-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 03:19:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/circuit-board-global-commercial-portable-ac-s>: HTTP status code is not handled or not allowed 2026-01-25 03:19:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-locker-double-tier-12x15x36-6-door-parchment>: HTTP status code is not handled or not allowed 2026-01-25 03:19:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-x-4-x-96-single-cedar-post-hardware>: HTTP status code is not handled or not allowed 2026-01-25 03:19:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd24sn-916-inch-band-heavy-duty-4-pc-hose-clamp-1-18-inch-1-1516-inch-clmpgdiam-10-pcs>: HTTP status code is not handled or not allowed 2026-01-25 03:19:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commercial-led-clf4-straight-100-150-straight-arm-mount-for-100-150w-led-flood-lights-bronze>: HTTP status code is not handled or not allowed 2026-01-25 03:19:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-48-tenv-1ph-3450rpm-c4c34nb2ee>: HTTP status code is not handled or not allowed 2026-01-25 03:19:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0008-inch-brass-thickness-gage-12-inch-x-12-inch-blades-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-25 03:19:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/boardwalk-mediumweight-polypropylene-cutlery-spork-white-1000carton>: HTTP status code is not handled or not allowed 2026-01-25 03:19:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/comfy-hand-wrist-orthosis-pediatric-medium-with-one-cover>: HTTP status code is not handled or not allowed 2026-01-25 03:19:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eco-products-bluestripe-recycled-content-clear-plastic-cold-drink-cups-9-oz-clear-1000ctn>: HTTP status code is not handled or not allowed 2026-01-25 03:19:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/general_purpose_electric_motors/single_phase_motors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:19:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22-1099-10-1-4-inch-door-latch-with-lock-and-strike-straight-handle>: HTTP status code is not handled or not allowed 2026-01-25 03:19:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-tecgen-flame-resistant-coverall-4xl-long-tall-navy>: HTTP status code is not handled or not allowed 2026-01-25 03:19:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-partition-flexible-five-panel-7>: HTTP status code is not handled or not allowed 2026-01-25 03:19:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-tecgen-flame-resistant-work-shirt-l-short-light-blue>: HTTP status code is not handled or not allowed 2026-01-25 03:19:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-v1100-spectrophotometer-320-1100nm-wavelength-range>: HTTP status code is not handled or not allowed 2026-01-25 03:19:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/general_purpose_electric_motors/single_phase_motors>: HTTP status code is not handled or not allowed 2026-01-25 03:19:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/00125-inch-black-plastic-color-coded-shim-stock-10-inch-x-20-inch-flat-sheet>: HTTP status code is not handled or not allowed 2026-01-25 03:19:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-flame-resistant-dh-air153-work-shirt-3xl-short-khaki>: HTTP status code is not handled or not allowed 2026-01-25 03:19:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metal-bookcase-two-shelf-34-1-2-w-x-12-5-8-d-x-29-h-black>: HTTP status code is not handled or not allowed 2026-01-25 03:20:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/263187-fry-basket-submerger>: HTTP status code is not handled or not allowed 2026-01-25 03:20:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-magnetic-floor-sweeper-36-w>: HTTP status code is not handled or not allowed 2026-01-25 03:20:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/bookcases already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:20:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/threaded-insert-installation-kit-for-hard-wood-brass-m3-0-5-ez-400-m3>: HTTP status code is not handled or not allowed 2026-01-25 03:20:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011697-hinge-kit-left-for-perlick-refrigeration>: HTTP status code is not handled or not allowed 2026-01-25 03:20:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elevated-sorting-cart-36-cu-ft-capacity-blue>: HTTP status code is not handled or not allowed 2026-01-25 03:20:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cove-heater-kcv2407-w-59-white-240-208v-700-525w>: HTTP status code is not handled or not allowed 2026-01-25 03:20:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unit-heater-kb2407-1-b2-eco-w-bracket-remote-and-thermostat-240v-7-5kw-1-ph>: HTTP status code is not handled or not allowed 2026-01-25 03:20:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glass-cubicle-calendar-board-24W-x-14H-monthly>: HTTP status code is not handled or not allowed 2026-01-25 03:20:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 57726 pages (at 97 pages/min), scraped 28808 items (at 53 items/min) 2026-01-25 03:20:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/isobar-ultra-8-outlet-12ft-2850-joules>: HTTP status code is not handled or not allowed 2026-01-25 03:20:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-flame-resistant-zip-front-hoodie-l-tall-navy>: HTTP status code is not handled or not allowed 2026-01-25 03:20:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/bookcases landed on page that is not a product page. 2026-01-25 03:20:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-foot-e-track-section-galvanized-2-per-carton>: HTTP status code is not handled or not allowed 2026-01-25 03:20:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/750va-low-profile-12-outlet>: HTTP status code is not handled or not allowed 2026-01-25 03:20:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/computers_electronics/audio_video_equipment/audio_visual_mounts/monitor_mounts_stands already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:20:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-inch-plastic-classroom-chair-set-of-five-green>: HTTP status code is not handled or not allowed 2026-01-25 03:20:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/twin-cluster-medium-base-socket>: HTTP status code is not handled or not allowed 2026-01-25 03:20:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/save-a-stud-rethread-die-716-20>: HTTP status code is not handled or not allowed 2026-01-25 03:20:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-1578-4-5-16-inch-x-1-1-8-inch-edge-mount-door-hinge-with-7-8-inch-offset>: HTTP status code is not handled or not allowed 2026-01-25 03:20:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/computers_electronics/audio_video_equipment/audio_visual_mounts/monitor_mounts_stands landed on page that is not a product page. 2026-01-25 03:20:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/collar-ft-15-1-ea-case>: HTTP status code is not handled or not allowed 2026-01-25 03:20:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010128-caster-swivel-no-brake4-inch-dia-x1-1-2-1-2-x-1-th>: HTTP status code is not handled or not allowed 2026-01-25 03:20:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storage-cabinet-36x20x78>: HTTP status code is not handled or not allowed 2026-01-25 03:20:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ufo-led25-led-emergency-battery-pack-powers-to-25vaw-or-led-fixtures-wfull-lumen-output>: HTTP status code is not handled or not allowed 2026-01-25 03:20:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dyn6-dynamo-industrial-led-emergency-unit-wet-rated-3w-led-lamps>: HTTP status code is not handled or not allowed 2026-01-25 03:20:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-panel-display-tower-6-ft-5-inch-high-fabric-walnut>: HTTP status code is not handled or not allowed 2026-01-25 03:20:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/06095-1-16-x-3-x-4-plastic-horseshoe-shim-56-500pc>: HTTP status code is not handled or not allowed 2026-01-25 03:20:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-ft-high-dry-erase-overlay>: HTTP status code is not handled or not allowed 2026-01-25 03:20:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ev2-led-compact-white-emergency-lighting-unit-1w-led-adjustable-heads>: HTTP status code is not handled or not allowed 2026-01-25 03:20:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stanley-stht71834-heavy-duty-narrow-crown-staples-5-16-inch-1250-pack>: HTTP status code is not handled or not allowed 2026-01-25 03:20:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-w-x-12-d-plastic-laminate-cantilever-shelf>: HTTP status code is not handled or not allowed 2026-01-25 03:20:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/outside-white-drum-glass-with-clear-sides-and-bottom-6-1-2-in-diameter>: HTTP status code is not handled or not allowed 2026-01-25 03:20:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/push-to-connect-tube-fitting-nylon-plastic-tee-adapter-532-tube-od-x-14-npt-female>: HTTP status code is not handled or not allowed 2026-01-25 03:20:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/phoenix-8-double-cantilever-steel-bench-without-back-surface-mount-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:20:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/room_dividers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:20:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-panel-divider-8-ft-high-x-5-ft-9-inch-long-fabric-black>: HTTP status code is not handled or not allowed 2026-01-25 03:20:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bleeding-triage-kit-plastic-case>: HTTP status code is not handled or not allowed 2026-01-25 03:20:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tractor-style-stool-nylon-black-no-tilt-no-foot-ring>: HTTP status code is not handled or not allowed 2026-01-25 03:20:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/push-to-connect-tube-fitting-nylon-plastic-90-degree-elbow-adapter-4mm-tube-od-x-18-npt-male>: HTTP status code is not handled or not allowed 2026-01-25 03:20:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/diamond-plate-1-2-thick-mat-24x36-black-yellow-border>: HTTP status code is not handled or not allowed 2026-01-25 03:20:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-flat-washer-sae-17-32-i-d-steel-zinc-grade-2-pkg-of-1-lb>: HTTP status code is not handled or not allowed 2026-01-25 03:20:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/partitions_dividers/room_dividers landed on page that is not a product page. 2026-01-25 03:20:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/250ml-filter-system-0-22um-pes-filter-75mm-diameter-sterile-12-case>: HTTP status code is not handled or not allowed 2026-01-25 03:20:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tribute-sauce-stock-pots-77522-16-quart-capacity-8-1-2-depth>: HTTP status code is not handled or not allowed 2026-01-25 03:20:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/best-value-plastic-platform-truck-36-l-x-24-w-1000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 03:21:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/snap-on-deck-magliner-gemini-senior-hand-truck>: HTTP status code is not handled or not allowed 2026-01-25 03:21:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washdown-motor-56-tefc-1ph-1725rpm-c6c17wk2pp>: HTTP status code is not handled or not allowed 2026-01-25 03:21:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mogul-to-medium-reducer>: HTTP status code is not handled or not allowed 2026-01-25 03:21:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-round-table-with-4-imme-armless-barstools-moonbeam-seatstudio-teak-top>: HTTP status code is not handled or not allowed 2026-01-25 03:21:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/341121-urn-heater-w-cut-out120v-1500w>: HTTP status code is not handled or not allowed 2026-01-25 03:21:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 57821 pages (at 95 pages/min), scraped 28865 items (at 57 items/min) 2026-01-25 03:21:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-flat-washer-sae-13-32-i-d-steel-zinc-grade-2-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 03:21:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reinforced-high-pressure-clear-pvc-tubing-1id-x-1-14od-x-100-ft>: HTTP status code is not handled or not allowed 2026-01-25 03:21:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-inch-club-grill-lp>: HTTP status code is not handled or not allowed 2026-01-25 03:21:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cpu-computer-cabinet-black>: HTTP status code is not handled or not allowed 2026-01-25 03:21:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pla-03244-128oz-3840ml-white-hdpe-f-style-jug-only-38-400-neck-finish-case-of-54>: HTTP status code is not handled or not allowed 2026-01-25 03:21:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bin-divider-2>: HTTP status code is not handled or not allowed 2026-01-25 03:21:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3m-nph43-lower-taping-head-2>: HTTP status code is not handled or not allowed 2026-01-25 03:21:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-bucket-with-lid-and-handle-polypropylene-one-gallon-capacity>: HTTP status code is not handled or not allowed 2026-01-25 03:21:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-56-truck-chest-single-lid-matte-black-79212438>: HTTP status code is not handled or not allowed 2026-01-25 03:21:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cdn-proaccurate-refrigerator-freezer-thermometer>: HTTP status code is not handled or not allowed 2026-01-25 03:21:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/321615-gh-2280zx-large-spindle-bore-lathe-w-acu-rite-300s-taper-attachment-collet-closer>: HTTP status code is not handled or not allowed 2026-01-25 03:21:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/261764-baffle-filter-25-x-20-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 03:21:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-width-wall-mounted-hanger-style-coat-rack-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:21:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12n40a03k-zinc-die-cast-adjustable-lever-w-steel-components-m12-x-40mm-stud>: HTTP status code is not handled or not allowed 2026-01-25 03:21:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fd-10-saver-cycle-cycling-refrigerated-dryer-21-cfm-1-phase-115v>: HTTP status code is not handled or not allowed 2026-01-25 03:21:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-best-value-wire-shelving-unit-48-w-x-18-d-x-74-h-600-lb-shelf-cap-chrome>: HTTP status code is not handled or not allowed 2026-01-25 03:21:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g7-10-hp-rotary-screw-compressor-120-gal-horizontal-125-psig-3-phase-208-230-460v>: HTTP status code is not handled or not allowed 2026-01-25 03:21:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motorola-1-wire-surveillance-earpiece>: HTTP status code is not handled or not allowed 2026-01-25 03:21:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pre-cut-polypropylene-strapping-with-plastic-buckle-1-2-x-17-black-skid-lot>: HTTP status code is not handled or not allowed 2026-01-25 03:21:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-18-push-broom-w-plastic-block-steel-handle-multi-surface-sweep>: HTTP status code is not handled or not allowed 2026-01-25 03:21:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/truck-powered-crate-turner-dumper-3000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 03:21:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-square-table-with-4-imme-armless-barstools-black-seatespresso-top>: HTTP status code is not handled or not allowed 2026-01-25 03:21:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-add-on-rack-60x24x96-with-3-levels-wood-deck-1300-lb-cap-per-level-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:21:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-round-table-with-4-imme-armless-caster-chairs-light-gray-seatstudio-teak-top>: HTTP status code is not handled or not allowed 2026-01-25 03:21:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-square-table-with-4-imme-armless-caster-chairs-light-gray-seatmaple-top>: HTTP status code is not handled or not allowed 2026-01-25 03:21:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/upholstered-textured-fabric-stacking-chair-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:21:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2281224-caster-stem-4-inch-female-thread-swivel>: HTTP status code is not handled or not allowed 2026-01-25 03:21:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-add-on-rack-96x36x96-with-3-levels-steel-deck-800lb-cap-per-level-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:21:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-round-table-with-4-imme-armless-barstools-white-seatriver-cherry-top>: HTTP status code is not handled or not allowed 2026-01-25 03:21:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/general_purpose_cleaners already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:21:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cm-lodestar-electric-chain-hoist-with-chain-container-1-000-lb-capacity-3>: HTTP status code is not handled or not allowed 2026-01-25 03:21:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silverware-soak-system-52617-20-x-15-x-5-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:21:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=690179-jtm-4vs-1-milling-machine-w-acu-rite-200s-dro-x-axis-powerfeed-2-hp already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:21:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wood-folding-table-96-x-30-mahogany>: HTTP status code is not handled or not allowed 2026-01-25 03:21:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/boil-in-vacuum-sealer-bags-23850-pack-of-100-6-x-8>: HTTP status code is not handled or not allowed 2026-01-25 03:21:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-square-table-with-4-imme-armless-caster-chairs-navy-seatstudio-teak-top>: HTTP status code is not handled or not allowed 2026-01-25 03:21:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/general_purpose_cleaners>: HTTP status code is not handled or not allowed 2026-01-25 03:21:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merit-66623340037-abrasotex-non-woven-quick-change-disc-2-inch-dia-med-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-25 03:21:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=690019-jtm-1-vertical-milling-machine-w-x-axis-powerfeed-2-hp already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:21:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-mold-on-8-rubber-hand-truck-wheel-kit>: HTTP status code is not handled or not allowed 2026-01-25 03:21:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=690179-jtm-4vs-1-milling-machine-w-acu-rite-200s-dro-x-axis-powerfeed-2-hp>: HTTP status code is not handled or not allowed 2026-01-25 03:22:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/norton-66261130333-paper-wp-sheet-9-inch-x-11-inch-400-grit-silicon-carbide>: HTTP status code is not handled or not allowed 2026-01-25 03:22:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-bin-rail>: HTTP status code is not handled or not allowed 2026-01-25 03:22:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-inch-hd-extended-angle-grinder>: HTTP status code is not handled or not allowed 2026-01-25 03:22:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=690019-jtm-1-vertical-milling-machine-w-x-axis-powerfeed-2-hp landed on page that is not a product page. 2026-01-25 03:22:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grid-wall-bin-stackable-1>: HTTP status code is not handled or not allowed 2026-01-25 03:22:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 57919 pages (at 98 pages/min), scraped 28920 items (at 55 items/min) 2026-01-25 03:22:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/norton-69957398015-neon-fiber-disc-9-1-8-inch-x-7-8-inch-seeded-gel-ceramic-24-grit>: HTTP status code is not handled or not allowed 2026-01-25 03:22:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/miramar-low-dome-cover-49427-fits-49430-satin-finish>: HTTP status code is not handled or not allowed 2026-01-25 03:22:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gearmotor-ac-parallel-22rpm-c4c17fz37c>: HTTP status code is not handled or not allowed 2026-01-25 03:22:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-square-table-with-4-imme-armless-caster-chairs-white-seatespresso-top>: HTTP status code is not handled or not allowed 2026-01-25 03:22:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/891817-7-flag-room-id-system-8-l-aluminum-flags-1-pk>: HTTP status code is not handled or not allowed 2026-01-25 03:22:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-square-table-with-4-imme-armless-barstools-black-seatespresso-top>: HTTP status code is not handled or not allowed 2026-01-25 03:22:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/item-ml1-025-sr2-rear-panel-mounting-extension-rod-coupling>: HTTP status code is not handled or not allowed 2026-01-25 03:22:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/display-case-40886-36-cubed-glass-refrigerated>: HTTP status code is not handled or not allowed 2026-01-25 03:22:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/item-ml2-063-dr3-rear-panel-mounting-disconnet-switches-door-coupling-interlock>: HTTP status code is not handled or not allowed 2026-01-25 03:22:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/production-straight-drill-6000>: HTTP status code is not handled or not allowed 2026-01-25 03:22:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/respirator-cleaning-kit-with-liquid-cleaner-1-gallon>: HTTP status code is not handled or not allowed 2026-01-25 03:22:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/appliances/refrigerators_freezers/refrigerators/refrigerated_merchandisers_display_cases already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:22:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-fit-test-kit-banana-oil>: HTTP status code is not handled or not allowed 2026-01-25 03:22:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/redco-instacut-3-1-2-corer-dicer-wedger-15006-8-section-wedge-cut-tabletop>: HTTP status code is not handled or not allowed 2026-01-25 03:22:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1031203-liner-flour-bin-un-notched-for-ultrafryer>: HTTP status code is not handled or not allowed 2026-01-25 03:22:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/graphic-facility-signs-no-smoking-vinyl-4x4>: HTTP status code is not handled or not allowed 2026-01-25 03:22:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_alarms_signals/forklift_alert_sensors_systems already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:22:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6x12x8-peak-style-shed-4-15-inch-augers-1-3-8-inch-frame-grey>: HTTP status code is not handled or not allowed 2026-01-25 03:22:34 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/appliances/refrigerators_freezers/refrigerators/refrigerated_merchandisers_display_cases landed on page that is not a product page. 2026-01-25 03:22:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1291050-filter-baffle-10x20-hd-type-i>: HTTP status code is not handled or not allowed 2026-01-25 03:22:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-d-wall-bracket-interlake-pallet-rack>: HTTP status code is not handled or not allowed 2026-01-25 03:22:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_alarms_signals/forklift_alert_sensors_systems already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:22:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-8482-46-child-size-square-outdoor-steel-picnic-table-expanded-metal-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:22:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/locksets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:22:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_alarms_signals/forklift_alert_sensors_systems>: HTTP status code is not handled or not allowed 2026-01-25 03:22:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/thresholds_bottom_sweeps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:22:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/thresholds_bottom_sweeps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:22:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5100-grade-1-heavy-duty-surface-door-closer-par-adj-2-6-alm-hdhos>: HTTP status code is not handled or not allowed 2026-01-25 03:22:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tote-box-heat-resistant-40-to-300-degrees-47>: HTTP status code is not handled or not allowed 2026-01-25 03:22:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-syle-2-23-3-4-x-23-3-4-pvc-lay-in-tile-in-polished-copper-pl5225>: HTTP status code is not handled or not allowed 2026-01-25 03:22:49 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_alarms_signals/forklift_alert_sensors_systems landed on page that is not a product page. 2026-01-25 03:22:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/fasteners/door_window_hardware/locksets>: HTTP status code is not handled or not allowed 2026-01-25 03:22:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-1-2-w-x-15-1-2-d-x-6-h-stackbin-hopper-front-container-black>: HTTP status code is not handled or not allowed 2026-01-25 03:22:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/door_window_hardware/thresholds_bottom_sweeps landed on page that is not a product page. 2026-01-25 03:22:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/fasteners/door_window_hardware/thresholds_bottom_sweeps>: HTTP status code is not handled or not allowed 2026-01-25 03:22:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-syle-3-23-3-4-x-23-3-4-pvc-lay-in-tile-in-argent-sliver-pl5409>: HTTP status code is not handled or not allowed 2026-01-25 03:22:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/281442-window-only-access-for-lincoln-manufacturing>: HTTP status code is not handled or not allowed 2026-01-25 03:22:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ptp212bu-tough-mark-strip>: HTTP status code is not handled or not allowed 2026-01-25 03:22:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-10-w-x-24-d-x-8-h-stackbin-hopper-front-container>: HTTP status code is not handled or not allowed 2026-01-25 03:23:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/locbin-wall-storage-3-220rws-w-rails-medium-red-26-pc>: HTTP status code is not handled or not allowed 2026-01-25 03:23:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/761148-4-range-burner-broiler-for-rocky-mountain-cookware>: HTTP status code is not handled or not allowed 2026-01-25 03:23:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kd-8-bushel-blue-vinyl-basket-bulk-truck>: HTTP status code is not handled or not allowed 2026-01-25 03:23:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10194977-harness-back-d-ring-tongue-buckle-leg-straps-standard>: HTTP status code is not handled or not allowed 2026-01-25 03:23:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-starter-72in-w-x-15in-d-x-60in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-25 03:23:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 58007 pages (at 88 pages/min), scraped 28970 items (at 50 items/min) 2026-01-25 03:23:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-starter-72in-w-x-15in-d-x-72in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-25 03:23:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-mat-ii-reversible-drainage-mat-3-ft-x-4-ft-red>: HTTP status code is not handled or not allowed 2026-01-25 03:23:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-x4-x72-telescoping-inner-boxes>: HTTP status code is not handled or not allowed 2026-01-25 03:23:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f-series-coax-connector-female-adapter>: HTTP status code is not handled or not allowed 2026-01-25 03:23:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6x6x1-1-4-literature-mailers>: HTTP status code is not handled or not allowed 2026-01-25 03:23:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10193128-standard-single-leg-energy-absorbing-6-tie-back-lanyard-ansi-z359-13-2013>: HTTP status code is not handled or not allowed 2026-01-25 03:23:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bristol-ridge-scraper-carpet-mat-4-ft-x-6-ft-slate-blue>: HTTP status code is not handled or not allowed 2026-01-25 03:23:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/marble-sof-tyle-grande-mat-4-ft-x-custom-length-black>: HTTP status code is not handled or not allowed 2026-01-25 03:23:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-extra-heavy-duty-boltless-shelving-starter-42in-w-x-15in-d-x-96in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-25 03:23:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-extra-heavy-duty-boltless-shelving-starter-42in-w-x-15in-d-x-60in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-25 03:23:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-inch-nylon-ball-carbon-steel-low-profile-flying-saucer-ball-transfer>: HTTP status code is not handled or not allowed 2026-01-25 03:23:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48x40x48-double-wall-gaylord-bottom>: HTTP status code is not handled or not allowed 2026-01-25 03:23:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10x6x57-telescoping-inner-boxes>: HTTP status code is not handled or not allowed 2026-01-25 03:23:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ratchet-lever-hoist-3-ton-capacity-5-lift>: HTTP status code is not handled or not allowed 2026-01-25 03:23:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ratchet-lever-hoist-3-4-ton-capacity-5-lift>: HTTP status code is not handled or not allowed 2026-01-25 03:23:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-wall-mount-shelf-with-one-lip>: HTTP status code is not handled or not allowed 2026-01-25 03:23:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dynamic-2-1-2-fluid-glycerine-filled-pressure-gage-panel-flange-60-psi>: HTTP status code is not handled or not allowed 2026-01-25 03:23:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-heavy-duty-boltless-shelving-add-on-72in-w-x-15in-d-x-72in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-25 03:23:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/impact-xps-153-3-phillips-1-insert-bit-matte-nickel-5-pack>: HTTP status code is not handled or not allowed 2026-01-25 03:23:39 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-25 03:23:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/impact-xps-153-1-phillips-1-insert-bit-matte-nickel-5-pack>: HTTP status code is not handled or not allowed 2026-01-25 03:23:40 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:29 2026-01-25 03:23:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inox-grinding-wheel-36-grit-type-27-9-dia-x-1-4-t-x-7-8-cntr-hole-dia-10-pk>: HTTP status code is not handled or not allowed 2026-01-25 03:23:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2491059-capacitor-091-uf-for-amana-litton>: HTTP status code is not handled or not allowed 2026-01-25 03:23:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-add-on-42in-w-x-15in-d-x-60in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-25 03:23:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-x2150-2-mil-poly-tubing>: HTTP status code is not handled or not allowed 2026-01-25 03:23:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/32-x2900-1-5-mil-poly-tubing>: HTTP status code is not handled or not allowed 2026-01-25 03:23:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-add-on-48in-w-x-36in-d-x-60in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-25 03:23:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dynamic-2-1-2-fluid-glycerine-filled-pressure-gage-panel-3000-psi>: HTTP status code is not handled or not allowed 2026-01-25 03:23:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/180-x-30-extra-long-bench-plastic-top>: HTTP status code is not handled or not allowed 2026-01-25 03:23:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/boards_easels/enclosed_bulletin_boards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:23:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/boards_easels/enclosed_bulletin_boards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:23:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kushion-walk-unslotted-black-3-8in-x-3ft-x-5ft>: HTTP status code is not handled or not allowed 2026-01-25 03:23:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-universal-roll-medium-weight-150-l-x-30-w-1-bag>: HTTP status code is not handled or not allowed 2026-01-25 03:23:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009330-bjwa-thermostat>: HTTP status code is not handled or not allowed 2026-01-25 03:23:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-36-x-72-red-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 03:24:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/boards_easels/enclosed_bulletin_boards landed on page that is not a product page. 2026-01-25 03:24:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-add-on-72in-w-x-24in-d-x-72in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-25 03:24:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wide-span-rack-48x36x96-with-3-shelves-wire-deck-1200-lb-capacity-per-level>: HTTP status code is not handled or not allowed 2026-01-25 03:24:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-00-rubber-voltage-gloves-black-size-10-dwh110010>: HTTP status code is not handled or not allowed 2026-01-25 03:24:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-2-single-solenoid-valve-ext-pilot-spring-return-g-1-8-110v-ac-5va-coil-black-conn>: HTTP status code is not handled or not allowed 2026-01-25 03:24:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cleaver-7-nylon>: HTTP status code is not handled or not allowed 2026-01-25 03:24:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/boards_easels/enclosed_bulletin_boards landed on page that is not a product page. 2026-01-25 03:24:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-narrow-aisle-18-fork-pallet-jack-truck-18-x-48-5500-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 03:24:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 58103 pages (at 96 pages/min), scraped 29026 items (at 56 items/min) 2026-01-25 03:24:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ladder-safety-strap-pack-slclssk>: HTTP status code is not handled or not allowed 2026-01-25 03:24:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durastripe-12-round-sign-pedestrian-traffic>: HTTP status code is not handled or not allowed 2026-01-25 03:24:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-dbl-solenoid-vlv-ext-pilot-pres-cntr-spr-cntred-g1-4-110v-ac-5va-coil-led>: HTTP status code is not handled or not allowed 2026-01-25 03:24:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-starter-72inw-x-48ind-x-60inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 03:24:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-high-capacity-bulk-rack-with-wood-decking-72x36x72-starter>: HTTP status code is not handled or not allowed 2026-01-25 03:24:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/kitchen_tools/knives_cutlery/utility_knives already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:24:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-starter-60inw-x-36ind-x-60inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 03:24:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-high-capacity-bulk-rack-with-wood-decking-96x24x96-starter>: HTTP status code is not handled or not allowed 2026-01-25 03:24:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/421942-circuit-breaker>: HTTP status code is not handled or not allowed 2026-01-25 03:24:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/shelving_accessories/shelving_enclosure_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:24:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durastripe-supreme-v-4-x-100-yellow>: HTTP status code is not handled or not allowed 2026-01-25 03:24:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-starter-96inw-x-36ind-x-60inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 03:24:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/kitchen_tools/knives_cutlery/utility_knives>: HTTP status code is not handled or not allowed 2026-01-25 03:24:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-starter-48inw-x-48ind-x-84inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 03:24:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lakeside-poly-top-work-height-can-rack-7210-965>: HTTP status code is not handled or not allowed 2026-01-25 03:24:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/shelving_accessories/shelving_enclosure_kits landed on page that is not a product page. 2026-01-25 03:24:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/881208-pressure-dual-control-for-johnson-controls>: HTTP status code is not handled or not allowed 2026-01-25 03:24:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1031132-switch-toggle-dpst-on-off-for-ultrafryer>: HTTP status code is not handled or not allowed 2026-01-25 03:24:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010643-ignition-module-for-frymaster>: HTTP status code is not handled or not allowed 2026-01-25 03:24:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-precision-self-centering-vise-4-inch-jaw-width-1-1-2-inch-depth>: HTTP status code is not handled or not allowed 2026-01-25 03:24:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/511130-thermocouple-baso-slmjim-72-for-johnson-controls>: HTTP status code is not handled or not allowed 2026-01-25 03:24:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-1749-infinite-control-switch-kit-15a-240v>: HTTP status code is not handled or not allowed 2026-01-25 03:24:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-heavy-duty-boltless-shelving-add-on-48inw-x-18ind-x-60inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 03:24:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-1-4-npt-double-solenoid-valve-pilot-24v-dc-2w-coil-black-connection>: HTTP status code is not handled or not allowed 2026-01-25 03:24:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-16-20-x-5-hex-head-cap-screw-steel-plain-unf-grade-8-usa-25-pack-452282>: HTTP status code is not handled or not allowed 2026-01-25 03:24:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tffn-16-gauge-building-wire-stranded-type-green>: HTTP status code is not handled or not allowed 2026-01-25 03:24:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-2-thermostat-black>: HTTP status code is not handled or not allowed 2026-01-25 03:24:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-vis-work-shirt-type-r-class-3-2xl-t-fluorescent-yellow-df2-ax3-324ls-hy-2xlt>: HTTP status code is not handled or not allowed 2026-01-25 03:24:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-heavy-duty-boltless-shelving-add-on-96inw-x-24ind-x-72inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 03:24:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-1-8-npt-double-solenoid-valve-pilot-12v-dc-3w-coil-black-connection>: HTTP status code is not handled or not allowed 2026-01-25 03:24:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-mesh-deck-96-wx36-d>: HTTP status code is not handled or not allowed 2026-01-25 03:24:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-powered-c-clamp-2-inch-bung-agitator-1-hp-2-5-inch-cl-propeller>: HTTP status code is not handled or not allowed 2026-01-25 03:24:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kit4sc403x08-40-cal-compliance-arc-flash-kit-w-short-coat-bib-overall-3xl-sz-08>: HTTP status code is not handled or not allowed 2026-01-25 03:24:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m8002-12-tubing-x-12-male-npt-fitting>: HTTP status code is not handled or not allowed 2026-01-25 03:25:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kit2cv112x10-12-cal-cm2-ultrasoft-arc-flash-kit-with-fr-coverall-2xl-glove-size-10>: HTTP status code is not handled or not allowed 2026-01-25 03:25:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/bar_restaurant_equipment/bartending_supplies/bar_mats_shelf_liners already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:25:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/staplers_staples/staplers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:25:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-closed-single-solenoid-valve-ext-pilot-g-1-4-12v-dc-3w-coil-led-connection>: HTTP status code is not handled or not allowed 2026-01-25 03:25:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kit2cv11-sm08-12-cal-cm2-ultrasoft-arc-flash-kit-with-fr-coverall-sm-glove-size-08>: HTTP status code is not handled or not allowed 2026-01-25 03:25:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2241212-blade-6-section-wedger-ii-for-nemco>: HTTP status code is not handled or not allowed 2026-01-25 03:25:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 58193 pages (at 90 pages/min), scraped 29079 items (at 53 items/min) 2026-01-25 03:25:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/structural-plastic-shelving-96x24x51>: HTTP status code is not handled or not allowed 2026-01-25 03:25:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/bar_restaurant_equipment/bartending_supplies/bar_mats_shelf_liners>: HTTP status code is not handled or not allowed 2026-01-25 03:25:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/staplers_staples/staplers landed on page that is not a product page. 2026-01-25 03:25:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kit2cv08-lg10-8-cal-cm2-arc-flash-kit-with-fr-coverall-lg-glove-size-10>: HTTP status code is not handled or not allowed 2026-01-25 03:25:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kit4sc40s12-40-cal-compliance-arc-flash-kit-w-short-coat-bib-overall-s-sz-12>: HTTP status code is not handled or not allowed 2026-01-25 03:25:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-hose-short-shank-coupling-set-268-30141-3-npsh-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 03:25:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/little-bin-plastic-stacking-bins-2-3-4-x-5-1-4-x-3-yellow>: HTTP status code is not handled or not allowed 2026-01-25 03:25:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-add-on-72inw-x-24ind-x-72inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 03:25:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-dbl-solenoid-vlv-ext-pilot-closed-cntr-spr-cntred-g1-4-110v-ac-5va-coil-black>: HTTP status code is not handled or not allowed 2026-01-25 03:25:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/snow-cone-syrups-grape>: HTTP status code is not handled or not allowed 2026-01-25 03:25:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-hose-female-swivel-x-male-adapter-363-2522561-2-1-2-nh-x-2-1-2-npt-brass>: HTTP status code is not handled or not allowed 2026-01-25 03:25:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-3294-spring-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 03:25:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/264616-pan-impedance-9-inch-for-duke-manufacturing>: HTTP status code is not handled or not allowed 2026-01-25 03:25:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-58-208-230v-1625rpm-ccwse>: HTTP status code is not handled or not allowed 2026-01-25 03:25:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-18-ga-36-wx30-dx73-h-open-clip-style-4-shelf>: HTTP status code is not handled or not allowed 2026-01-25 03:25:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jrs-5l-18w-straight-gal-steel-roller-conveyor-1-3-8-roller-dia-16-bf-6-axle-center>: HTTP status code is not handled or not allowed 2026-01-25 03:25:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jrs-5l-24w-straight-gal-steel-roller-conveyor-1-3-8-roller-dia-22-bf-6-axle-center>: HTTP status code is not handled or not allowed 2026-01-25 03:25:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mrs-10l-18w-straight-gal-steel-roller-conveyor-19-roller-dia-15-bf-3-axle-center>: HTTP status code is not handled or not allowed 2026-01-25 03:25:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-work-shirt-4x-navy-tcg01160231>: HTTP status code is not handled or not allowed 2026-01-25 03:25:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jrs-13w-1-3-8-dia-gal-steel-roller-for-15-roller-conveyors>: HTTP status code is not handled or not allowed 2026-01-25 03:25:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-add-on-48inw-x-18ind-x-84inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 03:25:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stanchion-2-yellow-w-c-hooks-incl-chain-extra-2-pole-14-base-41-height>: HTTP status code is not handled or not allowed 2026-01-25 03:25:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xerox-bold-digital-carbonless-paper-xer3r12424-8-1-2-11-pink-canary-white-5010-sheets>: HTTP status code is not handled or not allowed 2026-01-25 03:25:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-dbl-solenoid-vlv-ext-pilot-open-cntr-spr-cntred-g1-4-24v-dc-2w-coil-black-conn>: HTTP status code is not handled or not allowed 2026-01-25 03:25:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyester-web-sling-hd-double-ply-eye-and-eye-with-durable-edge-10-ft-l-x-2-in-w>: HTTP status code is not handled or not allowed 2026-01-25 03:25:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cfp4-25um-qualitative-cellulose-filter-paper-15-0cm-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 03:25:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-jumbo-unassembled-storage-cabinet-48w-x-18d-x-42h-putty>: HTTP status code is not handled or not allowed 2026-01-25 03:25:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/direct-drive-blower-1075rpm-277v-13hp>: HTTP status code is not handled or not allowed 2026-01-25 03:25:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kitchen-pegboard-pack-storage-organization-kit-red-blue-32-x-32-x-6>: HTTP status code is not handled or not allowed 2026-01-25 03:25:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-cal-flame-resistant-coveralls-l-ln-gray-tcg02150866>: HTTP status code is not handled or not allowed 2026-01-25 03:26:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/label-holder-1-in-36-in-clear-strip>: HTTP status code is not handled or not allowed 2026-01-25 03:26:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quick-adjust-wire-shelving-60x18x63>: HTTP status code is not handled or not allowed 2026-01-25 03:26:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-add-on-96inw-x-24ind-x-96inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 03:26:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-2-single-solenoid-valve-ext-pilot-spring-return-1-4-nptf-24v-dc-3w-coil-black-conn>: HTTP status code is not handled or not allowed 2026-01-25 03:26:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-high-metal-indoor-outdoor-counter-height-stool-with-back-white>: HTTP status code is not handled or not allowed 2026-01-25 03:26:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-add-on-72inw-x-30ind-x-84inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 03:26:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quick-adjust-wire-shelving-72x24x63>: HTTP status code is not handled or not allowed 2026-01-25 03:26:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m5-x-0-8-x-50mm-security-machine-screw-flat-torx-head-18-8-stainless-steel-ft-unc-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 03:26:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/laser-ink-jet-insert-letter-pref-8-1-2-in-x11-in-50-per-package>: HTTP status code is not handled or not allowed 2026-01-25 03:26:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-add-on-48inw-x-36ind-x-96inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 03:26:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-2-single-solenoid-valve-ext-pilot-spring-return-g-1-8-24v-dc-3w-coil-black-conn>: HTTP status code is not handled or not allowed 2026-01-25 03:26:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 58289 pages (at 96 pages/min), scraped 29135 items (at 56 items/min) 2026-01-25 03:26:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-16-x-1-1-2-security-machine-screw-button-hex-socket-head-alloy-steel-black-oxide-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 03:26:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/lockers/standard_lockers/steel_standard_lockers/standard_duty_steel_lockers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:26:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-retardant-silicone-foam-sheet-no-adhesive-1-4-x-36-x-36>: HTTP status code is not handled or not allowed 2026-01-25 03:26:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-32-x-5-8-security-machine-screw-button-torx-head-alloy-steel-black-oxide-ft-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 03:26:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securit-barrier-system-portable-15-dia-x-40-post-base-gold-plated>: HTTP status code is not handled or not allowed 2026-01-25 03:26:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-32-x-1-4-security-machine-screw-button-torx-head-alloy-steel-black-oxide-ft-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 03:26:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-394-1-light-pendant-alabaster-bell-brushed-nickel-9-75w-x-20-5h>: HTTP status code is not handled or not allowed 2026-01-25 03:26:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-pedestal-misting-fan-outdoor-rated-oscillating-7204-cfm-1-7-hp>: HTTP status code is not handled or not allowed 2026-01-25 03:26:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/printable-magnetic-sheet-ink-jet-letter>: HTTP status code is not handled or not allowed 2026-01-25 03:26:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/lockers/standard_lockers/steel_standard_lockers/standard_duty_steel_lockers>: HTTP status code is not handled or not allowed 2026-01-25 03:26:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14ab-x-1-1-4-security-sheet-metal-screw-button-torx-head-302hq-stainless-steel-usa-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 03:26:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8ab-x-1-1-2-security-sheet-metal-screw-flat-torx-head-302hq-stainless-steel-usa-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 03:26:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/faultless-swivel-plate-caster-5-phenolic-wheel>: HTTP status code is not handled or not allowed 2026-01-25 03:26:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010542-small-block-top-for-garland-range>: HTTP status code is not handled or not allowed 2026-01-25 03:26:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-king-sk3000-pallet-rack-120w-x-42d-x-120h-3320-lbs-cap-per-level-add-on-unit>: HTTP status code is not handled or not allowed 2026-01-25 03:26:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/locboard-pegboard--lochook-assortment-18w-x-916d-x-36h-red-pack-of-32>: HTTP status code is not handled or not allowed 2026-01-25 03:26:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/die-cast-aluminum-exit-sign-self-powered-double-face>: HTTP status code is not handled or not allowed 2026-01-25 03:26:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-2-inch-epdm-short-shank-gasket>: HTTP status code is not handled or not allowed 2026-01-25 03:26:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:26:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6ab-x-3-4-security-sheet-metal-screw-button-torx-head-302hq-stainless-steel-usa-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 03:26:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-20-x-2-3-4-security-machine-screw-button-hex-socket-head-302hq-18-8-ss-ft-unc-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 03:26:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sweatband-black-universal-4-x-10>: HTTP status code is not handled or not allowed 2026-01-25 03:26:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/77-191-8-light-vanity-wall-strip-polished-brass-48w-x-4-75h>: HTTP status code is not handled or not allowed 2026-01-25 03:26:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:26:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-1-8-inch-5-inch-stainless-steel-worm-gear-clamp-with-9-16-inch-band>: HTTP status code is not handled or not allowed 2026-01-25 03:26:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-4903-adams-1-light-outdoor-wall-frosted-textured-black-6-125w-x-18-25h>: HTTP status code is not handled or not allowed 2026-01-25 03:26:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spice-womens-safety-work-boots-steel-toe-7h-size-8w-auburn>: HTTP status code is not handled or not allowed 2026-01-25 03:26:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010936-bearing-for-hobart>: HTTP status code is not handled or not allowed 2026-01-25 03:26:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-inch-1-inch-stainless-steel-micro-worm-gear-clamp-with-5-16-inch-band>: HTTP status code is not handled or not allowed 2026-01-25 03:26:45 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits landed on page that is not a product page. 2026-01-25 03:26:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-2-inch-e-polypropylene-cam-and-groove-adapter-x-hose-shank>: HTTP status code is not handled or not allowed 2026-01-25 03:26:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-25 03:26:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-16-18-x-1-2-security-machine-screw-button-hex-socket-head-302hq-18-8-ss-ft-unc-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 03:26:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mela-scrub-sponge-white-48-sponges>: HTTP status code is not handled or not allowed 2026-01-25 03:26:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-1833-dupont-2-light-vanity-wall-satin-white-brushed-nickel-13w-x-8h>: HTTP status code is not handled or not allowed 2026-01-25 03:26:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-t-shirt-short-sleeve-1-pocket-fl-lime-small>: HTTP status code is not handled or not allowed 2026-01-25 03:26:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stanley-201-piece-drive-mechanics-tool-set>: HTTP status code is not handled or not allowed 2026-01-25 03:26:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/76-530-2-light-flood-light-w-adjust-swivel-motion-sensor-white-14w>: HTTP status code is not handled or not allowed 2026-01-25 03:26:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/floor_machine_accessories_supplies/vacuum_attachments_tools already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:27:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-shelving-36x12x60-with-4-shelves-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 03:27:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2661024-buster-ice-for-taylor-freezer>: HTTP status code is not handled or not allowed 2026-01-25 03:27:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cartridge-filter-cooling-air-9-989-190-0>: HTTP status code is not handled or not allowed 2026-01-25 03:27:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-steel-wheel-riser-102-x-18-x-6>: HTTP status code is not handled or not allowed 2026-01-25 03:27:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/floor_machine_accessories_supplies/vacuum_attachments_tools already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:27:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iprotect-reader-safety-glasses-erb-safety-17993-Smoke>: HTTP status code is not handled or not allowed 2026-01-25 03:27:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 58382 pages (at 93 pages/min), scraped 29188 items (at 53 items/min) 2026-01-25 03:27:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storability-wall-mount-shelving-unit-3-shelves-33w-x-16d-x-31-12h-white>: HTTP status code is not handled or not allowed 2026-01-25 03:27:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gravity-womens-safety-work-boots-steel-toe-4h-size-85w-vegan-blackout>: HTTP status code is not handled or not allowed 2026-01-25 03:27:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mk-296-nitrile-powder-free-disposable-gloves-4-7-mil-black-m-100-box>: HTTP status code is not handled or not allowed 2026-01-25 03:27:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mk-296-nitrile-powder-free-disposable-gloves-4-7-mil-black-xl-100-box>: HTTP status code is not handled or not allowed 2026-01-25 03:27:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/floor_machine_accessories_supplies/vacuum_attachments_tools landed on page that is not a product page. 2026-01-25 03:27:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:27:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glc-01705-16oz-clear-straight-sided-round-jar-with-89-400-black-phenolic-cap-case-of-12>: HTTP status code is not handled or not allowed 2026-01-25 03:27:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/floor_machine_accessories_supplies/vacuum_attachments_tools landed on page that is not a product page. 2026-01-25 03:27:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/76-131-2-light-ceiling-flush-mount-frosted-ribbed-textured-white-11-25w-x-5h>: HTTP status code is not handled or not allowed 2026-01-25 03:27:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:27:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deep-cleaning-ribbed-mat-3x10-blue-1>: HTTP status code is not handled or not allowed 2026-01-25 03:27:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/76-193-3-light-ceiling-flush-mount-clear-ribbed-textured-white-15-25w-x-6h>: HTTP status code is not handled or not allowed 2026-01-25 03:27:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/decorative_lighting/flush_mount_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:27:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-25 03:27:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-x-24-18-gauge-stainless-steel-workbench>: HTTP status code is not handled or not allowed 2026-01-25 03:27:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-boltless-shelving-add-on-48inw-x-18ind-x-60inh-no-deck>: HTTP status code is not handled or not allowed 2026-01-25 03:27:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-25 03:27:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-high-legal-adder-empty-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:27:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drink-mat-18794-mississippi-state-3-1-4-x-24-x-1>: HTTP status code is not handled or not allowed 2026-01-25 03:27:34 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/decorative_lighting/flush_mount_lights landed on page that is not a product page. 2026-01-25 03:27:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hss-tmx-concave-milling-cutter-7-32-inch-circle-dia-x-2-1-4-inch-cutter-dia-x-1-inch-hole-x-14-teeth>: HTTP status code is not handled or not allowed 2026-01-25 03:27:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-boltless-shelving-add-on-36inw-x-24ind-x-96inh-no-deck>: HTTP status code is not handled or not allowed 2026-01-25 03:27:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-edge-work-bench-top-plastic-48-w-x-30-d-x-1-5-8-thick>: HTTP status code is not handled or not allowed 2026-01-25 03:27:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hss-tmx-plain-slitting-saw-2-1-2-inch-dia-x-3-64-inch-face-x-3-8-inch-hole-x-28-teeth>: HTTP status code is not handled or not allowed 2026-01-25 03:27:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/warehouse-rack-bins-binw232042-23w-x-42d-x-20h-kraft-price-each>: HTTP status code is not handled or not allowed 2026-01-25 03:27:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esd-safe-vinyl-clean-room-chair-with-nylon-base-with-drag-chain-black>: HTTP status code is not handled or not allowed 2026-01-25 03:27:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-liner-for-medium-ultra-filter-pad-4-pack>: HTTP status code is not handled or not allowed 2026-01-25 03:27:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-ball-i-m-style-brass-coupler-1-4-x-3-8-standard-hose-barb-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 03:27:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qp-pro-two-stage-air-compressor-10-hp-120-gallon-horizontal-460v-3-phase>: HTTP status code is not handled or not allowed 2026-01-25 03:27:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/abrasives/sanding_discs/flap_sanding_discs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:27:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-mold-on-wheel-5316-ra-16-dia-3w-1400-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 03:27:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-shelf-single-rivet-laminate-deck-36inw-x-18ind-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:27:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/outdoor-8-multipurpose-led-lensed-striplight-9100l-80w-35k-dimable-120-277v>: HTTP status code is not handled or not allowed 2026-01-25 03:27:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cyclone-hand-chain-hoist-on-low-headroom-geared-trolley-6-ton-10-ft-lift>: HTTP status code is not handled or not allowed 2026-01-25 03:27:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pressure-wash-3-8-50-ft-4500-psi-pressure-wash>: HTTP status code is not handled or not allowed 2026-01-25 03:28:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/abrasives/sanding_discs/flap_sanding_discs>: HTTP status code is not handled or not allowed 2026-01-25 03:28:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zinc-plated-swivel-caster-zn22-4-prb-polyurethane-with-polyethylene-center-4-dia-500-lb>: HTTP status code is not handled or not allowed 2026-01-25 03:28:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shopair-chain-hoist-500-lbs-10-ft-lift-21-fpm-lift-55-fpm-lower>: HTTP status code is not handled or not allowed 2026-01-25 03:28:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-322-sanitation-container-rigid-caster-n32-5-mcp-phenolic-5-dia-1000-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 03:28:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-step-mobile-steel-step-stool-w-24-serrated-tread-wlsr001242>: HTTP status code is not handled or not allowed 2026-01-25 03:28:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-style-b-j-hook>: HTTP status code is not handled or not allowed 2026-01-25 03:28:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 58477 pages (at 95 pages/min), scraped 29246 items (at 58 items/min) 2026-01-25 03:28:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coil-lifter-1000-lb-capacity-14463>: HTTP status code is not handled or not allowed 2026-01-25 03:28:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/266312-sheave-door-rear-steel-tire>: HTTP status code is not handled or not allowed 2026-01-25 03:28:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0-4-bar-tong-lifter-1-000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 03:28:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ebf-615-8-bdm>: HTTP status code is not handled or not allowed 2026-01-25 03:28:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/material-handling/cranes_hoists/hoists/lever_chain_hoists already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:28:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-welded-aluminum-cleaning-sanitation-cart-39-inch-l-x-21-inch-w-x-41-inch-h>: HTTP status code is not handled or not allowed 2026-01-25 03:28:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adaptable-tricycle-1speed-coaster-brake-rear-wire-basket-orange>: HTTP status code is not handled or not allowed 2026-01-25 03:28:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1500t3q-cl-1500w-halogen-w-double-ended-base>: HTTP status code is not handled or not allowed 2026-01-25 03:28:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-a-frame-bin-cart-1-louvered-1-pegboard-panels-36w-x-30d-x-62h-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:28:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/material-handling/cranes_hoists/hoists/lever_chain_hoists landed on page that is not a product page. 2026-01-25 03:28:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caged-walk-through-fixed-access-ladder-gray-6>: HTTP status code is not handled or not allowed 2026-01-25 03:28:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-wall-corrugated-sheets-30l-x-24w-kraft-pack-of-5>: HTTP status code is not handled or not allowed 2026-01-25 03:28:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-48-inch-high-post-black-epoxy-4-pack>: HTTP status code is not handled or not allowed 2026-01-25 03:28:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/datum-tekstak-laptop-storage-locker-5-tier-electronic-lock-2>: HTTP status code is not handled or not allowed 2026-01-25 03:28:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-34-inch-high-post-chrome-4-pack>: HTTP status code is not handled or not allowed 2026-01-25 03:28:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40g25-220vv-40w-incandescent-w-medium-base>: HTTP status code is not handled or not allowed 2026-01-25 03:28:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-line-single-track-seal-top-bag-with-hang-hole-4in-x-4in-4-mil-pkg-qty-1000>: HTTP status code is not handled or not allowed 2026-01-25 03:28:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20mr11-sp-20w-halogen-w-sub-minature-2-pin-base>: HTTP status code is not handled or not allowed 2026-01-25 03:28:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-deluxe-literature-mailers-4l-x-4w-x-4h-kraft-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-25 03:28:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/floor-press-20-ton>: HTTP status code is not handled or not allowed 2026-01-25 03:28:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f32t8-865-env-32w-fluorescent-w-medium-bi-pin-base-daylight>: HTTP status code is not handled or not allowed 2026-01-25 03:28:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-double-wall-corrugated-boxes-16l-x-16w-x-14h-kraft-pack-of-15>: HTTP status code is not handled or not allowed 2026-01-25 03:28:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kx60cl-3m-mc-60w-halogen-w-mini-can-base>: HTTP status code is not handled or not allowed 2026-01-25 03:28:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/apc-smart-ups-rt-surt8000xlt>: HTTP status code is not handled or not allowed 2026-01-25 03:28:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hydraulic-axle-jack-30-15-ton-2-stage-air>: HTTP status code is not handled or not allowed 2026-01-25 03:28:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pic-gauges-4-pressure-gauge-liquid-fill-10000-psi-all-stainless-steel-lower-mount-301l-402u>: HTTP status code is not handled or not allowed 2026-01-25 03:28:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bottle-jack-12-ton-super-duty>: HTTP status code is not handled or not allowed 2026-01-25 03:29:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elegant-leatherplus-design-task-chair-black>: HTTP status code is not handled or not allowed 2026-01-25 03:29:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f15t8-blb-15w-fluorescent-w-medium-bi-pin-base-blacklight-blue>: HTTP status code is not handled or not allowed 2026-01-25 03:29:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/prem-met-air-fresh-66-oz-arsl-baby-powder-12>: HTTP status code is not handled or not allowed 2026-01-25 03:29:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/separable-roller-follower-metric-nast30zz-double-shielded-30-mm-bore-62-mm-od>: HTTP status code is not handled or not allowed 2026-01-25 03:29:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/110-ton-elite-ironworker-3-phase-380v-50-hz-elt110-3p380>: HTTP status code is not handled or not allowed 2026-01-25 03:29:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 58574 pages (at 97 pages/min), scraped 29312 items (at 66 items/min) 2026-01-25 03:29:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ivory-personal-bar-31oz-size-72>: HTTP status code is not handled or not allowed 2026-01-25 03:29:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/personal_care_hygiene/hand_sanitizers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:29:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/switchboard-non-conductive-corrugated-mat-3-wide-full-75-ft-roll-black-yellow>: HTTP status code is not handled or not allowed 2026-01-25 03:29:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-easy-fold-mailers-12l-x-11-12w-x-3h-white-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-25 03:29:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-abrasive-hose-413403>: HTTP status code is not handled or not allowed 2026-01-25 03:29:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-nitrile-gloves-large-powder-free>: HTTP status code is not handled or not allowed 2026-01-25 03:29:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/combination_metalworking_machines already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:29:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/combination_metalworking_machines already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:29:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tote-box-dividable-21>: HTTP status code is not handled or not allowed 2026-01-25 03:29:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/compact_fluorescent_light_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:29:18 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/personal_care_hygiene/hand_sanitizers landed on page that is not a product page. 2026-01-25 03:29:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_tables/outdoor_dining_bar_tables already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:29:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20v-60v-max-flexvolt-174-power-tool-battery-lithium-ion-9-0ah>: HTTP status code is not handled or not allowed 2026-01-25 03:29:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cut-resistant-gloves-micro-foam-nitrile-coated-ansi-a1-xl-blue>: HTTP status code is not handled or not allowed 2026-01-25 03:29:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cut-resistant-gloves-micro-foam-nitrile-coated-ansi-a1-2xl-blue>: HTTP status code is not handled or not allowed 2026-01-25 03:29:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/combination_metalworking_machines>: HTTP status code is not handled or not allowed 2026-01-25 03:29:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/combination_metalworking_machines landed on page that is not a product page. 2026-01-25 03:29:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tote-box-accessory-short-divider-25>: HTTP status code is not handled or not allowed 2026-01-25 03:29:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/compact_fluorescent_light_bulbs>: HTTP status code is not handled or not allowed 2026-01-25 03:29:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bar-isle-outdoor-mobile-trolley-39-38l-x-18-18w-x-29-18h-teakwhite>: HTTP status code is not handled or not allowed 2026-01-25 03:29:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_tables/outdoor_dining_bar_tables landed on page that is not a product page. 2026-01-25 03:29:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kdw-flush-access-doors-for-drywall-surf-lock-14wx14h-kdwp1414l>: HTTP status code is not handled or not allowed 2026-01-25 03:29:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hp431-premium-over-the-head-ear-muff-nrr-31-db-blackred>: HTTP status code is not handled or not allowed 2026-01-25 03:29:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/cable_management/cable_clamps_clips_hardware already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:29:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/28-inch-traffic-cone-reflective-solid-orange-base-7-lbs>: HTTP status code is not handled or not allowed 2026-01-25 03:29:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mx-keyed-exterior-door-key-latch-24wx24h-mxp2424clk>: HTTP status code is not handled or not allowed 2026-01-25 03:29:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-foxx-2hp-ufo-101m-dust-collector>: HTTP status code is not handled or not allowed 2026-01-25 03:29:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:29:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/158-3-36w-miniature-sealed-beams-w-mini-wedge-base>: HTTP status code is not handled or not allowed 2026-01-25 03:29:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-step-orange-extra-heavy-duty-steel-rolling-ladder-expanded-metal-tread>: HTTP status code is not handled or not allowed 2026-01-25 03:29:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/cable_management/cable_clamps_clips_hardware landed on page that is not a product page. 2026-01-25 03:29:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-step-thin-line-folding-step-ladder-300-lb-capacity-blue-tl318>: HTTP status code is not handled or not allowed 2026-01-25 03:29:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uhmw-polyethylene-plastic-rod-6-diameter-x-6-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 03:29:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:29:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-25 03:29:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:29:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-12-x-10-scratch-resistant-ribbed-clear-strip-strip-curtains>: HTTP status code is not handled or not allowed 2026-01-25 03:29:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibco-heavy-duty-electric-vibrator-2p-450-1>: HTTP status code is not handled or not allowed 2026-01-25 03:29:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:29:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibco-pneumatic-piston-vibrator-50-2l>: HTTP status code is not handled or not allowed 2026-01-25 03:29:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:29:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock>: HTTP status code is not handled or not allowed 2026-01-25 03:29:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/500-3-16-diameter-7x19-galvanized-aircraft-cable>: HTTP status code is not handled or not allowed 2026-01-25 03:29:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 03:30:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/250-1-16-diameter-vinyl-coated-1-8-diameter-7x7-galvanized-aircraft-cable>: HTTP status code is not handled or not allowed 2026-01-25 03:30:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/250-5-8-dia-6x26-extra-improved-plow-steel-galvanized-wire-rope-independent-wire-rope-center>: HTTP status code is not handled or not allowed 2026-01-25 03:30:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:30:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/omni-pir-ultrasonic-ceiling-low-voltage-sensor-with-500-sq-ft-range-off-white>: HTTP status code is not handled or not allowed 2026-01-25 03:30:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 03:30:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gh401-non-vented-safety-helmet-4-point-adjustable-ratchet-suspension-orange>: HTTP status code is not handled or not allowed 2026-01-25 03:30:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12inch-straight-tinner-sn-ip>: HTTP status code is not handled or not allowed 2026-01-25 03:30:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010693-temperature-controller-for-hatco-corp>: HTTP status code is not handled or not allowed 2026-01-25 03:30:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 58662 pages (at 88 pages/min), scraped 29360 items (at 48 items/min) 2026-01-25 03:30:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/fluorescent_linear_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:30:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/840-style-woven-wire-panels-10w-x-10h>: HTTP status code is not handled or not allowed 2026-01-25 03:30:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks landed on page that is not a product page. 2026-01-25 03:30:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-25 03:30:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-plastic-rod-1-diameter-x-1-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 03:30:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stackable-skid-box-green-62x31x21>: HTTP status code is not handled or not allowed 2026-01-25 03:30:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/fluorescent_linear_bulbs>: HTTP status code is not handled or not allowed 2026-01-25 03:30:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-1-thick-x-1-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 03:30:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdpe-plastic-sheet-1-8-thick-x-48-wide-x-96-long>: HTTP status code is not handled or not allowed 2026-01-25 03:30:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chillits-6710ct-evaporative-cooling-triangle-hat-with-cooling-towel-lime-12586>: HTTP status code is not handled or not allowed 2026-01-25 03:30:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wireway-pallet-rack-tear-drop-upright-frame-120x48>: HTTP status code is not handled or not allowed 2026-01-25 03:30:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2561147-probe-temperature-control-for-silver-king>: HTTP status code is not handled or not allowed 2026-01-25 03:30:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grip-16-w-3-step-steel-rolling-ladder-20-d-top-step-2>: HTTP status code is not handled or not allowed 2026-01-25 03:30:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-par30-short-neck-colored-recessed-light-bulb-e26-base-8w-green-pack-of-3>: HTTP status code is not handled or not allowed 2026-01-25 03:30:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/little-giant-6en-series-sump-pump-10feet-power-cord>: HTTP status code is not handled or not allowed 2026-01-25 03:30:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-glove-and-sanitizer-stand-22-diameter-star-base-40-height>: HTTP status code is not handled or not allowed 2026-01-25 03:30:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/led_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:30:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-forklift-pocket-lifter-5000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 03:30:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:30:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72x24-maple-square-edge-power-apron-production-bench-blue-32313811>: HTTP status code is not handled or not allowed 2026-01-25 03:30:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/garage_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:30:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-14-hss-taper-tmx-pipe-tap-npt-ground-rh-5-flutes>: HTTP status code is not handled or not allowed 2026-01-25 03:30:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/garage_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:30:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-16-wing-nut-1-5-8-wide-5-8-base-dia-malleable-iron-zinc-usa-pkg-of-25>: HTTP status code is not handled or not allowed 2026-01-25 03:30:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60x30-maple-safety-edge-mobile-power-apron-production-bench-tan-32313866>: HTTP status code is not handled or not allowed 2026-01-25 03:30:42 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/led_bulbs landed on page that is not a product page. 2026-01-25 03:30:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-25 03:30:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/cabinets/garage_cabinets landed on page that is not a product page. 2026-01-25 03:30:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/seh30073-30-hp-oil-less-scroll-compressor-tankless-116-psi-3-phase-230v>: HTTP status code is not handled or not allowed 2026-01-25 03:30:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/cabinets/garage_cabinets landed on page that is not a product page. 2026-01-25 03:30:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sed10074hp-10-hp-oil-less-scroll-compressor-tankless-145-psi-3-phase-460v>: HTTP status code is not handled or not allowed 2026-01-25 03:30:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/resealable-suffocation-warning-poly-bags-16-w-x-20-l-1-5-mil-clear-500-pack>: HTTP status code is not handled or not allowed 2026-01-25 03:30:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rt110af-safety-glasses-rt1-series-black-frame-clear-anti-fog-lens>: HTTP status code is not handled or not allowed 2026-01-25 03:30:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96-w-x-36-d-plastic-laminate-square-edge-with-drawer-riser>: HTTP status code is not handled or not allowed 2026-01-25 03:30:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/downlights_track_lighting already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:30:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hg-4c-tn-hss-4-flute-tin-square-single-end-mill-1-x-1-x-2-x-4-1-2>: HTTP status code is not handled or not allowed 2026-01-25 03:31:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/made-in-usa-w-1-water-hardening-drill-rod-25-32>: HTTP status code is not handled or not allowed 2026-01-25 03:31:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/jobsite_portable_lighting/string_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:31:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/linear-led-strip-light-fixture-11w-5000k-24l-super-white>: HTTP status code is not handled or not allowed 2026-01-25 03:31:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/artic-blue-icemelter-44-lb-bag-200-31043>: HTTP status code is not handled or not allowed 2026-01-25 03:31:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bca-363013-bl-bookcase-all-welded-36-wx13-dx30-h-black>: HTTP status code is not handled or not allowed 2026-01-25 03:31:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-fiberglass-platform-step-ladder>: HTTP status code is not handled or not allowed 2026-01-25 03:31:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-deck-truck-48-l-x-24-w-1000-lb-capacity-with-ergo-handle>: HTTP status code is not handled or not allowed 2026-01-25 03:31:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 58753 pages (at 91 pages/min), scraped 29415 items (at 55 items/min) 2026-01-25 03:31:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/newport-patio-umbrella-with-silver-pole-108w-sunbrella-1a-taupe-fabric>: HTTP status code is not handled or not allowed 2026-01-25 03:31:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/jobsite_portable_lighting/string_lights landed on page that is not a product page. 2026-01-25 03:31:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-rotatable-wall-pack-outdoor-vanity-light-fixture-80w-10000-lumens-10l-black>: HTTP status code is not handled or not allowed 2026-01-25 03:31:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-deck-truck-60-l-x-30-w-2000-lb-capacity-with-ergo-handle>: HTTP status code is not handled or not allowed 2026-01-25 03:31:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cable-ties-uv-black-4-100-pieces>: HTTP status code is not handled or not allowed 2026-01-25 03:31:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-step-steel-safety-stairway-slope-rolling-ladder-weight-actuated-lock-16-w-serr-step-w-cal-osha-handrail-cal-wa-sw-72414g>: HTTP status code is not handled or not allowed 2026-01-25 03:31:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/respiratory-protection-and-safety-dvd-program>: HTTP status code is not handled or not allowed 2026-01-25 03:31:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-cable-ties-uv-black-3-50-pieces>: HTTP status code is not handled or not allowed 2026-01-25 03:31:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-step-steel-safety-rolling-ladder-w-weight-actuated-lock-16-w-perforated-step-orange-w-cal-osha-handrail-cal-wa072414p-o>: HTTP status code is not handled or not allowed 2026-01-25 03:31:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-cable-ties-uv-black-48-50-pieces>: HTTP status code is not handled or not allowed 2026-01-25 03:31:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aah-36rbmag3-h-grn-storage-cabinet-all-welded-36-wx24-dx78-h-hunter-green>: HTTP status code is not handled or not allowed 2026-01-25 03:31:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/cable_management/cable_ties_wraps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:31:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/cable_management/cable_ties_wraps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:31:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-linear-bar-vanity-light-fixture-182530w-3035-lumens-36l-brushed-nickel>: HTTP status code is not handled or not allowed 2026-01-25 03:31:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/cable_management/cable_ties_wraps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:31:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aah-36rbmag1-wr-storage-cabinet-all-welded-36-wx24-dx72-h-wine-red>: HTTP status code is not handled or not allowed 2026-01-25 03:31:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/educational_materials_books/safety_training_materials already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:31:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/downlights_track_lighting landed on page that is not a product page. 2026-01-25 03:31:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25-w-x-14-h-x-1-d-pleated-merv-8-standard-capacity-air-filter-wire-back-global-industrial>: HTTP status code is not handled or not allowed 2026-01-25 03:31:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fr-pullover-hoodie-xl>: HTTP status code is not handled or not allowed 2026-01-25 03:31:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fr-pullover-hoodie-xlt>: HTTP status code is not handled or not allowed 2026-01-25 03:31:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/cable_management/cable_ties_wraps>: HTTP status code is not handled or not allowed 2026-01-25 03:31:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zip-a-duct-fixing-strap-with-galvanized-steel-hardwarefor-20-to-36-inch-diameter-ducts>: HTTP status code is not handled or not allowed 2026-01-25 03:31:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/cable_management/cable_ties_wraps>: HTTP status code is not handled or not allowed 2026-01-25 03:31:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/graphic-signs-danger-high-voltage-plastic-7-w-x-10-h>: HTTP status code is not handled or not allowed 2026-01-25 03:31:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/educational_materials_books/safety_training_materials>: HTTP status code is not handled or not allowed 2026-01-25 03:31:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/filter-5012b6006-1-npt-5-0200-psig-275-scfm>: HTTP status code is not handled or not allowed 2026-01-25 03:31:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bca-365213-n-bookcase-all-welded-36-wx13-dx52-h-navy>: HTTP status code is not handled or not allowed 2026-01-25 03:31:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/cable_management/cable_ties_wraps landed on page that is not a product page. 2026-01-25 03:31:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-step-16-wx78-d-stainless-steel-rolling-safety-ladder-serrated-grating-w-cal-osha-handrail-cal-ss082428g>: HTTP status code is not handled or not allowed 2026-01-25 03:31:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bronze-hose-braid-w-copper-sweat-ends-1-5-x-12>: HTTP status code is not handled or not allowed 2026-01-25 03:31:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/import-copper-fitting-with-90-long-radius-elbow-c-x-c-1-18-od-x-1-id-10-pack>: HTTP status code is not handled or not allowed 2026-01-25 03:31:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-w-x-16-h-x-1-d-pleated-merv-10-high-capacity-air-filter-global-industrial>: HTTP status code is not handled or not allowed 2026-01-25 03:31:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-step-extra-heavy-duty-steel-rolling-safety-ladder-heavy-duty-serrated-grating-w-cal-osha-handrail-cal-ml123221g>: HTTP status code is not handled or not allowed 2026-01-25 03:31:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glance174-na-glass--multi-surface-cleaner-unscented-25-l-bottle-2case>: HTTP status code is not handled or not allowed 2026-01-25 03:31:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stride174-neutral-floor-cleaner-citrus-scent-5-gallon-envirobox>: HTTP status code is not handled or not allowed 2026-01-25 03:31:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polymer-pilaster-w-shoe-24w-x-82h-charcoal>: HTTP status code is not handled or not allowed 2026-01-25 03:31:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-pilaster-w-shoe-18w-x-82h-azure>: HTTP status code is not handled or not allowed 2026-01-25 03:31:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1002106-web-adjustable-tie-off-adaptor-fixed-6l-310-cap-lbs>: HTTP status code is not handled or not allowed 2026-01-25 03:31:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bl-364pt-l-counter-high-storage-cabinet-w--plastic-top-all-welded-36x18x42-green>: HTTP status code is not handled or not allowed 2026-01-25 03:31:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3500213-nano-lok-edge-tie-back-quick-connect-self-retracting-lifeline-9l>: HTTP status code is not handled or not allowed 2026-01-25 03:31:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/remote-speaker-microphone-with-3-5mm-audio-jack-for-ht-series-portable-radios>: HTTP status code is not handled or not allowed 2026-01-25 03:31:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-inch-fan-shroud-merv-6-air-filter-global-industrial>: HTTP status code is not handled or not allowed 2026-01-25 03:31:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-locker-triple-tier-12x12x24-9-doors-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:32:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/web-sling-1-ply-eye-eye-3-long-1-wide>: HTTP status code is not handled or not allowed 2026-01-25 03:32:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-pilaster-w-shoe-18w-x-82h-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:32:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bradley-lavatory-system-low-on-left-2-people>: HTTP status code is not handled or not allowed 2026-01-25 03:32:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bradley-wash-fountain-circular-110-24-vac-18>: HTTP status code is not handled or not allowed 2026-01-25 03:32:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1801-1-4-e-hss-heavy-duty-steam-oxide-135-split-point-jobber-length-drill>: HTTP status code is not handled or not allowed 2026-01-25 03:32:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-inches-steel-wire-buckles-b-3a-for-3-8-inches-polypropylene-strapping>: HTTP status code is not handled or not allowed 2026-01-25 03:32:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 58846 pages (at 93 pages/min), scraped 29462 items (at 47 items/min) 2026-01-25 03:32:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b151117-12-single-leg-chain-sling-w-sling-grab-hook>: HTTP status code is not handled or not allowed 2026-01-25 03:32:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-8-inches-heavy-duty-wire-buckles-hdb-5-for-5-8-inches-polypropylene-5-8-inches-cord-strapping>: HTTP status code is not handled or not allowed 2026-01-25 03:32:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1818-7-8-hss-heavy-duty-sand-blasted-118-point-carbide-tipped-masonry-drill>: HTTP status code is not handled or not allowed 2026-01-25 03:32:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gray-elevated-plastic-box-truck-4-bushel-capacity>: HTTP status code is not handled or not allowed 2026-01-25 03:32:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-d-flanged-lacing-bar-1903-1-101-00>: HTTP status code is not handled or not allowed 2026-01-25 03:32:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-signs-notice-cylinders-must-be-chained-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 03:32:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ground-pad-for-containment-berm-12l-x-40w-black>: HTTP status code is not handled or not allowed 2026-01-25 03:32:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leverset-single-step-roses-passage-lock-dull-chrome>: HTTP status code is not handled or not allowed 2026-01-25 03:32:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/en53375-solid-five-lobed-knob-075-14-20-3t75ag6>: HTTP status code is not handled or not allowed 2026-01-25 03:32:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-squeegee-set-urethane-for-nobles-tennant-9015738>: HTTP status code is not handled or not allowed 2026-01-25 03:32:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1878-1-4-e-hss-heavy-duty-black-gold-135-split-point-3-flatted-shank-jobber-length-drill>: HTTP status code is not handled or not allowed 2026-01-25 03:32:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1877-14-00mm-black-gold-118-point-3-flat-1-2-reduced-shank-silver-deming-drill>: HTTP status code is not handled or not allowed 2026-01-25 03:32:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/sockets_bits/wood_drill_bits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:32:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacment-actuator-for-nobles-tennant-9005784-nobles-tennant-1050461-nobles-tennant-397721>: HTTP status code is not handled or not allowed 2026-01-25 03:32:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelate-silver-epoxy-wire-divider-24x8>: HTTP status code is not handled or not allowed 2026-01-25 03:32:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1877-11-16-black-gold-118-point-3-flat-1-2-reduced-shank-silver-deming-drill>: HTTP status code is not handled or not allowed 2026-01-25 03:32:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1880-7-16-cobalt-heavy-duty-straw-135-left-hand-mechanics-length-drill>: HTTP status code is not handled or not allowed 2026-01-25 03:32:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u-shaped-reception-station-with-window-88w-x-44d-x-44h-cherry-counter-blue-panel>: HTTP status code is not handled or not allowed 2026-01-25 03:32:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-locker-five-tier-3-wide-12-3-4w-x-18d-x-14-5-8h-tan-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 03:32:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/max-double-long-scissor-lift-table-foot-operated-2000-lb-cap-96l-x-26w-115230v>: HTTP status code is not handled or not allowed 2026-01-25 03:32:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-climax-1c-081-s>: HTTP status code is not handled or not allowed 2026-01-25 03:32:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1896-13-00mm-hss-heavy-duty-steam-oxide-135-split-point-screw-machine-drill>: HTTP status code is not handled or not allowed 2026-01-25 03:32:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0-3-micron-sensor-air-quality-microfilter-for-cvu-30-1-36-1-46-1-8-600-522-0>: HTTP status code is not handled or not allowed 2026-01-25 03:32:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/side-broom-w-water-resistant-bristles-for-km-70-20-km-70-30-6-966-005-0>: HTTP status code is not handled or not allowed 2026-01-25 03:32:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/lockers/standard_lockers/plastic_standard_lockers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:32:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1898-5-10mm-hss-general-purpose-bright-118-point-jobber-length-drill>: HTTP status code is not handled or not allowed 2026-01-25 03:32:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dunnage-shelf-18x60-1>: HTTP status code is not handled or not allowed 2026-01-25 03:32:44 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/sockets_bits/wood_drill_bits landed on page that is not a product page. 2026-01-25 03:32:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-couplings-recessed-screw-climax-cc-150-100>: HTTP status code is not handled or not allowed 2026-01-25 03:32:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-partition-panel-wo-brackets-5712w-red>: HTTP status code is not handled or not allowed 2026-01-25 03:32:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-unground-carbide-blank-316x716x1316-1310>: HTTP status code is not handled or not allowed 2026-01-25 03:32:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1872-tn-7-32-hss-heavy-duty-tin-135-hss-heavy-duty-parabolic-jobber-length-drill>: HTTP status code is not handled or not allowed 2026-01-25 03:32:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/science-table-with-casters-phenolic-top-adjustable-height-24-x-48-black-oak>: HTTP status code is not handled or not allowed 2026-01-25 03:32:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-cotton-flame-resistant-henley-shirt-l-grey>: HTTP status code is not handled or not allowed 2026-01-25 03:32:51 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/lockers/standard_lockers/plastic_standard_lockers landed on page that is not a product page. 2026-01-25 03:32:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacment-vacuum-bag-for-nobles-tennant-613020-proteam-103191>: HTTP status code is not handled or not allowed 2026-01-25 03:32:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polymer-pilaster-w-shoe-5w-x-82h-burgundy>: HTTP status code is not handled or not allowed 2026-01-25 03:32:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wide-mouth-ldpe-wash-bottles-500ml-case-of-24>: HTTP status code is not handled or not allowed 2026-01-25 03:32:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-x-5-x-5-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-25 03:32:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/tables/laboratory_tables already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:32:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polymer-partition-panel-wo-brackets-5412w-moss>: HTTP status code is not handled or not allowed 2026-01-25 03:33:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slotted-cap-with-1-touch-suspension-hi-viz-yellow-green>: HTTP status code is not handled or not allowed 2026-01-25 03:33:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/psg-14-bowl-shape-scraper-14l-silicone>: HTTP status code is not handled or not allowed 2026-01-25 03:33:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slotted-cap-with-staz-on-suspension-black>: HTTP status code is not handled or not allowed 2026-01-25 03:33:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/abc-industries-thermohose-1095-heater-duct-12-dia-x-25l-x-4-pitch-white>: HTTP status code is not handled or not allowed 2026-01-25 03:33:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/tables/laboratory_tables landed on page that is not a product page. 2026-01-25 03:33:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-shovel-14-in-blade-blue>: HTTP status code is not handled or not allowed 2026-01-25 03:33:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 58938 pages (at 92 pages/min), scraped 29510 items (at 48 items/min) 2026-01-25 03:33:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/glass_cleaners already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:33:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/abc-industries-enviroduct-15wk-portable-heater-duct-20-dia-x-25l-x-6-pitch-white>: HTTP status code is not handled or not allowed 2026-01-25 03:33:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-climax-c-500>: HTTP status code is not handled or not allowed 2026-01-25 03:33:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/shelf_trucks/aluminum_shelf_trucks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:33:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrosion-resistant-set-screw-collar-cr-climax-crc-012-s>: HTTP status code is not handled or not allowed 2026-01-25 03:33:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cordless-inspection-light-bare-tool-24v>: HTTP status code is not handled or not allowed 2026-01-25 03:33:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cast-iron-centrifugal-pump-3-4-hp>: HTTP status code is not handled or not allowed 2026-01-25 03:33:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-climax-1c-256>: HTTP status code is not handled or not allowed 2026-01-25 03:33:18 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/glass_cleaners landed on page that is not a product page. 2026-01-25 03:33:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/shelf_trucks/aluminum_shelf_trucks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:33:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-recessed-screw-climax-h1c-193-s>: HTTP status code is not handled or not allowed 2026-01-25 03:33:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/shelf_trucks/aluminum_shelf_trucks landed on page that is not a product page. 2026-01-25 03:33:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/accessory-arc-kit-white-for-slotted-caps-with-chin-protector>: HTTP status code is not handled or not allowed 2026-01-25 03:33:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cap-with-1-touch-suspension-hi-viz-orange>: HTTP status code is not handled or not allowed 2026-01-25 03:33:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-canvas-ce-phenolic-sheet-1-250-x-48-000-x-48-000>: HTTP status code is not handled or not allowed 2026-01-25 03:33:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slotted-full-brim-hat-with-fas-trac-iii-suspension-hi-viz-orange>: HTTP status code is not handled or not allowed 2026-01-25 03:33:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/carts-trucks/shelf_trucks/aluminum_shelf_trucks>: HTTP status code is not handled or not allowed 2026-01-25 03:33:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pp-ssm-screened-pourer-midnight-smoke>: HTTP status code is not handled or not allowed 2026-01-25 03:33:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nashville-wire-steel-cargo-cart-open-front-63w-x-29d-x-72h>: HTTP status code is not handled or not allowed 2026-01-25 03:33:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-linen-le-phenolic-rod-1-500-dia-x-48-000-inch>: HTTP status code is not handled or not allowed 2026-01-25 03:33:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ex4106tcrdbk-41-w-x-25-3-4-d-x-43-7-8-h-6-drawer-red-flip-top-tool-cart-black-pulls>: HTTP status code is not handled or not allowed 2026-01-25 03:33:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/patriot-fully-framed-ta-50-locker-wexpanded-metal-sides-48wx24dx78h-rdunassembled>: HTTP status code is not handled or not allowed 2026-01-25 03:33:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/packaging/stretch_wrap_shrink_wrap/shrink_wrap already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:33:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o-201-general-purpose-polyolefin-cf-shrink-film-50-ga-22w-x-5250l-clear-1-roll>: HTTP status code is not handled or not allowed 2026-01-25 03:33:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/packaging/stretch_wrap_shrink_wrap/shrink_wrap already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:33:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pmcd-14k-double-fold-menu-cover-8-1-2w-14h-plastic-w-vinyl-binding-double-page-black>: HTTP status code is not handled or not allowed 2026-01-25 03:33:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7in-quick-tool-for-use-with-16in-surfacing-machines-3-pack-298239>: HTTP status code is not handled or not allowed 2026-01-25 03:33:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-climax-2c-150>: HTTP status code is not handled or not allowed 2026-01-25 03:33:49 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/packaging/stretch_wrap_shrink_wrap/shrink_wrap landed on page that is not a product page. 2026-01-25 03:33:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-x-30-kra-3-piece-telescoping-tubes-case>: HTTP status code is not handled or not allowed 2026-01-25 03:33:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-linen-phenolic-sheet-0-062-thick-36-x-48>: HTTP status code is not handled or not allowed 2026-01-25 03:33:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/packaging/stretch_wrap_shrink_wrap/shrink_wrap landed on page that is not a product page. 2026-01-25 03:33:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/patriot-fully-framed-duty-locker-24wx24dx90h-red-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 03:33:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ptp-16c-pebbled-tumblers-16-oz-13h>: HTTP status code is not handled or not allowed 2026-01-25 03:33:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kwik-seal-ultra-5-5-oz-clear-7079818915>: HTTP status code is not handled or not allowed 2026-01-25 03:33:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wsecurity-box-footlocker-24wx18dx78hrdall-welded>: HTTP status code is not handled or not allowed 2026-01-25 03:34:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cbr-yl-cutting-board-brush-yellow-looped-handle-for-hanging>: HTTP status code is not handled or not allowed 2026-01-25 03:34:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0030-07-shangarila-oyster-fork>: HTTP status code is not handled or not allowed 2026-01-25 03:34:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qf-router-hood-6-dia-galvanized-steel>: HTTP status code is not handled or not allowed 2026-01-25 03:34:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/afp-8xc-fry-pan>: HTTP status code is not handled or not allowed 2026-01-25 03:34:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wsecurity-box-footlocker-18wx18dx78hblall-welded>: HTTP status code is not handled or not allowed 2026-01-25 03:34:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arborpants-chainsaw-pants-weljeap1xl32-x-large-32-l-black>: HTTP status code is not handled or not allowed 2026-01-25 03:34:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/king-economy-3-section-privacy-screen>: HTTP status code is not handled or not allowed 2026-01-25 03:34:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 59033 pages (at 95 pages/min), scraped 29567 items (at 57 items/min) 2026-01-25 03:34:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/asfp-11ns-non-stick-stir-fry-pan>: HTTP status code is not handled or not allowed 2026-01-25 03:34:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-two-piece-clamping-collar-climax-m2c-24>: HTTP status code is not handled or not allowed 2026-01-25 03:34:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vinyl-stack-chair-black-ace-series>: HTTP status code is not handled or not allowed 2026-01-25 03:34:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-poly-bag-rolls-16x20-2-mil-1000-rl>: HTTP status code is not handled or not allowed 2026-01-25 03:34:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wsecurity-box-footlocker-24wx24dx78hblall-welded>: HTTP status code is not handled or not allowed 2026-01-25 03:34:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coil-cleaner-cleaner-for-ac-coils-fins-cherry-scent-gallon-4-case-nl294-g4>: HTTP status code is not handled or not allowed 2026-01-25 03:34:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commercial-turf-series-gas-engine-vertical-shaft-1-18-x-4-1964-25-hp>: HTTP status code is not handled or not allowed 2026-01-25 03:34:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelf-footlocker-24wx21dx72h-red-all-welded>: HTTP status code is not handled or not allowed 2026-01-25 03:34:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelf-18x30-nexelon>: HTTP status code is not handled or not allowed 2026-01-25 03:34:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-door-cell-phone-locker-cp12-091572-dmgy-4-wide-whasp-36w-x-15d-x-82-34h-med-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:34:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-one-piece-clamping-collar-climax-m1c-28>: HTTP status code is not handled or not allowed 2026-01-25 03:34:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-wire-shelf-21x36>: HTTP status code is not handled or not allowed 2026-01-25 03:34:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tb600-hydraulic-gate-closer>: HTTP status code is not handled or not allowed 2026-01-25 03:34:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-gx4-5ppm-co-cartridge-battery-clip>: HTTP status code is not handled or not allowed 2026-01-25 03:34:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelf-security-box-24wx21dx78h-gray-all-welded>: HTTP status code is not handled or not allowed 2026-01-25 03:34:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:34:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-2-2-non-flammable-gas-tagboard-placard>: HTTP status code is not handled or not allowed 2026-01-25 03:34:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0488-insulated-quilted-sweatshirt-regular-navy-xl>: HTTP status code is not handled or not allowed 2026-01-25 03:34:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-climax-2c-212-z>: HTTP status code is not handled or not allowed 2026-01-25 03:34:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0344r-iron-tuff-minus-50-suit-regular-navy-xl>: HTTP status code is not handled or not allowed 2026-01-25 03:34:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-wire-shelf-54x24>: HTTP status code is not handled or not allowed 2026-01-25 03:34:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/azar-300271-slv-snap-frame-for-freestanding-unit-22-w-x-1-d-x-28-h>: HTTP status code is not handled or not allowed 2026-01-25 03:34:42 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats landed on page that is not a product page. 2026-01-25 03:34:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0381r-iron-tuff-minus-50-suit-regular-navy-xl>: HTTP status code is not handled or not allowed 2026-01-25 03:34:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-performance-starter-battery-s6523b-auto-vm>: HTTP status code is not handled or not allowed 2026-01-25 03:34:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-nova-2000-side-wings-large>: HTTP status code is not handled or not allowed 2026-01-25 03:34:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0442-chillbreaker-parka-regular-navy-2xl>: HTTP status code is not handled or not allowed 2026-01-25 03:34:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-l-x-30-w-14-gauge-stainless-workbench-w-undershelf-le-sink-1>: HTTP status code is not handled or not allowed 2026-01-25 03:34:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-l-x-30-w-14-gauge-stainless-workbench-w-undershelf-le-sink-2>: HTTP status code is not handled or not allowed 2026-01-25 03:34:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fsfp-11-french-fry-pan-carbon-steel>: HTTP status code is not handled or not allowed 2026-01-25 03:34:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0934-econo-tuff-frock-liner-regular-navy-large>: HTTP status code is not handled or not allowed 2026-01-25 03:34:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-locker-single-tier-12x15x72-3-door-assembled-parchment>: HTTP status code is not handled or not allowed 2026-01-25 03:35:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-locker-single-tier-18x18x72-3-door-assembled-parchment>: HTTP status code is not handled or not allowed 2026-01-25 03:35:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galvanized-steel-pallet-40-l-x-48-w-x-4-3-4-h-8000-capacity>: HTTP status code is not handled or not allowed 2026-01-25 03:35:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-z3-magnifying-lens-holder>: HTTP status code is not handled or not allowed 2026-01-25 03:35:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sanding-belt-aluminum-oxide-180-grit-36l-x-4w-pack-of-3>: HTTP status code is not handled or not allowed 2026-01-25 03:35:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:35:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/respiratory_protection/welding_face_masks_respirators already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:35:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metal-storage-cabinet-36x24x72-tan>: HTTP status code is not handled or not allowed 2026-01-25 03:35:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=jtm-4vs-milling-machine-with-newall-dp700-dro-x-y-axis-powerfeeds-3-hp-230460v already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:35:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 59127 pages (at 94 pages/min), scraped 29624 items (at 57 items/min) 2026-01-25 03:35:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-recessed-screw-climax-h2c-268-s>: HTTP status code is not handled or not allowed 2026-01-25 03:35:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-inch-x-5-yards-self-adhesive-bandage-non-latex-dark-blue-pack-of-8>: HTTP status code is not handled or not allowed 2026-01-25 03:35:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-panel-monitor-track-72-lan-station-1>: HTTP status code is not handled or not allowed 2026-01-25 03:35:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-hole-set-screw-collar-c-2h-series-climax-c-025-2h-90>: HTTP status code is not handled or not allowed 2026-01-25 03:35:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/129c-enduramax-boot-regular-black-7>: HTTP status code is not handled or not allowed 2026-01-25 03:35:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n20s-swivel-hose-adapter-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 03:35:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-climax-c-025-s>: HTTP status code is not handled or not allowed 2026-01-25 03:35:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats>: HTTP status code is not handled or not allowed 2026-01-25 03:35:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/respiratory_protection/welding_face_masks_respirators landed on page that is not a product page. 2026-01-25 03:35:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=jtm-4vs-milling-machine-with-newall-dp700-dro-x-y-axis-powerfeeds-3-hp-230460v>: HTTP status code is not handled or not allowed 2026-01-25 03:35:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/129c-enduramax-boot-regular-black-11-5>: HTTP status code is not handled or not allowed 2026-01-25 03:35:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-recessed-screw-climax-h1c-093-s>: HTTP status code is not handled or not allowed 2026-01-25 03:35:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-debris-bags-713165-canister-vacuum-5-pack>: HTTP status code is not handled or not allowed 2026-01-25 03:35:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-inch-x-5-yards-self-adhesive-bandage-non-latex-dark-purple-pack-of-8>: HTTP status code is not handled or not allowed 2026-01-25 03:35:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/surge_protectors_extension_cords/extension_cords already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:35:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cantilever-rack-double-sided-upright-36>: HTTP status code is not handled or not allowed 2026-01-25 03:35:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/636-deluxe-in-line-lubricator-14-npt-150-psi>: HTTP status code is not handled or not allowed 2026-01-25 03:35:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1450110-airflow-conveyor-chain-ps-8e-40012>: HTTP status code is not handled or not allowed 2026-01-25 03:35:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/garage_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:35:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reference-shelf-30-wx15-d-shelf-with-hardware>: HTTP status code is not handled or not allowed 2026-01-25 03:35:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:35:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/surge_protectors_extension_cords/extension_cords>: HTTP status code is not handled or not allowed 2026-01-25 03:35:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-removable-safety-bollard-6-dia-x-36-h-yellow>: HTTP status code is not handled or not allowed 2026-01-25 03:35:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:35:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ba102-universal-back-up-alarm>: HTTP status code is not handled or not allowed 2026-01-25 03:35:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:35:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mmpo-138-oval-melamine-platter-oval-melamine-tan-13-x-8>: HTTP status code is not handled or not allowed 2026-01-25 03:35:44 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/cabinets/garage_cabinets landed on page that is not a product page. 2026-01-25 03:35:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets>: HTTP status code is not handled or not allowed 2026-01-25 03:35:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:35:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-green-3-tier-wire-shelving-starter-unit-60-w-x-14-d-x-34-h>: HTTP status code is not handled or not allowed 2026-01-25 03:35:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-recessed-screw-climax-h1c-112-a>: HTTP status code is not handled or not allowed 2026-01-25 03:35:50 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-25 03:35:51 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-25 03:35:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stand-for-bdb-929-lathe>: HTTP status code is not handled or not allowed 2026-01-25 03:35:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yellow-parking-curb-with-hardware-72-l-x-4-h-x-6-w>: HTTP status code is not handled or not allowed 2026-01-25 03:35:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-tier-shelving-starter-unit-solid-galvanized-steel-42w-x-24d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 03:35:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:35:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls>: HTTP status code is not handled or not allowed 2026-01-25 03:35:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-hose-spring-rewind-hose-reel-for-oxy-acetylene-14-i-d-100-hose-200-psi>: HTTP status code is not handled or not allowed 2026-01-25 03:35:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/room-divider-6-height-9-sections-black>: HTTP status code is not handled or not allowed 2026-01-25 03:36:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arp-13-rolling-pin>: HTTP status code is not handled or not allowed 2026-01-25 03:36:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-side-and-back-panel-kit-in-the-color-black-includes-1-back-panel-and-2-side-panels>: HTTP status code is not handled or not allowed 2026-01-25 03:36:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:36:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:36:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-hardware-kit-for-caster-installation-for-rubbermaid-trademaster-carts>: HTTP status code is not handled or not allowed 2026-01-25 03:36:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/viton-rubber-roll-no-adhesive-75a-132-thick-x-36-wide-x-4-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 03:36:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:36:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-25 03:36:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/70049-2a-flap-wheel-2-inch-x-1-inch-x-1-4-inch-40-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-25 03:36:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fiber-trays-pla-lined-pfas-free-1-compartment-18-x-14-x-1-100-carton>: HTTP status code is not handled or not allowed 2026-01-25 03:36:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 59210 pages (at 83 pages/min), scraped 29667 items (at 43 items/min) 2026-01-25 03:36:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hac-142-deep-cake-pan>: HTTP status code is not handled or not allowed 2026-01-25 03:36:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls>: HTTP status code is not handled or not allowed 2026-01-25 03:36:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fiberglass-tray-13-1-4x10-5-8x1-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:36:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/35422-hook-and-loop-disc-6a-5-inch-dia-120-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-25 03:36:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-climax-c-168-s>: HTTP status code is not handled or not allowed 2026-01-25 03:36:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-25 03:36:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:36:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0010-05-lisa-dinner-fork>: HTTP status code is not handled or not allowed 2026-01-25 03:36:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/gloves_sleeves/disposable_gloves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:36:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fiberglass-tote-lid-42-1-2x20-red-2>: HTTP status code is not handled or not allowed 2026-01-25 03:36:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/notchers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:36:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-25 03:36:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g-450-galvanized-clip-drop-forged-1>: HTTP status code is not handled or not allowed 2026-01-25 03:36:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/l-320cn-s-c-carbon-hook-eye-w-latch-1-1-2t-wll>: HTTP status code is not handled or not allowed 2026-01-25 03:36:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/antimicrobial-gaming-chair-high-back-black-ruby-red>: HTTP status code is not handled or not allowed 2026-01-25 03:36:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-25 03:36:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/utility-bench-vise-with-swivel-base-6-12-jaw-width-3-1316-throat-depth>: HTTP status code is not handled or not allowed 2026-01-25 03:36:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/gloves_sleeves/disposable_gloves landed on page that is not a product page. 2026-01-25 03:36:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/notchers>: HTTP status code is not handled or not allowed 2026-01-25 03:36:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:36:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/120-ton-hydraulic-ironworker-w-powerlink-5-tooling-stations-3ph-208-v>: HTTP status code is not handled or not allowed 2026-01-25 03:36:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/respiratory_protection/disposable_face_masks_respirators already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:36:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-sheet-no-adhesive-50a-14-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 03:36:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/drilling_hole_making/punches_dies/knockout_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:36:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/confectionary-scoop>: HTTP status code is not handled or not allowed 2026-01-25 03:36:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/binders_covers/binders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:36:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-standard-soft-face-hammer-131-lb-head-9-38-steel-handle>: HTTP status code is not handled or not allowed 2026-01-25 03:36:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/din-508-t-slot-nuts-steel-blackened-m12-1-w-x-11-16-h>: HTTP status code is not handled or not allowed 2026-01-25 03:36:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/binders_covers/binders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:36:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-roll-no-adhesive-50a-14-thick-x-36-wide-x-7-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 03:36:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/machining/drilling_hole_making/punches_dies/knockout_kits>: HTTP status code is not handled or not allowed 2026-01-25 03:36:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-25 03:36:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gn-251-6-setting-bolts-steel-w-retaining-magnet-zinc-pltd-m6-3-8-mag-dia-5-8-l>: HTTP status code is not handled or not allowed 2026-01-25 03:36:45 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/respiratory_protection/disposable_face_masks_respirators landed on page that is not a product page. 2026-01-25 03:36:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g-450-galvanized-clip-drop-forged-1-8>: HTTP status code is not handled or not allowed 2026-01-25 03:36:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/binders_covers/binders landed on page that is not a product page. 2026-01-25 03:36:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-cart-poly-green-24l-x-18w-x-40h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 03:36:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/workholding_clamps_clamp_components/clamp_components_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:36:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/record-storage-rack-gray-42-x30-x60>: HTTP status code is not handled or not allowed 2026-01-25 03:36:50 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/binders_covers/binders landed on page that is not a product page. 2026-01-25 03:36:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/241-punch-assembly-for-65t-75t-100t-ironworkers>: HTTP status code is not handled or not allowed 2026-01-25 03:36:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pullman-holt-hepa-filter-asy-16-for-102asb>: HTTP status code is not handled or not allowed 2026-01-25 03:36:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/hand_tools_1/hammers_mallets/dead_blow_hammers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:36:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-carbon-cartridge-for-aerosolv-dual-price-for-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-25 03:36:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/l100-150wo-10-hand-chain-hoist-with-overload-protection-1-12-ton-capacity-10-lift>: HTTP status code is not handled or not allowed 2026-01-25 03:36:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delfin-12-gallon-hepa-vacuum-1-5-hp-mistral-451-bl-d2>: HTTP status code is not handled or not allowed 2026-01-25 03:36:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leeson-3-phase-washguard-duty-motor-5-33hp-1140-940rpm-56h-tenv-208-230-460v-60-50hz-rigid-c>: HTTP status code is not handled or not allowed 2026-01-25 03:37:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-tarp-system-with-mesh-tarp-5-ft-x-18-ft-dtr8018>: HTTP status code is not handled or not allowed 2026-01-25 03:37:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=hydraulic-h-frame-press-3-hp-3-phase-220v-hsp-110m-hd already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:37:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/hand_tools_1/hammers_mallets/dead_blow_hammers>: HTTP status code is not handled or not allowed 2026-01-25 03:37:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-sheet-no-adhesive-60a-132-thick-x-36-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 03:37:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=multi-purpose-manual-sheet-metal-shear-14-gauge-mild-steel-capacity already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:37:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bridges_returns already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:37:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tyvek-400-sfr-coverall-open-wrists--ankle-storm-flap-serged-seams-3xl-white-25pk>: HTTP status code is not handled or not allowed 2026-01-25 03:37:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/workholding_clamps_clamp_components/clamp_components_accessories landed on page that is not a product page. 2026-01-25 03:37:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pem-aluminum-saddle-threshold-72-l-x-6-w-1-2-h-mill-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 03:37:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 59290 pages (at 80 pages/min), scraped 29704 items (at 37 items/min) 2026-01-25 03:37:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-tier-15-door-clear-view-locker-digital-lock-36-inch-w-x-12-inch-d-x-66-inch-h-gray-assembled>: HTTP status code is not handled or not allowed 2026-01-25 03:37:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=hydraulic-h-frame-press-3-hp-3-phase-220v-hsp-110m-hd>: HTTP status code is not handled or not allowed 2026-01-25 03:37:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-tier-6-door-clear-view-locker-digital-lock12-inch-w-x-18-inch-d-x-78-inch-hgray-assembled>: HTTP status code is not handled or not allowed 2026-01-25 03:37:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-general-security-laminated-padlocks-no-25kalf>: HTTP status code is not handled or not allowed 2026-01-25 03:37:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/313-carton-sealing-tape-2-x-100-yds-25-mil-clear-36pack>: HTTP status code is not handled or not allowed 2026-01-25 03:37:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=multi-purpose-manual-sheet-metal-shear-14-gauge-mild-steel-capacity landed on page that is not a product page. 2026-01-25 03:37:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m18-redlithium-high-output-cp3-0-starter-kit>: HTTP status code is not handled or not allowed 2026-01-25 03:37:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bridges_returns landed on page that is not a product page. 2026-01-25 03:37:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:37:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/23-2-l-x-11-1-w-vinyl-wall-tile-venetian-marble-10-pack>: HTTP status code is not handled or not allowed 2026-01-25 03:37:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/311-carton-sealing-tape-2-x-110-yds-205-mil-clear-6pack>: HTTP status code is not handled or not allowed 2026-01-25 03:37:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/94-l-inside-corner-trim-in-adobe-drift-5-pack>: HTTP status code is not handled or not allowed 2026-01-25 03:37:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:37:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caster-steer-trailer-36-x-60>: HTTP status code is not handled or not allowed 2026-01-25 03:37:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25-6-l-x-14-8-w-vinyl-wall-tile-adobe-drift-8-pack>: HTTP status code is not handled or not allowed 2026-01-25 03:37:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-25 03:37:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/375-carton-sealing-tape-3-x-55-yds-31-mil-tan-24pack>: HTTP status code is not handled or not allowed 2026-01-25 03:37:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carton-sealing-tape-if-seal-has-been-3-x-110-yds-bluewhite-6pack>: HTTP status code is not handled or not allowed 2026-01-25 03:37:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-25 03:37:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/itembk200-200-amp-main-breaker-conversion-kit>: HTTP status code is not handled or not allowed 2026-01-25 03:37:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vpkubif0235n-circuit-breaker-type-ubif-thin-2-pole-35a-clamshell-packaged>: HTTP status code is not handled or not allowed 2026-01-25 03:37:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-part-carbonless-inventory-tag-pre-wired-5500-5999-8-6-14l-x-3-18w-500pack>: HTTP status code is not handled or not allowed 2026-01-25 03:37:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:37:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-shelf-truck-5-lip-shelves-30-x-48-w-writing-shelf-document-pocket>: HTTP status code is not handled or not allowed 2026-01-25 03:37:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-high-security-steel-weather-resistant-covered-laminated-padlocks-no-6121kab>: HTTP status code is not handled or not allowed 2026-01-25 03:37:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o12008-magnaprene-plain-front-overall-green-2xl>: HTTP status code is not handled or not allowed 2026-01-25 03:37:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-shelving-closed-shelving-36x18x75-3>: HTTP status code is not handled or not allowed 2026-01-25 03:37:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/j26119-bomber-ii-hooded-jacket-fluorescent-orangeredblack-lg>: HTTP status code is not handled or not allowed 2026-01-25 03:37:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-25 03:37:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:37:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o22001-iron-eagle-plain-front-overall-blue-snaplock-buckles-3xl>: HTTP status code is not handled or not allowed 2026-01-25 03:37:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/permaloc-wire-rope-sling-1-4x6-eye-to-eye>: HTTP status code is not handled or not allowed 2026-01-25 03:37:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-roll-with-acrylic-adhesive-50a-132-thick-x-36-wide-x-10-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 03:37:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:37:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hytrol-5-ft-28-inch-w-1-9-inch-dia-galvanized-rollers-25-inch-between-rail-4-5-inch-rollers-centers>: HTTP status code is not handled or not allowed 2026-01-25 03:37:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hytrol-5-ft-18-inch-w-1-9-inch-dia-galvanized-rollers-15-inch-between-rail-3-inch-rollers-centers>: HTTP status code is not handled or not allowed 2026-01-25 03:37:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-delrin-homopolymer-sheet-2-000-thick-x-12-w-x-24-l>: HTTP status code is not handled or not allowed 2026-01-25 03:37:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-25 03:37:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-delrin-homopolymer-sheet-0-250-thick-x-12-w-x-24-l>: HTTP status code is not handled or not allowed 2026-01-25 03:38:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-delrin-homopolymer-sheet-0-375-thick-x-24-000-w-x-48-000-l>: HTTP status code is not handled or not allowed 2026-01-25 03:38:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-delrin-homopolymer-sheet-1-500-thick-x-24-000-w-x-48-000-l>: HTTP status code is not handled or not allowed 2026-01-25 03:38:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips landed on page that is not a product page. 2026-01-25 03:38:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paramount-locker-six-tier-12x12x12-6-door-assembled-tan>: HTTP status code is not handled or not allowed 2026-01-25 03:38:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-parts-storage-drawer-5-3-8-w-x-11-1-4-d-x-3-1-2-h>: HTTP status code is not handled or not allowed 2026-01-25 03:38:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:38:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/pneumatic_hoses_connectors/pneumatic_quick_connect_couplings already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:38:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-315-coupling-1-2-hose-barb-asian-interchange>: HTTP status code is not handled or not allowed 2026-01-25 03:38:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 59379 pages (at 89 pages/min), scraped 29753 items (at 49 items/min) 2026-01-25 03:38:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:38:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-delrin-homopolymer-sheet-1-250-thick-x-12-w-x-12-l>: HTTP status code is not handled or not allowed 2026-01-25 03:38:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merit-red-brass-seamless-standard-14-Close-npt-male>: HTTP status code is not handled or not allowed 2026-01-25 03:38:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-garment-tag-1-78l-x-1-14w-green-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 03:38:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hiland-fire-pit-waterproof-cover-hvd-rfp-cvr-tan-37l-x-29w-x-22h-fits-gsf-rfp>: HTTP status code is not handled or not allowed 2026-01-25 03:38:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/pneumatic_hoses_connectors/pneumatic_quick_connect_couplings>: HTTP status code is not handled or not allowed 2026-01-25 03:38:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-25 03:38:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-roll-no-adhesive-50a-14-thick-x-36-wide-x-8-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 03:38:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-delrin-homopolymer-sheet-0-500-thick-x-24-w-x-48-l>: HTTP status code is not handled or not allowed 2026-01-25 03:38:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merchandise-tag-pre-strung-1316l-x-38w-white-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 03:38:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/cable_management/cable_protectors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:38:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-channel-cable-protector-one-inch-channel-orange-black-lg1100>: HTTP status code is not handled or not allowed 2026-01-25 03:38:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-25 03:38:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-x5-8-11-z3-60-t27-reg100-za-flap-disc>: HTTP status code is not handled or not allowed 2026-01-25 03:38:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-shipping-tag-pre-wired-6-14l-x-3-18w-green-100pack>: HTTP status code is not handled or not allowed 2026-01-25 03:38:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/j56107-durascrim-storm-fly-front-hooded-jacket-yellow-md>: HTTP status code is not handled or not allowed 2026-01-25 03:38:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gn471-crank-handles-aluminum-revolving-handle-through-round-bore-12mm-3-94-long>: HTTP status code is not handled or not allowed 2026-01-25 03:38:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v41108-safetyflex-zipper-fly-front-hooded-coverall-lg>: HTTP status code is not handled or not allowed 2026-01-25 03:38:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/cable_management/cable_protectors landed on page that is not a product page. 2026-01-25 03:38:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-x7-8-z3-40-t27-xl100-za-flap-disc>: HTTP status code is not handled or not allowed 2026-01-25 03:38:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/abrasives/flap_specialty_wheels/deburring_surface_conditioning_wheels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:38:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-tag-pre-wired-4-4-14l-x-2-18w-fluorescent-orange-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 03:38:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slatwall-4-way-display-fixture-display-fixture-black-with-spinner-base-and-casters>: HTTP status code is not handled or not allowed 2026-01-25 03:38:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/35211-steel-toe-pvc-overshoes-blackyellow-sm>: HTTP status code is not handled or not allowed 2026-01-25 03:38:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-ft-usb-20-extension-cable-a-to-a-m-f-black>: HTTP status code is not handled or not allowed 2026-01-25 03:38:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-round-outdoor-cafe-table-29-h-blue>: HTTP status code is not handled or not allowed 2026-01-25 03:38:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/power_tools/power_sanders/abrasive_tool_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:38:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/j56207-durascrim-storm-fly-front-jacket-yellow-hood-snaps-3xl>: HTTP status code is not handled or not allowed 2026-01-25 03:38:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-roll-no-adhesive-50a-18-thick-x-36-wide-x-6-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 03:38:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/abrasives/flap_specialty_wheels/deburring_surface_conditioning_wheels>: HTTP status code is not handled or not allowed 2026-01-25 03:38:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-ketron-peek-rod-q-1-500-dia>: HTTP status code is not handled or not allowed 2026-01-25 03:38:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grease-system-120-lb>: HTTP status code is not handled or not allowed 2026-01-25 03:38:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tygon-b44-3-tube-aab00062-1-000-id-x-1-250-od-x-50-l>: HTTP status code is not handled or not allowed 2026-01-25 03:38:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/13-shelf-steel-shelving-with-81-akrobins-36x12x75>: HTTP status code is not handled or not allowed 2026-01-25 03:38:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/power_tools/power_sanders/abrasive_tool_accessories>: HTTP status code is not handled or not allowed 2026-01-25 03:38:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-oh-two-mid-back-task-chairs-250-lbs-cap-black-seat-black-frame>: HTTP status code is not handled or not allowed 2026-01-25 03:38:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-oil-filled-cast-nylon-tube-6-id-x-8-od-x-13-l>: HTTP status code is not handled or not allowed 2026-01-25 03:38:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ketron-hpv-bearing-grade-peek-0-250-thick-x-20-670-w-x-39-400-l>: HTTP status code is not handled or not allowed 2026-01-25 03:38:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:38:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3m-s-b-8x1x3-8smed048011-18279>: HTTP status code is not handled or not allowed 2026-01-25 03:38:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-sheet-no-adhesive-50a-14-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 03:38:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s56307-durascrim-3-pc-suit-yellow-detachable-hood-md>: HTTP status code is not handled or not allowed 2026-01-25 03:38:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-rack-wood-deck-starter-4-level-gray-7>: HTTP status code is not handled or not allowed 2026-01-25 03:38:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/abrasives/grinding_cutoff_wheels/grinding_cut_off_wheel_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:38:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-tag-12-8l-x-4w-dark-blue-500pack>: HTTP status code is not handled or not allowed 2026-01-25 03:38:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-25 03:39:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11-step-industrial-access-stairway-ladder-grip-strut>: HTTP status code is not handled or not allowed 2026-01-25 03:39:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylatron-nsm-sheet-2-000-thick-x-24-w-x-48-l>: HTTP status code is not handled or not allowed 2026-01-25 03:39:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triple-wall-corrugated-sheets-48l-x-40w-kraft-5pack>: HTTP status code is not handled or not allowed 2026-01-25 03:39:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mb922b-neoprene-steel-toe-snugleg-boots-brick-redbrown-size-11>: HTTP status code is not handled or not allowed 2026-01-25 03:39:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o31007-webdri-plain-front-overall-yellow-2xl>: HTTP status code is not handled or not allowed 2026-01-25 03:39:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/abrasives/grinding_cutoff_wheels/grinding_cut_off_wheel_accessories>: HTTP status code is not handled or not allowed 2026-01-25 03:39:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/heaters/gas_heaters_furnaces/portable_gas_oil_kerosene_heaters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:39:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gorbel-hd-wall-bracket-jib-crane-14-ft-span-and-200and-176-rotation-2000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 03:39:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 59473 pages (at 94 pages/min), scraped 29800 items (at 47 items/min) 2026-01-25 03:39:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wide-span-rack-96x24x84-gray-with-3-shelves-laminated-deck-1100-lb-cap-per-level>: HTTP status code is not handled or not allowed 2026-01-25 03:39:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mh5011l-30-magnetic-wall-mount-barrier-w-light-kit-do-not-enter-arc-flash-boundary>: HTTP status code is not handled or not allowed 2026-01-25 03:39:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gorbel-steel-gantry-crane-25-ft-span-and-15-ft-fixed-height-2000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 03:39:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/53-natural-rubber-tape-2-x-110-yds-21-mil-tan-6pack>: HTTP status code is not handled or not allowed 2026-01-25 03:39:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gorbel-steel-gantry-crane-20-ft-span-and-15-ft-18-ft-adjustable-height-4000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 03:39:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/heaters/gas_heaters_furnaces/portable_gas_oil_kerosene_heaters>: HTTP status code is not handled or not allowed 2026-01-25 03:39:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-height-welded-workbench-w-open-base-30-x-48>: HTTP status code is not handled or not allowed 2026-01-25 03:39:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gorbel-steel-gantry-crane-12-ft-span-and-9-ft-12-ft-adjustable-height-4000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 03:39:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gift-boxes-9l-x-9w-x-5-12h-white-50pack>: HTTP status code is not handled or not allowed 2026-01-25 03:39:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-caster-kit-wood-steel-deck-narrow-aisle-platform-trucks>: HTTP status code is not handled or not allowed 2026-01-25 03:39:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/gym_fitness_equipment/sports_equipment already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:39:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/work-benches/workbenches_worktables/open_base_workbenches/stationary_open_base_workbenches/heavy_duty_stationary_workbenches already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:39:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premier-high-durability-no-buff-finish-unscented-5-gallon-box-1-box-904015>: HTTP status code is not handled or not allowed 2026-01-25 03:39:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/welded-steel-shelving-5-shelves-24-x-36>: HTTP status code is not handled or not allowed 2026-01-25 03:39:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nomex-410-sheet-0-007-thick-x-24-000-w-x-36-000-l>: HTTP status code is not handled or not allowed 2026-01-25 03:39:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-vinyl-tubing-5-8-i-d-x-1-16-wall-100-feet-reel>: HTTP status code is not handled or not allowed 2026-01-25 03:39:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/office_furniture/literature_mail_sorters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:39:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/gym_fitness_equipment/sports_equipment>: HTTP status code is not handled or not allowed 2026-01-25 03:39:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/work-benches/workbenches_worktables/open_base_workbenches/stationary_open_base_workbenches/heavy_duty_stationary_workbenches>: HTTP status code is not handled or not allowed 2026-01-25 03:39:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-welded-storage-locker-24-x-60>: HTTP status code is not handled or not allowed 2026-01-25 03:39:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g5-116t-aff-7-5hp-oil-injected-rotary-screw-71-gal-tank-116psi-1ph-230v-w-dryer>: HTTP status code is not handled or not allowed 2026-01-25 03:39:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machinery-guard-round-48x48>: HTTP status code is not handled or not allowed 2026-01-25 03:39:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/office_furniture/literature_mail_sorters>: HTTP status code is not handled or not allowed 2026-01-25 03:39:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fresh-products-easy-fresh-dispenser-white-12-dispensers-case>: HTTP status code is not handled or not allowed 2026-01-25 03:39:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/walls_ceilings/ceiling_tiles already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:39:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-cd-mailers-5-58l-x-5w-x-2-916h-white-50pack>: HTTP status code is not handled or not allowed 2026-01-25 03:39:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/wire_shelving/poly_zbrite_wire_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:39:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/great-lakes-tin-toledo-2-x-2-nail-up-tin-ceiling-tile-in-matte-white-t55-01>: HTTP status code is not handled or not allowed 2026-01-25 03:39:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/libman-commercial-18-inch-smooth-sweep-complete-set>: HTTP status code is not handled or not allowed 2026-01-25 03:39:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/54h-nexel-chrome-wire-shelving-add-on-24wx21d>: HTTP status code is not handled or not allowed 2026-01-25 03:39:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fresh-products-wave-3d-urinal-deodorizer-screen-blue-cotton-blossom-10-screens-box>: HTTP status code is not handled or not allowed 2026-01-25 03:39:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hopper-bins-10-7-8w-x-17-1-2d-x-12-1-2h-4>: HTTP status code is not handled or not allowed 2026-01-25 03:39:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/walls_ceilings/ceiling_tiles landed on page that is not a product page. 2026-01-25 03:39:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fresh-products-tidal-wave-urinal-screens-cucumber-melon-scent-6-box>: HTTP status code is not handled or not allowed 2026-01-25 03:39:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/walls_ceilings/ceiling_tiles already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:39:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultrasonic-table-top-parts-cleaning-system-digital-timer-heat-2-5-gal-45-khz-230v>: HTTP status code is not handled or not allowed 2026-01-25 03:39:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clutch-upper-gear-1230r-5234111>: HTTP status code is not handled or not allowed 2026-01-25 03:39:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-dunnage-rack-23x17x8>: HTTP status code is not handled or not allowed 2026-01-25 03:39:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/wire_shelving/poly_zbrite_wire_shelves landed on page that is not a product page. 2026-01-25 03:39:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/collomix-mk160hf-heavy-duty-3-blade-paddle-large-batch>: HTTP status code is not handled or not allowed 2026-01-25 03:40:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bnwl1818-18-x-18-drywall-bead-flange-key-operated-lock>: HTTP status code is not handled or not allowed 2026-01-25 03:40:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/walls_ceilings/ceiling_tiles>: HTTP status code is not handled or not allowed 2026-01-25 03:40:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/newaire-plugin-room-deodorizer>: HTTP status code is not handled or not allowed 2026-01-25 03:40:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-shaft-1230r-5235111>: HTTP status code is not handled or not allowed 2026-01-25 03:40:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/plumbing_fittings/pipe_clamps_fittings/brass_pipe_fittings already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:40:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/panel-kit-72-w-workbench-with-18-w-pegboard-36-w-louver-mounting-rail-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:40:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 59564 pages (at 91 pages/min), scraped 29852 items (at 52 items/min) 2026-01-25 03:40:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bitk2222-22-x-22-exposed-flange-knurled-knob>: HTTP status code is not handled or not allowed 2026-01-25 03:40:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-purpose-cleaner-morning-meadow-23-oz-trigger-spray-bottle-8-case>: HTTP status code is not handled or not allowed 2026-01-25 03:40:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shampoo-light-green-gold-pleasant-scent-0-75-oz-bottle-288-case>: HTTP status code is not handled or not allowed 2026-01-25 03:40:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/mailboxes_drop_boxes/multi_unit_mailboxes_enclosures already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:40:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chain-adapter-1230r-5235981>: HTTP status code is not handled or not allowed 2026-01-25 03:40:18 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/plumbing/plumbing_fittings/pipe_clamps_fittings/brass_pipe_fittings landed on page that is not a product page. 2026-01-25 03:40:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/dunnage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:40:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-dunnage-rack-48-w-x-18-d-2>: HTTP status code is not handled or not allowed 2026-01-25 03:40:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/embassy-sic-90-inside-corner-90-deg>: HTTP status code is not handled or not allowed 2026-01-25 03:40:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnum-pro-250lx-gt-spool-gun-25l-cable>: HTTP status code is not handled or not allowed 2026-01-25 03:40:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keyguard-lab-coat-no-pockets-elastic-wrists-snap-front-single-collar-white-lg-30case>: HTTP status code is not handled or not allowed 2026-01-25 03:40:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/mailboxes_drop_boxes/multi_unit_mailboxes_enclosures>: HTTP status code is not handled or not allowed 2026-01-25 03:40:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/utility_racks/dunnage_racks>: HTTP status code is not handled or not allowed 2026-01-25 03:40:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7020-stop-switch-5713351>: HTTP status code is not handled or not allowed 2026-01-25 03:40:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-standard-particulate-filter-w-metal-bowl>: HTTP status code is not handled or not allowed 2026-01-25 03:40:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/welding/arc_welding/multi_process_welding_machines already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:40:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-zone-portable-fridge-freezer>: HTTP status code is not handled or not allowed 2026-01-25 03:40:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-antibacterial-dishwashing-liquid-20-oz-bottle>: HTTP status code is not handled or not allowed 2026-01-25 03:40:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/lockers/bulk_storage_lockers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:40:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavenly-soft-center-pull-towel-2-ply-7-6-x-9-white-600-towels-roll-6-rolls-cs>: HTTP status code is not handled or not allowed 2026-01-25 03:40:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/lockers/bulk_storage_lockers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:40:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dispatcher-locker-triple-front-door-single-rear-door-36w-x-25-1-2d-x-79-1-2h>: HTTP status code is not handled or not allowed 2026-01-25 03:40:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disinfectant-toilet-bowl-cleaner-with-bleach-24-oz-bottle-9-case>: HTTP status code is not handled or not allowed 2026-01-25 03:40:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/welding/arc_welding/multi_process_welding_machines landed on page that is not a product page. 2026-01-25 03:40:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hyperfill-reveal153-power-feed-84-dual-multi-process-welder-red>: HTTP status code is not handled or not allowed 2026-01-25 03:40:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lightweight-nylon-hairnet-100-latex-free-black-28-144box-10-boxescase>: HTTP status code is not handled or not allowed 2026-01-25 03:40:44 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/lockers/bulk_storage_lockers landed on page that is not a product page. 2026-01-25 03:40:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/medical_cabinets/multi_use_wall_mount_medical_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:40:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/lockers/bulk_storage_lockers>: HTTP status code is not handled or not allowed 2026-01-25 03:40:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd-polypropylene-coverall-elastic-wrists-ankles-attached-hood-zipper-front-white-5xl-25cs>: HTTP status code is not handled or not allowed 2026-01-25 03:40:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-cutter-for-3-4-1-1-4-steel-strapping>: HTTP status code is not handled or not allowed 2026-01-25 03:40:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crimper-for-steel-strapping-1-2-w-x-023-thickness>: HTTP status code is not handled or not allowed 2026-01-25 03:40:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/otg-visitor-safety-glasses-clear-lens-frame>: HTTP status code is not handled or not allowed 2026-01-25 03:40:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/cabinets/medical_cabinets/multi_use_wall_mount_medical_cabinets>: HTTP status code is not handled or not allowed 2026-01-25 03:40:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/decorative-bollard-cover-stone-finish-6-in-dia-60-in-h>: HTTP status code is not handled or not allowed 2026-01-25 03:40:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vogue-outdoor-stackable-bar-stool-black-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-25 03:41:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_bottles_jars already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:41:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-adjustable-solid-brute-shelf-18-wx42-l>: HTTP status code is not handled or not allowed 2026-01-25 03:41:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-l-deluxe-save-t-loc-strip-curtain-door-mounting-bracket-kit>: HTTP status code is not handled or not allowed 2026-01-25 03:41:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/task_desk_chairs/task_office_chairs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:41:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corner-steel-stakes-mpcr-4s-19-14-l-x-1-14-dia-set-of-4>: HTTP status code is not handled or not allowed 2026-01-25 03:41:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jet-jtm-949evs-mill-3-axis-newall-dp700-dro-quill-x-y-and-z-axis-powerfeeds-690650>: HTTP status code is not handled or not allowed 2026-01-25 03:41:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/square-waste-windshield-center-black>: HTTP status code is not handled or not allowed 2026-01-25 03:41:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 59655 pages (at 91 pages/min), scraped 29908 items (at 56 items/min) 2026-01-25 03:41:09 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_bottles_jars landed on page that is not a product page. 2026-01-25 03:41:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-32w-x-44l-2-mil-clear-200pack>: HTTP status code is not handled or not allowed 2026-01-25 03:41:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/drilling_hole_making/metalworking_drill_bits/drill_blanks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:41:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-adjustable-solid-brute-shelf-20-wx66-l>: HTTP status code is not handled or not allowed 2026-01-25 03:41:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triumph-style-t1d-hss-jobbers-drill-black-oxide-18-12>: HTTP status code is not handled or not allowed 2026-01-25 03:41:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/task_desk_chairs/task_office_chairs landed on page that is not a product page. 2026-01-25 03:41:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triumph-style-t9-hss-reduced-shank-drill-black-oxide-3564>: HTTP status code is not handled or not allowed 2026-01-25 03:41:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/tool_storage/chests_roller_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:41:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/short-sleeve-wicking-birdseye-t-shirt-w-pocket-hi-vis-yellow-4xl>: HTTP status code is not handled or not allowed 2026-01-25 03:41:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:41:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/machining/drilling_hole_making/metalworking_drill_bits/drill_blanks>: HTTP status code is not handled or not allowed 2026-01-25 03:41:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/wall_mount_belts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:41:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-corner-guard-with-safety-first-message-24-h-center-24-l-x-12-h-side>: HTTP status code is not handled or not allowed 2026-01-25 03:41:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16oz-natural-hdpe-dairy-jug-with-38-400-white-pp-pe-foam-lined-cap-400pk>: HTTP status code is not handled or not allowed 2026-01-25 03:41:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-4010-double-strap-wrist-support-black-medium-left>: HTTP status code is not handled or not allowed 2026-01-25 03:41:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/neoprene-steel-pad-with-friction-pad-8-x-8-x-1-1-4>: HTTP status code is not handled or not allowed 2026-01-25 03:41:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions landed on page that is not a product page. 2026-01-25 03:41:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/19-x-65mm-3-dram-clear-vial-w-15-425-black-phenolic-polycone-lined-cap-attached-144pk>: HTTP status code is not handled or not allowed 2026-01-25 03:41:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/tool_storage/chests_roller_cabinets landed on page that is not a product page. 2026-01-25 03:41:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/wall_mount_belts>: HTTP status code is not handled or not allowed 2026-01-25 03:41:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:41:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25-x-56mm-7-dram-clear-ps-snap-cap-vial-with-32mm-white-pp-linerless-snap-cap-144pk>: HTTP status code is not handled or not allowed 2026-01-25 03:41:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/queuepro-twin-black-post-retracting-belt-barrier-ada-compliant-11-ft-black-belt>: HTTP status code is not handled or not allowed 2026-01-25 03:41:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wallpro-twin-yellow-post-retracting-belt-barrier-13-ft-yellow-authorized-belt>: HTTP status code is not handled or not allowed 2026-01-25 03:41:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8915-class-e-rain-pants-lime-2-x-large>: HTTP status code is not handled or not allowed 2026-01-25 03:41:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:41:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/breathable-rain-jacket-class-3-hi-vis-yellow-l>: HTTP status code is not handled or not allowed 2026-01-25 03:41:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions landed on page that is not a product page. 2026-01-25 03:41:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nut-lock-n-06-6549006>: HTTP status code is not handled or not allowed 2026-01-25 03:41:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slat-wall-mouinting-bracket-adjustable-for-small-medium-or-large-dispensing-bins>: HTTP status code is not handled or not allowed 2026-01-25 03:41:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/retainer-ring-jdp-14jmjf14d-10606801>: HTTP status code is not handled or not allowed 2026-01-25 03:41:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8981-filament-tape-34-x-60-yds-66-mil-clear-12pack>: HTTP status code is not handled or not allowed 2026-01-25 03:41:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions landed on page that is not a product page. 2026-01-25 03:41:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bellman-hotel-cart-41x24-satin-aluminum-with-black-carpet-pneumatic-wheels>: HTTP status code is not handled or not allowed 2026-01-25 03:41:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_vials_tubes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:41:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-3w-x-30l-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 03:41:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grid-max-100-sq-ft-kit-vinyl-argent-bronze-282-28>: HTTP status code is not handled or not allowed 2026-01-25 03:41:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nut-9180-232>: HTTP status code is not handled or not allowed 2026-01-25 03:41:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-16-dia-1-8-shank-3-16-loc-1-1-2-oal-2-flute-solid-carbide-ball-single-end-mill-uncoated>: HTTP status code is not handled or not allowed 2026-01-25 03:41:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/binders_covers/dividers_tabs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:41:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/128oz-clear-jug-w-38-400-black-phenolic-polycone-cap-4pk>: HTTP status code is not handled or not allowed 2026-01-25 03:41:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/punches_1 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:41:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-16-dia-1-8-shank-3-16-loc-1-1-2-oal-4-flute-solid-carbide-single-end-mill-altin>: HTTP status code is not handled or not allowed 2026-01-25 03:41:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_vials_tubes landed on page that is not a product page. 2026-01-25 03:42:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/windshield-towel-9-1-8-x-10-1-4-blue-140-pack-16-packs-carton-192122>: HTTP status code is not handled or not allowed 2026-01-25 03:42:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/notebooks_pads already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:42:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-26w-x-36l-4-mil-clear-200pack>: HTTP status code is not handled or not allowed 2026-01-25 03:42:04 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/binders_covers/dividers_tabs landed on page that is not a product page. 2026-01-25 03:42:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caution-watch-your-step-7x10-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 03:42:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/sticky_notes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:42:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-18w-x-26l-2-mil-clear-500pack>: HTTP status code is not handled or not allowed 2026-01-25 03:42:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 59741 pages (at 86 pages/min), scraped 29954 items (at 46 items/min) 2026-01-25 03:42:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/sticky_notes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:42:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/colored-paper-20lb-812-x-11-green-500-sheetsream>: HTTP status code is not handled or not allowed 2026-01-25 03:42:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/punches_1 landed on page that is not a product page. 2026-01-25 03:42:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/danger-no-trespassing-7x10-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 03:42:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/paper_stationery/notebooks_pads landed on page that is not a product page. 2026-01-25 03:42:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nemesis-indoor-outdoor-lens-safety-glasses-3000357>: HTTP status code is not handled or not allowed 2026-01-25 03:42:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/cash_register_pos_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:42:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/paper_stationery/sticky_notes>: HTTP status code is not handled or not allowed 2026-01-25 03:42:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/paper_stationery/sticky_notes>: HTTP status code is not handled or not allowed 2026-01-25 03:42:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/411kas3teal-safety-padlock-1-12w-x-1-12h-teal-3set>: HTTP status code is not handled or not allowed 2026-01-25 03:42:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/notebooks_pads already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:42:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-6w-x-14l-4-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 03:42:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spacer-mdl-bdb-1340a-bdb1340a-02124>: HTTP status code is not handled or not allowed 2026-01-25 03:42:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sparkplugs-disp-earplugcorded-2000-per-case-6654>: HTTP status code is not handled or not allowed 2026-01-25 03:42:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-6w-x-28l-4-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 03:42:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/406kas3ylw-safety-padlock-yellow-3set>: HTTP status code is not handled or not allowed 2026-01-25 03:42:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-jaw-self-centering-puller-1ton-capacity-1-4-to-3-3-16-open-3-3-16-reach>: HTTP status code is not handled or not allowed 2026-01-25 03:42:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fork-separator-3-4-18-mm-opening>: HTTP status code is not handled or not allowed 2026-01-25 03:42:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/paper_stationery/cash_register_pos_rolls landed on page that is not a product page. 2026-01-25 03:42:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rope-grab-cobra-lg-eye-ac202d>: HTTP status code is not handled or not allowed 2026-01-25 03:42:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/paper_stationery/notebooks_pads landed on page that is not a product page. 2026-01-25 03:42:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/probes-44-1220-sou1181996>: HTTP status code is not handled or not allowed 2026-01-25 03:42:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metaltech-steel-maxi-drywall-baker-scaffold-73l-x-29w-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 03:42:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s33red-snap-lock-safety-padlock-1-38w-x-1-12h-red>: HTTP status code is not handled or not allowed 2026-01-25 03:42:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delta-no-tangle-body-harness-vest-style-1102000>: HTTP status code is not handled or not allowed 2026-01-25 03:42:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ez-line-60-ft-retractable-7605060>: HTTP status code is not handled or not allowed 2026-01-25 03:42:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-kevlar-tube-sleeve-kvs-2-18th>: HTTP status code is not handled or not allowed 2026-01-25 03:42:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rods-22-1667-ber400825-0075a>: HTTP status code is not handled or not allowed 2026-01-25 03:42:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-sand-fillable-drum-base>: HTTP status code is not handled or not allowed 2026-01-25 03:42:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/food_beverages/snacks_candies/nutrition_bars_mixes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:42:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/automatic-soap-dispenser-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 03:42:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/relays-44-1774-tuc101273>: HTTP status code is not handled or not allowed 2026-01-25 03:42:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-plug-body-with-high-flow-profile-1-4-fnpt-connection-used-with-esi-coupling>: HTTP status code is not handled or not allowed 2026-01-25 03:42:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11-step-24in-w-steel-safety-angle-rolling-ladder-perforated-tread-gray-kdec111246>: HTTP status code is not handled or not allowed 2026-01-25 03:42:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/air_compressors/compressed_air_treatment_dryers_frls/compressed_air_separators already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:42:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/food_beverages/snacks_candies/nutrition_bars_mixes>: HTTP status code is not handled or not allowed 2026-01-25 03:42:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulbs-38-1483-henbl01-004>: HTTP status code is not handled or not allowed 2026-01-25 03:42:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-fnpt-2pc-frl>: HTTP status code is not handled or not allowed 2026-01-25 03:42:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-step-24in-w-steel-safety-angle-rolling-ladder-grip-strut-gray-kdec106242>: HTTP status code is not handled or not allowed 2026-01-25 03:42:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/furniture-bags-28w-x-17d-x-61l-1-mil-clear-250roll>: HTTP status code is not handled or not allowed 2026-01-25 03:42:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ball-peen-hammers-102g>: HTTP status code is not handled or not allowed 2026-01-25 03:42:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gusseted-poly-bags-10w-x-8d-x-20h-3-mil-clear-500pack>: HTTP status code is not handled or not allowed 2026-01-25 03:42:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyurethane-spiral-hose-20-with-non-scratch-fittings-and-1-4-mnpt-thd-connections>: HTTP status code is not handled or not allowed 2026-01-25 03:43:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/air_compressors/compressed_air_treatment_dryers_frls/compressed_air_separators landed on page that is not a product page. 2026-01-25 03:43:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-099-classic-99-retractable-blade-utility-knife-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:43:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/lathe_chucks_accessories/lathe_chuck_jaws already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:43:04 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-25 03:43:04 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:30 2026-01-25 03:43:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-1-hp-roof-ventilator-2>: HTTP status code is not handled or not allowed 2026-01-25 03:43:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motorola-swivel-earpiece-with-inline-ptt-mic-long-cord-black>: HTTP status code is not handled or not allowed 2026-01-25 03:43:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 59835 pages (at 94 pages/min), scraped 30006 items (at 52 items/min) 2026-01-25 03:43:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-jaw-2-piece-scroll-chuck-semi-steel-body-plain-back-5-inch>: HTTP status code is not handled or not allowed 2026-01-25 03:43:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/afk1216-floor-stand-kit-qty-2-1200x1606-steelgray>: HTTP status code is not handled or not allowed 2026-01-25 03:43:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/lathe_chucks_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:43:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11-step-aluminum-rolling-ladder-24in-w-grip-strut-14in-d-top-step-32in-handrails-wlar111245>: HTTP status code is not handled or not allowed 2026-01-25 03:43:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-26w-x-28l-2-mil-clear-250pack>: HTTP status code is not handled or not allowed 2026-01-25 03:43:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spring-pin-15x4-jsm4140-27>: HTTP status code is not handled or not allowed 2026-01-25 03:43:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-6w-x-24l-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 03:43:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-step-16in-w-20in-d-top-step-steel-rolling-ladder-grip-strut-tread-36in-handrail-kdsr103162-d2>: HTTP status code is not handled or not allowed 2026-01-25 03:43:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pesticide-cabinet-self-close-double-door-30-gallon>: HTTP status code is not handled or not allowed 2026-01-25 03:43:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/lathe_chucks_accessories landed on page that is not a product page. 2026-01-25 03:43:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-32-od-polyurethane-tubing-black-color-100-roll-125-200-psi>: HTTP status code is not handled or not allowed 2026-01-25 03:43:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-long-stacking-box-48-x-15-x-7-1-2-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:43:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a12p10al-panel-junction-box-1075x888-fits-12x10-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 03:43:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/measurement_layout_tools/micrometers_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:43:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a30p30-panel-nema-12-2700x2700-fits-30x30-steelwhite>: HTTP status code is not handled or not allowed 2026-01-25 03:43:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-chrome-21w-x-54l-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 03:43:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-32-od-nylon-tubing-natural-color-100-roll-160-500-psi>: HTTP status code is not handled or not allowed 2026-01-25 03:43:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cylinder-cap-jsm705a-18>: HTTP status code is not handled or not allowed 2026-01-25 03:43:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-nexelon-21w-x-72l-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 03:43:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/measurement_layout_tools/micrometers_accessories landed on page that is not a product page. 2026-01-25 03:43:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/healthsmart-rollator-walker-with-seat-and-backrest-adjustable-handle-height-removable-storage-bag-and-a-durable-lightweight-frame-that-easily-folds-while-supporting-up-to-300-pounds-purple>: HTTP status code is not handled or not allowed 2026-01-25 03:43:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/taper-pin-6x100-zx-lathes-zx-b31>: HTTP status code is not handled or not allowed 2026-01-25 03:43:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jaw-ins-assy-1206-cam-actvise-3120521>: HTTP status code is not handled or not allowed 2026-01-25 03:43:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-designer-cane-with-offset-handle-and-strap-leopard>: HTTP status code is not handled or not allowed 2026-01-25 03:43:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30mm-non-metallic-non-illuminated-flush-head-momentary-op-push-button-green>: HTTP status code is not handled or not allowed 2026-01-25 03:43:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manual-threading-pipe-and-bolt-dies-only-38345>: HTTP status code is not handled or not allowed 2026-01-25 03:43:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ck16-310-contactor-3-pole-230v>: HTTP status code is not handled or not allowed 2026-01-25 03:43:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/screw-1820361>: HTTP status code is not handled or not allowed 2026-01-25 03:43:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-plastic-dunnage-rack-1500-lbs-capacity-1>: HTTP status code is not handled or not allowed 2026-01-25 03:43:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-plastic-dunnage-rack-2000-lbs-capacity-1>: HTTP status code is not handled or not allowed 2026-01-25 03:43:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-polypropylene-coverall-open-wrists-ankles-white-2x-large-25-case>: HTTP status code is not handled or not allowed 2026-01-25 03:43:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pin-spanner-wrenches-460>: HTTP status code is not handled or not allowed 2026-01-25 03:43:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-poly-z-brite-18w-x-48l-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 03:43:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/l-c-d-mobile-console-computer-cabinet-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:43:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metal-luggage-rack-matte-finish>: HTTP status code is not handled or not allowed 2026-01-25 03:43:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-cap-style-6-point-snap-lock-suspension-hard-hat>: HTTP status code is not handled or not allowed 2026-01-25 03:43:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/boards_easels/whiteboard_bulletin_board_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:44:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/model-no-848-aluminum-straight-pipe-wrench-48>: HTTP status code is not handled or not allowed 2026-01-25 03:44:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/task_desk_chairs/task_office_chairs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:44:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-rack-add-on-w-steel-decking-96-w-x-24-d-x-144-h>: HTTP status code is not handled or not allowed 2026-01-25 03:44:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silicone-conical-vibration-shock-absorption-mount-10-24-thread-25-ft-lbs-max-load>: HTTP status code is not handled or not allowed 2026-01-25 03:44:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/183010d-x-large-acrylic-refrigerator-lock-box-keyed-differently-clear>: HTTP status code is not handled or not allowed 2026-01-25 03:44:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/memesd-memory-foam-anti-fatigue-insoles-10-11-molded-for-closed-shoes-antistatic-threads>: HTTP status code is not handled or not allowed 2026-01-25 03:44:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enriched-lotion-free-n-clean-soap-1600ml-fg402364>: HTTP status code is not handled or not allowed 2026-01-25 03:44:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 59926 pages (at 91 pages/min), scraped 30057 items (at 51 items/min) 2026-01-25 03:44:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/impactoe-steel-toe-cap-overshoes-xs-6-5-8-5-vulcanized-rbbr-pvc-free-waterproof-antislip>: HTTP status code is not handled or not allowed 2026-01-25 03:44:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/contemporary-design-electric-pencil-sharpener-black>: HTTP status code is not handled or not allowed 2026-01-25 03:44:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-w-x-30-d-production-workbench-maple-butcher-block-safety-edge-with-drawers-shelf-tan>: HTTP status code is not handled or not allowed 2026-01-25 03:44:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/task_desk_chairs/task_office_chairs landed on page that is not a product page. 2026-01-25 03:44:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pocket-clip-highlighter-chisel-tip-fluorescent-orange-ink-1-dozen>: HTTP status code is not handled or not allowed 2026-01-25 03:44:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yellow-cap-style-4-point-ratchet-suspension-hard-hat>: HTTP status code is not handled or not allowed 2026-01-25 03:44:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/pens already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:44:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rmt-4000-series-rotary-tool-3-attachments-34-accessories>: HTTP status code is not handled or not allowed 2026-01-25 03:44:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-cabinets-bca-367213-b-bookcase-assembled-36x13x72-black>: HTTP status code is not handled or not allowed 2026-01-25 03:44:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/boards_easels/whiteboard_bulletin_board_accessories landed on page that is not a product page. 2026-01-25 03:44:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ever-ghb-1340text-7531>: HTTP status code is not handled or not allowed 2026-01-25 03:44:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stone-panel-trash-weather-urn-gray-13-1-2-square-x-35-h>: HTTP status code is not handled or not allowed 2026-01-25 03:44:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fabricated-aluminum-cable-hose-protection-bridges-44-x-24-x-4-1-2>: HTTP status code is not handled or not allowed 2026-01-25 03:44:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-chain-3-4-link-50-l-hdpe-brown>: HTTP status code is not handled or not allowed 2026-01-25 03:44:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/pens>: HTTP status code is not handled or not allowed 2026-01-25 03:44:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-storage-cabinet-36-x-18-x-72>: HTTP status code is not handled or not allowed 2026-01-25 03:44:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1550-strapping-tape-12-x-60-yds-59-mil-clear-12pack>: HTTP status code is not handled or not allowed 2026-01-25 03:44:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/breakfast-biscuits-blueberry-64-carton>: HTTP status code is not handled or not allowed 2026-01-25 03:44:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/guest_chairs/guest_reception_chairs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:44:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fibre-disc-782c-60-tr-4-in-die-r400bb-25-per-inner-100-per-case>: HTTP status code is not handled or not allowed 2026-01-25 03:44:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/concrete-72-commercial-flat-bench-1>: HTTP status code is not handled or not allowed 2026-01-25 03:44:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-x-36-x-30-standard-shop-stand-maple-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:44:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/column-flange-base-jmd15-209>: HTTP status code is not handled or not allowed 2026-01-25 03:44:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25-foot-x-45-foot-blue-tarp>: HTTP status code is not handled or not allowed 2026-01-25 03:44:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/minigrip174-reclosable-poly-bags-2w-x-2l-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 03:44:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-5w-x-7l-4-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 03:44:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-lower-plastic-cover-for-761217-761218>: HTTP status code is not handled or not allowed 2026-01-25 03:44:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/clips_fasteners/rubber_bands already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:44:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fiberglass-tote-box-25-1-4-x-18-x-6-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:44:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-tarp-repair-tape-2-inch-x-35-foot>: HTTP status code is not handled or not allowed 2026-01-25 03:44:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/minigrip174-reclosable-poly-bags-2w-x-5l-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 03:44:51 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/guest_chairs/guest_reception_chairs landed on page that is not a product page. 2026-01-25 03:44:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-precision-ansi-cottered-pin-roller-chain-60-1c-3-4-pitch-10ft-box>: HTTP status code is not handled or not allowed 2026-01-25 03:44:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-kit-for-manual-pallet-jack-gwk-272136-lw-fits-wesco-model-272136>: HTTP status code is not handled or not allowed 2026-01-25 03:44:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/clips_fasteners/rubber_bands landed on page that is not a product page. 2026-01-25 03:44:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poly-tubing-10w-x-725l-6-mil-black-1-roll>: HTTP status code is not handled or not allowed 2026-01-25 03:44:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:44:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-kit-for-electric-pallet-jack-gwk-rj50-lw-fits-raymond-model-rj50>: HTTP status code is not handled or not allowed 2026-01-25 03:45:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gondola-slider-system-with-56-tip-out-white-bins>: HTTP status code is not handled or not allowed 2026-01-25 03:45:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/05040su-f4t5cw-4w-fluorescent-t5-bulb-mini-bipin-cool-white>: HTTP status code is not handled or not allowed 2026-01-25 03:45:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/05023su-fc16t9ww-40w-fluorescent-t9-circline-bulb-4pin-warm-white>: HTTP status code is not handled or not allowed 2026-01-25 03:45:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-kit-for-electric-pallet-jack-gwk-pw23-lw-fits-mitsubishi-model-pw23>: HTTP status code is not handled or not allowed 2026-01-25 03:45:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits>: HTTP status code is not handled or not allowed 2026-01-25 03:45:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 60018 pages (at 92 pages/min), scraped 30110 items (at 53 items/min) 2026-01-25 03:45:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/full-apron>: HTTP status code is not handled or not allowed 2026-01-25 03:45:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-deluxe-3-shelf-plastic-utility-service-cart-5-rubber-casters>: HTTP status code is not handled or not allowed 2026-01-25 03:45:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-direct-drive-agricultural-box-fan-230v-1-hp-motor-3-phase>: HTTP status code is not handled or not allowed 2026-01-25 03:45:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-cal-elephant-bark-rubber-flooring-3-8-x-4-x-8-floor-mat-green-dot>: HTTP status code is not handled or not allowed 2026-01-25 03:45:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/55-gallon-drum-liner-4-mil-38-x-53>: HTTP status code is not handled or not allowed 2026-01-25 03:45:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11604-economy-gun-w-silver-nozzel-cast-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 03:45:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-block-reclosable-poly-bags-2w-x-6l-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 03:45:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-oz-alum-scoops>: HTTP status code is not handled or not allowed 2026-01-25 03:45:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4150059-siphon-gun-steel>: HTTP status code is not handled or not allowed 2026-01-25 03:45:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sup-2xa-extruder-for-method-x>: HTTP status code is not handled or not allowed 2026-01-25 03:45:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/student-sit-stand-desk-manual-height-adjustment-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:45:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8210z-xs-orange-type-r-class-2-economy-mesh-vest>: HTTP status code is not handled or not allowed 2026-01-25 03:45:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/abrasives/sandblasting_tumblers/portable_blasters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:45:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8262frz-4xl-5xl-lime-hi-vis-fr-surveyor-vest-class-2-nfpa-70e-mesh>: HTTP status code is not handled or not allowed 2026-01-25 03:45:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprng-pin-5x25-jtm-1050-d-11>: HTTP status code is not handled or not allowed 2026-01-25 03:45:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/complete-mobile-packing-workbench-esd-square-edge-60-x-36>: HTTP status code is not handled or not allowed 2026-01-25 03:45:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vh5-personal-vortex-space-heater-black>: HTTP status code is not handled or not allowed 2026-01-25 03:45:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/wire_cable/electrical_wire_cable already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:45:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/17-sand-screen-disc-80-grit-10-per-case>: HTTP status code is not handled or not allowed 2026-01-25 03:45:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/table-set-plastic-banquet-roll-table-cover-cut-to-fit-40-x-100-white>: HTTP status code is not handled or not allowed 2026-01-25 03:45:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/abrasives/sandblasting_tumblers/portable_blasters landed on page that is not a product page. 2026-01-25 03:45:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merit-ss-304-40-welded-pipe-nipple-14x2-npt-male>: HTTP status code is not handled or not allowed 2026-01-25 03:45:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/resealable-gusseted-poly-bags-6w-x-2d-x-9h-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 03:45:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lookout-tagout-station>: HTTP status code is not handled or not allowed 2026-01-25 03:45:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stanco-flame-resistant-13-atpv-jacket-frc9630orcs-m>: HTTP status code is not handled or not allowed 2026-01-25 03:45:45 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/wire_cable/electrical_wire_cable landed on page that is not a product page. 2026-01-25 03:45:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-packing-workbench-esd-square-edge-72-x-36-with-lower-shelf-kit>: HTTP status code is not handled or not allowed 2026-01-25 03:45:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powr-flite-1-4-hp-downdra-floor-dryer-with-handle-wheels>: HTTP status code is not handled or not allowed 2026-01-25 03:45:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a24n24blp-medium-type-1-enclosure-2400x2400x862-steelgray>: HTTP status code is not handled or not allowed 2026-01-25 03:45:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ws60hs3716ss-watershed-wall-mount-enclosure-type-4x-6000x3700x1600>: HTTP status code is not handled or not allowed 2026-01-25 03:45:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/csd24246-concept-wall-mount-encl-2400x2400x600-steelgray>: HTTP status code is not handled or not allowed 2026-01-25 03:45:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-x-30-4-drawer-with-plastic-top-workbench>: HTTP status code is not handled or not allowed 2026-01-25 03:45:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-x-30-2-drawer-with-steel-top-workbench>: HTTP status code is not handled or not allowed 2026-01-25 03:45:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sierrus-dinner-platenarrow-rim-9red>: HTTP status code is not handled or not allowed 2026-01-25 03:46:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/table-fixer-screw-20-disc-sander-5644851>: HTTP status code is not handled or not allowed 2026-01-25 03:46:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:46:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pringles-original-potato-chips-single-serving-1-75-oz-snack-pack-36-cans-ct>: HTTP status code is not handled or not allowed 2026-01-25 03:46:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/gloves_sleeves/coated_gloves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:46:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waterhog-fashion-diamond-mat-dark-brown-4-x-12>: HTTP status code is not handled or not allowed 2026-01-25 03:46:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/packaging/envelopes_mailers/shipping_envelopes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:46:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mapa-temp-dex-710-nitrile-palm-coated-thermal-gloves-w-dots-light-weight-1-pair-size-11>: HTTP status code is not handled or not allowed 2026-01-25 03:46:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waterhog-fashion-mat-med-gray-4-x-6>: HTTP status code is not handled or not allowed 2026-01-25 03:46:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 60112 pages (at 94 pages/min), scraped 30167 items (at 57 items/min) 2026-01-25 03:46:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waterhog-fashion-diamond-mat-red-black-4-x-6>: HTTP status code is not handled or not allowed 2026-01-25 03:46:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats>: HTTP status code is not handled or not allowed 2026-01-25 03:46:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/gloves_sleeves/coated_gloves landed on page that is not a product page. 2026-01-25 03:46:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ingersoll-rand-109xpa-3-8-club-style-ratchet>: HTTP status code is not handled or not allowed 2026-01-25 03:46:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/packaging/envelopes_mailers/shipping_envelopes>: HTTP status code is not handled or not allowed 2026-01-25 03:46:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-horizontal-rail-mirror>: HTTP status code is not handled or not allowed 2026-01-25 03:46:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tubing-kit-34-id-x-100l>: HTTP status code is not handled or not allowed 2026-01-25 03:46:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ccs2ijlg-intermediate-joint-pend-arm-fits-45x60mm-alumltgray>: HTTP status code is not handled or not allowed 2026-01-25 03:46:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exq-pro-roller-cabinet-55w-x-30d-x-46-58h-blue-with-chrome-drawer-pulls-trim>: HTTP status code is not handled or not allowed 2026-01-25 03:46:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/ductwork/duct_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:46:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/back-up-alarm-with-beep-beep-sound>: HTTP status code is not handled or not allowed 2026-01-25 03:46:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/emergency_vehicle_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:46:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/emergency_vehicle_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:46:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-x-34-x-1-tubing-reducing-tee-drop-leg-fitting>: HTTP status code is not handled or not allowed 2026-01-25 03:46:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a903636fsda-free-stand-singledual-access-1-door-9006x3606x3606-steelpaint>: HTTP status code is not handled or not allowed 2026-01-25 03:46:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90-176-ss-corner-guard-16-ga-3-8-wings-sq-corners-48-lx1-1-2-wx1-1-2-h>: HTTP status code is not handled or not allowed 2026-01-25 03:46:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90-176-ss-corner-guard-16-ga-3-8-wings-sq-corners-96-lx3-wx3-h>: HTTP status code is not handled or not allowed 2026-01-25 03:46:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/ductwork/duct_boots>: HTTP status code is not handled or not allowed 2026-01-25 03:46:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/lockers/standard_lockers/steel_standard_lockers/heavy_duty_steel_lockers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:46:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/open-sided-wire-exchange-truck-four-wire-one-galvanized-shelf-800-lb-cap-5>: HTTP status code is not handled or not allowed 2026-01-25 03:46:34 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/emergency_vehicle_lights landed on page that is not a product page. 2026-01-25 03:46:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/emergency_vehicle_lights>: HTTP status code is not handled or not allowed 2026-01-25 03:46:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-unit-5-tier-54w-x-18d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 03:46:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-5-tier-72w-x-24d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 03:46:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/weathershield-commercial-canopy-18w-x-20l-white>: HTTP status code is not handled or not allowed 2026-01-25 03:46:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/lockers/standard_lockers/steel_standard_lockers/heavy_duty_steel_lockers landed on page that is not a product page. 2026-01-25 03:46:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hydrabarrier-standarf-6-ft-long-4-in-high>: HTTP status code is not handled or not allowed 2026-01-25 03:46:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bttn-hd-screw-xlift-e0040>: HTTP status code is not handled or not allowed 2026-01-25 03:46:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modine-18000-btu-steam-or-hot-water-unit-heater-with-explosion-proof-motor>: HTTP status code is not handled or not allowed 2026-01-25 03:46:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-computer-station-with-monitor-arm-36w-x-24d-x-33-1-2h-black>: HTTP status code is not handled or not allowed 2026-01-25 03:46:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cast-acrylic-sheet-24-l-x-24-w-x-1-4-thick-green-ps-cact-98>: HTTP status code is not handled or not allowed 2026-01-25 03:46:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carly-utility-tong-8-2732clear>: HTTP status code is not handled or not allowed 2026-01-25 03:46:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/lockers/standard_lockers/steel_standard_lockers/heavy_duty_steel_lockers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:46:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-black-steel-base-modular-locker>: HTTP status code is not handled or not allowed 2026-01-25 03:46:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/index-card-holder-4-x-6-black>: HTTP status code is not handled or not allowed 2026-01-25 03:46:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classical-a-v-belt-a45>: HTTP status code is not handled or not allowed 2026-01-25 03:46:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30x12x72-solar-star-greenhouse-w-poly-ends-and-roll-up-sides>: HTTP status code is not handled or not allowed 2026-01-25 03:46:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/krylon-crystal-tint-base-u-v-gloss>: HTTP status code is not handled or not allowed 2026-01-25 03:47:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shaft-model-pm15-6296320>: HTTP status code is not handled or not allowed 2026-01-25 03:47:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/105892-lockout-padlock-keyed-alike-1-12-plastic-covered-steel-yellow-6pack>: HTTP status code is not handled or not allowed 2026-01-25 03:47:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/97615-floor-warning-forklift-traffic-sign-yellowblack-anit-skid-vinyl-17dia>: HTTP status code is not handled or not allowed 2026-01-25 03:47:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-seal-bubble-mailers-4-9-12w-x-14-12l-kraft-25pack>: HTTP status code is not handled or not allowed 2026-01-25 03:47:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/lockers/standard_lockers/steel_standard_lockers/heavy_duty_steel_lockers landed on page that is not a product page. 2026-01-25 03:47:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classical-cogged-bx-v-belt-bx43>: HTTP status code is not handled or not allowed 2026-01-25 03:47:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labels-with-fragile-handle-with-care-print-4l-x-1-12w-redwhite-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-25 03:47:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classical-cogged-bx-v-belt-bx70>: HTTP status code is not handled or not allowed 2026-01-25 03:47:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 60205 pages (at 93 pages/min), scraped 30218 items (at 51 items/min) 2026-01-25 03:47:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10x10-heavy-duty-commercial-canopy-12-5oz-green>: HTTP status code is not handled or not allowed 2026-01-25 03:47:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vinyl-uphostered-restaurant-chair-with-ladder-back-black-1>: HTTP status code is not handled or not allowed 2026-01-25 03:47:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magic-tape-value-pack-3-4-x-1000-1-core-clear-20-pack>: HTTP status code is not handled or not allowed 2026-01-25 03:47:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heat-seal-bubble-mailers-6-12-12w-x-19l-kraft-25pack>: HTTP status code is not handled or not allowed 2026-01-25 03:47:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ingredient-bin-27-galwhite>: HTTP status code is not handled or not allowed 2026-01-25 03:47:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cylinder-jsg-04700472-829022>: HTTP status code is not handled or not allowed 2026-01-25 03:47:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bubble-lined-poly-mailers-9-12w-x-14-12l-2-mil-white-25pack>: HTTP status code is not handled or not allowed 2026-01-25 03:47:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/market-tray-17-34-12-34-1pearl-white>: HTTP status code is not handled or not allowed 2026-01-25 03:47:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/spill_control_supplies/chemical_sorbents_spill_kits/spill_kits_stations already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:47:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/76226-danger-do-not-operate-tag-2-sided-25pkg-polyester-3w-x-5-34h>: HTTP status code is not handled or not allowed 2026-01-25 03:47:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/locks_lockout_devices/lockout_tagout_devices/lockout_padlocks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:47:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/86537-stop-do-not-operate-tag-2-sided-10pkg-polyester-3w-x-5-34h>: HTTP status code is not handled or not allowed 2026-01-25 03:47:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96908-fire-extinguisher-v-sign-2-sided-polyethylene-18w-x-12h>: HTTP status code is not handled or not allowed 2026-01-25 03:47:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:47:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96264-rearview-mirror-hanging-tags-301-400-parking-permits-blue-2-34w-x-4-34h>: HTTP status code is not handled or not allowed 2026-01-25 03:47:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-buna-n-strip-36-l-x-7-w-x-3-16-thick-blue-detectable-60a-acrylic-adhesive-bulk-rs-h60md-84>: HTTP status code is not handled or not allowed 2026-01-25 03:47:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/spill_control_supplies/chemical_sorbents_spill_kits/spill_kits_stations>: HTTP status code is not handled or not allowed 2026-01-25 03:47:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/locks_lockout_devices/lockout_tagout_devices/lockout_padlocks>: HTTP status code is not handled or not allowed 2026-01-25 03:47:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelving-add-on-unit-5-tier-54w-x-18d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 03:47:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-ace-tuf-wheel-8x250-58-ball>: HTTP status code is not handled or not allowed 2026-01-25 03:47:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents>: HTTP status code is not handled or not allowed 2026-01-25 03:47:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/appliances/food_holding_warming_equipment/hot_cold_food_wells already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:47:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelate-silver-epoxy-wire-shelving-add-on-unit-5-tier-30w-x-21d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 03:47:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/foam/foam_tubes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:47:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/solder-16-oz>: HTTP status code is not handled or not allowed 2026-01-25 03:47:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/86533-danger-do-not-use-tag-2-sided-25pkg-polyester-3w-x-5-34h>: HTTP status code is not handled or not allowed 2026-01-25 03:47:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/self_luminous_emergency_exit_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:47:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mailing-tubes-with-caps-2dia-x-18l-0060-thick-black-50pack>: HTTP status code is not handled or not allowed 2026-01-25 03:47:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/appliances/food_holding_warming_equipment/hot_cold_food_wells landed on page that is not a product page. 2026-01-25 03:47:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-ldpe-film-1200-l-x-6-3-4-w-x-0-006-thick-clear-bulk-pf-ldpe-13>: HTTP status code is not handled or not allowed 2026-01-25 03:47:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/foam/foam_tubes>: HTTP status code is not handled or not allowed 2026-01-25 03:47:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-x-12-x-18-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-25 03:47:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bevel-gear-model-201planer-6012099>: HTTP status code is not handled or not allowed 2026-01-25 03:47:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/146122-cable-safety-padlock-with-label-4-316h-clearance-steel-cable-blue>: HTTP status code is not handled or not allowed 2026-01-25 03:47:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/self_luminous_emergency_exit_signs>: HTTP status code is not handled or not allowed 2026-01-25 03:47:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sectional-chipbreaker-mdl-201-planer-6012174>: HTTP status code is not handled or not allowed 2026-01-25 03:47:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-250-watt-ceramic-infra-red-lamp-with-stationary-base>: HTTP status code is not handled or not allowed 2026-01-25 03:47:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-seal-bubble-mailers-6-12-12w-x-19l-kraft-25pack>: HTTP status code is not handled or not allowed 2026-01-25 03:47:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-pressure-regulator-pres-reg-01-for-all-jetstream-hurricane-units>: HTTP status code is not handled or not allowed 2026-01-25 03:47:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simoniz-ultra-line-floor-stripper-5-gallon>: HTTP status code is not handled or not allowed 2026-01-25 03:48:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-x-18-x-12-double-wall-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-25 03:48:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regulated-labels-health-flammability-reactivity-1l-x-2w-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-25 03:48:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/switch-rev-1ph-6289115>: HTTP status code is not handled or not allowed 2026-01-25 03:48:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30a-2-gear-20t-6289057>: HTTP status code is not handled or not allowed 2026-01-25 03:48:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-m8-x-125p-x-35lg-model-201-6012242>: HTTP status code is not handled or not allowed 2026-01-25 03:48:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/28-x-18-x-18-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-25 03:48:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interion-174-36-wide-desk-mahogany>: HTTP status code is not handled or not allowed 2026-01-25 03:48:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-pvc-rectangular-tube-72-l-x-2-w-gray-bulk-rpt-pvc-3>: HTTP status code is not handled or not allowed 2026-01-25 03:48:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 60297 pages (at 92 pages/min), scraped 30269 items (at 51 items/min) 2026-01-25 03:48:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-dunnage-rack-20-inch-w-60-inch-l-8-inch-h>: HTTP status code is not handled or not allowed 2026-01-25 03:48:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/-light-duty-non-lighted-warning-whip-green>: HTTP status code is not handled or not allowed 2026-01-25 03:48:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labels-with-made-in-america-by-print-8l-x-2w-redwhiteblue-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-25 03:48:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glo-worm-light-for-heavy-duty-whip-amber>: HTTP status code is not handled or not allowed 2026-01-25 03:48:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-dia-round-paper-labels-with-jan-print-fluorescent-yellow-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-25 03:48:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-dia-round-paper-labels-with-recycle-print-green-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-25 03:48:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/server-7-quart-66-l-food-warmer>: HTTP status code is not handled or not allowed 2026-01-25 03:48:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-buna-n-sheet-36-l-x-36-w-x-1-8-thick-blue-detectable-60a-acrylic-adhesive>: HTTP status code is not handled or not allowed 2026-01-25 03:48:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/consecutive-numbered-labels-1001-1500-1-12l-x-1w-blackwhite-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-25 03:48:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-x-4-x-4-mailers>: HTTP status code is not handled or not allowed 2026-01-25 03:48:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-sheet-48-l-x-48-w-x-1-8-thick-black>: HTTP status code is not handled or not allowed 2026-01-25 03:48:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-x-3-x-2-mailers>: HTTP status code is not handled or not allowed 2026-01-25 03:48:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/diamondback-tunnel-only-for-45-line>: HTTP status code is not handled or not allowed 2026-01-25 03:48:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/olympic-hex-bar-2-handle>: HTTP status code is not handled or not allowed 2026-01-25 03:48:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/116in-w-x80in-d-reception-station-w-non-electric-raceway-cherry-counter-blue-panel>: HTTP status code is not handled or not allowed 2026-01-25 03:48:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-x-4-x-4-mailers>: HTTP status code is not handled or not allowed 2026-01-25 03:48:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/work-height-platform-truck-rnb2-3048-6pyah-with-lower-shelf-30-x-48-adj-height>: HTTP status code is not handled or not allowed 2026-01-25 03:48:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pick-set-4-piece-w-neon-orange-handles>: HTTP status code is not handled or not allowed 2026-01-25 03:48:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-plain-self-seal-envelopes-w-security-tint-4-18w-x-9-12l-white-2500pk>: HTTP status code is not handled or not allowed 2026-01-25 03:48:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iowa-hitch-cover-4-1-2-x-3-3-8-15076>: HTTP status code is not handled or not allowed 2026-01-25 03:48:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-dia-round-number-paper-labels-with-0-print-dark-blue-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-25 03:48:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/impact-socket-1-2-drive-1-6-point-short>: HTTP status code is not handled or not allowed 2026-01-25 03:48:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/neurological-buck-hammer-with-256-cps-tuning-fork>: HTTP status code is not handled or not allowed 2026-01-25 03:48:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-office-partition-panel-w-partial-window-and-raceway-48-1-4w-x-101-1-2h-blue-gl>: HTTP status code is not handled or not allowed 2026-01-25 03:48:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/grill_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:48:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nba-brooklyn-nets-grill-mat-14197>: HTTP status code is not handled or not allowed 2026-01-25 03:48:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ax13098-keypad-security-safe-1334-x-1934-x-13-145-cubic-ft>: HTTP status code is not handled or not allowed 2026-01-25 03:48:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-adjustable-wire-shelving-24-w-x-18-d-x-63-h-nexelon>: HTTP status code is not handled or not allowed 2026-01-25 03:48:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/circle-mailing-labels-1-12-dia-clear-roll-of-20000>: HTTP status code is not handled or not allowed 2026-01-25 03:48:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-circle-mailing-labels-1-dia-clear-roll-of-5000>: HTTP status code is not handled or not allowed 2026-01-25 03:48:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/floor_mats/grill_mats>: HTTP status code is not handled or not allowed 2026-01-25 03:48:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labels-w-do-not-inventory-print-2l-x-1-14w-fluorescent-red-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-25 03:48:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-door-ovation-letter-board-black-w-gray-frame-46-7-8w-x-33-3-4h>: HTTP status code is not handled or not allowed 2026-01-25 03:48:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:48:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-outdoor-in-ground-steel-bench-with-backrest-perforated-metal-blue>: HTTP status code is not handled or not allowed 2026-01-25 03:48:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-socket-3-8-drive-13mm-short>: HTTP status code is not handled or not allowed 2026-01-25 03:49:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kdd824-steering-wheel-message-cover-vinyl-20-diameter>: HTTP status code is not handled or not allowed 2026-01-25 03:49:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/htp-trolley-1-2-ton-plain-push-wide-flange-width>: HTTP status code is not handled or not allowed 2026-01-25 03:49:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/htg-trolley-3-ton-geared>: HTTP status code is not handled or not allowed 2026-01-25 03:49:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-door-enclosed-recycled-rubber-tackboard-60w-x36h-black-w-silver-frame>: HTTP status code is not handled or not allowed 2026-01-25 03:49:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/sockets_bits/power_drill_driver_bits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:49:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-bags-for-ai-series-commercial-wet-dry-vacuum-3-bags-per-pack>: HTTP status code is not handled or not allowed 2026-01-25 03:49:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 60393 pages (at 96 pages/min), scraped 30326 items (at 57 items/min) 2026-01-25 03:49:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-oz-claw-hammer-w-fiberglass-handle>: HTTP status code is not handled or not allowed 2026-01-25 03:49:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-x-2150-2-mil-anti-static-poly-tubing>: HTTP status code is not handled or not allowed 2026-01-25 03:49:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/decor_linens/rugs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:49:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mrst813va-notice-sign-wash-hands-before-returning-to-work-14w-x-10h-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 03:49:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30x25-liter-floor-base-cabinet-fully-lined-top-tray-31w-x-20d-x-36-5-8h>: HTTP status code is not handled or not allowed 2026-01-25 03:49:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-25 03:49:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/sockets_bits/power_drill_driver_bits>: HTTP status code is not handled or not allowed 2026-01-25 03:49:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:49:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wrestling-marial-arts-roll-mats-24-x-6-x-1-5-8-black-111r-bk>: HTTP status code is not handled or not allowed 2026-01-25 03:49:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/decor_linens already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:49:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mb-2530-bb-mobile-service-bench-20-bin-box-and-locking-cabinet-33-w-x-25-d-x-40-h-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:49:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:49:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-post-86in-h-leveler-installed-pack-of-four>: HTTP status code is not handled or not allowed 2026-01-25 03:49:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-service-bench-20-bin-box-locking-cabinet-and-3-drawers-48-w-x-25-d-x-40-h-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:49:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sck116-on-the-job-safety-electronic-scoreboard-20w-x-28h-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 03:49:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/array-conference-table-with-power-data-grommet-96l-x-48w-x-29h-noble-oakwhite>: HTTP status code is not handled or not allowed 2026-01-25 03:49:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 03:49:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/31-polyolefin-heat-shrink-150-dia-25-black>: HTTP status code is not handled or not allowed 2026-01-25 03:49:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-enclosed-wire-stockpicker-truck-48-x-18-x-69-800-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 03:49:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gp3300i-49st-co-portable-inverter-generator-with-recoil-start-gasoline-2500-rated-watts>: HTTP status code is not handled or not allowed 2026-01-25 03:49:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 03:49:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15x30-4-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 03:49:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/decor_linens landed on page that is not a product page. 2026-01-25 03:49:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-sheet-96-l-x-48-w-x-3-16-thick-clear-scratch-and-uv-resistant>: HTTP status code is not handled or not allowed 2026-01-25 03:49:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pm1800b-3t-18-bandsaw-with-armorglide-5-hp-3-phase-460v>: HTTP status code is not handled or not allowed 2026-01-25 03:49:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powderfree-textured-exam-grade-nitrile-gloves-indigo-small-100box-10-boxcs>: HTTP status code is not handled or not allowed 2026-01-25 03:49:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-degree-45-degree-adjustable-cross-3>: HTTP status code is not handled or not allowed 2026-01-25 03:49:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thermashield-aluminized-fiberglass-tube-125-dia-250-silver>: HTTP status code is not handled or not allowed 2026-01-25 03:49:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bridges_returns already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:49:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cpvc-bar-36-l-x-1-2-w-x-1-2-thick-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:49:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_cabinets_pedestals already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:49:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/male-corner-swivel-socket-member-4>: HTTP status code is not handled or not allowed 2026-01-25 03:49:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mdt623ctp-caution-tag-caution-pf-cardstock-25pack>: HTTP status code is not handled or not allowed 2026-01-25 03:49:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-5-16-x-4-1-8-x-13-3-8-10-hardware-bag>: HTTP status code is not handled or not allowed 2026-01-25 03:49:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-1-8-x-4-1-2-x-13-3-4-12-hardware-bag>: HTTP status code is not handled or not allowed 2026-01-25 03:49:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-socket-3-8-drive-7-8-6-point-short>: HTTP status code is not handled or not allowed 2026-01-25 03:49:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-sheet-48-l-x-48-w-x-3-16-thick-clear-antistatic>: HTTP status code is not handled or not allowed 2026-01-25 03:49:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-1-2x4-3-4x19-1-mil-gusseted-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 03:49:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/z-line-boltless-shelving-w-steel-industrial-shelves-42-w-x-36-d-x-84-h-add-on-5-shelves>: HTTP status code is not handled or not allowed 2026-01-25 03:49:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_film_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:49:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8x5x15-1-mil-gusseted-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 03:49:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_bridges_returns landed on page that is not a product page. 2026-01-25 03:49:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_cabinets_pedestals landed on page that is not a product page. 2026-01-25 03:49:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4x4-1-5-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 03:49:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-bar-48-l-x-4-w-x-1-2-thick-clear-antistatic>: HTTP status code is not handled or not allowed 2026-01-25 03:49:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/malleable-plug-4>: HTTP status code is not handled or not allowed 2026-01-25 03:50:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/security-safe-electronic-lock-21-11-16w-x-19-13-16d-x-37-11-16h-black>: HTTP status code is not handled or not allowed 2026-01-25 03:50:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_film_stock landed on page that is not a product page. 2026-01-25 03:50:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-two-socket-cross-2>: HTTP status code is not handled or not allowed 2026-01-25 03:50:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mext591vs-exit-safety-sign-10w-x-7h-4-mil-adhesive-vinyl>: HTTP status code is not handled or not allowed 2026-01-25 03:50:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-double-swivel-socket-1>: HTTP status code is not handled or not allowed 2026-01-25 03:50:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-wall-flange-2>: HTTP status code is not handled or not allowed 2026-01-25 03:50:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 60483 pages (at 90 pages/min), scraped 30371 items (at 45 items/min) 2026-01-25 03:50:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-fleece-bag-for-zephyr153-backpack-vacuum-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 03:50:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16x20-1-5-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 03:50:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/li-ion-battery-for-strike-upright-zephyr-backpack--tempest-sweeper-vac-36v-75ah>: HTTP status code is not handled or not allowed 2026-01-25 03:50:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durawrap-1050-ballistic-nylon-wrap-w-velcro-closure-250-dia-150-black>: HTTP status code is not handled or not allowed 2026-01-25 03:50:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mcpg611va-caution-sign-hot-10w-x-7h-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 03:50:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11x12-1-5-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 03:50:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/freestanding-2-panel-corner-room-divider-with-partial-window-48-x-60-blue>: HTTP status code is not handled or not allowed 2026-01-25 03:50:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/power_tools/planers_jointers/woodworking_jointers_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:50:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speco-8-channel-network-video-recorder-with-facial-recognition--smart-analytics-6tb-storage>: HTTP status code is not handled or not allowed 2026-01-25 03:50:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cpvc-sheet-12-l-x-12-w-x-1-thick-gray-ps-cpvc-29>: HTTP status code is not handled or not allowed 2026-01-25 03:50:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-x-12-open-end-static-shielding-bags>: HTTP status code is not handled or not allowed 2026-01-25 03:50:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:50:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jct-1602-benchtop-sand-rammer-24-cfm-800-bpm-90-psi>: HTTP status code is not handled or not allowed 2026-01-25 03:50:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hurricane-360-hoist-and-trolley-combo-geared-10-ton-capacity-15-lift>: HTTP status code is not handled or not allowed 2026-01-25 03:50:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/power_tools/planers_jointers/woodworking_jointers_accessories>: HTTP status code is not handled or not allowed 2026-01-25 03:50:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-a-cylinders-with-reverse-graduations-10ml-case-of-6>: HTTP status code is not handled or not allowed 2026-01-25 03:50:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7x12-3-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 03:50:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-sheet-24-l-x-12-w-x-1-16-thick-clear-scratch-and-uv-resistant>: HTTP status code is not handled or not allowed 2026-01-25 03:50:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6x9-3-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 03:50:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-wall-pack-switchable-lumen-output-switchable-cct-large-size>: HTTP status code is not handled or not allowed 2026-01-25 03:50:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-25 03:50:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dax-brow-pad-suspension-replacement-black>: HTTP status code is not handled or not allowed 2026-01-25 03:50:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jlh-series-lever-hoist-w-overload-protection-3-4-ton-20-ft-lift>: HTTP status code is not handled or not allowed 2026-01-25 03:50:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:50:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/indooroutdoor-lcd-guardian-tv-enclosure-for-44-50-monitors-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 03:50:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-freestanding-3-panel-corner-room-divider-48-x-61-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:50:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/fans_blowers/cooling_fans/wall_fans already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:50:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classic-entrance-mat-waffle-pattern-3-8-thick-6-x-12-blue>: HTTP status code is not handled or not allowed 2026-01-25 03:50:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-x-5-x-30-square-mailing-tubes>: HTTP status code is not handled or not allowed 2026-01-25 03:50:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 03:50:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bt-4040-all-purpose-tarp-40-x-40-blue>: HTTP status code is not handled or not allowed 2026-01-25 03:50:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ice-cream-cone-dispenser-ic2200sm>: HTTP status code is not handled or not allowed 2026-01-25 03:50:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pn-30005-privacy-netting-wreinforced-hem-68-x-150-green>: HTTP status code is not handled or not allowed 2026-01-25 03:50:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/fans_blowers/cooling_fans/wall_fans landed on page that is not a product page. 2026-01-25 03:50:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelate-wire-shelving-add-on-a36725ep-72w-x-36d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 03:50:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-polypropylene-instrument-tray-with-cover-6-x-3-3-4-x-2-1-4>: HTTP status code is not handled or not allowed 2026-01-25 03:50:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/145-165-percent-alcohol-proof---ethyl-alcohol-hydrometer>: HTTP status code is not handled or not allowed 2026-01-25 03:50:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/electrical_power_testing/insulation_testers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:50:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dry-zone-shadow-board-combo-kit-blue-blue-68-x-30-pro-series-acrylic-sbk131fg>: HTTP status code is not handled or not allowed 2026-01-25 03:50:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:50:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0-50-degree-baume-hydrometer>: HTTP status code is not handled or not allowed 2026-01-25 03:51:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-rod-96-l-x-1-2-dia-white>: HTTP status code is not handled or not allowed 2026-01-25 03:51:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pennsylvania-7600-series-hdhigh-resolution-db-capable-counting-scale-50-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 03:51:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluke-ac175-threaded-alligator-clip-set>: HTTP status code is not handled or not allowed 2026-01-25 03:51:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eemax-electric-mini-tank-1-gallon-capacity>: HTTP status code is not handled or not allowed 2026-01-25 03:51:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/testing_inspection_instruments/electrical_power_testing/insulation_testers landed on page that is not a product page. 2026-01-25 03:51:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock>: HTTP status code is not handled or not allowed 2026-01-25 03:51:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-air-foam-rolls-12w-x-2000l-x-132-thick-white-6-rolls>: HTTP status code is not handled or not allowed 2026-01-25 03:51:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 60577 pages (at 94 pages/min), scraped 30423 items (at 52 items/min) 2026-01-25 03:51:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18in-extension-for-manually-or-air-operated-grease-guns-5818>: HTTP status code is not handled or not allowed 2026-01-25 03:51:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coated-lead-shield-8-1-2-x-3-x-6>: HTTP status code is not handled or not allowed 2026-01-25 03:51:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-4-x-2-7-8-7-orange-shipping-tag>: HTTP status code is not handled or not allowed 2026-01-25 03:51:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:51:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/atx9-powersports-battery-8-amps-12v-b-terminals>: HTTP status code is not handled or not allowed 2026-01-25 03:51:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-48-x-36-adj-height-workbench-w-drawer-black-plastic-laminate-square-top>: HTTP status code is not handled or not allowed 2026-01-25 03:51:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/open-top-receptacle-30-gallon>: HTTP status code is not handled or not allowed 2026-01-25 03:51:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-epoxy-wire-shelving-21606gz-60w-x-21d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 03:51:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h-b-durac-plus-calibrated-liquid-in-glass-thermometer-20-to-110c-76mm-immersion-organic-liquid-fill>: HTTP status code is not handled or not allowed 2026-01-25 03:51:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-96-x-30-adj-height-workbench-w-drawer-black-maple-square-edge-top>: HTTP status code is not handled or not allowed 2026-01-25 03:51:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-rod-36-l-x-9-16-dia-white>: HTTP status code is not handled or not allowed 2026-01-25 03:51:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24h-double-sided-floor-stand-with-128-drawer-interlocking-storage-cabinet>: HTTP status code is not handled or not allowed 2026-01-25 03:51:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 03:51:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roundup-1-1-4w-14l-endless-poly-roundsling-yellow>: HTTP status code is not handled or not allowed 2026-01-25 03:51:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:51:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h-b-durac-plus-calibrated-liquid-in-glass-thermometer-20-to-110c-organic-liquid-fill>: HTTP status code is not handled or not allowed 2026-01-25 03:51:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-graphite-sheet-6-l-x-6-w-x-1-4-thick-black>: HTTP status code is not handled or not allowed 2026-01-25 03:51:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-16-alloy-latching-j-hook-style-a-230-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 03:51:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plus-astm-like-liquid-in-glass-thermometer-1f-partial-immersion-76mm-0-to-302f>: HTTP status code is not handled or not allowed 2026-01-25 03:51:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h-b-frio-temp-refrigerator-verification-thermometer-with-individual-calibration-report-2-to-10c>: HTTP status code is not handled or not allowed 2026-01-25 03:51:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plus-astm-like-liquid-in-glass-thermometer-117c-bomb-calorimeter-239-to-301c>: HTTP status code is not handled or not allowed 2026-01-25 03:51:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:51:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buyers-aluminum-underbody-box-w-t-handle-14x12x24>: HTTP status code is not handled or not allowed 2026-01-25 03:51:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metal-indoor-outdoor-restaurant-chair-silver>: HTTP status code is not handled or not allowed 2026-01-25 03:51:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_microscopes/accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:51:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mro-industrial-enamel-20-oz-ryder-yellow-6-cans-case-620-1446>: HTTP status code is not handled or not allowed 2026-01-25 03:51:50 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock landed on page that is not a product page. 2026-01-25 03:51:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mro-industrial-enamel-20-oz-stainless-steel-6-cans-case-620-1460>: HTTP status code is not handled or not allowed 2026-01-25 03:51:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-x-12-2-mil-reclosable-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 03:51:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers landed on page that is not a product page. 2026-01-25 03:51:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hplc-reservoir-secondary-container-10-liters>: HTTP status code is not handled or not allowed 2026-01-25 03:51:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_microscopes/accessories landed on page that is not a product page. 2026-01-25 03:51:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stripe-renew-concrete-coating-20-oz-gray-12-cans-case-20-600>: HTTP status code is not handled or not allowed 2026-01-25 03:52:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-rigid-caster-for-dandux-bulk-trucks-986835>: HTTP status code is not handled or not allowed 2026-01-25 03:52:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-x-14-4-mil-reclosable-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 03:52:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-36-l-x-6-w-x-1-3-4-thick-white>: HTTP status code is not handled or not allowed 2026-01-25 03:52:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyethylene-dispensing-jug-20-liters-5-gallons-polyethylene-spigot>: HTTP status code is not handled or not allowed 2026-01-25 03:52:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-wall-mount-wire-shelving-72w-x-18d-x-34h-2-shelf-starter-poly-green>: HTTP status code is not handled or not allowed 2026-01-25 03:52:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:52:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 60671 pages (at 94 pages/min), scraped 30483 items (at 60 items/min) 2026-01-25 03:52:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-wire-shelf-platform-poly-z-brite-18-w-x-30-l-x-14-h>: HTTP status code is not handled or not allowed 2026-01-25 03:52:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-24-l-x-6-w-x-1-4-thick-black>: HTTP status code is not handled or not allowed 2026-01-25 03:52:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-shelf-wall-mount-wine-rack-double-wide-chrome-14-d-x-96-w-x-34-h>: HTTP status code is not handled or not allowed 2026-01-25 03:52:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergency-instruction-device-eid>: HTTP status code is not handled or not allowed 2026-01-25 03:52:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-24-14-drawers-plastic>: HTTP status code is not handled or not allowed 2026-01-25 03:52:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 03:52:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glass-door-back-bar-refrigerator-bb-g-79w-bb78g>: HTTP status code is not handled or not allowed 2026-01-25 03:52:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyethylene-jerrican-with-spigot-5-liters-125-gallons-screw-cap-1-id-spout>: HTTP status code is not handled or not allowed 2026-01-25 03:52:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kendall-howard-3u-flat-spacer-blank>: HTTP status code is not handled or not allowed 2026-01-25 03:52:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3ft-x-5ft-bulldog-174-cotton-us-flag-with-sewn-stripes-embroidered-stars>: HTTP status code is not handled or not allowed 2026-01-25 03:52:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:52:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/drinking_fountains_bottle_fillers_water_coolers/indoor_drinking_fountains_bottle_fillers/indoor_drinking_fountains already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:52:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heartsine-samaritan-automated-external-defibrillator-with-prescription>: HTTP status code is not handled or not allowed 2026-01-25 03:52:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-ptfe-sheet-24-l-x-24-w-x-1-thick-off-white-glass-filled>: HTTP status code is not handled or not allowed 2026-01-25 03:52:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/geh-12c-spherical-plain-bearing-metric-heavy-series-maintenance-free>: HTTP status code is not handled or not allowed 2026-01-25 03:52:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:52:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40-70120-gourmet-freeze-dried-meat-buckets-2-buckets-120-servings>: HTTP status code is not handled or not allowed 2026-01-25 03:52:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:52:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/01-160-entree-only-grab-and-go-kit-60-servings>: HTTP status code is not handled or not allowed 2026-01-25 03:52:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers>: HTTP status code is not handled or not allowed 2026-01-25 03:52:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/plumbing/drinking_fountains_bottle_fillers_water_coolers/indoor_drinking_fountains_bottle_fillers/indoor_drinking_fountains landed on page that is not a product page. 2026-01-25 03:52:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ge-180es-spherical-plain-bearing-metric>: HTTP status code is not handled or not allowed 2026-01-25 03:52:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leviton-5362-l0-20-amp-125v-heavy-duty-grade-duplex-receptacle-grounding-brown>: HTTP status code is not handled or not allowed 2026-01-25 03:52:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fixed-straight-3-shelf-bookcase-36wx15dx35h>: HTTP status code is not handled or not allowed 2026-01-25 03:52:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kid-dress-up-storage-unit-with-colored-tubs-48wx15dx415h>: HTTP status code is not handled or not allowed 2026-01-25 03:52:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-25 03:52:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1610-surface-mount-spike-system-3-foot-spike-sections-yellow>: HTTP status code is not handled or not allowed 2026-01-25 03:52:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-25 03:52:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-galvanized-feed-bunk-with-square-ends-60l-x-34w-x-19h>: HTTP status code is not handled or not allowed 2026-01-25 03:52:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-silicone-rubber-chuck-for-pipette-pump-green-pipettors>: HTTP status code is not handled or not allowed 2026-01-25 03:52:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/parts_cleaning_treating/parts_cleaners_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:52:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gezm-200es-spherical-plain-bearing-inch-extended-inner-ring>: HTTP status code is not handled or not allowed 2026-01-25 03:52:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:52:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ap1-20-half-frame-safety-glasses-smoke-lens-black-frame>: HTTP status code is not handled or not allowed 2026-01-25 03:52:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:52:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-sheet-12-l-x-12-w-x-2-3-4-thick-black-bulk-ps-acb-2207>: HTTP status code is not handled or not allowed 2026-01-25 03:52:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:52:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/lockers/bulk_storage_lockers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:52:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/prevention-security-cabinet-96w-36d-84h-double-door-assembled>: HTTP status code is not handled or not allowed 2026-01-25 03:52:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-5-in-1-straight-tubing-connectors-3-1-2-polypropylene-12pk>: HTTP status code is not handled or not allowed 2026-01-25 03:52:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/parts_cleaning_treating/parts_cleaners_accessories landed on page that is not a product page. 2026-01-25 03:52:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/stainless_steel_bathroom_partitions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:52:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:52:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-25 03:52:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 03:52:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-72w-x-14d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 03:53:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 03:53:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/geg-125es-spherical-plain-bearing-metric-extended-inner-ring>: HTTP status code is not handled or not allowed 2026-01-25 03:53:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-5-tier-72w-x-36d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 03:53:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-neoprene-roll-600-l-x-12-w-x-1-16-thick-black-fabric-reinforced-70a-bulk-rs-nfr70-232>: HTTP status code is not handled or not allowed 2026-01-25 03:53:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/concrete-3bin-recycle-unit-with-blue-push-door-lid-75-x-25-x-47-gray-trashbottlescanspaper>: HTTP status code is not handled or not allowed 2026-01-25 03:53:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/concrete-waste-receptacle-with-black-aluminum-ash-n-trash-top24-dia-x-33-sand>: HTTP status code is not handled or not allowed 2026-01-25 03:53:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 03:53:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-pump-isolation-flanges-pair-npt>: HTTP status code is not handled or not allowed 2026-01-25 03:53:06 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/lockers/bulk_storage_lockers landed on page that is not a product page. 2026-01-25 03:53:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-12-l-x-1-1-2-w-x-5-8-thick-black>: HTTP status code is not handled or not allowed 2026-01-25 03:53:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-24-l-x-2-1-2-w-x-2-1-4-thick-white>: HTTP status code is not handled or not allowed 2026-01-25 03:53:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 60756 pages (at 85 pages/min), scraped 30522 items (at 39 items/min) 2026-01-25 03:53:09 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/stainless_steel_bathroom_partitions landed on page that is not a product page. 2026-01-25 03:53:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-plastic-6-compartment-storage-box-13-1-8-x-9-x-2-5-16>: HTTP status code is not handled or not allowed 2026-01-25 03:53:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oreck-17-inch-carpet-bonnet>: HTTP status code is not handled or not allowed 2026-01-25 03:53:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:53:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-5-tier-54w-x-24d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 03:53:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/label_tapes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:53:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-36w-x-21d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 03:53:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/staplers_staples/staplers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:53:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/staplers_staples/staplers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:53:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:53:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-54w-x-24d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 03:53:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hytrol-stationary-floor-support-18w-max-leg-35-5-8h>: HTTP status code is not handled or not allowed 2026-01-25 03:53:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-underbed-truck-box-18x18x36-black-1732305>: HTTP status code is not handled or not allowed 2026-01-25 03:53:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/room_dividers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:53:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-all-purpose-heavy-duty-blade-bracket-chrome>: HTTP status code is not handled or not allowed 2026-01-25 03:53:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/label_tapes landed on page that is not a product page. 2026-01-25 03:53:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/basic_supplies/staplers_staples/staplers>: HTTP status code is not handled or not allowed 2026-01-25 03:53:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/staplers_staples/staplers landed on page that is not a product page. 2026-01-25 03:53:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8d-x-12w-straight-shelf-semi-gloss-black>: HTTP status code is not handled or not allowed 2026-01-25 03:53:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-304-stainless-steel-locker-18w-x-18d-x-78h-double-tier-1-wide-assembled>: HTTP status code is not handled or not allowed 2026-01-25 03:53:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/splash-shield-2-38-articulating-arms-15-x-24-plexiglass>: HTTP status code is not handled or not allowed 2026-01-25 03:53:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-25 03:53:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-16-gauge-heavy-duty-classic-duratough-combination-cabinet-36w-x-24d-x-78h>: HTTP status code is not handled or not allowed 2026-01-25 03:53:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/partitions_dividers/room_dividers>: HTTP status code is not handled or not allowed 2026-01-25 03:53:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tempered-glass-for-cubbies-10-x-16>: HTTP status code is not handled or not allowed 2026-01-25 03:53:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 03:53:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lkwobg1617ss-bottom-grid-ss-16316lx141116w>: HTTP status code is not handled or not allowed 2026-01-25 03:53:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/140104-idis-1-act-flat-8l-x-4w-x-3h>: HTTP status code is not handled or not allowed 2026-01-25 03:53:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/food-grade-silicone-lube-5-gallon>: HTTP status code is not handled or not allowed 2026-01-25 03:53:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-in-i-d-zinc-plated-steel-closed-hammer-pliers-holder-for-locboard-3-pack>: HTTP status code is not handled or not allowed 2026-01-25 03:53:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sterile-cloning-cylinders-7mm-top-x-8mm-bottom-od-plastic-50pk>: HTTP status code is not handled or not allowed 2026-01-25 03:53:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-polypropylene-vortexer-attachment-for-microcentrifuge-tubes-for-15ml-tubes-20-places>: HTTP status code is not handled or not allowed 2026-01-25 03:53:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-double-tier-locker-3-wide-12w-x-12d-x-36h-assembled-blue-frost>: HTTP status code is not handled or not allowed 2026-01-25 03:53:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-x-48-melamine-shelf-black>: HTTP status code is not handled or not allowed 2026-01-25 03:53:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/countertop-costumer-w-shoulder-bar-matte-black>: HTTP status code is not handled or not allowed 2026-01-25 03:53:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-way-connector-chrome>: HTTP status code is not handled or not allowed 2026-01-25 03:53:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:53:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-way-connector-chrome>: HTTP status code is not handled or not allowed 2026-01-25 03:53:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tissue-culture-flask-rack-12-places-for-75ml-flasks>: HTTP status code is not handled or not allowed 2026-01-25 03:53:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-x-30-maple-safety-edge-mobile-production-bench-tan>: HTTP status code is not handled or not allowed 2026-01-25 03:53:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-neoprene-roll-720-l-x-12-w-x-1-8-thick-black-ultra-strength-50a>: HTTP status code is not handled or not allowed 2026-01-25 03:53:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:53:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/charging-bucket-lid-replaceable-black-3486465>: HTTP status code is not handled or not allowed 2026-01-25 03:54:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-fiberglass-extension-ladder-300-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 03:54:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-25 03:54:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2w-x-7h-grid-panel-semi-gloss-white>: HTTP status code is not handled or not allowed 2026-01-25 03:54:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/VTT-1884-84-W-x-18-D-T-430-18ga-Stainless-Steel-Workbench-w-Galvanized-Legs>: HTTP status code is not handled or not allowed 2026-01-25 03:54:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/material-handling/ladders_personnel_lifts/step_ladders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:54:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-x-40-3-gauge-peva-taffeta-w-window-zipper-burgundy-w-burgundy-trim>: HTTP status code is not handled or not allowed 2026-01-25 03:54:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cantilever-single-sided-starter-unit-5>: HTTP status code is not handled or not allowed 2026-01-25 03:54:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls>: HTTP status code is not handled or not allowed 2026-01-25 03:54:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5l-round-tubing-w-1-1-16-dia-chrome>: HTTP status code is not handled or not allowed 2026-01-25 03:54:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/laboratory_heating_cooling/laboratory_hotplates_stirrers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:54:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:54:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 60846 pages (at 90 pages/min), scraped 30569 items (at 47 items/min) 2026-01-25 03:54:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waterbase-street-utility-marking-paint-20-oz-orange-fluorescent-20-657-12-case>: HTTP status code is not handled or not allowed 2026-01-25 03:54:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/material-handling/ladders_personnel_lifts/step_ladders landed on page that is not a product page. 2026-01-25 03:54:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precision-brass-thumb-screw-8-32-3-8-thread-3-8-head-dia-3-16-head-h-pkg-of-10>: HTTP status code is not handled or not allowed 2026-01-25 03:54:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-buna-n-sheet-12-l-x-12-w-x-1-16-thick-black-high-strength-80a-acrylic-adhesive-bulk-rs-h80-45>: HTTP status code is not handled or not allowed 2026-01-25 03:54:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-heavy-weight-single-slot-w-1-2-slots-on-1-centers-satin-zinc>: HTTP status code is not handled or not allowed 2026-01-25 03:54:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/laboratory_heating_cooling/laboratory_hotplates_stirrers landed on page that is not a product page. 2026-01-25 03:54:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls>: HTTP status code is not handled or not allowed 2026-01-25 03:54:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:54:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-buna-n-sheet-12-l-x-12-w-x-3-32-thick-black-high-strength-80a-bulk-rs-h80-17>: HTTP status code is not handled or not allowed 2026-01-25 03:54:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-rack-netting-two-bay-1>: HTTP status code is not handled or not allowed 2026-01-25 03:54:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-94dv2r-n5-division-2-xenon-strobe-red-120v-ac>: HTTP status code is not handled or not allowed 2026-01-25 03:54:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6l-x-1-2-x-1-1-2-rectangular-tubing-chrome>: HTTP status code is not handled or not allowed 2026-01-25 03:54:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:54:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-pvc-type-1-tube-36-l-x-3-4-id-x-1-1-4-od-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:54:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/die_cut_labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:54:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-25 03:54:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b0277-orbit-shaft-balancer-5-x-316-orbit-1-pkg-qty>: HTTP status code is not handled or not allowed 2026-01-25 03:54:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/briggs-stratton-44s977-0032-g1-gas-engine-professional-v-twin-vertical-shaft-1-crank-shaft>: HTTP status code is not handled or not allowed 2026-01-25 03:54:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/die_cut_labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:54:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c-frame-motor-1-100-hp-1260-rpm-240v-oao>: HTTP status code is not handled or not allowed 2026-01-25 03:54:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/locks_lockout_devices/lockout_tagout_devices/lockout_padlocks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:54:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-25 03:54:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-50a-n5-40wh-flashing-beacon-amber-120v-ac>: HTTP status code is not handled or not allowed 2026-01-25 03:54:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:54:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/draft-inducer-blower-3139-rpm-115v-oao>: HTTP status code is not handled or not allowed 2026-01-25 03:54:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cl3514-11-2-hp-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 03:54:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-270fg24240a-incandescentled-bulb-module-green-24-240v-ac>: HTTP status code is not handled or not allowed 2026-01-25 03:54:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cjm3104-33-hp-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 03:54:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/locks_lockout_devices/lockout_tagout_devices/lockout_padlocks landed on page that is not a product page. 2026-01-25 03:54:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/apron-style-walk-ramp-38-w-x-14-l-1600-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 03:54:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:54:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mighty-products-mn-ms90-b0614-90-shade-cloth-shade-tarp-6-x-14>: HTTP status code is not handled or not allowed 2026-01-25 03:54:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-hinged-top-basket-truck-18-bu-blue-vinyl-wood-base-all-swivel>: HTTP status code is not handled or not allowed 2026-01-25 03:54:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/room-air-conditioner-motor-1-20-hp-1075-rpm-115v-oao-42y-frame>: HTTP status code is not handled or not allowed 2026-01-25 03:54:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-hinged-top-basket-truck-14-bu-yellow-vinyl-wood-base-all-swivel>: HTTP status code is not handled or not allowed 2026-01-25 03:54:56 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels landed on page that is not a product page. 2026-01-25 03:54:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/banner-the-safety-way-is-the-only-way-3-x-10>: HTTP status code is not handled or not allowed 2026-01-25 03:54:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels>: HTTP status code is not handled or not allowed 2026-01-25 03:55:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-coated-tape-476xl-1-x-540-yds-6-mil-translucent>: HTTP status code is not handled or not allowed 2026-01-25 03:55:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/sockets_bits/socket_sets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:55:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ecp3764t-5-3-hp-1165-rpm>: HTTP status code is not handled or not allowed 2026-01-25 03:55:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/general_purpose_electric_motors/three_phase_motors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:55:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flexible-duct-25-ft-length-of-12-inch-diameter-indoor-outdoor>: HTTP status code is not handled or not allowed 2026-01-25 03:55:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cfdl3507m-75-hp-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 03:55:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 60928 pages (at 82 pages/min), scraped 30616 items (at 47 items/min) 2026-01-25 03:55:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-inch-v-jaw-dipped-handle-tongue-groove-pliers>: HTTP status code is not handled or not allowed 2026-01-25 03:55:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/work-gloves-15-gauge-gray-nylon-shell-black-hpt-palm-and-fingertips-xl>: HTTP status code is not handled or not allowed 2026-01-25 03:55:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rolling-push-cart-hopper-green-32-x-40-x-24-4000lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 03:55:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/condenser-fans-motor-2-hp-1140-rpm-208-230-460v-teao>: HTTP status code is not handled or not allowed 2026-01-25 03:55:14 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/sockets_bits/socket_sets landed on page that is not a product page. 2026-01-25 03:55:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/general_purpose_electric_motors/three_phase_motors landed on page that is not a product page. 2026-01-25 03:55:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/he-i-li-hydraulic-li-positioner-4>: HTTP status code is not handled or not allowed 2026-01-25 03:55:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/long-deck-cart-1>: HTTP status code is not handled or not allowed 2026-01-25 03:55:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-afm3529-1-hp-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-25 03:55:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multipurpose-paper-white-8-1-2in-x-11in-20-lb-2500-sheets-carton>: HTTP status code is not handled or not allowed 2026-01-25 03:55:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cylinder-storage-cabinet-double-door-horizontal-16-cylinder-capacity-assembled>: HTTP status code is not handled or not allowed 2026-01-25 03:55:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keyed-trigger-lock-keyed-alike-no-90kadspt>: HTTP status code is not handled or not allowed 2026-01-25 03:55:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-union-6mm-tube-1-1-3-l>: HTTP status code is not handled or not allowed 2026-01-25 03:55:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-uniform-cabinet-with-4-shelves-36-w-x-24-d-x-75-h-brown>: HTTP status code is not handled or not allowed 2026-01-25 03:55:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-shift-industrial-cabinet-60-w-x-24-d-x-75-h-black>: HTTP status code is not handled or not allowed 2026-01-25 03:55:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clearview-industrial-cabinet-60-w-x-24-d-x-75-h-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:55:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-boltless-shelving-96-inch-w-x-24-inch-d-x-84-inch-h-3-levels-starter-w-ez-decking-black>: HTTP status code is not handled or not allowed 2026-01-25 03:55:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-5-tier-starter-unit-with-solid-bottom-shelf-48-w-x-24-d-x-74-h>: HTTP status code is not handled or not allowed 2026-01-25 03:55:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-4-tier-starter-unit-24-w-x-24-d-x-63-h>: HTTP status code is not handled or not allowed 2026-01-25 03:55:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cdpswd3410-0-25-hp-1750-rpm>: HTTP status code is not handled or not allowed 2026-01-25 03:55:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-swivel-run-tee-4mm-tube-x-1-4-swift-fit-universal-thread-5-6-l>: HTTP status code is not handled or not allowed 2026-01-25 03:55:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mastercool-53010-nitrogen-pressure-regulator-kit>: HTTP status code is not handled or not allowed 2026-01-25 03:55:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-boltless-shelving-96-inch-w-x-30-inch-d-1-level-w-wood-decking-black>: HTTP status code is not handled or not allowed 2026-01-25 03:55:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oem-replacement-motor-1-2-hp-1090-rpm-208-230v-teao>: HTTP status code is not handled or not allowed 2026-01-25 03:55:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-stackable-shelf-ledge-back-24-w-x-2-h>: HTTP status code is not handled or not allowed 2026-01-25 03:55:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wood-end-table-24-x-24-black>: HTTP status code is not handled or not allowed 2026-01-25 03:55:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-boltless-shelving-48-inch-w-x-30-inch-d-x-84-inch-h-3-levels-add-on-w-wire-decking-black>: HTTP status code is not handled or not allowed 2026-01-25 03:55:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-idvsnm2333t-15-hp-5400-rpm>: HTTP status code is not handled or not allowed 2026-01-25 03:55:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cabinet-with-electronic-lock-with-digital-screen-60-w-x-24-d-x-75-h-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:55:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cabinet-with-keyless-entry-lock-60-w-x-24-d-x-75-h-white>: HTTP status code is not handled or not allowed 2026-01-25 03:56:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/long-divider-14-l-x-4-w-x-2-h-gray-polyprolylene>: HTTP status code is not handled or not allowed 2026-01-25 03:56:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v-grip-pre-configured-wire-shelving-spill-containment-system-36-w-x-24-d-x-84-h-add-on-red>: HTTP status code is not handled or not allowed 2026-01-25 03:56:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-jl3510a-1-hp-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 03:56:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bowl-brush-polypro-12-case-96311>: HTTP status code is not handled or not allowed 2026-01-25 03:56:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 61027 pages (at 99 pages/min), scraped 30681 items (at 65 items/min) 2026-01-25 03:56:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylon-safety-flag-orange-24-x24>: HTTP status code is not handled or not allowed 2026-01-25 03:56:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bassinet-kit-w-closed-cabinet-includes-bassinet-basket-w-cardholder-and-mattress-pad>: HTTP status code is not handled or not allowed 2026-01-25 03:56:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-tab-poly-binder-index-dividers-with-slant-pockets-assorted-5-st>: HTTP status code is not handled or not allowed 2026-01-25 03:56:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-floor-pads-white-13>: HTTP status code is not handled or not allowed 2026-01-25 03:56:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-tear-drop-pallet-rack-add-on-96-w-x-42-d-x-96-h-796573n>: HTTP status code is not handled or not allowed 2026-01-25 03:56:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-ring-poly-binder-1-inch-capacity>: HTTP status code is not handled or not allowed 2026-01-25 03:56:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xtra-st1600h-ultrasonic-cleaner-208v-3-ph-dual-freq-25-45-khz-high-power-35-gallons>: HTTP status code is not handled or not allowed 2026-01-25 03:56:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insulated-hinged-lid-for-st1400h-st1600h>: HTTP status code is not handled or not allowed 2026-01-25 03:56:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/decontamination-station-116-w-x-30-d-base-cabinets-3-sink-bowls-deck-mounted-faucets>: HTTP status code is not handled or not allowed 2026-01-25 03:56:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-pocket-heavyweight-poly-portfolio-folder-orange>: HTTP status code is not handled or not allowed 2026-01-25 03:56:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dupont-tyvek-security-wristbands-blue-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 03:56:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slider-pencil-case-assorted-colors>: HTTP status code is not handled or not allowed 2026-01-25 03:56:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:56:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-hotbox-48-x-40-x-48-38-c-100-f>: HTTP status code is not handled or not allowed 2026-01-25 03:56:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:56:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spill-magnet-drain-cover-24-inches-by-36-inches-l-yellow-black-3112ls>: HTTP status code is not handled or not allowed 2026-01-25 03:56:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-station-wall-mounted-lodi-scrub-sink-knee-action-control-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 03:56:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perma-lined-panel-jacket-regular-2xl-spruce-green-jt50>: HTTP status code is not handled or not allowed 2026-01-25 03:56:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-3-8-industrial-coupler-x-3-8-male-nptf-2-1-3-l>: HTTP status code is not handled or not allowed 2026-01-25 03:56:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_labels landed on page that is not a product page. 2026-01-25 03:56:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:56:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perma-lined-panel-jacket-regular-4xl-navy-jt50>: HTTP status code is not handled or not allowed 2026-01-25 03:56:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits landed on page that is not a product page. 2026-01-25 03:56:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/desk-chair-mat-stonewash-in-grey-3-x-4>: HTTP status code is not handled or not allowed 2026-01-25 03:56:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-smock-loose-fit-short-sleeve-light-blue-s-tp23>: HTTP status code is not handled or not allowed 2026-01-25 03:56:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/23210-blank-valve-tag-25-pkg-brass-1-and-a-half-inches-dia>: HTTP status code is not handled or not allowed 2026-01-25 03:56:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-em3311t-8-7-5-hp-1770-rpm>: HTTP status code is not handled or not allowed 2026-01-25 03:56:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:56:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/locks_lockout_devices/lockout_tagout_devices/lockout_devices already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:56:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-yard-ramp-forkli-dock-ramp-36-l-x-84-w-25-000-cap-lbs-with-tow-bar>: HTTP status code is not handled or not allowed 2026-01-25 03:56:45 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_labels landed on page that is not a product page. 2026-01-25 03:56:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-dock-board-with-lock-pins-72-w-x-72-l-15-000-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 03:56:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-steel-mobile-storage-bins-ms4-1564-12-openings-64-l-x-20-w-x-45-1-2-h>: HTTP status code is not handled or not allowed 2026-01-25 03:56:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-shipping-tag-pre-wired-4-3-4-x-2-3-8-white-100-pack>: HTTP status code is not handled or not allowed 2026-01-25 03:56:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_labels>: HTTP status code is not handled or not allowed 2026-01-25 03:56:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/horizontal-baby-changing-table-white-granite-350lb-cap-surface-200-eh-03>: HTTP status code is not handled or not allowed 2026-01-25 03:56:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/locks_lockout_devices/lockout_tagout_devices/lockout_devices landed on page that is not a product page. 2026-01-25 03:56:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultrasidewinder-1-ft-extension-black>: HTTP status code is not handled or not allowed 2026-01-25 03:56:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bloodborne-pathogen-spill-clean-up-kit-plastic-case-24-piece>: HTTP status code is not handled or not allowed 2026-01-25 03:56:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-gas-heater-infrared-positive-pressure-straight-tube-tough-guy-40000-btu-20l>: HTTP status code is not handled or not allowed 2026-01-25 03:56:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-circle-tags-3-diamter-white-100-pack>: HTTP status code is not handled or not allowed 2026-01-25 03:56:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coverall-zipper-front-elastic-wrist-and-ankle-stormflap-bound-seam-yellow-5x>: HTTP status code is not handled or not allowed 2026-01-25 03:56:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-11in-clear-combination-spot-flood-light-bar-with-12-led-1492160>: HTTP status code is not handled or not allowed 2026-01-25 03:56:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/big-d-handheld-aerosol-room-deodorant-cinnamon>: HTTP status code is not handled or not allowed 2026-01-25 03:56:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-strap-kit-for-led-modular-light-bar-ford-f-250-to-550-1999-2016-3026115>: HTTP status code is not handled or not allowed 2026-01-25 03:57:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/big-d-flat-urinal-screen-apple-blue>: HTTP status code is not handled or not allowed 2026-01-25 03:57:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-middle-take-down-light-module-with-9-led-3024640>: HTTP status code is not handled or not allowed 2026-01-25 03:57:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ejmm4106t-20-hp-3520-rpm>: HTTP status code is not handled or not allowed 2026-01-25 03:57:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metalin-zipper-bag-8-x-8-100-pack>: HTTP status code is not handled or not allowed 2026-01-25 03:57:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coverallzipper-fronthoodelastic-wrist-and-anklestormflapbound-seamyellowxl>: HTTP status code is not handled or not allowed 2026-01-25 03:57:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_alarms_signals/forklift_alert_sensors_systems already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:57:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-water-filter-bracket-rotocast-outdoor-drinking-fountains>: HTTP status code is not handled or not allowed 2026-01-25 03:57:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/100-pure-coffee-decaffeinated-1-5-oz-pack-42-packs-carton>: HTTP status code is not handled or not allowed 2026-01-25 03:57:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 61115 pages (at 88 pages/min), scraped 30722 items (at 41 items/min) 2026-01-25 03:57:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precoiled-strap-on-pipe-marker-gas-1>: HTTP status code is not handled or not allowed 2026-01-25 03:57:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-powered-lcd-computer-cabinet-100ah-battery-dark-gray-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 03:57:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lawnmower-adapter-for-use-with-2-post-lifts-ateatlma-fpd>: HTTP status code is not handled or not allowed 2026-01-25 03:57:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_alarms_signals/forklift_alert_sensors_systems>: HTTP status code is not handled or not allowed 2026-01-25 03:57:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-metric-master-cylinder-bleeder-kit-thx804>: HTTP status code is not handled or not allowed 2026-01-25 03:57:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brake-bleeder-mitmv6830>: HTTP status code is not handled or not allowed 2026-01-25 03:57:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precoiled-strap-on-pipe-marker-gas-3>: HTTP status code is not handled or not allowed 2026-01-25 03:57:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precoiled-strap-on-pipe-marker-storm-drain-8>: HTTP status code is not handled or not allowed 2026-01-25 03:57:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/overall-green-40mm-pvc-on-polyester-lg>: HTTP status code is not handled or not allowed 2026-01-25 03:57:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precoiled-strap-on-pipe-marker-hydrogen-5>: HTTP status code is not handled or not allowed 2026-01-25 03:57:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/plumbing_parts_accessories/pipe_hose_tubing_accessories/pipe_markers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:57:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6in-red-oval-stop-turn-tail-surface-mount-light-kit-with-10-led-5626552>: HTTP status code is not handled or not allowed 2026-01-25 03:57:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/cutting_tools/paper_trimmers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:57:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mlrs8mclk-breast-milk-refrigerator-with-4-lockers-8-cu-ft>: HTTP status code is not handled or not allowed 2026-01-25 03:57:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stat-i-d-interim-identification-bracelet-250-box>: HTTP status code is not handled or not allowed 2026-01-25 03:57:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/19pc-driver-bit-set-gunmetal-bronze-s2-modified-steel>: HTTP status code is not handled or not allowed 2026-01-25 03:57:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/plumbing/plumbing_parts_accessories/pipe_hose_tubing_accessories/pipe_markers>: HTTP status code is not handled or not allowed 2026-01-25 03:57:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/basic_supplies/cutting_tools/paper_trimmers>: HTTP status code is not handled or not allowed 2026-01-25 03:57:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/square-3-power-bit-x-3-1-2-s2-modified-steel-gunmetal-grey>: HTTP status code is not handled or not allowed 2026-01-25 03:57:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-venm3546t-1-hp-1745-rpm>: HTTP status code is not handled or not allowed 2026-01-25 03:57:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-4-x-3-1-2-welded-pipe-nipple-schedule-40-316-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 03:57:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ambidex-disposable-nitrile-industrial-grade-4-mil-textured-xl>: HTTP status code is not handled or not allowed 2026-01-25 03:57:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/phillips-2-insert-bit-x-1-drywall-indenter-s2-modified-steel-gunmetal-grey>: HTTP status code is not handled or not allowed 2026-01-25 03:57:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-wheelsecure-mobile-shredder-cart-175-gallon-capacity-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:57:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-plastic-sign-danger-explosives-keep-out>: HTTP status code is not handled or not allowed 2026-01-25 03:57:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-partition-3-drawer-pedestal-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:57:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-m3543t-5-75-hp-1140-rpm>: HTTP status code is not handled or not allowed 2026-01-25 03:57:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-vinyl-sign-caution-do-not-operate-this-machine-without-guards>: HTTP status code is not handled or not allowed 2026-01-25 03:57:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esp-kit-1-pair-black-esp-glove-1-pair-goat-class-0-size-7>: HTTP status code is not handled or not allowed 2026-01-25 03:57:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torq-set-4-insert-bit-x-1-s2-modified-steel-gunmetal-grey>: HTTP status code is not handled or not allowed 2026-01-25 03:57:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-8-precision-flat-washer-1-3-8-o-d-1-8-thick-steel-black-oxide-pkg-of-10>: HTTP status code is not handled or not allowed 2026-01-25 03:58:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-vinyl-sign-caution-eye-protection-required-1>: HTTP status code is not handled or not allowed 2026-01-25 03:58:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3408-hip-lr-drill-kit-type-iii-barricade-w-plastic-uprights-angle-iron-feet-hip-striped-sheeting>: HTTP status code is not handled or not allowed 2026-01-25 03:58:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torx-174-25-insert-bit-x-1-gunmetal-bronze-s2-modified-steel>: HTTP status code is not handled or not allowed 2026-01-25 03:58:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wsl2-3060-ah-welded-steel-workbenches-w-back-and-end-stops-30-x-60-adj-height>: HTTP status code is not handled or not allowed 2026-01-25 03:58:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 61211 pages (at 96 pages/min), scraped 30785 items (at 63 items/min) 2026-01-25 03:58:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cam-follower-sealed-hex-head-eccentric-stud-3-4-l-stud-04375-w-roller-0625-roller-dia>: HTTP status code is not handled or not allowed 2026-01-25 03:58:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-plastic-sign-no-smoking>: HTTP status code is not handled or not allowed 2026-01-25 03:58:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-3-tier-wire-shelving-add-on-unit-72w-x-21d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 03:58:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-2-tier-wire-shelving-add-on-unit-36w-x-21d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 03:58:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/water_filtration/inline_beverage_ice_machine_filters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:58:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-5-tier-wire-shelving-starter-unit-48w-x-14d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 03:58:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/le-3-26-flexible-leader>: HTTP status code is not handled or not allowed 2026-01-25 03:58:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/r-3uc-10-3-u-cutter-w-r-cables>: HTTP status code is not handled or not allowed 2026-01-25 03:58:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelf-truck-esd-21w-x-60l-x-80h-polyurethane-4-swivel-2-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 03:58:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelf-truck-esd-24w-x-72l-x-80h-polyurethane-4-swivel-2-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 03:58:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/water_filtration already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:58:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-plastic-sign-caution-no-smoking-beyond-this-point>: HTTP status code is not handled or not allowed 2026-01-25 03:58:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/summit-commercial-45-capable-chest-freezer-wdigital-thermostat-48-cuft-capacity>: HTTP status code is not handled or not allowed 2026-01-25 03:58:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/light-duty-floor-crane-4l0007-500-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 03:58:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v8770160-intrinsic-style-mid-sole-ice-cleat-original-profile>: HTTP status code is not handled or not allowed 2026-01-25 03:58:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-hepa-filter-for-portable-air-conditioner-293149>: HTTP status code is not handled or not allowed 2026-01-25 03:58:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/impromptu-machine-stand-gray>: HTTP status code is not handled or not allowed 2026-01-25 03:58:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/plumbing/water_filtration landed on page that is not a product page. 2026-01-25 03:58:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oty-42075-2-101-pnc-series-pvc-no-calk-shower-drain-w-plastic-strainer>: HTTP status code is not handled or not allowed 2026-01-25 03:58:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-hose-wagon>: HTTP status code is not handled or not allowed 2026-01-25 03:58:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-4-tier-esd-36w-x-72l-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 03:58:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:58:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/danger-sign-14x10-vinyl-non-potable-water>: HTTP status code is not handled or not allowed 2026-01-25 03:58:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-4-tier-esd-14w-x-54l-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 03:58:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergency-sign-14x10-aluminum-first-aid-station>: HTTP status code is not handled or not allowed 2026-01-25 03:58:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/valiant-outdoor-parcel-locker-black>: HTTP status code is not handled or not allowed 2026-01-25 03:58:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11883-cartridge-roll-1-2-x-1-x-1-8-aluminum-oxide-medium>: HTTP status code is not handled or not allowed 2026-01-25 03:58:49 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs landed on page that is not a product page. 2026-01-25 03:58:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honeywell-single-rod-direct-spark-ignition-control-6-second-trial-timing>: HTTP status code is not handled or not allowed 2026-01-25 03:58:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40435-satin-disc-wheel-1-8-mandrel-1-aluminum-oxide-fine>: HTTP status code is not handled or not allowed 2026-01-25 03:58:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8x18x4-1-2-kraft-open-top-bin-boxes>: HTTP status code is not handled or not allowed 2026-01-25 03:58:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-only-1-8-thick-polytough-plastic-white>: HTTP status code is not handled or not allowed 2026-01-25 03:58:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40332-qc-disc-type-r-2-aluminum-oxide-medium>: HTTP status code is not handled or not allowed 2026-01-25 03:58:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-esd-21w-x-36l-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 03:58:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/goffs-welding-screen-8w-x-8h-green>: HTTP status code is not handled or not allowed 2026-01-25 03:59:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-inch-x-12-inch-x-6-inch-doublewall-boxes>: HTTP status code is not handled or not allowed 2026-01-25 03:59:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-lav-faucet-w-fittings>: HTTP status code is not handled or not allowed 2026-01-25 03:59:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silicone-foam-no-adhesive-14-thick-x-12-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 03:59:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-non-electric-office-partition-panel-with-raceway-48-14-w-x-65-12-h-tan>: HTTP status code is not handled or not allowed 2026-01-25 03:59:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-esd-21w-x-24l-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 03:59:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mw-481812-hd-polymight-dunnage-rack-48w-x-18d-x-12h-1500-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-25 03:59:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-dual-heat-electric-garage-heater-w-bluetooth-51-180-btu-15-000-watts>: HTTP status code is not handled or not allowed 2026-01-25 03:59:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/martin-yale-high-speed-letter-opener>: HTTP status code is not handled or not allowed 2026-01-25 03:59:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/individual-character-stencil-12-letter-w>: HTTP status code is not handled or not allowed 2026-01-25 03:59:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 61307 pages (at 96 pages/min), scraped 30840 items (at 55 items/min) 2026-01-25 03:59:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/individual-character-stencil-12-letter-z>: HTTP status code is not handled or not allowed 2026-01-25 03:59:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conductive-silicone-o-ring-dash-013-pack-of-5>: HTTP status code is not handled or not allowed 2026-01-25 03:59:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-24x1-2-flat-undercut-head-thread-cutting-screws-type-f-1208FPU188>: HTTP status code is not handled or not allowed 2026-01-25 03:59:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rust-oleum-m1800-sys-wtr-based-prec-line-invert-mark-pnt-aero-wh>: HTTP status code is not handled or not allowed 2026-01-25 03:59:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-turn-straight-stop-valve-w-cross-handle-brushed-nickel-1-2-fip-x-3-8-od-comp>: HTTP status code is not handled or not allowed 2026-01-25 03:59:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-4-tier-esd-12w-x-48l-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 03:59:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silicone-rubber-strip-with-high-temp-adhesive-60a-18-thick-x-1-wide-x-10-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 03:59:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-link-t-link-hard-hat-assembly-tychem-2000-hood-hx5-papr>: HTTP status code is not handled or not allowed 2026-01-25 03:59:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rule-index-cards-40279-3-x-5-glow-assorted-100pack>: HTTP status code is not handled or not allowed 2026-01-25 03:59:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-20x1-2-pan-head-machine-screws-1408MCP188>: HTTP status code is not handled or not allowed 2026-01-25 03:59:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-safety-sign-fire-blanket-vinyl>: HTTP status code is not handled or not allowed 2026-01-25 03:59:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:59:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clearance-bar-kit-7-d-x-96-l-white-bar-red-tapes>: HTTP status code is not handled or not allowed 2026-01-25 03:59:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rust-oleum-as5600-system-acrylic-anti-slip-floor-and-deck-coating-blk>: HTTP status code is not handled or not allowed 2026-01-25 03:59:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-inch-tow-lawn-sweeper-67-inch-l-x-51-inch-w-x-24-inch-h-black>: HTTP status code is not handled or not allowed 2026-01-25 03:59:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-safety-sign-fire-sprinkler-shut-off-valve-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 03:59:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/entrance-mat-3-8-thick-6-l-x-8-w-black-cleated-backing>: HTTP status code is not handled or not allowed 2026-01-25 03:59:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs>: HTTP status code is not handled or not allowed 2026-01-25 03:59:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:59:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-safety-sign-fire-door-keep-closed-at-all-times-plastic>: HTTP status code is not handled or not allowed 2026-01-25 03:59:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-safety-sign-fire-extinguisher-with-le-arrow-vinyl>: HTTP status code is not handled or not allowed 2026-01-25 03:59:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hardware-pack-3-chains-for-panel-fans>: HTTP status code is not handled or not allowed 2026-01-25 03:59:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rule-index-cards-35810-5-x-8-assorted-100pack>: HTTP status code is not handled or not allowed 2026-01-25 03:59:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silicone-o-ring-dash-430-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-25 03:59:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/entrance-mat-3-8-thick-2-l-x-3-w-black-cleated-backing>: HTTP status code is not handled or not allowed 2026-01-25 03:59:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-safety-sign-fire-extinguisher-do-not-block-plastic>: HTTP status code is not handled or not allowed 2026-01-25 03:59:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7x7-8-flat-self-tapping-screws-type-a-thread-0714APF>: HTTP status code is not handled or not allowed 2026-01-25 03:59:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/sticky_notes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:59:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-gloss-photo-paper-99550-8-12-x-11-white-50pack>: HTTP status code is not handled or not allowed 2026-01-25 03:59:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/floor_mats/entrance_mats>: HTTP status code is not handled or not allowed 2026-01-25 03:59:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evs-1440b-evs-lathe-with-taper-attachment-collet-closer>: HTTP status code is not handled or not allowed 2026-01-25 03:59:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-rubber-overboots-mens-black-size-1719-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 03:59:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 03:59:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tingley-174-pilot-g2-knee-boot-composite-safety-toe-15-h-size-13-black>: HTTP status code is not handled or not allowed 2026-01-25 03:59:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/paper_stationery/sticky_notes landed on page that is not a product page. 2026-01-25 03:59:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10x3-modified-truss-head-self-drilling-screws-1048KPM>: HTTP status code is not handled or not allowed 2026-01-25 03:59:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/415-1-1-4-tubeless-tire-valve>: HTTP status code is not handled or not allowed 2026-01-25 04:00:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10x3-flat-self-drilling-screws-1048KQF>: HTTP status code is not handled or not allowed 2026-01-25 04:00:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards>: HTTP status code is not handled or not allowed 2026-01-25 04:00:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ingersoll-rand-non-cycling-air-dryer-64-cfm>: HTTP status code is not handled or not allowed 2026-01-25 04:00:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grit-anti-slip-tape-white-4-w>: HTTP status code is not handled or not allowed 2026-01-25 04:00:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-20x1-1-2-truss-head-machine-screws-1424MPTB>: HTTP status code is not handled or not allowed 2026-01-25 04:00:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10x3-flat-head-self-drilling-screws-1048KPF410>: HTTP status code is not handled or not allowed 2026-01-25 04:00:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 61399 pages (at 92 pages/min), scraped 30893 items (at 53 items/min) 2026-01-25 04:00:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tribeca-queen-tufted-upholstered-platform-bed-beige-with-pocket-spring-mattress>: HTTP status code is not handled or not allowed 2026-01-25 04:00:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:00:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/utility-post-yellow-2>: HTTP status code is not handled or not allowed 2026-01-25 04:00:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/toilets_urinals/bidets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:00:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/stamp_pads already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:00:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buna-n-o-ring-dash-226-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 04:00:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/heaters/stoves_fireplaces_log_sets_fire_pits/fireplaces_log_sets_fireboxes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:00:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/stamps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:00:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premier-pharmacy-stainless-steel-solid-door-refrigerator-72-cu-ft-ph-abt-hc-ssp-72>: HTTP status code is not handled or not allowed 2026-01-25 04:00:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:00:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brown-viton-o-ring-dash-372-pack-of-1>: HTTP status code is not handled or not allowed 2026-01-25 04:00:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/heaters/stoves_fireplaces_log_sets_fire_pits/stove_heaters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:00:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges>: HTTP status code is not handled or not allowed 2026-01-25 04:00:21 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/plumbing/toilets_urinals/bidets landed on page that is not a product page. 2026-01-25 04:00:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/basic_supplies/stamp_pads>: HTTP status code is not handled or not allowed 2026-01-25 04:00:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/heaters/stoves_fireplaces_log_sets_fire_pits/fireplaces_log_sets_fireboxes landed on page that is not a product page. 2026-01-25 04:00:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/stamps landed on page that is not a product page. 2026-01-25 04:00:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-desk-cherry-2>: HTTP status code is not handled or not allowed 2026-01-25 04:00:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges>: HTTP status code is not handled or not allowed 2026-01-25 04:00:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/heaters/stoves_fireplaces_log_sets_fire_pits/stove_heaters>: HTTP status code is not handled or not allowed 2026-01-25 04:00:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/comfort-grip-retractable-ballpoint-pen-1mm-black-ink-silver-barrel-dozen>: HTTP status code is not handled or not allowed 2026-01-25 04:00:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quicknotes-recycled-mini-erasable-wall-planner-16-x-12-2015-aagpm550b28>: HTTP status code is not handled or not allowed 2026-01-25 04:00:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hirsh-industries-26in-deep-vertical-file-cabinet-4-drawer-letter-size-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:00:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fda-neoprene-rubber-strip-no-adhesive-60a-18-thick-x-2w-x-10l>: HTTP status code is not handled or not allowed 2026-01-25 04:00:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hirsh-industries-10000-series-lateral-file-36in-wide-3-drawer-putty>: HTTP status code is not handled or not allowed 2026-01-25 04:00:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-8-split-lock-washers-medium-62WS316>: HTTP status code is not handled or not allowed 2026-01-25 04:00:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/hand_tools_1/wrenches/combination_wrenches already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:00:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flux-wrap-cooling-jacket-system-w-insulation-wrap-tubing-connectors-15-gallon-drum>: HTTP status code is not handled or not allowed 2026-01-25 04:00:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10x3-8-round-spacers-310610RS303>: HTTP status code is not handled or not allowed 2026-01-25 04:00:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4x4-toggle-anchors-1464TBCR>: HTTP status code is not handled or not allowed 2026-01-25 04:00:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/print-or-write-removable-multi-use-labels-3-4-x-1-white-1000-pack-05428>: HTTP status code is not handled or not allowed 2026-01-25 04:00:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pres-a-ply-laser-address-labels-3-1-3-x-4-white-600-box-30604>: HTTP status code is not handled or not allowed 2026-01-25 04:00:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/556-professional-rolling-trimmer-37in-cutting-length>: HTTP status code is not handled or not allowed 2026-01-25 04:00:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/hand_tools_1/wrenches/combination_wrenches>: HTTP status code is not handled or not allowed 2026-01-25 04:00:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/odorstop-72-watt-uv-air-treatment-system-with-16-bulbs>: HTTP status code is not handled or not allowed 2026-01-25 04:00:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-six-drawer-narrow-procedure-cart-tall-height-mini-width-key-lock-hammertone-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:00:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cardboard-carton-truck-single-level-48-x-24-5-polyurethane-casters>: HTTP status code is not handled or not allowed 2026-01-25 04:00:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-tank-bottle-lifter-yellow-500-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 04:00:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/automatic-label-applicator-for-square-rectangular-items-up-to-12-w-boxmate-612>: HTTP status code is not handled or not allowed 2026-01-25 04:00:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-lift-beam-yellow-42-l-x-16-w-x-6-h-80lbs-10000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 04:00:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-flush-door-welded-bin-cabinet-14-bins-2-shelves>: HTTP status code is not handled or not allowed 2026-01-25 04:00:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/champion-fmp41mc-rd-fmpro-41-w-x-20-1-4-d-x-10-h-red-mobile-maintenance-cart-base>: HTTP status code is not handled or not allowed 2026-01-25 04:01:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pourover-coffee-brewer-2-warmers-vp17-2-sst>: HTTP status code is not handled or not allowed 2026-01-25 04:01:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-x-6-working-length-j-hook-style-b>: HTTP status code is not handled or not allowed 2026-01-25 04:01:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-lift-beam-yellow-12-l-x-14-w-x-12-h-9lbs-500-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 04:01:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/19-litre-thermal-carafes-thermal-carafeorn-185l-12pk>: HTTP status code is not handled or not allowed 2026-01-25 04:01:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 61490 pages (at 91 pages/min), scraped 30946 items (at 53 items/min) 2026-01-25 04:01:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-cart-standard-width-tall-cabinet-6-drawers-41-2-5-wx22-lx64-1-10-h-hammertone-green>: HTTP status code is not handled or not allowed 2026-01-25 04:01:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-safe-130eh-electronic-lock-28w-x-30-1-2d-x-57h-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:01:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/security_access_control/safes_lockboxes/security_safes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:01:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/depository-safe-mb2014il-electronic-lock-16-1-2w-x-14d-x-20h-black>: HTTP status code is not handled or not allowed 2026-01-25 04:01:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/61825-s383p-class-2-mesh-vest-hi-vis-orange-contrasting-trim-xl>: HTTP status code is not handled or not allowed 2026-01-25 04:01:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-twin-hoist-lift-beam-yellow-10000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 04:01:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-series-aluminium-medical-cart-tall-height-36-3-4-w-x-22-d-x-43-3-4-h-brushed-silver>: HTTP status code is not handled or not allowed 2026-01-25 04:01:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/security_access_control/safes_lockboxes/security_safes landed on page that is not a product page. 2026-01-25 04:01:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-feet-x-10-feet-sf-9-93-oz-water-resist-canvas-tarp-tan>: HTTP status code is not handled or not allowed 2026-01-25 04:01:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dot-placard-radioactive-7-1>: HTTP status code is not handled or not allowed 2026-01-25 04:01:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nestable-tote-24-x-17-1-2-x-13-cardboard-brown-qty-100-plus>: HTTP status code is not handled or not allowed 2026-01-25 04:01:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-x-6-working-length-j-hook-style-9>: HTTP status code is not handled or not allowed 2026-01-25 04:01:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-diamond-series-built-in-undercounter-manual-defrost-freezer-ss-door-right-hinged>: HTTP status code is not handled or not allowed 2026-01-25 04:01:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-freestanding-undercounter-freezer-4-cu-ft>: HTTP status code is not handled or not allowed 2026-01-25 04:01:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plus-plastic-stanchion-stake-22-42-adjustable-height-red>: HTTP status code is not handled or not allowed 2026-01-25 04:01:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poly-z-brite-heavy-duty-wire-shelving-ds18606z-60w-x-18d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 04:01:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-frame-72-x-74-yellow-800-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 04:01:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/horn-120vac-350-120-30>: HTTP status code is not handled or not allowed 2026-01-25 04:01:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/selectone-signal-24vac-dc-hazardous-location-gain-control-300gcx-024>: HTTP status code is not handled or not allowed 2026-01-25 04:01:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/litestak-led-light-module-24vdc-green-lsld-024g>: HTTP status code is not handled or not allowed 2026-01-25 04:01:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/litestak-led-light-module-120vac-amber-lsld-120a>: HTTP status code is not handled or not allowed 2026-01-25 04:01:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vynckier-vj1210hw-vj-12-inch-x-10-inch-non-metallic-enclosure-hinge-2-corner-screws>: HTTP status code is not handled or not allowed 2026-01-25 04:01:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/strobe-light-male-pipe-mount-12-48vdc-green-lp3m-012-048g>: HTTP status code is not handled or not allowed 2026-01-25 04:01:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/250mm-dia-slewing-ring-bearing-17984-lbs-max-axial-static>: HTTP status code is not handled or not allowed 2026-01-25 04:01:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-trade-show-supertrunk-foam-lined-with-wheels-843-36-fl-ww-38x26x26-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:01:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-handbook-lockout-tagout-controlling-the-beast>: HTTP status code is not handled or not allowed 2026-01-25 04:01:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-2-drylin-r-twin-bearing-block-with-polymer-liner>: HTTP status code is not handled or not allowed 2026-01-25 04:02:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyjohn-gap-compliant-portable-restroom>: HTTP status code is not handled or not allowed 2026-01-25 04:02:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triple-column-storage-cabinet-electronic-keypad-lock-60-w-x-28-d-x-81-h-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:02:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-z-brite-4-tier-microwave-cart-24-l-x-18-w-x-63-h>: HTTP status code is not handled or not allowed 2026-01-25 04:02:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/square-mailing-tubes-5-x-5-x-48-200-ect-32-kraft>: HTTP status code is not handled or not allowed 2026-01-25 04:02:05 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-25 04:02:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triple-column-storage-cabinet-electronic-keypad-lock-60-w-x-28-d-x-81-h-cream>: HTTP status code is not handled or not allowed 2026-01-25 04:02:06 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:31 2026-01-25 04:02:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-28-insert-for-hard-wood-brass-400-428>: HTTP status code is not handled or not allowed 2026-01-25 04:02:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/boltless-heavy-duty-die-rack-96-w-x-24-d-x-84-h-5-shelves-medium-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:02:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 61581 pages (at 91 pages/min), scraped 31003 items (at 57 items/min) 2026-01-25 04:02:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/octagon-sign-base-60-lbs-w-wheels-u-channel-adaptor-4-u-channel-post>: HTTP status code is not handled or not allowed 2026-01-25 04:02:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/engraved-occupancy-sign-meeting-room-use-vacant-red>: HTTP status code is not handled or not allowed 2026-01-25 04:02:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/prowler-pool-66-gallon>: HTTP status code is not handled or not allowed 2026-01-25 04:02:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/half-ht-double-column-med-storage-cabinet-keylock-40-1-10-wx29-1-4-dx40-3-4-h-yellow>: HTTP status code is not handled or not allowed 2026-01-25 04:02:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-6ml-cf-cryogenic-vial-internal-thread-round-bottom-sterile-polypropylene-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 04:02:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m4-07-thin-wall-insert-for-metal-319-m4>: HTTP status code is not handled or not allowed 2026-01-25 04:02:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-32-tapered-ultrasonic-insert-single-vane-th-132-sv>: HTTP status code is not handled or not allowed 2026-01-25 04:02:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/half-ht-double-column-med-storage-cabinet-keylock-40-1-10-wx29-1-4-dx40-3-4-h-forest-green>: HTTP status code is not handled or not allowed 2026-01-25 04:02:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wedgebit-drill-bit-sdsplus-size-58-usable-6-l-overall-8-l>: HTTP status code is not handled or not allowed 2026-01-25 04:02:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/511-rotunda-graphics-case-with-wheels-trade-show-case-14l-x-14w-x-55h-black>: HTTP status code is not handled or not allowed 2026-01-25 04:02:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/office-facility-signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:02:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/suredry-high-vol-16-scope-cabinet-driscope-aid-key-lock-44-wx24-dx93-h-hammertone-red>: HTTP status code is not handled or not allowed 2026-01-25 04:02:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/emergency_vehicle_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:02:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-solutions-neutra-sul-oxidizer-4-1-gallon-bottles>: HTTP status code is not handled or not allowed 2026-01-25 04:02:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fc250c-manual-post-bottomer>: HTTP status code is not handled or not allowed 2026-01-25 04:02:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/patient_support_immobilization/splints_supports already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:02:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roller-bag-for-davit-cranes>: HTTP status code is not handled or not allowed 2026-01-25 04:02:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/k6000-control-key-for-5-pin-cylinder-padlocks>: HTTP status code is not handled or not allowed 2026-01-25 04:02:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/office-facility-signs>: HTTP status code is not handled or not allowed 2026-01-25 04:02:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/emergency_vehicle_lights>: HTTP status code is not handled or not allowed 2026-01-25 04:02:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spring-return-pneumatic-actuator-205-in-lbs-spring-end>: HTTP status code is not handled or not allowed 2026-01-25 04:02:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cp354518-concepthmi-encl-wblack-trim-type-4-50x450x180mm>: HTTP status code is not handled or not allowed 2026-01-25 04:02:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-lug-style-butterfly-valve-w-buna-seals-and-10-position-handle>: HTTP status code is not handled or not allowed 2026-01-25 04:02:42 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/patient_support_immobilization/splints_supports landed on page that is not a product page. 2026-01-25 04:02:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/siesta-miranda-high-back-dining-chair-black>: HTTP status code is not handled or not allowed 2026-01-25 04:02:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/think-osha-7x10-vinyl-on-the-job-safety-begins-here-2>: HTTP status code is not handled or not allowed 2026-01-25 04:02:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c8c12ss-consolet-sloped-cover-type-12-800x1200x709in>: HTTP status code is not handled or not allowed 2026-01-25 04:02:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jpa-vertical-lifting-180-degree-turning-plate-clamp-steel-4000-lbs-cap-1-opening>: HTTP status code is not handled or not allowed 2026-01-25 04:02:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wb-mfg-30-x-60-lobo-heavy-duty-table-with-armor-edge-laminate-top-fixed-legs-wild-cherry>: HTTP status code is not handled or not allowed 2026-01-25 04:02:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:02:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-2-pc-ss-npt-ball-valve-w-da-pneumatic>: HTTP status code is not handled or not allowed 2026-01-25 04:02:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:02:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/restricted-area-aluminum-bilingual-unauthorized-persons-keep-out>: HTTP status code is not handled or not allowed 2026-01-25 04:02:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:02:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-bar-1-14-thick-x-3-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 04:02:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/american-lock-non-rekeyable-solid-aluminum-padlock-no-a11>: HTTP status code is not handled or not allowed 2026-01-25 04:02:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-jack-100-ft-ct-zn>: HTTP status code is not handled or not allowed 2026-01-25 04:03:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 04:03:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs>: HTTP status code is not handled or not allowed 2026-01-25 04:03:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/american-lock-solid-aluminum-rectangular-padlock-purple-no-a1207prp>: HTTP status code is not handled or not allowed 2026-01-25 04:03:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-320-compressed-air-hose-kit-157-12-25-l>: HTTP status code is not handled or not allowed 2026-01-25 04:03:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-spreader-beam-10-000-lbs-capacity-240-chain-top-rigging-yellow-steel>: HTTP status code is not handled or not allowed 2026-01-25 04:03:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-16-1-1-2-lap-lk-zinc-ctn>: HTTP status code is not handled or not allowed 2026-01-25 04:03:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-uhmw-polyethylene-plastic-bar-1-14-thick-x-2-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 04:03:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 04:03:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 61675 pages (at 94 pages/min), scraped 31059 items (at 56 items/min) 2026-01-25 04:03:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/neoprene-rubber-sheet-no-adhesive-60a-38-thick-x-6-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 04:03:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tri-flow-synthetic-food-grade-grease-n-l-g-i-grade-2-1-4-oz-tube>: HTTP status code is not handled or not allowed 2026-01-25 04:03:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unisex-excel-flame-resistant-lab-coat-7-oz-light-blue-cotton-3xl>: HTTP status code is not handled or not allowed 2026-01-25 04:03:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/lab_coats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:03:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-lab-coat-navy-polycombed-cotton-regular-54>: HTTP status code is not handled or not allowed 2026-01-25 04:03:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xx-garolite-sheet-58-thick-x-12-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 04:03:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brake-control-wiring-harness-chevy-gmc-14-16-bchgm04>: HTTP status code is not handled or not allowed 2026-01-25 04:03:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-2-thick-x-6-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 04:03:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xx-garolite-sheet-1-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 04:03:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/lab_coats landed on page that is not a product page. 2026-01-25 04:03:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:03:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caster-wheel-for-6hbw30-8hbw30-8hbe30-8hbe40-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 04:03:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lube-fitting-for-mpw060e-a897-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 04:03:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:03:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/axle-for-pe-4000-pallet-trucks-cr-128648-002>: HTTP status code is not handled or not allowed 2026-01-25 04:03:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-built-in-combo-lock-ada-compliant-lift-handle-green>: HTTP status code is not handled or not allowed 2026-01-25 04:03:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-security-combination-padlock-silver>: HTTP status code is not handled or not allowed 2026-01-25 04:03:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits landed on page that is not a product page. 2026-01-25 04:03:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bushing-for-wp-2000-pallet-trucks-cr-055084-009>: HTTP status code is not handled or not allowed 2026-01-25 04:03:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-built-in-combination-lock-with-long-bolt-right-hinged>: HTTP status code is not handled or not allowed 2026-01-25 04:03:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:03:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-axle-for-m-series-stacker-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 04:03:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:03:42 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits landed on page that is not a product page. 2026-01-25 04:03:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-for-wrt-60-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 04:03:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dumpster-dolly-single-caster-250-if-container-has-2-wheels-350lbs-capacity-lbs>: HTTP status code is not handled or not allowed 2026-01-25 04:03:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:03:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-acetal-plastic-bar-18-thick-x-34-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 04:03:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-key-for-1695mkada>: HTTP status code is not handled or not allowed 2026-01-25 04:03:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storage-container-with-lid-36-x-6-x-4-1-5-green>: HTTP status code is not handled or not allowed 2026-01-25 04:03:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-ner-dual-speed-elec-hoist-push-trolley-3-20-173>: HTTP status code is not handled or not allowed 2026-01-25 04:03:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fuse-for-w-40-xl-w-40-xt-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 04:03:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-25 04:03:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microflex-174-onyx-174-n64-nitrile-gloves-powder-free-beaded-size-m-100-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:03:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits landed on page that is not a product page. 2026-01-25 04:03:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits landed on page that is not a product page. 2026-01-25 04:03:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-axle-for-st-3000-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 04:03:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-ed-dual-speed-electric-chain-hoist-400-lbs-10-2610>: HTTP status code is not handled or not allowed 2026-01-25 04:03:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-ner-dual-speed-elec-hoist-motor-trolley-14-10-539>: HTTP status code is not handled or not allowed 2026-01-25 04:04:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-ner-electric-chain-hoist-hook-suspension-18-20-55>: HTTP status code is not handled or not allowed 2026-01-25 04:04:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-w-lse-acrylic-adhesive-18-thick-x-12-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 04:04:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caster-wheel-assembly-for-8300-8400-8500-pallet-trucks-ra-671-041>: HTTP status code is not handled or not allowed 2026-01-25 04:04:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:04:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/27u-linier-server-cabinet-glass-vented-doors-24-depth>: HTTP status code is not handled or not allowed 2026-01-25 04:04:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:04:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ships-stair-30-w-tread-floor-heights-9-6-to-10-5-796925>: HTTP status code is not handled or not allowed 2026-01-25 04:04:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 61763 pages (at 88 pages/min), scraped 31109 items (at 50 items/min) 2026-01-25 04:04:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hygienic-wall-bracket-grip-band-module-black-polypropylene-tpe-rubber-polyamide>: HTTP status code is not handled or not allowed 2026-01-25 04:04:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x42x74-chrome-shelving-with-13-magnum-giant-hopper-bins-black>: HTTP status code is not handled or not allowed 2026-01-25 04:04:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nps-8ft-mobile-convertible-bench-unit-particleboard-top-t-mold-edge-cherry>: HTTP status code is not handled or not allowed 2026-01-25 04:04:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-for-111-112-113-114-pallet-trucks-ra-632-069-007>: HTTP status code is not handled or not allowed 2026-01-25 04:04:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 04:04:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nps-10ft-mobile-cafeteria-fixed-bench-unit-with-mdf-core-top-protect-edge-oak>: HTTP status code is not handled or not allowed 2026-01-25 04:04:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gripper-front-butcher-frock-wo-pockets-white-polyestercotton-twill-4xl>: HTTP status code is not handled or not allowed 2026-01-25 04:04:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-5212zz-double-row-angular-contact-bearing-double-shielded-60mm-bore-x-110mm-od-x-36-5mm-w>: HTTP status code is not handled or not allowed 2026-01-25 04:04:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gripper-front-butcher-coat-winterior-top-pocket-spun-polyester-white-2xl>: HTTP status code is not handled or not allowed 2026-01-25 04:04:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gripper-front-butcher-coat-wknit-cuffs-pocket-less-spun-polyester-white-m>: HTTP status code is not handled or not allowed 2026-01-25 04:04:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-5201zz-double-row-angular-contact-bearing-double-shielded-12mm-bore-x-32mm-od-x-15-9mm-w>: HTTP status code is not handled or not allowed 2026-01-25 04:04:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-25 04:04:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90-degree-heavy-duty-flat-corner-bumper-guard-type-h-plus-yellow-black>: HTTP status code is not handled or not allowed 2026-01-25 04:04:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stock-pot-range-natural-gas-1-burner>: HTTP status code is not handled or not allowed 2026-01-25 04:04:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drug-test-container-90ml-sterile-tab-seal-id-label-celsius-thermometer-strip-300-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:04:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/concrete-dual-drinking-fountain-ada-accessible-tan-river-rock-2>: HTTP status code is not handled or not allowed 2026-01-25 04:04:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/panel-fan-18-diameter-three-phase-240-460v-with-grill>: HTTP status code is not handled or not allowed 2026-01-25 04:04:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crc-brakeleen-non-chlorinated-brake-parts-cleaners-14-oz-aerosol-can-less-45-percent-voc>: HTTP status code is not handled or not allowed 2026-01-25 04:04:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ballymore-6-step-perforated-cantilever-ladder-28-overhang>: HTTP status code is not handled or not allowed 2026-01-25 04:04:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ballymore-9-step-serrated-cantilever-ladder-42-overhang>: HTTP status code is not handled or not allowed 2026-01-25 04:04:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/repair-kit-vibco-rk-bvs-60>: HTTP status code is not handled or not allowed 2026-01-25 04:04:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ballymore-15-step-serrated-cantilever-ladder-14-overhang>: HTTP status code is not handled or not allowed 2026-01-25 04:04:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:04:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-assembly-for-111-112-113-114-pallet-trucks-ra-632-076-a>: HTTP status code is not handled or not allowed 2026-01-25 04:04:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/dry_cleaning_laundry_equipment/irons_garment_steamers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:04:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rcl-7200vb>: HTTP status code is not handled or not allowed 2026-01-25 04:04:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cam-for-pe-3000-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 04:04:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-sheet-3-thick-x-24-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 04:04:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits landed on page that is not a product page. 2026-01-25 04:04:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eggcrate-grille-panel-aluminum-black-pk5>: HTTP status code is not handled or not allowed 2026-01-25 04:04:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/guest_chairs/guest_reception_chairs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:04:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-men-s-boot-16-hazmax-ez-fit-green-yellow-steel-toe-mid-sole-pvc-size-15>: HTTP status code is not handled or not allowed 2026-01-25 04:04:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-men-s-10-viking-yellow-overshoe-w-4-way-cleated-outsole-pvc-size-extra-small>: HTTP status code is not handled or not allowed 2026-01-25 04:04:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-men-s-boot-16-sureflex-brown-cream-steel-toe-pvc-size-11>: HTTP status code is not handled or not allowed 2026-01-25 04:04:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/dry_cleaning_laundry_equipment/irons_garment_steamers landed on page that is not a product page. 2026-01-25 04:04:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-bar-1-14-thick-x-5-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 04:04:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-men-s-boot-16-hazmax-ez-fit-green-yellow-steel-toe-mid-sole-pvc-size-12>: HTTP status code is not handled or not allowed 2026-01-25 04:04:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/guest_chairs/guest_reception_chairs>: HTTP status code is not handled or not allowed 2026-01-25 04:05:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x36x39-steel-shelving-with-16-6-h-shelf-bins-yellow>: HTTP status code is not handled or not allowed 2026-01-25 04:05:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-kit-for-electric-pallet-jack-truck-gwk-wp2300-nosg-lw-fits-crown-model-wp2300>: HTTP status code is not handled or not allowed 2026-01-25 04:05:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/desk_accessories/desk_organizers_letter_trays already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:05:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-for-6hbw23-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 04:05:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12x48x74-chrome-wire-shelving-with-110-shelf-bins-blue-2>: HTTP status code is not handled or not allowed 2026-01-25 04:05:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/cart_truck_accessories/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:05:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-webtex-48-yellow-coat-w-hood-snaps-pvc-size-small>: HTTP status code is not handled or not allowed 2026-01-25 04:05:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/male-mannequin-hands-by-side-legs-straight-gloss-finish-white>: HTTP status code is not handled or not allowed 2026-01-25 04:05:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 61854 pages (at 91 pages/min), scraped 31158 items (at 49 items/min) 2026-01-25 04:05:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-stacking-bin-4-1-8x5-3-8x3-black>: HTTP status code is not handled or not allowed 2026-01-25 04:05:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:05:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/basic_supplies/desk_accessories/desk_organizers_letter_trays>: HTTP status code is not handled or not allowed 2026-01-25 04:05:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-stacking-bin-16-1-2x14-3-4x7-green>: HTTP status code is not handled or not allowed 2026-01-25 04:05:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/cart_truck_accessories/pallet_truck_parts_accessories/pallet_truck_wheel_kits landed on page that is not a product page. 2026-01-25 04:05:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sunglass-rack-32-pc-with-sign-holder-24high>: HTTP status code is not handled or not allowed 2026-01-25 04:05:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milwaukee-chuck-key-holder-48-66-4040>: HTTP status code is not handled or not allowed 2026-01-25 04:05:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 04:05:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v-gard-wire-mesh-visor-with-plastic-edge-10116557>: HTTP status code is not handled or not allowed 2026-01-25 04:05:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-hinged-lid-5-8-cu-yd-self-dumping-plastic-hopper>: HTTP status code is not handled or not allowed 2026-01-25 04:05:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v-gard-hard-hats-full-brim-fas-trac-suspension-green-475370>: HTTP status code is not handled or not allowed 2026-01-25 04:05:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-men-s-boot-16-flex-3-black-plain-toe-pvc-size-11>: HTTP status code is not handled or not allowed 2026-01-25 04:05:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ramp-cleat-non-slip-outdoor-rubber-mat-18-thick-3-x-20-black>: HTTP status code is not handled or not allowed 2026-01-25 04:05:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/shoe_covers_guards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:05:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-head-cap-screw-m10-x-1-5-x-35mm-steel-zinc-yellow-class-10-9-iso-4017-pkg-of-100-aap10035>: HTTP status code is not handled or not allowed 2026-01-25 04:05:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-deluxe-office-partition-panel-24-1-4w-x-73-1-2h-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:05:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-head-cap-screw-m10-x-1-5-x-40mm-steel-zinc-clear-class-8-8-din-931-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 04:05:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/flooring/rubber_flooring already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:05:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slide-box-for-12-slides-cork-lined-white>: HTTP status code is not handled or not allowed 2026-01-25 04:05:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/diamond-plate-rubber-floor-mat-3mm-thick-4-x-10-black>: HTTP status code is not handled or not allowed 2026-01-25 04:05:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xx-garolite-sheet-1-12-thick-x-12-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 04:05:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/louvered-panel-portable-bin-cart-black>: HTTP status code is not handled or not allowed 2026-01-25 04:05:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/flooring/rubber_flooring already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:05:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/shoe_covers_guards landed on page that is not a product page. 2026-01-25 04:05:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pediatric-medical-bag-purple>: HTTP status code is not handled or not allowed 2026-01-25 04:05:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plate-adapter-round-100mm-gvm-series-for-use-w-foam-tube-adapters-dimpled-pad>: HTTP status code is not handled or not allowed 2026-01-25 04:05:42 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/flooring/rubber_flooring landed on page that is not a product page. 2026-01-25 04:05:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-removable-push-button-and-rotary-switch-cover>: HTTP status code is not handled or not allowed 2026-01-25 04:05:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-sheet-2-12-thick-x-36-long-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 04:05:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transfer-pipet-wide-bore-large-bulb-124mm-sterile-polypropylenes-500-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:05:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/flooring/rubber_flooring>: HTTP status code is not handled or not allowed 2026-01-25 04:05:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/management-card-for-rfid-locks-supervisory-access-only>: HTTP status code is not handled or not allowed 2026-01-25 04:05:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/control-card-for-rfid-locks-supervisory-access-and-code-reset>: HTTP status code is not handled or not allowed 2026-01-25 04:05:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-safety-padlock-short-zenex-316-body-x-1-12-shackle-black>: HTTP status code is not handled or not allowed 2026-01-25 04:05:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-safety-padlock-short-zenex-316-body-x-1-12-shackle-green>: HTTP status code is not handled or not allowed 2026-01-25 04:05:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/omega-ii-full-brim-with-6-point-slide-lock-suspension-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:05:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reflective-strip-1-x-4-orange-order-by-the-sheet-16-strips>: HTTP status code is not handled or not allowed 2026-01-25 04:05:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-sloped-shelf-portable-bin-cart-putty>: HTTP status code is not handled or not allowed 2026-01-25 04:05:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-cable-lockout-3>: HTTP status code is not handled or not allowed 2026-01-25 04:05:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lightduty-absorbent-rolls-43-gallon-capacity-2812-x-150-1-rollcase>: HTTP status code is not handled or not allowed 2026-01-25 04:05:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oil-only-20-gallon-drum-spill-kit-22-gallon-capacity-1-spill-kitcase>: HTTP status code is not handled or not allowed 2026-01-25 04:05:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/balaclava-fleece-head-wear-ski-mask-hardhat-liner-black-one-size-bmsk-s1>: HTTP status code is not handled or not allowed 2026-01-25 04:05:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gun-safe-dehumidifier-md12-19-1-2-h-100-cubic-ft-coverage-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 04:05:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flip-training-table-w-casters-30-x-72-concrete-groovz-top-w-black-base-r-style-series>: HTTP status code is not handled or not allowed 2026-01-25 04:05:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:05:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slush-boots-rubber-17-height-yellow-size-17-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 04:06:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/turbine-tree-spray-gun>: HTTP status code is not handled or not allowed 2026-01-25 04:06:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gh-1440-3-with-acu-rite-203-dro>: HTTP status code is not handled or not allowed 2026-01-25 04:06:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/gauge_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:06:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-drawer-counter-high-60-w-cabinet-everest-blue-1>: HTTP status code is not handled or not allowed 2026-01-25 04:06:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-25 04:06:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-drawer-full-height-48-w-cabinet-everest-blue-2>: HTTP status code is not handled or not allowed 2026-01-25 04:06:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 61947 pages (at 93 pages/min), scraped 31206 items (at 48 items/min) 2026-01-25 04:06:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jtm-1050-mill-with-newall-dp700-dro-x-z-powerfeeds>: HTTP status code is not handled or not allowed 2026-01-25 04:06:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/tables/laboratory_tables already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:06:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/gauge_accessories>: HTTP status code is not handled or not allowed 2026-01-25 04:06:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crescent-shaped-wall-guard-w-rec-plastic-clip-retainer-system-5-h-x-12-l-monterey>: HTTP status code is not handled or not allowed 2026-01-25 04:06:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-rated-uninsulated-prime-coat-standard-latch-36x36-fr36x36pc-rtl>: HTTP status code is not handled or not allowed 2026-01-25 04:06:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=e-1236vs-with-newall-dp700-dro-with-collet-closer already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:06:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drywall-door-prime-coat-with-screwdriver-latch-16-ga-dw16x24pc-sdl>: HTTP status code is not handled or not allowed 2026-01-25 04:06:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tru-ray-construction-paper-9-x-12-violet50-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:06:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/back-bar-cooler-3-section-73-18w-24-12d-34-14h-tbb-24gal-72-s>: HTTP status code is not handled or not allowed 2026-01-25 04:06:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gh-1440-1-with-newall-dp700-dro-with-taper-attachment>: HTTP status code is not handled or not allowed 2026-01-25 04:06:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-tabs-repositionable-tabs-2-x-1-1-2-patterns-blue-green-pink-24-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:06:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/tables/laboratory_tables landed on page that is not a product page. 2026-01-25 04:06:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=e-1236vs-with-newall-dp700-dro-with-collet-closer>: HTTP status code is not handled or not allowed 2026-01-25 04:06:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=e-1440vs-with-acu-rite-300s-css-dro already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:06:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-tabs-repositionable-tabs-2-x-1-1-2-neon-green-orange-pink-24-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:06:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/color_coding_labels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:06:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-milk-cooler-8-crts-dual-sided-34w-33-38d-tmc-34-s-ds-ss>: HTTP status code is not handled or not allowed 2026-01-25 04:06:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-milk-cooler-16-crts-dual-sided-58w-33-38d-tmc-58-ds-ss>: HTTP status code is not handled or not allowed 2026-01-25 04:06:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triton-lbc-4ssh-48w-x46w-ss-frame-ss-locboard-tool-cart-w-tray>: HTTP status code is not handled or not allowed 2026-01-25 04:06:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=e-1440vs-with-acu-rite-300s-css-dro landed on page that is not a product page. 2026-01-25 04:06:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/etm-949-mill-with-x-y-axis-jet-powerfeeds>: HTTP status code is not handled or not allowed 2026-01-25 04:06:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/welding/fume_extractors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:06:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/labels_label_makers/labels/color_coding_labels>: HTTP status code is not handled or not allowed 2026-01-25 04:06:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-foam-latex-coated-gloves-black-navy-x-large-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 04:06:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/etm-949-mill-with-2-axis-newall-dp700-dro-and-x-y-z-axis-jet-powerfeeds>: HTTP status code is not handled or not allowed 2026-01-25 04:06:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tube-stand-reinforced-polypropylene-13mm-31-place-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:06:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/welding/fume_extractors landed on page that is not a product page. 2026-01-25 04:06:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-glamour-bubble-mailer-7-x-6-3-4>: HTTP status code is not handled or not allowed 2026-01-25 04:06:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/market-tray-13x18-black>: HTTP status code is not handled or not allowed 2026-01-25 04:06:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evs-949-mill-with-3-axis-acu-rite-303-knee-dro-and-x-y-z-axis-jet-powerfeeds>: HTTP status code is not handled or not allowed 2026-01-25 04:06:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=evs-949-mill-with-3-axis-newall-dp700-knee-dro already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:06:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_pipettes_pipettors_syringe_tips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:06:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25-2699-1-inch-trans-flo-gold-aodd-pump-aluminum-with-santoprene-diaphragms-and-fnpt-ports>: HTTP status code is not handled or not allowed 2026-01-25 04:06:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-step-stock-picking-ladder-perforated-tread-spl-7-14np>: HTTP status code is not handled or not allowed 2026-01-25 04:06:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-corrugated-boxes-15-x-10-x-4>: HTTP status code is not handled or not allowed 2026-01-25 04:06:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/service-cart-three-shelves-black>: HTTP status code is not handled or not allowed 2026-01-25 04:07:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rv-pette-dispenser-tip-for-repeat-volume-pipettors-1-25ml-100-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:07:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=evs-949-mill-with-3-axis-newall-dp700-knee-dro>: HTTP status code is not handled or not allowed 2026-01-25 04:07:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_pipettes_pipettors_syringe_tips landed on page that is not a product page. 2026-01-25 04:07:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1849-sg-1-6-00mm-x-15-9mm-double-cut-pointed-tree-bur>: HTTP status code is not handled or not allowed 2026-01-25 04:07:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-sided-cantilever-144-inch-upright-72-inch-brace-kit-adder>: HTTP status code is not handled or not allowed 2026-01-25 04:07:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-totally-enclosed-high-pressure-exhaust-fan-3-phase-2-hp-1>: HTTP status code is not handled or not allowed 2026-01-25 04:07:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 62037 pages (at 90 pages/min), scraped 31260 items (at 54 items/min) 2026-01-25 04:07:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carson-soldermag-solder-station-magnifier-45x-spot-lens>: HTTP status code is not handled or not allowed 2026-01-25 04:07:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/file_cabinets/lateral_file_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:07:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-sided-smoking-shelter-half-side-panels-13-ft-9-inch-w-x-3-ft-6-inch-d-x-7-ft-11-inch-h-clear-roof>: HTTP status code is not handled or not allowed 2026-01-25 04:07:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/work-height-platform-truck-with-lower-shelf-24-x-48-fixed-height>: HTTP status code is not handled or not allowed 2026-01-25 04:07:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-totally-enclosed-direct-drive-duct-fan-3-phase-2-hp>: HTTP status code is not handled or not allowed 2026-01-25 04:07:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-1-2-silver-base-anti-microbial-binder-side-open-burgundy>: HTTP status code is not handled or not allowed 2026-01-25 04:07:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/cabinets/file_cabinets/lateral_file_cabinets>: HTTP status code is not handled or not allowed 2026-01-25 04:07:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kraft-paper-sheets-40-lb-18-x-18>: HTTP status code is not handled or not allowed 2026-01-25 04:07:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/preprinted-dividers-220911-side-open-16-tab-set>: HTTP status code is not handled or not allowed 2026-01-25 04:07:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-step-aluminum-rolling-ladder-24w-grip-tread-14d-top-step>: HTTP status code is not handled or not allowed 2026-01-25 04:07:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-molded-ring-binder-top-open-medium-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:07:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unex-additional-level-for-gravity-flow-roller-rack-with-span-track-96w-x-72d>: HTTP status code is not handled or not allowed 2026-01-25 04:07:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yellow-jacket-protective-gauge-boot-2-12>: HTTP status code is not handled or not allowed 2026-01-25 04:07:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stackbin-16-long-drawer-cabinet-pedestal-workbench-butcher-block-square-edge-48w-x-36d-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:07:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/statshield-metal-in-bag-12in-x-16in-100-bagspack>: HTTP status code is not handled or not allowed 2026-01-25 04:07:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bottle-narrow-mouth-hdpe-attached-polypropylene-screw-cap-500ml-12-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:07:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magsnap-360-dual-wire-metal-wristband-medium>: HTTP status code is not handled or not allowed 2026-01-25 04:07:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/common-point-quad-ground-cord-for-workmat-banana-jacks-10-mm-stud-w-resistor-10ft-cord>: HTTP status code is not handled or not allowed 2026-01-25 04:07:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/column-protector-36-h>: HTTP status code is not handled or not allowed 2026-01-25 04:07:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/supco-commerial-condensate-evaporator-pan-7qt-120v-1500-w>: HTTP status code is not handled or not allowed 2026-01-25 04:07:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pipestake-fittings-75l-12dia-furniture-grade-pvc-white>: HTTP status code is not handled or not allowed 2026-01-25 04:07:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-key-cabinet-60-keys-sand>: HTTP status code is not handled or not allowed 2026-01-25 04:07:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fsk-insulation-tape-3-inch-x-50-yards>: HTTP status code is not handled or not allowed 2026-01-25 04:07:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/electrical_power_testing/insulation_testers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:07:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rotationally-molded-plastic-tray-19-1-2x15-1-4x2-1-2-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:07:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-inch-square-bistro-table-4-barstool-set-studio-teak-table-top-with-natural-barstools>: HTTP status code is not handled or not allowed 2026-01-25 04:07:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ul181b-fx-flexible-duct-closure-tape-2-inch-x-120-yards-sliver-printed>: HTTP status code is not handled or not allowed 2026-01-25 04:07:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buffet-bar-24x41-navy-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:07:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/welding/welding_blankets_curtains already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:07:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/881605-type-k-immersion-probe-type-k-stainless-steel-59l>: HTTP status code is not handled or not allowed 2026-01-25 04:07:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/testing_inspection_instruments/electrical_power_testing/insulation_testers landed on page that is not a product page. 2026-01-25 04:08:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ep-730-48-desk-mount-poly-tubing-dispenser-48>: HTTP status code is not handled or not allowed 2026-01-25 04:08:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-double-rivet-angle-beam-84-w-x-1-d-x-2-3-4-h-14-gauge>: HTTP status code is not handled or not allowed 2026-01-25 04:08:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kac-lc0-nbe-nw-hd-v-4xl-navy>: HTTP status code is not handled or not allowed 2026-01-25 04:08:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:08:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:08:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ice-caddies-beige-100-lbs-cap-1>: HTTP status code is not handled or not allowed 2026-01-25 04:08:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/welding/welding_blankets_curtains landed on page that is not a product page. 2026-01-25 04:08:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ptm-500-ae403-drum-platform-scale-with-lcd-indicator-1100-lb-x-0-2-lb>: HTTP status code is not handled or not allowed 2026-01-25 04:08:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 62128 pages (at 91 pages/min), scraped 31318 items (at 58 items/min) 2026-01-25 04:08:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-size-w-post-mix-system-bag-box-syrup-two-tone-brown-mahogany>: HTTP status code is not handled or not allowed 2026-01-25 04:08:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carbon-monoxide-gas-cylinder-1016-10-ppm-bal-air-17l-p>: HTTP status code is not handled or not allowed 2026-01-25 04:08:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves>: HTTP status code is not handled or not allowed 2026-01-25 04:08:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/chemical_storage_buildings already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:08:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/workstations/lan_stations already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:08:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves landed on page that is not a product page. 2026-01-25 04:08:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-o-smith-pool-filter-motor-115-230-volts-3450-rpm-1-2hp-1>: HTTP status code is not handled or not allowed 2026-01-25 04:08:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/siesta-victor-xl-resin-outdoor-arm-chair-olive-green>: HTTP status code is not handled or not allowed 2026-01-25 04:08:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/workstations/lan_stations already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:08:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50bs32hx1-58-pitch-1-finished-bore-32-teeth>: HTTP status code is not handled or not allowed 2026-01-25 04:08:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-25bs15hx12-14-pitch-12-finished-bore-15-teeth>: HTTP status code is not handled or not allowed 2026-01-25 04:08:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/chemical_storage_buildings landed on page that is not a product page. 2026-01-25 04:08:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/workstations/lan_stations landed on page that is not a product page. 2026-01-25 04:08:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-o-smith-single-phase-jet-pump-motor-115-230-volts-3450-rpm-1-1-2hp-2>: HTTP status code is not handled or not allowed 2026-01-25 04:08:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/replacement_shelves_decks/plastic_shelving_additional_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:08:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-5-5-8-permanent-split-capacitor-motor-208-230-volts-1625-rpm-3>: HTTP status code is not handled or not allowed 2026-01-25 04:08:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/workstations/lan_stations landed on page that is not a product page. 2026-01-25 04:08:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-5-5-8-permanent-split-capacitor-motor-208-230-volts-1075-rpm-2>: HTTP status code is not handled or not allowed 2026-01-25 04:08:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-care-20-x24-x1-electrostatic-air-filter-merv-8>: HTTP status code is not handled or not allowed 2026-01-25 04:08:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-care-16-x20-x4-wide-frame-electrostatic-air-filter-merv-8>: HTTP status code is not handled or not allowed 2026-01-25 04:08:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd-trident-bellman-cart-stainless-gray-carpet-black-bumper-8-inch-black-pneumatic-casters>: HTTP status code is not handled or not allowed 2026-01-25 04:08:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/replacement_shelves_decks/plastic_shelving_additional_shelves landed on page that is not a product page. 2026-01-25 04:08:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cotton-knit-thermal-blankets-white-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 04:08:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tube-urine-centrifuge-12ml-sediment-bulb-and-flared-top-ps-graduated-to-10-ml-1500-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:08:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-ton-air-over-hydraulic-flr-jack-w-air>: HTTP status code is not handled or not allowed 2026-01-25 04:08:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50bs35hx11516-58-pitch-1-1516-finished-bore-35-teeth>: HTTP status code is not handled or not allowed 2026-01-25 04:08:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-ton-shorty-hydraulic-bottle-jack>: HTTP status code is not handled or not allowed 2026-01-25 04:08:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/graduated-cylinder-pmp-tpx-printed-graduations-250ml-12-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:08:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microcentrifuge-tube-6-x-47mm-400ul-ldpe-attached-plug-cap-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:08:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/number-8-adjustable-height-economy-packing-workbench-butcher-block-top-square-edge-60x30-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:08:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beaker-disposable-polystyrene-20ml-500-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:08:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stinger-1-ton-capacity-folding-engine-crane>: HTTP status code is not handled or not allowed 2026-01-25 04:08:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-cart-for-signs-shadow-boards-up-to-72-h-x-36-w-anodized-aluminum-frame>: HTTP status code is not handled or not allowed 2026-01-25 04:08:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-tier-3-door-premium-steel-locker-12-w-x-18-d-x-72-h-black-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 04:08:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regulated-medical-waste-cart-with-bio-hazard-logo-and-casters-32-gallon-red>: HTTP status code is not handled or not allowed 2026-01-25 04:08:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wide-angle-magnetic-mount-forklift-mirror>: HTTP status code is not handled or not allowed 2026-01-25 04:08:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/curbside-truck-silver-ep-21w-x-48l-x-69h-4-wire-shelves-polyurethane-2-rigid-casters>: HTTP status code is not handled or not allowed 2026-01-25 04:08:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/to-go-rack-chrome-21w-x-24l-x-63h-4-wire-shelves-dividers-ledges>: HTTP status code is not handled or not allowed 2026-01-25 04:09:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50bs24hx138-58-pitch-1-38-finished-bore-24-teeth>: HTTP status code is not handled or not allowed 2026-01-25 04:09:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/hvac_electric_motors/unit_bearing_fan_motors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:09:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50bs25hx112-58-pitch-1-12-finished-bore-25-teeth>: HTTP status code is not handled or not allowed 2026-01-25 04:09:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keep-frozen-3-4-white-red-black>: HTTP status code is not handled or not allowed 2026-01-25 04:09:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-polypropylene-camlock-fitting-female-coupler-x-fpt-thread>: HTTP status code is not handled or not allowed 2026-01-25 04:09:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 62220 pages (at 92 pages/min), scraped 31370 items (at 52 items/min) 2026-01-25 04:09:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-12a27h-metric-a-plate-34-pitch-19mm-bore-27-teeth>: HTTP status code is not handled or not allowed 2026-01-25 04:09:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-two-tone-poly-filled-winter-insulated-bibs-w-multi-pockets-4xl-5xl>: HTTP status code is not handled or not allowed 2026-01-25 04:09:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/hvac_electric_motors/unit_bearing_fan_motors landed on page that is not a product page. 2026-01-25 04:09:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/to-go-rack-silver-ep-18w-x-30l-x-34h-3-wire-shelves-dividers-ledges>: HTTP status code is not handled or not allowed 2026-01-25 04:09:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-seal-bubble-mailers-6-12-1-2-x-19-gold-50-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:09:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/sneeze_guards_distancing_barriers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:09:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hot-2-fluorescent-red-black>: HTTP status code is not handled or not allowed 2026-01-25 04:09:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/incident-command-vest-orange-vest-w-lime-prismatic-tape-one-size-fits-all>: HTTP status code is not handled or not allowed 2026-01-25 04:09:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-3-night-glow-sherpa-line-heavy-weight-sierra-jacket-md>: HTTP status code is not handled or not allowed 2026-01-25 04:09:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-120a22x114-a-plate-1-12-pitch-1-14-bore-22-teeth>: HTTP status code is not handled or not allowed 2026-01-25 04:09:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-kit-floor-stands>: HTTP status code is not handled or not allowed 2026-01-25 04:09:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/desktop_dividers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:09:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/touch-free-condiment-dispenser-with-frame-silver>: HTTP status code is not handled or not allowed 2026-01-25 04:09:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1627-500bk-500-1-4-id-push-on-and-lock-hose>: HTTP status code is not handled or not allowed 2026-01-25 04:09:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/605w-1-4-mnpt-3-8-id-hose-end-fitting>: HTTP status code is not handled or not allowed 2026-01-25 04:09:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-drawer-mobile-cabinet-36-wx24-dx45-1-2-h-avalanche-blue-1>: HTTP status code is not handled or not allowed 2026-01-25 04:09:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mounting-bracket-for-mantis-1x2-white>: HTTP status code is not handled or not allowed 2026-01-25 04:09:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/partitions_dividers/desktop_dividers landed on page that is not a product page. 2026-01-25 04:09:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transformer-replacemnet-for-nemesis-cm-80>: HTTP status code is not handled or not allowed 2026-01-25 04:09:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rx-19-drawer-roller-cabinet-55-w-x-25-d-x-h-46-black-w-red-handles-trim>: HTTP status code is not handled or not allowed 2026-01-25 04:09:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-eu2002b-2-hp-1-phase-3450-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:09:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/credenzas already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:09:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-wdp34s2ahc-3-4-hp-3-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 04:09:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/candy-jar-assortment-5-lb-gr>: HTTP status code is not handled or not allowed 2026-01-25 04:09:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arizona-juice-variety-pack-20-oz-24-count-gr>: HTTP status code is not handled or not allowed 2026-01-25 04:09:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/narrow-collapsible-hamper-steel-beige-vinyl-bag>: HTTP status code is not handled or not allowed 2026-01-25 04:09:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=us-motor-eu2002b-2-hp-1-phase-3450-rpm landed on page that is not a product page. 2026-01-25 04:09:50 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/credenzas landed on page that is not a product page. 2026-01-25 04:09:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-chrome-linen-cart-with-solid-bottom-shelf-36-l-x-24-w-x-78-h>: HTTP status code is not handled or not allowed 2026-01-25 04:09:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-t32cm1jh-1-1-2-hp-1-phase-3450-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:09:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-t14b3p9-1-4-hp-1-phase-1140-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:09:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-t14c2pcr-1-4-hp-1-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 04:09:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-t12c3jcr-1-2-hp-1-phase-1140-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:09:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oreo-chocolate-sandwich-cookies-2-pack-120-count-gr>: HTTP status code is not handled or not allowed 2026-01-25 04:09:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-6178-1-hp-1-phase-1725-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:09:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-step-extra-rolling-safety-ladder-expanded-metal-tread>: HTTP status code is not handled or not allowed 2026-01-25 04:09:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ocean-spray-craisins-fruit-clusters-cranberry-granola-2-oz-10-count-gr>: HTTP status code is not handled or not allowed 2026-01-25 04:09:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=us-motor-t32cm1jh-1-1-2-hp-1-phase-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-25 04:10:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=us-motor-t14b3p9-1-4-hp-1-phase-1140-rpm landed on page that is not a product page. 2026-01-25 04:10:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-t1c3jcr-1-hp-1-phase-1140-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:10:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-step-16-w-stainless-steel-tilt-roll-ladder-perforated-tread>: HTTP status code is not handled or not allowed 2026-01-25 04:10:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=us-motor-t12c3jcr-1-2-hp-1-phase-1140-rpm landed on page that is not a product page. 2026-01-25 04:10:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=us-motor-6178-1-hp-1-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 04:10:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dw6hd-1bz-smart-wi-fi-universal-dimmer>: HTTP status code is not handled or not allowed 2026-01-25 04:10:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hog-heaven-anti-fatigue-mat-7-8-thick-58x95-black>: HTTP status code is not handled or not allowed 2026-01-25 04:10:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anti-bacterial-terry-cloth-rags-white-1-lbs-99803>: HTTP status code is not handled or not allowed 2026-01-25 04:10:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8000-series-direct-feed-jewelry-steam-cleaner>: HTTP status code is not handled or not allowed 2026-01-25 04:10:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 62308 pages (at 88 pages/min), scraped 31422 items (at 52 items/min) 2026-01-25 04:10:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-t1cm2jh-1-hp-1-phase-1725-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:10:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/fans_blowers/exhaust_fans/duct_fans already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:10:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-in-7-outlet-surege-protector-6-ft-cord>: HTTP status code is not handled or not allowed 2026-01-25 04:10:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=us-motor-t1c3jcr-1-hp-1-phase-1140-rpm landed on page that is not a product page. 2026-01-25 04:10:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-lab-chair-backless-black>: HTTP status code is not handled or not allowed 2026-01-25 04:10:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-wdp12s2ahc-1-2-hp-3-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 04:10:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smoker-bucket-liners-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 04:10:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-d13b2n4z9-1-3-hp-1-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 04:10:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=us-motor-t1cm2jh-1-hp-1-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 04:10:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/fans_blowers/exhaust_fans/duct_fans landed on page that is not a product page. 2026-01-25 04:10:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ilb-cp10-he-emergency-driver-50-60hz-10-watts-ca-title-20-silver>: HTTP status code is not handled or not allowed 2026-01-25 04:10:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-u3p2dc-3-hp-3-phase-1765-rpm>: HTTP status code is not handled or not allowed 2026-01-25 04:10:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uberlight-26-flex-led-task-light-with-clamp-black>: HTTP status code is not handled or not allowed 2026-01-25 04:10:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beaker-globe-glass-low-form-griffin-style-dual-graduations-astm-e960-250ml-12-box>: HTTP status code is not handled or not allowed 2026-01-25 04:10:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flask-volumetric-globe-glass-class-b-to-contain-tc-astme288-50ml-6-box>: HTTP status code is not handled or not allowed 2026-01-25 04:10:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/khn-helical-bevel-reducer-double-shaft-kit-shaft-dia-1-0-l-9-137-1-97-usable-hardw-incl>: HTTP status code is not handled or not allowed 2026-01-25 04:10:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-d10p2dc-10-hp-3-phase-1765-rpm>: HTTP status code is not handled or not allowed 2026-01-25 04:10:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-u20e2h-20-hp-3-phase-1770-rpm>: HTTP status code is not handled or not allowed 2026-01-25 04:10:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/automatic-foam-hand-sanitizer-soap-dispenser-800-ml-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 04:10:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:10:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-x10e1b-10-hp-3-phase-3500-rpm>: HTTP status code is not handled or not allowed 2026-01-25 04:10:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-master-link-2-inch-heavy-duty-link-safety-orange-10-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:10:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-xs14ca3p-1-4-hp-1-phase-1140-rpm>: HTTP status code is not handled or not allowed 2026-01-25 04:10:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-galvanized-steel-blower-wheel-4-17-32-diameter-5-16-bore-2>: HTTP status code is not handled or not allowed 2026-01-25 04:10:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-fd1cm2pcr-1-hp-1-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 04:10:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:10:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-d2p1d-2-hp-3-phase-3515-rpm>: HTTP status code is not handled or not allowed 2026-01-25 04:10:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lau-4-blade-condenser-propeller-10-diameter-ccw-rotation-4>: HTTP status code is not handled or not allowed 2026-01-25 04:10:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-bags-2x5-2-mil-1000-ctn>: HTTP status code is not handled or not allowed 2026-01-25 04:10:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-rod-6-90-degree-bend-1>: HTTP status code is not handled or not allowed 2026-01-25 04:10:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-loop-8>: HTTP status code is not handled or not allowed 2026-01-25 04:10:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves landed on page that is not a product page. 2026-01-25 04:10:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/heavy_duty_boltless_shelves>: HTTP status code is not handled or not allowed 2026-01-25 04:10:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/food-storage-container-18x26x3-1-2-5-gallon-capacity-natural-white>: HTTP status code is not handled or not allowed 2026-01-25 04:10:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-x-30-maple-top-with-open-leg>: HTTP status code is not handled or not allowed 2026-01-25 04:10:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-horizontal-6-upright-sections>: HTTP status code is not handled or not allowed 2026-01-25 04:10:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/cubicle_partition_panels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:10:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/combination-rack-3-upright-3-horizontal>: HTTP status code is not handled or not allowed 2026-01-25 04:10:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-locker-double-tier-1-wide-12-x-15-x-36-beige>: HTTP status code is not handled or not allowed 2026-01-25 04:11:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/locker-flat-top-15-x-12-beige>: HTTP status code is not handled or not allowed 2026-01-25 04:11:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-locker-three-tier-1-wide-12-x-12-x-20-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:11:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/bins_totes_containers/bulk_containers/plastic_bulk_containers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:11:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/partitions_dividers/cubicle_partition_panels>: HTTP status code is not handled or not allowed 2026-01-25 04:11:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dock-door-seal-58-heavy-duty-40-oz-with-wear-pleats-8w-x-9h-10-projection-black>: HTTP status code is not handled or not allowed 2026-01-25 04:11:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camshelving-elements-mobile-starter-unit-21-w-x-42-l-x-70-h-4-shelf-4-premium-casters-brushed-graphite>: HTTP status code is not handled or not allowed 2026-01-25 04:11:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 62403 pages (at 95 pages/min), scraped 31477 items (at 55 items/min) 2026-01-25 04:11:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camshelving-elements-dunnage-support-14-w-brushed-graphite>: HTTP status code is not handled or not allowed 2026-01-25 04:11:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-female-metal-release-collet-6mm-tube-x-1-8-bspp>: HTTP status code is not handled or not allowed 2026-01-25 04:11:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fan-forced-horizontal-discharge-unit-heater-5kw-208v-1ph>: HTTP status code is not handled or not allowed 2026-01-25 04:11:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camshelving-elements-mobile-starter-unit-24-w-x-48-l-x-70-h-4-shelf-4-premium-casters-brushed-graphite>: HTTP status code is not handled or not allowed 2026-01-25 04:11:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reducing-manifold-10mm-tube-x-8mm-tube-x-3-8-swift-fit>: HTTP status code is not handled or not allowed 2026-01-25 04:11:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/bins_totes_containers/bulk_containers/plastic_bulk_containers landed on page that is not a product page. 2026-01-25 04:11:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camshelving-elements-mobile-starter-unit-18-w-x-48-l-x-70-h-4-shelf-4-premium-casters-brushed-graphite->: HTTP status code is not handled or not allowed 2026-01-25 04:11:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-workbench-2-3-4-backsplash-galv-steel-undershelf-72-w-x-30-d>: HTTP status code is not handled or not allowed 2026-01-25 04:11:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manifold-6mm-tube-x-1-8-bspp>: HTTP status code is not handled or not allowed 2026-01-25 04:11:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulkhead-union-metal-release-collet-5mm>: HTTP status code is not handled or not allowed 2026-01-25 04:11:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cast-iron-jaw-coupling-hub-style-l110-3-4-inch-bore-diameter-3-16-x-3-32-keyway>: HTTP status code is not handled or not allowed 2026-01-25 04:11:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forced-air-circulation-incubator-programmable-749-liters-115-volt>: HTTP status code is not handled or not allowed 2026-01-25 04:11:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cast-aluminum-level-10-inch-l>: HTTP status code is not handled or not allowed 2026-01-25 04:11:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/micrometer-setting-standard-2-inch-dia->: HTTP status code is not handled or not allowed 2026-01-25 04:11:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ninja-series-planetary-gear-winch-w-synthetic-rope-4500-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 04:11:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ecp-industrial-dispenser-60-inch-l>: HTTP status code is not handled or not allowed 2026-01-25 04:11:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gutter-point-bird-spike-25l-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-25 04:11:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/synthetic-winch-line-extension-w-integrated-shackle-916-dia-x-75l>: HTTP status code is not handled or not allowed 2026-01-25 04:11:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/bulk_container_trucks_tilt_trucks/basket_trucks/vinyl_basket_trucks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:11:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbw-antimicrobial-vinyl-basket-truck-w-bumper-18-bushel-capacity-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:11:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbw-vinyl-bumper-truck-all-swivel-20-bushel-capacity-forest-green>: HTTP status code is not handled or not allowed 2026-01-25 04:11:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_seating/adirondack_lounger_chairs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:11:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/muskoka-adirondack-3-piece-bistro-dining-set-30h-table-weathered-acorn>: HTTP status code is not handled or not allowed 2026-01-25 04:11:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:11:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:11:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/key-west-silver-lines-slab-door-wood-28w-x-80h-ice-maple>: HTTP status code is not handled or not allowed 2026-01-25 04:11:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:11:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/key-west-night-lines-slab-door-wood-24w-x-80h-whiskey-oak>: HTTP status code is not handled or not allowed 2026-01-25 04:11:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_plates_braces already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:11:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/liquid-absorbing-sandless-sandbag-22l-x-16w-x-1h-pack-of-40>: HTTP status code is not handled or not allowed 2026-01-25 04:11:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hog-heavenanti-fatigue-mat-78-thick-30l-x-2w-black>: HTTP status code is not handled or not allowed 2026-01-25 04:11:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/carts-trucks/bulk_container_trucks_tilt_trucks/basket_trucks/vinyl_basket_trucks>: HTTP status code is not handled or not allowed 2026-01-25 04:11:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hog-heavenanti-fatigue-mat-58-thick-20l-x-3w-black-striped-border>: HTTP status code is not handled or not allowed 2026-01-25 04:11:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hog-heavenanti-fatigue-mat-78-thick-30l-x-3w-black-striped-border>: HTTP status code is not handled or not allowed 2026-01-25 04:11:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_seating/adirondack_lounger_chairs>: HTTP status code is not handled or not allowed 2026-01-25 04:11:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors>: HTTP status code is not handled or not allowed 2026-01-25 04:11:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:11:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors>: HTTP status code is not handled or not allowed 2026-01-25 04:11:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_plates_braces>: HTTP status code is not handled or not allowed 2026-01-25 04:11:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:11:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extreme-hiker-boots-size-12-black>: HTTP status code is not handled or not allowed 2026-01-25 04:11:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:11:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/foodservice_disposables/disposable_plates_bowls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:11:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/soak-proof-foam-plates-tableware-8-7-8-dia-white-pack-of-600>: HTTP status code is not handled or not allowed 2026-01-25 04:11:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-impact-plastic-dinnerware-plate-w-3-compartment-10-dia-black-pack-of-500>: HTTP status code is not handled or not allowed 2026-01-25 04:11:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors>: HTTP status code is not handled or not allowed 2026-01-25 04:11:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/foodservice_disposables/disposable_baking_pans already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:11:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/continuous-cling-food-film-roll-1000-l-x-12-w-clear>: HTTP status code is not handled or not allowed 2026-01-25 04:11:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 04:12:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/doors_windows/doors>: HTTP status code is not handled or not allowed 2026-01-25 04:12:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/foodservice_disposables/disposable_plates_bowls>: HTTP status code is not handled or not allowed 2026-01-25 04:12:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-cold-drink-cups-polypropylene-9-oz-translucent-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 04:12:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foam-container-w-3-compartment-perforated-lid-9-3-8-l-x-9-w-x-3-h-white-pack-of-200>: HTTP status code is not handled or not allowed 2026-01-25 04:12:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lift-n-lock-hot-drink-cup-lids-w-straw-slots-fits-10-oz-to-14-oz-cups-pack-of-1000>: HTTP status code is not handled or not allowed 2026-01-25 04:12:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hot-coffee-cups-8-oz-pack-of-1000>: HTTP status code is not handled or not allowed 2026-01-25 04:12:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spiral-wound-paper-hot-cup-lids-for-16-oz-cups-white-pack-of-500>: HTTP status code is not handled or not allowed 2026-01-25 04:12:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-port-charger>: HTTP status code is not handled or not allowed 2026-01-25 04:12:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/foodservice_disposables/disposable_baking_pans>: HTTP status code is not handled or not allowed 2026-01-25 04:12:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oscillating-41-piece-sanding-kit>: HTTP status code is not handled or not allowed 2026-01-25 04:12:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-torque-impact-wrench-bare-tool-24v>: HTTP status code is not handled or not allowed 2026-01-25 04:12:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/colormaxx-paint-primer-12-oz-satin-italian-olive-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 04:12:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/colormaxx-paint-primer-12-oz-gloss-regal-blue-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 04:12:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 62493 pages (at 90 pages/min), scraped 31516 items (at 39 items/min) 2026-01-25 04:12:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stereoblue-binocular-zoom-microscope-w-ergonomic-stand-led-illumination-7x-to-45x>: HTTP status code is not handled or not allowed 2026-01-25 04:12:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/infinity-eis-60-mm-plan-pli-20x040-objective-working-distance-12-mm>: HTTP status code is not handled or not allowed 2026-01-25 04:12:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-compartment-drawer-organizer-23w-x-16-516d-x-1-38h-black>: HTTP status code is not handled or not allowed 2026-01-25 04:12:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ductless-mini-split-air-conditioner-w-heat-pump-wifi-enabled-36000-btu-3-ton-12ft-lineset>: HTTP status code is not handled or not allowed 2026-01-25 04:12:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-visibility-softshell-water-resistant-jacket-type-r-class-3-lime-2xl>: HTTP status code is not handled or not allowed 2026-01-25 04:12:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/curad-performance-ironman-adjustable-wraparound-ankle-support-open-heel-universal-4-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:12:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/canes-for-the-visually-impaired-aluminum-white>: HTTP status code is not handled or not allowed 2026-01-25 04:12:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/wheelchairs_physical_aids/canes_crutches already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:12:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pediatric-wheelchair-w-elevating-leg-rests-14w-seat>: HTTP status code is not handled or not allowed 2026-01-25 04:12:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/basic-steel-rollator-w-6-casters-black>: HTTP status code is not handled or not allowed 2026-01-25 04:12:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/wheelchairs_physical_aids/walkers_rollators already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:12:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/basic-aluminum-transport-wheelchair-w-detachable-footrests-18w-seat-red-frame>: HTTP status code is not handled or not allowed 2026-01-25 04:12:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/comprecares-reusable-compression-garment-kit-large-tall-black>: HTTP status code is not handled or not allowed 2026-01-25 04:12:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/personal_care_hygiene/bath_body/oral_care already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:12:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sterile-alcohol-swabsticks-25-per-box10-boxes-per-case>: HTTP status code is not handled or not allowed 2026-01-25 04:12:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pathway-straight-wheelchair-ramp-kit-aluminum-4-top-platform-20l-x-36w>: HTTP status code is not handled or not allowed 2026-01-25 04:12:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/wheelchairs_physical_aids/canes_crutches landed on page that is not a product page. 2026-01-25 04:12:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-creeper-w-adj-headrest-and-20-wide-bed-and-parts-tray-500-lb-cap-black>: HTTP status code is not handled or not allowed 2026-01-25 04:12:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vinexflame-retardant-coat-50inchl-navy-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:12:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/wheelchairs_physical_aids/walkers_rollators>: HTTP status code is not handled or not allowed 2026-01-25 04:12:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flexible-stainless-steel-faucet-connector-20l-12-x-12-fip-x-fip-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-25 04:12:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/personal_care_hygiene/bath_body/oral_care>: HTTP status code is not handled or not allowed 2026-01-25 04:12:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-pantry-deck-mount-pre-rinse-faucet>: HTTP status code is not handled or not allowed 2026-01-25 04:12:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proelite-stability-ball-21-58-dia-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:12:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kettlebell-prime-70-lb>: HTTP status code is not handled or not allowed 2026-01-25 04:12:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/seating/reception_sofas_sectionals already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:12:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/business-furniture-coventry-sofa-85-1-16w-x-33-7-16d-x-35-3-4h-beige>: HTTP status code is not handled or not allowed 2026-01-25 04:12:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/contemporary-classic-chrome-refrigerator-11-cuft-capacity-midnight-black>: HTTP status code is not handled or not allowed 2026-01-25 04:12:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-in-l-ball-transfer-table-21-in-w-between-frame-3-in-ball-center>: HTTP status code is not handled or not allowed 2026-01-25 04:12:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/counter-height-cabinet-w-drawer--laminate-top-30w-x-18d-x-36h-dove-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:12:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cabinet-w-fixed-shelves-assembled-48w-x-18d-x-72h-navy-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:12:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/organizer-cabinet-assembled-24w-x-18d-x-72h-wine-red>: HTTP status code is not handled or not allowed 2026-01-25 04:12:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wardrobe-cabinet-assembled-42w-x-24d-x-72h-wine-red>: HTTP status code is not handled or not allowed 2026-01-25 04:12:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bookcase-assembled-36w-x-18d-x-75h-hunter-green>: HTTP status code is not handled or not allowed 2026-01-25 04:12:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bookcase-assembled-36w-x-18d-x-33h-red>: HTTP status code is not handled or not allowed 2026-01-25 04:12:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/seating/reception_sofas_sectionals landed on page that is not a product page. 2026-01-25 04:12:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bookcase-assembled-36w-x-18d-x-33h-tropic-sand>: HTTP status code is not handled or not allowed 2026-01-25 04:12:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bookcase-assembled-36w-x-18d-x-60h-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:12:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/full_height_cabinet_w_drawers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:12:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/screw-cap-microtube-w-o-ring-sterile-2-ml-capacity-pack-of-1000>: HTTP status code is not handled or not allowed 2026-01-25 04:12:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/testculture-tube-polypropylene-non-sterile-12-x-60-mm-3-ml-cap-pack-of-5000>: HTTP status code is not handled or not allowed 2026-01-25 04:12:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fr-waterproof-flame-resistant-insulated-bib-pant-2xl3xl-lime>: HTTP status code is not handled or not allowed 2026-01-25 04:12:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onyx-non-ansi-rip-stop-rain-coat-w-teflon-coating-lxl-navy>: HTTP status code is not handled or not allowed 2026-01-25 04:12:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onyx-standard-safety-vest-w-black-contrasting-trim-class-2-3xl-lime>: HTTP status code is not handled or not allowed 2026-01-25 04:12:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forkliftable-truck-w-2-shelves-3600-lb-cap-53-12l-x-24w-x-70-12h-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:12:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-hi-2-reflective-strips-4-pockets-lime-size-2xl>: HTTP status code is not handled or not allowed 2026-01-25 04:12:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/cabinets/full_height_cabinet_w_drawers>: HTTP status code is not handled or not allowed 2026-01-25 04:12:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/hats_headwear already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:12:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8287-hi-vis-winter-jacket--vest-w-detachable-sleeves-4xl-lime>: HTTP status code is not handled or not allowed 2026-01-25 04:13:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/squids-3100-standard-lanyard-w-carabiner--cinch-loop-lime-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 04:13:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-7024-cut-resistant-gloves-polyurethane-coated-ansi-a2-l-gray-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 04:13:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8379-hi-vis-3-in-1-winter-bomber-jacket-medium-lime>: HTTP status code is not handled or not allowed 2026-01-25 04:13:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/skullerz174-odin-anti-fog-safety-glasses-clear-lens-kryptek-highlander-frame>: HTTP status code is not handled or not allowed 2026-01-25 04:13:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-step-narrow-hybrid-rolling-safety-ladder-450-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 04:13:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-bulk-gown-dispenser-14-58w-x-4d-x-15-516h-transparent>: HTTP status code is not handled or not allowed 2026-01-25 04:13:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/hats_headwear>: HTTP status code is not handled or not allowed 2026-01-25 04:13:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 62585 pages (at 92 pages/min), scraped 31559 items (at 43 items/min) 2026-01-25 04:13:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vertical-air-grinder-58-in-11-thread-5-or-6-type-6-11-wheel-6000-rpm-side-exhaust-3-hp>: HTTP status code is not handled or not allowed 2026-01-25 04:13:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/right-angle-air-die-grinder-14-collet-14000-rpm-rear-exhaust-033-hp>: HTTP status code is not handled or not allowed 2026-01-25 04:13:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-drill-reversible-roll-throttle-300-rpm-3hp-no-3-morse-taper>: HTTP status code is not handled or not allowed 2026-01-25 04:13:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-tier-6-door-digital-locker-12in-w-x-12in-d-x-78in-h-gray-assembled>: HTTP status code is not handled or not allowed 2026-01-25 04:13:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-tier-1-door-digital-locker-12in-w-x-12in-d-x-78in-h-blue-assembled>: HTTP status code is not handled or not allowed 2026-01-25 04:13:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-tier-1-door-digital-locker-12in-w-x-12in-d-x-78in-h-tan-assembled>: HTTP status code is not handled or not allowed 2026-01-25 04:13:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5000-arbor-series-kit-w-4-ft-barrel-225-cfm>: HTTP status code is not handled or not allowed 2026-01-25 04:13:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5000-arbor-series-w-4-ft-barrel-105-cfm>: HTTP status code is not handled or not allowed 2026-01-25 04:13:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5000-utility-series-w-4-ft-barrel-105-cfm>: HTTP status code is not handled or not allowed 2026-01-25 04:13:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-low-chest-freezer-48-cu-ft-capacity-white>: HTTP status code is not handled or not allowed 2026-01-25 04:13:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/built-in-all-freezer-32-cu-ft-capacity-white>: HTTP status code is not handled or not allowed 2026-01-25 04:13:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silicone-rubber-set-for-yc-450hl>: HTTP status code is not handled or not allowed 2026-01-25 04:13:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-kit-for-kf-300f>: HTTP status code is not handled or not allowed 2026-01-25 04:13:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-kit-for-kf-600f>: HTTP status code is not handled or not allowed 2026-01-25 04:13:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sds-max-combination-rotary-hammer-1-34-chuck-size-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:13:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/impact-tough-1l-20-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-25 04:13:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a21-led-light-bulb-medium-base-20w-2550-lumens-4000k-cool-white>: HTTP status code is not handled or not allowed 2026-01-25 04:13:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cylindrical-w-endcut-bur-single-cut-1-12l-x-18-shank-dia>: HTTP status code is not handled or not allowed 2026-01-25 04:13:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/radius-cone-shaped-bur-double-cut-7-18l-x-14-shank-dia>: HTTP status code is not handled or not allowed 2026-01-25 04:13:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cylindrical-w-endcut-bur-double-cut-2l-x-14-shank-dia-14-cut-dia>: HTTP status code is not handled or not allowed 2026-01-25 04:13:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bur-box-diamond-cut-set-of-12>: HTTP status code is not handled or not allowed 2026-01-25 04:13:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wallet-pouch-bur-set-aluma-cut-powerz-finish-14-shank-dia-set-of-4>: HTTP status code is not handled or not allowed 2026-01-25 04:13:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/led_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:13:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plv-led-plug-play-light-bulb-g24q-base-9w-1000-lumens-4000k-cool-white>: HTTP status code is not handled or not allowed 2026-01-25 04:13:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/block-for-15-x-5ml-centrifuge-tubes>: HTTP status code is not handled or not allowed 2026-01-25 04:13:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nextpette-variable-volume-pipette-sterile-2-200l>: HTTP status code is not handled or not allowed 2026-01-25 04:13:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/biomixer-3d-nutating-shaker-w-dimpled-mat-115v-5060-hz-24-rpm>: HTTP status code is not handled or not allowed 2026-01-25 04:13:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compact-balance-500g-capacity-01g-readability-230v>: HTTP status code is not handled or not allowed 2026-01-25 04:13:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/medical_laboratory_refrigeration/medical_freezers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:13:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compact-balance-5000g-capacity-01g-readability-230v>: HTTP status code is not handled or not allowed 2026-01-25 04:13:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-head-led-work-light-w-tripod-20000-lumens-black>: HTTP status code is not handled or not allowed 2026-01-25 04:13:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbi-sala174-nano-lok-personal-self-retracting-lifeline-w-aluminum-web-rebar-hook-72l>: HTTP status code is not handled or not allowed 2026-01-25 04:13:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tread-plate-deck-platform-truck-aluminum-6-polyurethane-caster-75l-x-36w-2600-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 04:13:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/led_bulbs>: HTTP status code is not handled or not allowed 2026-01-25 04:13:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbi-sala-nano-lok-personal-twin-leg-self-retracting-lifeline-w-aluminum-carabiner-72l>: HTTP status code is not handled or not allowed 2026-01-25 04:13:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-nitrile-exam-gloves-powder-free-xl-blue-pack-of-1000>: HTTP status code is not handled or not allowed 2026-01-25 04:13:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-starter-36l-x-24w-x-48h-antique-white>: HTTP status code is not handled or not allowed 2026-01-25 04:13:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-sided-starter-36l-x-20w-x-54h-black>: HTTP status code is not handled or not allowed 2026-01-25 04:13:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-add-on-48l-x-40w-x-84h-black>: HTTP status code is not handled or not allowed 2026-01-25 04:13:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-add-on-48l-x-28w-x-84h-antique-white>: HTTP status code is not handled or not allowed 2026-01-25 04:13:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/medical_laboratory_cold_storage already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:13:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trex153-6326-spikeless-shoe-traction-device-slip--oil-resistant-small-black>: HTTP status code is not handled or not allowed 2026-01-25 04:13:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-starter-48l-x-40w-x-60h-antique-white>: HTTP status code is not handled or not allowed 2026-01-25 04:13:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/optimus-power-scissor-lift-table-foot-operated-5000-cap-3ph-230v-56l-x-32w>: HTTP status code is not handled or not allowed 2026-01-25 04:13:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/optimus-power-scissor-lift-table-hand-operated-5000-cap-1ph-230v-56l-x-32w>: HTTP status code is not handled or not allowed 2026-01-25 04:13:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/skullerz-vali-afas-frameless-safety-glasses-clear-lens-matte-black-frame>: HTTP status code is not handled or not allowed 2026-01-25 04:13:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mmb-yb12a-a-12v-12ah-165-cca-battery>: HTTP status code is not handled or not allowed 2026-01-25 04:13:56 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/medical_laboratory_cold_storage landed on page that is not a product page. 2026-01-25 04:13:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/identification-tag-4-inch-x-2-inch-gray-37083>: HTTP status code is not handled or not allowed 2026-01-25 04:13:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spout-cap-yellow-medium-34410>: HTTP status code is not handled or not allowed 2026-01-25 04:14:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h1-2-bar-aluminum-ladder-roof-rack-for-ram-promaster-city-2014-on-white>: HTTP status code is not handled or not allowed 2026-01-25 04:14:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-containment-wall-m2-straight-section-86l-x-24w-x-24h>: HTTP status code is not handled or not allowed 2026-01-25 04:14:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h1-3-bar-steel-ladder-roof-rack-for-ford-transit-connect-2014-on-black>: HTTP status code is not handled or not allowed 2026-01-25 04:14:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wg-composite-bulkhead-ford-transit-mid-roof-high-roof-base>: HTTP status code is not handled or not allowed 2026-01-25 04:14:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/military-grade-cargo-lifting-net-4500-lb-capacity-10-opening-120l-x-120w-black>: HTTP status code is not handled or not allowed 2026-01-25 04:14:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/horizontal-heavy-duty-sign-frame-black-8-x-11>: HTTP status code is not handled or not allowed 2026-01-25 04:14:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polytec153-recycling-container-w-riverstone-panels-42-gal-cap-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:14:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caulk-guns-revolving-barrel-18-to-1-thrust-29-oz-cartridge>: HTTP status code is not handled or not allowed 2026-01-25 04:14:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 62681 pages (at 96 pages/min), scraped 31600 items (at 41 items/min) 2026-01-25 04:14:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v-neck-scrub-shirt-nonwoven-xl-dark-green-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-25 04:14:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-ceramic-fireplace-heater-1500w-black>: HTTP status code is not handled or not allowed 2026-01-25 04:14:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pedi-padz-ii-defibrillator-pads-white>: HTTP status code is not handled or not allowed 2026-01-25 04:14:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-safety-crafted-waterproof-work-boots-steel-toe-size-14m-6-inch-h-dark-brown>: HTTP status code is not handled or not allowed 2026-01-25 04:14:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_tools/cleaning_cloths_towels/shop_towels_cloth_rags/microfiber_cloths_towels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:14:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oven-and-grill-cleaner-24-oz-aerosol-6-carton>: HTTP status code is not handled or not allowed 2026-01-25 04:14:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-16bs20hx50-metric-1-pitch-50mm-finished-bore-20-teeth>: HTTP status code is not handled or not allowed 2026-01-25 04:14:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:14:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trailgrip-work-athletic-hiker-boots-alloy-toe-size-11m-4-inch-h-coyote-black>: HTTP status code is not handled or not allowed 2026-01-25 04:14:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/healthcare-surface-disinfectant-fragrance-free-128-oz-bottle-4-carton>: HTTP status code is not handled or not allowed 2026-01-25 04:14:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-safety-crafted-waterproof-logger-work-boots-steel-toe-size-12m-8-inch-h-dark-brown>: HTTP status code is not handled or not allowed 2026-01-25 04:14:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e1-foam-handwash-fragrance-free-1250-ml-4-carton>: HTTP status code is not handled or not allowed 2026-01-25 04:14:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-safety-crafted-waterproof-wellington-work-boots-steel-toe-size-9w-10-inch-h-brown>: HTTP status code is not handled or not allowed 2026-01-25 04:14:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/glass_cleaners already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:14:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/truflex-fly-skylar-work-zip-bootie-composite-toe-size-12w-4-inch-h-black>: HTTP status code is not handled or not allowed 2026-01-25 04:14:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:14:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/advanced-gel-hand-sanitizer-clean-scent-for-es1-450-ml-refill-clean-scent-6-pk>: HTTP status code is not handled or not allowed 2026-01-25 04:14:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_tools/cleaning_cloths_towels/shop_towels_cloth_rags/microfiber_cloths_towels landed on page that is not a product page. 2026-01-25 04:14:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/truflex-fly-skylar-work-zip-bootie-composite-toe-size-7w-4-inch-h-black>: HTTP status code is not handled or not allowed 2026-01-25 04:14:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-25 04:14:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-safety-crafted-waterproof-work-boots-steel-toe-size-14m-6-inch-h-brown>: HTTP status code is not handled or not allowed 2026-01-25 04:14:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:14:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-safety-crafted-waterproof-work-boots-steel-toe-size-7w-6-inch-h-dark-brown>: HTTP status code is not handled or not allowed 2026-01-25 04:14:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/truflex-fly-skylar-work-zip-bootie-composite-toe-size-12m-4-inch-h-black>: HTTP status code is not handled or not allowed 2026-01-25 04:14:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80bs33x11516-1-pitch-1-1516-finished-bore-33-teeth>: HTTP status code is not handled or not allowed 2026-01-25 04:14:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:14:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_chemicals_lubricants/glass_cleaners landed on page that is not a product page. 2026-01-25 04:14:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots>: HTTP status code is not handled or not allowed 2026-01-25 04:14:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-24bs13hx60-metric-1-12-pitch-60mm-finished-bore-13-teeth>: HTTP status code is not handled or not allowed 2026-01-25 04:14:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/futuro-performance-comfort-wrist-support-adjustable-2-pk-6-pk-case>: HTTP status code is not handled or not allowed 2026-01-25 04:14:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/futuro-wrist-stabilizer-left-hand-small-medium-3-pk-4-pk-case>: HTTP status code is not handled or not allowed 2026-01-25 04:14:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-25 04:14:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-20bs20hx38-metric-1-14-pitch-38mm-finished-bore-20-teeth>: HTTP status code is not handled or not allowed 2026-01-25 04:14:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zig-elusion-heritage-work-men-s-low-cut-sneaker-composite-toe-size-11-5m-gray-black>: HTTP status code is not handled or not allowed 2026-01-25 04:14:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/groundbreaker-work-boots-w-external-met-guard-steel-toe-size-9-5m-8-inch-h-brown>: HTTP status code is not handled or not allowed 2026-01-25 04:14:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-25 04:14:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-16bs25x35-metric-1-pitch-35mm-finished-bore-25-teeth>: HTTP status code is not handled or not allowed 2026-01-25 04:14:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vitals-skate-inspired-work-shoes-composite-toe-size-9w-wintermoss-whitecap-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:14:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35bs21hx34-38-pitch-34-finished-bore-21-teeth>: HTTP status code is not handled or not allowed 2026-01-25 04:14:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-12bs11hx30-metric-34-pitch-30mm-finished-bore-11-teeth>: HTTP status code is not handled or not allowed 2026-01-25 04:14:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-10bs14hx32-metric-58-pitch-32mm-finished-bore-14-teeth>: HTTP status code is not handled or not allowed 2026-01-25 04:14:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zig-elusion-heritage-work-women-s-low-cut-sneaker-composite-toe-size-7m-black>: HTTP status code is not handled or not allowed 2026-01-25 04:14:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:14:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-d40bs15hx118-double-12-pitch-1-18-finished-bore-15-teeth>: HTTP status code is not handled or not allowed 2026-01-25 04:14:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-08a35h-metric-a-plate-12-pitch-15mm-bore-35-teeth>: HTTP status code is not handled or not allowed 2026-01-25 04:14:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-clearseal-microcentrifuge-tubes-with-conical-bottom-sterile-0-5-ml-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:14:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35bs27hx118-38-pitch-1-18-finished-bore-27-teeth>: HTTP status code is not handled or not allowed 2026-01-25 04:14:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35bs30hx1-38-pitch-1-finished-bore-30-teeth>: HTTP status code is not handled or not allowed 2026-01-25 04:14:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-readystrain-sterile-cell-straining-kit-100um-50-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:14:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qline-retractable-belt-barrier-x2-matte-post-red-restricted-area>: HTTP status code is not handled or not allowed 2026-01-25 04:14:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-25 04:14:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/true-skate-inspired-work-shoes-composite-toe-size-11-5w-black-gum>: HTTP status code is not handled or not allowed 2026-01-25 04:15:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-40sh21h-qd-bushed-12-pitch-21-teeth>: HTTP status code is not handled or not allowed 2026-01-25 04:15:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stone-op-art-skate-inspired-work-shoes-composite-toe-size-13m-black-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:15:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-crafted-western-work-boots-w-steel-toe-and-internal-met-guard-size-12w-brown>: HTTP status code is not handled or not allowed 2026-01-25 04:15:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-crafted-western-work-boots-w-steel-toe-and-internal-met-guard-size-12m-brown>: HTTP status code is not handled or not allowed 2026-01-25 04:15:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stone-skate-inspired-work-shoes-composite-toe-size-8-5m-black-red>: HTTP status code is not handled or not allowed 2026-01-25 04:15:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:15:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:15:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80bs13hx178-1-pitch-1-78-finished-bore-13-teeth>: HTTP status code is not handled or not allowed 2026-01-25 04:15:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80bs13hx11516-1-pitch-1-1516-finished-bore-13-teeth>: HTTP status code is not handled or not allowed 2026-01-25 04:15:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 62773 pages (at 92 pages/min), scraped 31639 items (at 39 items/min) 2026-01-25 04:15:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-air-bubble-rolls-24-x-375-x-5-16-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:15:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-08bs22hx32-metric-12-pitch-32mm-finished-bore-22-teeth>: HTTP status code is not handled or not allowed 2026-01-25 04:15:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bubble-bags-4-x-5-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:15:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kraft-literature-mailer-18-x-12-x-3-50-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:15:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paper-twist-ties-10-x-5-32-blue-2000-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:15:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots>: HTTP status code is not handled or not allowed 2026-01-25 04:15:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-25 04:15:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-twist-tie-spool-5-32-x-7000-blue->: HTTP status code is not handled or not allowed 2026-01-25 04:15:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adi-654-alarm-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 04:15:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stone-op-art-skate-inspired-work-shoes-composite-toe-size-14m-dark-brown-khaki>: HTTP status code is not handled or not allowed 2026-01-25 04:15:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stone-op-art-skate-inspired-work-shoes-composite-toe-size-13w-dark-brown-khaki>: HTTP status code is not handled or not allowed 2026-01-25 04:15:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bubble-bags-15-x-15-200-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:15:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/r-d-5417-sealed-lead-acid-replacement-battery-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 04:15:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60bs15hx118-34-pitch-1-18-finished-bore-15-teeth>: HTTP status code is not handled or not allowed 2026-01-25 04:15:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-41bs30hx58-12-pitch-58-finished-bore-30-teeth>: HTTP status code is not handled or not allowed 2026-01-25 04:15:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-10bs18hx32-metric-58-pitch-32mm-finished-bore-18-teeth>: HTTP status code is not handled or not allowed 2026-01-25 04:15:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-10bs18hx35-metric-58-pitch-35mm-finished-bore-18-teeth>: HTTP status code is not handled or not allowed 2026-01-25 04:15:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nanoflex-tr-athletic-work-shoes-composite-toe-size-11-5w-black>: HTTP status code is not handled or not allowed 2026-01-25 04:15:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/altronix-smp7pmp4-alarm-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 04:15:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/altronix-maxim55d-alarm-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 04:15:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/altronix-al600ulm-alarm-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 04:15:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rb3317-nanoflex-tr-athletic-work-shoes-composite-toe-size-9-5w-black>: HTTP status code is not handled or not allowed 2026-01-25 04:15:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nano-x1-adventure-athletic-work-mid-cut-shoes-composite-toe-size-10-5w-black>: HTTP status code is not handled or not allowed 2026-01-25 04:15:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-08a19h-metric-a-plate-12-pitch-10mm-bore-19-teeth>: HTTP status code is not handled or not allowed 2026-01-25 04:15:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/altronix-al1024ulmr-alarm-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 04:15:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ag10-series-2-1-4-x-10-solid-alum-fixed-bar-supply-return-air-vent-grille-brushed-satin>: HTTP status code is not handled or not allowed 2026-01-25 04:15:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nano-x1-adventure-athletic-work-mid-cut-shoes-composite-toe-size-4m-black>: HTTP status code is not handled or not allowed 2026-01-25 04:15:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lavante-trail-2-running-work-shoes-composite-toe-size-14m-gray-black>: HTTP status code is not handled or not allowed 2026-01-25 04:15:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lavante-trail-2-running-work-shoes-composite-toe-size-15w-army-green-black-yellow>: HTTP status code is not handled or not allowed 2026-01-25 04:15:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lavante-trail-2-running-work-shoes-composite-toe-size-10-5m-gray-black>: HTTP status code is not handled or not allowed 2026-01-25 04:15:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-hook-with-acrylic-adhesive-4-inch-x-75-feet>: HTTP status code is not handled or not allowed 2026-01-25 04:15:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-40bs16hx118-12-pitch-1-18-finished-bore-16-teeth>: HTTP status code is not handled or not allowed 2026-01-25 04:15:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tork-40-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 04:15:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35b19hx12mpb-38-pitch-12-plain-bore-19-teeth>: HTTP status code is not handled or not allowed 2026-01-25 04:15:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:15:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35a22hx12-a-plate-38-pitch-12-bore-22-teeth>: HTTP status code is not handled or not allowed 2026-01-25 04:15:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mounted-work-station-with-foldable-keyboard-tray>: HTTP status code is not handled or not allowed 2026-01-25 04:15:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:15:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:15:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hivis-insulated-softshell-high-bib-black-lime-10f-comfort-rating-3xl>: HTTP status code is not handled or not allowed 2026-01-25 04:15:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dyna-ray-556-emergency-light-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 04:15:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:15:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/irontuff-jackoat-black-hivis-lime-50-comfort-rating-4xl-regular>: HTTP status code is not handled or not allowed 2026-01-25 04:15:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elan-np66a-emergency-light-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 04:15:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evolve-skate-inspired-work-shoes-composite-toe-size-4-5m-triple-black>: HTTP status code is not handled or not allowed 2026-01-25 04:15:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-25 04:15:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/winch-strap-423021-with-flat-hook-30-x-4-gold>: HTTP status code is not handled or not allowed 2026-01-25 04:15:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-drum-spillpallet-5116-bd-low-profile-2000-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 04:15:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-25 04:15:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evolve-skate-inspired-work-shoes-composite-toe-size-11-5w-olive-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:16:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats landed on page that is not a product page. 2026-01-25 04:16:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-ds250-250cc-atv-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 04:16:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-25 04:16:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gs-gtx7l-bs-powersports-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 04:16:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-sump-pump-replacement-battery-100ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 04:16:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evolve-skate-inspired-high-top-work-shoes-composite-toe-size-8w-black-yellow>: HTTP status code is not handled or not allowed 2026-01-25 04:16:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/daystart-slip-on-work-shoes-steel-toe-size-9w-navy>: HTTP status code is not handled or not allowed 2026-01-25 04:16:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rieju-mrx-50cc-motorcycle-replacement-battery-1998-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 04:16:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:16:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/left-turn-for-ns5140-36-cable-guard-orange-black>: HTTP status code is not handled or not allowed 2026-01-25 04:16:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oil-well-pump-motor-wo50-12-404t-tefc-rigid-3-ph-404t-230-460-796v-50-hp>: HTTP status code is not handled or not allowed 2026-01-25 04:16:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 62864 pages (at 91 pages/min), scraped 31675 items (at 36 items/min) 2026-01-25 04:16:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pm-dc-motor-wpmdc15-18-180v-145tcb-tenv-tefc-145tc-180v-15-hp-1800-rpm>: HTTP status code is not handled or not allowed 2026-01-25 04:16:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-slide-base-w364t>: HTTP status code is not handled or not allowed 2026-01-25 04:16:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/d-flange-kit-pew280td-prem-eff-284t-286t>: HTTP status code is not handled or not allowed 2026-01-25 04:16:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:16:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-3d-780cc-personal-watercraft-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 04:16:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-25 04:16:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/razor-electric-scooter-replacement-battery-14ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 04:16:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/colored-carton-sealing-tape-3-x-55-yds-white-pack-of-24>: HTTP status code is not handled or not allowed 2026-01-25 04:16:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-frame-motor-wwht100-18-405t-f2-gp-tefc-rigid-3-ph-f2-405t-115-fla-rb>: HTTP status code is not handled or not allowed 2026-01-25 04:16:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cc-pump-motor-wwe25-36-284jm-tefc-rigid-c-3-ph-284jm-25-hp-3600-rpm>: HTTP status code is not handled or not allowed 2026-01-25 04:16:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dv-180605-door-viewer-180-916installation-hole-polish-brass>: HTTP status code is not handled or not allowed 2026-01-25 04:16:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polaris-550-lx-544cc-snowmobile-replacement-battery-2008-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 04:16:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-25 04:16:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arctic-cat-zr-800-cc-785cc-snowmobile-replacement-battery-12v-i>: HTTP status code is not handled or not allowed 2026-01-25 04:16:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-foot-shin-guards-steel-toe-clip-rubber-strap-5w-standard-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 04:16:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cotton-cover-for-floor-mop-261564>: HTTP status code is not handled or not allowed 2026-01-25 04:16:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-rated-insulated-metal-access-panel-36l-x-48w>: HTTP status code is not handled or not allowed 2026-01-25 04:16:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-purpose-metal-access-panel-key-lock-24l-x-24w>: HTTP status code is not handled or not allowed 2026-01-25 04:16:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/concealed-frame-access-panel-for-wallboard-cam-latch-8l-x-8w>: HTTP status code is not handled or not allowed 2026-01-25 04:16:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/greenline-626d-hand-truck-w-single-loop-handle-pe-pneumatic-wheels-600-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 04:16:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/upg-ub1234-sealed-lead-acid-replacement-battery-9ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 04:16:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elk-12120-sealed-lead-acid-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 04:16:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ajc-prescolite-em-exits-6v-4-5ah-emergency-light-battery>: HTTP status code is not handled or not allowed 2026-01-25 04:16:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-flex-a-prene-1gauge-welding>: HTTP status code is not handled or not allowed 2026-01-25 04:16:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/epdm-commercial-grade-60a-rubber-sheet-3-16-thick-x-3-x-12-length-black>: HTTP status code is not handled or not allowed 2026-01-25 04:16:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-6-center-wheel-for-500-lb-capacity-stainless-steel-bulk-truck>: HTTP status code is not handled or not allowed 2026-01-25 04:16:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bb-bp1-2-6-sealed-lead-acid-replacement-battery-1-3ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 04:16:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jump-n-carry-jnc100-jump-starter-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 04:16:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/painting_supplies/specialty_coatings_paint already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:16:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-blade-for-sugoi-360mm>: HTTP status code is not handled or not allowed 2026-01-25 04:16:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/surface-mount-valve-box-satin-polish-12-12-6>: HTTP status code is not handled or not allowed 2026-01-25 04:16:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ready-built-ii-2-tier-2-doors-locker-12w-x-12d-x-82h-light-gray-assembled>: HTTP status code is not handled or not allowed 2026-01-25 04:16:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:16:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-waterfall-deck-shelving-5-shelves-48w-x-18d-x-96h-midnight-ebony>: HTTP status code is not handled or not allowed 2026-01-25 04:16:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-tech-closed-style-metal-shelving-6-shelves-350-lb-cap-48w-x-24d-x-87h-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:16:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-tech-open-shelving-6-shelves-starter-unit-350-lb-cap-48w-x-24d-x-87h-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:16:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/digital-wheelchair-dual-ramp-scale-1000lb-capacity-emr-connectivity>: HTTP status code is not handled or not allowed 2026-01-25 04:16:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/painting_supplies/specialty_coatings_paint landed on page that is not a product page. 2026-01-25 04:16:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-tech-open-shelving-6-shelves-starter-unit-400-lb-cap-48w-x-12d-x-87h-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:16:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powered-drum-truck-f89484-standard-chime-hook-12-wheels>: HTTP status code is not handled or not allowed 2026-01-25 04:16:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions>: HTTP status code is not handled or not allowed 2026-01-25 04:16:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jasco-rb4100-sealed-lead-acid-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 04:16:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-step-heavy-duty-double-entry-work-platform-24w-steps-dep5-2472>: HTTP status code is not handled or not allowed 2026-01-25 04:17:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/978s08-catwalk-120-high-unit-walkway-96-x-48>: HTTP status code is not handled or not allowed 2026-01-25 04:17:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-tech-open-stand-alone-shelving-6-shelves-800-lb-cap-36w-x-24d-x-87h-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:17:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-tech-open-style-metal-shelving-sway-braces-8-shelves-48w-x-12d-x-87h-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:17:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-in-drainable-blade-fixed-louver-with-flange>: HTTP status code is not handled or not allowed 2026-01-25 04:17:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-in-combination-louver-damper-with-flange>: HTTP status code is not handled or not allowed 2026-01-25 04:17:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mighty-products-mt-dt-751600-7-5-x-16-dump-truck-tarp-industrial-grade>: HTTP status code is not handled or not allowed 2026-01-25 04:17:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-in-high-velocity-wall-exhaust-shutter>: HTTP status code is not handled or not allowed 2026-01-25 04:17:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-purpose-e-track-tie-down-strap-4400-lb-capacity-2-x-12>: HTTP status code is not handled or not allowed 2026-01-25 04:17:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shockstrap-tie-downs-ratchet-strap-6000-lb-capacity-9l-x-2w-black>: HTTP status code is not handled or not allowed 2026-01-25 04:17:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/csb-gpl12400-sealed-lead-acid-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 04:17:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caterpillar-ap1050-industrial-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 04:17:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 62957 pages (at 93 pages/min), scraped 31717 items (at 42 items/min) 2026-01-25 04:17:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-tech-open-stand-alone-shelving-5-shelves-375-lb-cap-48w-x-18d-x-87h-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:17:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-in-aluminum-modular-fixed-louver>: HTTP status code is not handled or not allowed 2026-01-25 04:17:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cam-tie-down-cinch-strap-1500-lb-capacity-1-x-2>: HTTP status code is not handled or not allowed 2026-01-25 04:17:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stair-tread-rubber-square-nose-48l-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:17:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b-b-bp17-12-nb-sealed-lead-acid-replacement-battery-18ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 04:17:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-sonic-ps-12180-nb-ps12180-12v-18ah-emergency-light>: HTTP status code is not handled or not allowed 2026-01-25 04:17:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cove-base-vinyl-4x18x48-black>: HTTP status code is not handled or not allowed 2026-01-25 04:17:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/maintenance-free-quiet-2-tier-2-doors-locker-15w-x-24d-x-78h-dark-gray-assembled>: HTTP status code is not handled or not allowed 2026-01-25 04:17:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gallagher-solar-fence-industrial-replacement-battery-8ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 04:17:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/maintenance-free-quiet-1-tier-1-door-locker-15w-x-24d-x-78h-dark-gray-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 04:17:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-reel-cart-with-cabinet-RCM-2448-5PYTL>: HTTP status code is not handled or not allowed 2026-01-25 04:17:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labclean-cleanroom-wipes-10037-polyester-9l-x-9w-pack-of-150>: HTTP status code is not handled or not allowed 2026-01-25 04:17:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spark-resistant-manual-chain-hoist-12-ton-capacity-20-lift>: HTTP status code is not handled or not allowed 2026-01-25 04:17:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bg-series-gear-adjustable-magazine-rack-plastic-1258w-14d-22316h>: HTTP status code is not handled or not allowed 2026-01-25 04:17:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/karryall-model-79-adjustable-conductive-medium-pcb-rack-15w-11d-1514h>: HTTP status code is not handled or not allowed 2026-01-25 04:17:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yuasa-np24-12t-sealed-lead-acid-replacement-battery-26ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 04:17:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exair-soft-grip-safety-air-gun-wmini-super-air-nozzle-and-chip-shield-zinc>: HTTP status code is not handled or not allowed 2026-01-25 04:17:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/altronix-al400ulxpd16-alarm-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 04:17:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/centrifuge-tubes-sterile-15ml-capacity-amber-pack-of-500>: HTTP status code is not handled or not allowed 2026-01-25 04:17:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-push-beam-trolley-5-ton-capacity>: HTTP status code is not handled or not allowed 2026-01-25 04:17:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-in-2-1-2-in-black-steel-pipe-nipple-150-psi-lead-free>: HTTP status code is not handled or not allowed 2026-01-25 04:17:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/napco-alarms-gem-p816-alarm-replacement-battery-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 04:17:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cant1260-cantilever-shelf-adjustable-solid-shelf-12w-x-2h-x-60l-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 04:17:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eye-face-wash-station-pedestal-mount-abs-plastic-bowl-stainless-steel-pipe>: HTTP status code is not handled or not allowed 2026-01-25 04:17:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/potter-electric-pfc-5004e-alarm-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 04:17:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powervar-54811-01-12v-12ah-ups-battery>: HTTP status code is not handled or not allowed 2026-01-25 04:17:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnetic-drive-pump-1md-115v-180-gph-at-1>: HTTP status code is not handled or not allowed 2026-01-25 04:17:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/type-i-steel-safety-can-with-funnel-1-gallon-4l-self-close-lid-yellow-7110210>: HTTP status code is not handled or not allowed 2026-01-25 04:17:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/material-handling/lift_tables_work_positioners/stationary_lift_tables already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:17:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/63092rs-p53-deep-groove-ball-bearing-double-sealed-abec-5-45mm-bore-100mm-od>: HTTP status code is not handled or not allowed 2026-01-25 04:17:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/invacare-xterra-g-wheelchair-replacement-battery-55ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 04:17:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bungee-cords-14-inch-epdm-rubber-strap-10-pack-black>: HTTP status code is not handled or not allowed 2026-01-25 04:17:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c-clamp-8-inch>: HTTP status code is not handled or not allowed 2026-01-25 04:17:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wardrobe-z-locker-w-padlock-hasp-2-tier-assembled-15w-x-15d-x-60h-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:17:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reagent-bottle-narrow-mouth-pp-125ml-capacity-clear-pack-of-500>: HTTP status code is not handled or not allowed 2026-01-25 04:17:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60182rsnr-deep-groove-ball-bearing-double-sealed-snap-ring-90mm-bore-140mm-od>: HTTP status code is not handled or not allowed 2026-01-25 04:17:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30206-tapered-roller-bearing-metric-30mm-bore-62mm-od>: HTTP status code is not handled or not allowed 2026-01-25 04:17:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/production-booster-452t72bk-72w-x-24h-2-shelves-black>: HTTP status code is not handled or not allowed 2026-01-25 04:17:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/production-booster-454t60bk-60w-x-14h-1-shelf-black>: HTTP status code is not handled or not allowed 2026-01-25 04:17:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pneumatic-impact-vibrator-p-60-cast-iron-body>: HTTP status code is not handled or not allowed 2026-01-25 04:17:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluidization-nozzle-steel-nipple-with-plastic-insert->: HTTP status code is not handled or not allowed 2026-01-25 04:17:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-tier-3-door-digital-wood-locker-36-w-x-15-d-x-72-h-gray-assembled>: HTTP status code is not handled or not allowed 2026-01-25 04:18:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/material-handling/lift_tables_work_positioners/stationary_lift_tables landed on page that is not a product page. 2026-01-25 04:18:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-tier-2-door-wood-locker-12-w-x-15-d-x-72-h-gray-assembled>: HTTP status code is not handled or not allowed 2026-01-25 04:18:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pneumatic-vibrator-turbine-ot-13-anodized-aluminum-body>: HTTP status code is not handled or not allowed 2026-01-25 04:18:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/permobil-chairman-hd-wheelchair-replacement-battery-75ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 04:18:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drop-in-strapping-cart>: HTTP status code is not handled or not allowed 2026-01-25 04:18:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pride-mobility-1104-wheelchair-replacement-battery-75ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 04:18:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6306-deep-groove-ball-bearing-open-30mm-bore-72mm-od>: HTTP status code is not handled or not allowed 2026-01-25 04:18:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/material-handling/ladders_personnel_lifts/step_ladders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:18:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelf-truck-60l-x-24w-x-60h-polyurethane-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 04:18:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelf-truck-24l-x-18w-x-60h-polyurethane-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 04:18:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 63051 pages (at 94 pages/min), scraped 31761 items (at 44 items/min) 2026-01-25 04:18:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:18:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelf-truck-36l-x-24w-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 04:18:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelf-truck-24l-x-24w-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 04:18:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:18:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wheelhorse-lawn-and-garden-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 04:18:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zebra-m301-mechanical-pencil-leaderaser-refillable-05mm-blacksilver>: HTTP status code is not handled or not allowed 2026-01-25 04:18:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:18:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/material-handling/ladders_personnel_lifts/step_ladders landed on page that is not a product page. 2026-01-25 04:18:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-cvt3gb5i-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 04:18:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60440227464-341d-1-2dia-x-1-2w-36-grit-100-case>: HTTP status code is not handled or not allowed 2026-01-25 04:18:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simplicity-5212-lawn-and-garden-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 04:18:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:18:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers>: HTTP status code is not handled or not allowed 2026-01-25 04:18:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bisley-womens-long-sleeve-shirt-l-hi-vis-yellow>: HTTP status code is not handled or not allowed 2026-01-25 04:18:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-25 04:18:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8020-10-series-1-34-x-716-lower-door-slide-track-120l-stock-bar>: HTTP status code is not handled or not allowed 2026-01-25 04:18:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ewf3000-1500-gallon-lead-sediment-reduction-filter-kit>: HTTP status code is not handled or not allowed 2026-01-25 04:18:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers>: HTTP status code is not handled or not allowed 2026-01-25 04:18:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8020-40-series-1-916-x-1-916-lite-bi-t-slotted-extrusion-profile-118-18l-stock-bar>: HTTP status code is not handled or not allowed 2026-01-25 04:18:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9682121-bench-grinder-wpedestal-floor-stand-12-wheel-dia-2hp-220v>: HTTP status code is not handled or not allowed 2026-01-25 04:18:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers>: HTTP status code is not handled or not allowed 2026-01-25 04:18:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zapper-ts-950-g-lawn-and-garden-replacement-battery-9ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 04:18:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bisley-womens-short-sleeve-t-shirt-class-2-l-hi-vis-yellow>: HTTP status code is not handled or not allowed 2026-01-25 04:18:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/toro-titan-mx4800-lawn-and-garden-replacement-battery-9ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 04:18:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/grinders_buffers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:18:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/energel-liquid-gel-refill-07mm-needle-tip-black-ink>: HTTP status code is not handled or not allowed 2026-01-25 04:18:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:18:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dyna-ray-dr70712-emergency-light-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 04:18:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/permanent-marker-waterproof-ultra-fine-black-ink-5pack>: HTTP status code is not handled or not allowed 2026-01-25 04:18:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/476-in3-hydraulic-hand-pump-p8701-2speed-wcarry-handle>: HTTP status code is not handled or not allowed 2026-01-25 04:18:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-body-hydraulic-cylinder-kit-hf3005b-30-ton>: HTTP status code is not handled or not allowed 2026-01-25 04:18:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mpsr1-dhtw-ge-single-action-ex-screw-reset-weatherproof>: HTTP status code is not handled or not allowed 2026-01-25 04:18:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12v24a-battery-12-v-24-amp>: HTTP status code is not handled or not allowed 2026-01-25 04:18:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electrostatic-air-filter-merv-8-14w-x-1d-x-30h>: HTTP status code is not handled or not allowed 2026-01-25 04:18:44 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-25 04:18:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/egc-hdvmh-multi-high-cdhorn-strobe-95-177cd-ceiling-mount-white>: HTTP status code is not handled or not allowed 2026-01-25 04:18:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hyperlite-extension-ladder-w-cable-hooks-claw-v-bar-16-type-iaa-375-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 04:18:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/grinders_buffers landed on page that is not a product page. 2026-01-25 04:18:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-hi-density-rounded-foam-mini-roller-cover-10-pack-6-case-4foamq-10>: HTTP status code is not handled or not allowed 2026-01-25 04:18:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dyna-ray-s18210-emergency-light-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 04:18:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-trouble-light-sl425a-25l-cord-163-ga-yellow-6pk>: HTTP status code is not handled or not allowed 2026-01-25 04:18:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sumostance-extension-ladder-w-hyperlite-tech-fiberglass-24-type-ia-300-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 04:18:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/jobsite_portable_lighting/job_site_work_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:18:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-person-standard-ladder-telescoping-plank-121l-x-13w-x-3h>: HTTP status code is not handled or not allowed 2026-01-25 04:18:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-12-706-emergency-light-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 04:18:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clip-finished-end-panel-12-inch-d-x-87-inch-h-champagne>: HTTP status code is not handled or not allowed 2026-01-25 04:18:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reverse-bolt-on-hitch-bar-led-flood-light-for-2011-2016-ford-f250-f450-44a>: HTTP status code is not handled or not allowed 2026-01-25 04:19:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interlake-bulk-storage-rack-galvanized-metal-shelf-12-inch-l-x-48-inch-w>: HTTP status code is not handled or not allowed 2026-01-25 04:19:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/panasonic-lcr064r5p-sealed-lead-acid-agm-vrla-battery>: HTTP status code is not handled or not allowed 2026-01-25 04:19:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portalac-pe5f1-emergency-light-replacement-battery-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 04:19:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/jobsite_portable_lighting/job_site_work_lights>: HTTP status code is not handled or not allowed 2026-01-25 04:19:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergi-lite-6m5-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 04:19:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kung-long-wp1812-sealed-lead-acid-agm-vrla-battery>: HTTP status code is not handled or not allowed 2026-01-25 04:19:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dyna-ray-s18166-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 04:19:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-120706-emergency-light-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 04:19:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-lm80--emergency-light-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 04:19:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 63142 pages (at 91 pages/min), scraped 31803 items (at 42 items/min) 2026-01-25 04:19:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/square-base-w-wheels-for-libra-cast-aluminum-100-lb-weight-bronze>: HTTP status code is not handled or not allowed 2026-01-25 04:19:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/locksets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:19:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dyna-ray-540d2-emergency-light-replacement-battery-3-2ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 04:19:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commercial-cylindrical-lockset-lever-storeroom-brushed-chrome>: HTTP status code is not handled or not allowed 2026-01-25 04:19:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/151597-112-x-112-tslotted-profile-97-stock-bar>: HTTP status code is not handled or not allowed 2026-01-25 04:19:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-panel-class-a-4w-x-12h-white>: HTTP status code is not handled or not allowed 2026-01-25 04:19:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sure-lites-9-01-emergency-light-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 04:19:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-12361-emergency-light-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 04:19:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2545-45-degree-support-18l>: HTTP status code is not handled or not allowed 2026-01-25 04:19:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/fasteners/door_window_hardware/locksets>: HTTP status code is not handled or not allowed 2026-01-25 04:19:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/umbrellas_awnings_shades/outdoor_umbrellas_bases/umbrella_bases already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:19:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modern-equipment-meco-kl1-steel-tilt-hopper-truck-1-cu-yd-capacity>: HTTP status code is not handled or not allowed 2026-01-25 04:19:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kimwipe-holder-w-magnet-mount-small-white>: HTTP status code is not handled or not allowed 2026-01-25 04:19:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/core-cl-cart-w-gray-drawers-27-12l-x-19-12w-x-35-58h-white>: HTTP status code is not handled or not allowed 2026-01-25 04:19:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/separator-puller-with-2-jaws-11-ton-capacity-5-78-opening-12-78-adjustable-reach>: HTTP status code is not handled or not allowed 2026-01-25 04:19:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/analog-bench-hardness-tester-8-12w-x-20d-x-30h>: HTTP status code is not handled or not allowed 2026-01-25 04:19:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergi-lite-2vm-6v-45ah-alarm-battery>: HTTP status code is not handled or not allowed 2026-01-25 04:19:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jump-n-carry-jnc660-jump-starter-12v-18ah-jump-starter-battery>: HTTP status code is not handled or not allowed 2026-01-25 04:19:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/outdoor_furniture/umbrellas_awnings_shades/outdoor_umbrellas_bases/umbrella_bases>: HTTP status code is not handled or not allowed 2026-01-25 04:19:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/this-is-a-non-smoking-vehicle-safety-sign-adhesive-vinyl-3-x-7-whitered-pack-of-5>: HTTP status code is not handled or not allowed 2026-01-25 04:19:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nerp003s-15-ner-electric-hoist-push-trolley-14-ton-15-lift-36-ftmin-208v>: HTTP status code is not handled or not allowed 2026-01-25 04:19:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3378-end-fastener-double-wing-clip>: HTTP status code is not handled or not allowed 2026-01-25 04:19:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nerp003h-20-ner-electric-hoist-push-trolley-14-ton-20-lift-53-ftmin-208v>: HTTP status code is not handled or not allowed 2026-01-25 04:19:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nerp003hd-10-ner-electric-hoist-push-trolley-14-ton-10-lift-539-ftmin-208v>: HTTP status code is not handled or not allowed 2026-01-25 04:19:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polaris-scrambler-1000cc-atv-replacement-battery-2014-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 04:19:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ion-audio-house-party-speaker-replacement-battery-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 04:19:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6425-double-flange-linear-bearing-brake-kit-ready>: HTTP status code is not handled or not allowed 2026-01-25 04:19:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caution-this-vehicle-makes-frequent-stops-sign-adh-reflective-sheet-10-x-14-yellow>: HTTP status code is not handled or not allowed 2026-01-25 04:19:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ner003hd-10-ner-dual-speed-electric-chain-hoist-14-ton-10-lift-539-ftmin-230v>: HTTP status code is not handled or not allowed 2026-01-25 04:19:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3859-slidein-economy-tnut>: HTTP status code is not handled or not allowed 2026-01-25 04:19:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ner003sd-15-ner-dual-speed-electric-chain-hoist-14-ton-15-lift-366-ftmin-208v>: HTTP status code is not handled or not allowed 2026-01-25 04:19:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ner003s-20-ner-electric-hoist-hook-suspension-14-ton-20-lift-36-ftmin-208v>: HTTP status code is not handled or not allowed 2026-01-25 04:19:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20673-compact-fluorescent-pin-based-cf26dd-e-835-eco-t4x2>: HTTP status code is not handled or not allowed 2026-01-25 04:19:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7305bmuc3-angular-contact-ball-bearing-flush-ground-25mm-bore-x-62mm-od-x-17mm-w>: HTTP status code is not handled or not allowed 2026-01-25 04:19:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/contoured-reversible-puffer-vest-class-2-s-hi-vis-yellow>: HTTP status code is not handled or not allowed 2026-01-25 04:19:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/husaberg-fx650e-650cc-motorcycle-replacement-battery-2003-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 04:19:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ner015s-15-ner-electric-hoist-hook-suspension-1-12-ton-15-lift-18-ftmin-230v>: HTTP status code is not handled or not allowed 2026-01-25 04:19:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/husqvarna-smr-510cc-motorcycle-replacement-battery-2010-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 04:19:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nerm030c-l-20-ner-electric-hoist-motor-trolley-3-ton-20-lift-17-ftmin-208v>: HTTP status code is not handled or not allowed 2026-01-25 04:19:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/atk-250-cc-250cc-motorcycle-replacement-battery-1991-1996-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 04:19:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ducati-gtl-350cc-motorcycle-replacement-battery-1974-1977-12v-i>: HTTP status code is not handled or not allowed 2026-01-25 04:19:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ducati-gtl-500cc-motorcycle-replacement-battery-1976-1978-12v-i>: HTTP status code is not handled or not allowed 2026-01-25 04:19:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9800-system-340-voc-dtm-urethane-mastic-safety-blue-9825419>: HTTP status code is not handled or not allowed 2026-01-25 04:19:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/import-1-4-inch-extra-long-drill-bit-12-inch-long>: HTTP status code is not handled or not allowed 2026-01-25 04:20:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nerp010l-10-ner-electric-chain-hoist-push-trolley-1-ton-10-lift-14-ftmin-208v>: HTTP status code is not handled or not allowed 2026-01-25 04:20:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conserve-water-label-adhesive-vinyl-14-x-10>: HTTP status code is not handled or not allowed 2026-01-25 04:20:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-mobile-instrument-table-20-x-20-undershelf-304-grade>: HTTP status code is not handled or not allowed 2026-01-25 04:20:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:20:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/infinity-1-tier-1-door-ventilated-locker12-inch-wx18-inch-dx72-inch-h-assembled>: HTTP status code is not handled or not allowed 2026-01-25 04:20:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wus-tech-m4a-mobility-scooter-replacement-battery-55ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 04:20:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roof-ventguard-black-painted-galvanized-steel-25-x-25-x-12-rvg2525g>: HTTP status code is not handled or not allowed 2026-01-25 04:20:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pulse-grt-11-electric-scooter-replacement-battery-7ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 04:20:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 63237 pages (at 95 pages/min), scraped 31848 items (at 45 items/min) 2026-01-25 04:20:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/linen-cover-stock-paper-8-1-2-inch-x-11-inch-65-lb-linen-ivory-100-sheets-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:20:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-subject-notebook-8-inch-x-10-1-2-inch-wide-ruled-bright-white-180-sheets-pad>: HTTP status code is not handled or not allowed 2026-01-25 04:20:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-cycle-hydraulic-drum-crusher-compactor-3-phase-460v-38000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 04:20:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/building_safety_signs landed on page that is not a product page. 2026-01-25 04:20:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/picker-international-meteor-portable-x-ray-medical-replacement-battery-35ah-12v>: HTTP status code is not handled or not allowed 2026-01-25 04:20:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durapro-xcp-knee-boots-chevron-outsole-steel-toe-size-14-16h-brown>: HTTP status code is not handled or not allowed 2026-01-25 04:20:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-round-pedestal-table-with-crisp-linen-top-round-black-base-bistro-height>: HTTP status code is not handled or not allowed 2026-01-25 04:20:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/Aire-3000-Mattress-Only-For-Aire-3000-80032>: HTTP status code is not handled or not allowed 2026-01-25 04:20:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ferno-ille-193-194-196-197-lift-chair-medical-replacement-battery-26ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 04:20:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dress-and-shirt-hanger-w-swivel-hook-17-l-plastic-clear>: HTTP status code is not handled or not allowed 2026-01-25 04:20:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/customizable-print-on-divider-85x11-8-tabs-25-sets-white-white>: HTTP status code is not handled or not allowed 2026-01-25 04:20:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/schumacher-electric-ipd-1800-jump-starter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 04:20:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-collated-legal-divider-a-to-z-85x11-26-tabs-white-white>: HTTP status code is not handled or not allowed 2026-01-25 04:20:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fixed-height-steel-gantry-crane-6000-lb-capacity-15w-x-10-1516h>: HTTP status code is not handled or not allowed 2026-01-25 04:20:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/welded-wire-open-deck-pallet-galvanized-steel-42l-x-46w-2500-lb-dynamic-capacity-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:20:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stackable-shelf-bin-11w-x-14d-x-7h-black>: HTTP status code is not handled or not allowed 2026-01-25 04:20:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-electric-chain-hoist-2-ton-capacity-15-lift-16-fpm-440v>: HTTP status code is not handled or not allowed 2026-01-25 04:20:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mechanical-edge-o-dock-leveler-78-ramp-width-steel-20000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 04:20:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelf-unit-36w-x-14d-x-74h-with-28-ssb443-stackable-shelf-bins-black>: HTTP status code is not handled or not allowed 2026-01-25 04:20:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/construction-paper-58-lbs-24-x-36-black-50-sheets-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:20:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cap-style-industrial-climbing-helmet-non-vented-hdpe-suspension-black>: HTTP status code is not handled or not allowed 2026-01-25 04:20:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jtm-1050-millacu-rite-200s-dropowerfeeds-power-draw-bar>: HTTP status code is not handled or not allowed 2026-01-25 04:20:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jtm-4vs-mill3-axis-newall-dp700-knee-dro-powerfeeds-power-draw-bar>: HTTP status code is not handled or not allowed 2026-01-25 04:20:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lincoln-electric-company-classic-ii-industrial-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 04:20:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anti-short-plastic-bushing-polyethylene-516-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 04:20:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kelvinator-scientific-freezer-uc26f-6-040-alarm-replacement-battery-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 04:20:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/46-square-perforated-picnic-table-green-694551GN>: HTTP status code is not handled or not allowed 2026-01-25 04:20:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conduit-hanger-w-hex-nut-and-bolt-stainless-steel-2-12-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 04:20:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/935-0006-935-case-220l-x-140w-x-90h-olive>: HTTP status code is not handled or not allowed 2026-01-25 04:20:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-super-size-16-1-2w-x-23-7-8d-x-11h>: HTTP status code is not handled or not allowed 2026-01-25 04:20:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-pump-jack-wide-end-rail-kit-pj-erw>: HTTP status code is not handled or not allowed 2026-01-25 04:20:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-blower-9539-16-16-dia-1hp-3200-cfm>: HTTP status code is not handled or not allowed 2026-01-25 04:20:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-liquid-tight-connector-alloy-steel-1-12-size-5-pk>: HTTP status code is not handled or not allowed 2026-01-25 04:20:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-person-stage-24-inch-w-x-32-ft-l-2632>: HTTP status code is not handled or not allowed 2026-01-25 04:20:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conduit-body-gasket-nitrile-2-12--3-size>: HTTP status code is not handled or not allowed 2026-01-25 04:20:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conduit-body-gasket-nitrile-3-12--4-size>: HTTP status code is not handled or not allowed 2026-01-25 04:20:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ev-rider-breeze-iv-cargo-edition-wheelchair-replacement-battery-75ah-12v-it>: HTTP status code is not handled or not allowed 2026-01-25 04:20:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ef-ct-fr-cargo-pocket-work-pants-plc2-navy-9-oz-size-33-x-37u>: HTTP status code is not handled or not allowed 2026-01-25 04:20:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/armored-ground-clamp-w-swivel-bronze-1-14-2-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 04:20:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/invacare-at-m-qt-take-along-chair-wheelchair-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 04:20:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:20:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s-hook-low-carbon-steel-1-12-size-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 04:20:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/head-machine-screw-kit-round-phillipsslotted-14-20-size-assorted>: HTTP status code is not handled or not allowed 2026-01-25 04:20:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-precision-bearing-7015cydup4-universal-ground-duplex-75mm-bore-115mm-od>: HTTP status code is not handled or not allowed 2026-01-25 04:20:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5502-moisture-wicking-short-sleeve-safety-tshirt-with-chest-pocket-orange-medium>: HTTP status code is not handled or not allowed 2026-01-25 04:21:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6805-class-e-premium-bibs-2-side-pockets-1-cargo-pocket-lime-sm>: HTTP status code is not handled or not allowed 2026-01-25 04:21:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/murray-17-5hp-46-17-5-hp-35ah-lawn-and-garden-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 04:21:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ibt-technologies-bt15-12-emergency-light-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 04:21:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers>: HTTP status code is not handled or not allowed 2026-01-25 04:21:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perf-16-w-3-step-steel-roll-ladder-10-d-top-stp-orange-318po>: HTTP status code is not handled or not allowed 2026-01-25 04:21:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slotted-flat-head-machine-screw-10-32-x-1-14-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 04:21:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perf-24-w-7-step-steel-roll-ladder-14-d-top-stp-hndrl-orange-fsh726po>: HTTP status code is not handled or not allowed 2026-01-25 04:21:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 63333 pages (at 96 pages/min), scraped 31893 items (at 45 items/min) 2026-01-25 04:21:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bombardier-outlander-400-xt-400cc-atv-replacement-battery-2004-2015-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 04:21:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drop-in-anchor-stainless-steel-12-13-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 04:21:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/snap-in-knockout-seal-carbon-steel-1-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 04:21:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wallpro-300-black-wall-mount-retracting-barrier-10-yellowblack-out-of-service-belt>: HTTP status code is not handled or not allowed 2026-01-25 04:21:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/wall_mount_belts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:21:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trio-lighting-lightning-tl930110-emergency-light-replacement-battery-9ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 04:21:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pneumatic-silencer-5500b90012-npt-female-thread>: HTTP status code is not handled or not allowed 2026-01-25 04:21:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/can-am-outlander-max-650-efi-xt-650cc-atv-replacement-battery-2011-2014-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 04:21:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/signage-system-short-side-open-red-frame-10-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:21:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/pneumatic_valves/directional_control_valves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:21:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/peg-perego-john-deere-utililty-tractor-iged1064-ride-on-toys-battery-35ah-12v>: HTTP status code is not handled or not allowed 2026-01-25 04:21:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inventory-cone-orange-3-sided-with-magnets-dry-erase-decal>: HTTP status code is not handled or not allowed 2026-01-25 04:21:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_tools/mops/mop_buckets_wringers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:21:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-gallon-square-waste-receptacle-green>: HTTP status code is not handled or not allowed 2026-01-25 04:21:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2690gp-heavy-duty-steel-strap-cutter-for-straps-up-to-2-24l-2-x-05051-x-1-mm-cap>: HTTP status code is not handled or not allowed 2026-01-25 04:21:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/wall_mount_belts>: HTTP status code is not handled or not allowed 2026-01-25 04:21:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-gallon-screw-top-plastic-pail-lidwhite-987413>: HTTP status code is not handled or not allowed 2026-01-25 04:21:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zamac-alloy-anchor-phillips-flat-head-14-x-1-size-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 04:21:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrs175101r-cast-iron-right-angle-worm-gear-reducer-101-ratio>: HTTP status code is not handled or not allowed 2026-01-25 04:21:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrf206201l56c-cast-iron-right-angle-worm-gear-reducer-201-ratio-56c-frame>: HTTP status code is not handled or not allowed 2026-01-25 04:21:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/pneumatic_valves/directional_control_valves landed on page that is not a product page. 2026-01-25 04:21:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/cleaning_tools/mops/mop_buckets_wringers>: HTTP status code is not handled or not allowed 2026-01-25 04:21:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/peg-perego-case-ih-lil-tractor-trailer-iged1112-ride-on-toys-battery-35ah-12v>: HTTP status code is not handled or not allowed 2026-01-25 04:21:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90-degree-insulated-liquid-tight-connector-zinc-die-cast-3-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 04:21:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrf154101l56c-cast-iron-right-angle-worm-gear-reducer-101-ratio-56c-frame>: HTTP status code is not handled or not allowed 2026-01-25 04:21:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hello-my-name-is-name-badge-labels-2-11-32-x-3-3-8-blue-100-labels-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:21:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/can-am-spyder-st-se5-1000cc-motorcycle-replacement-battery-2013-2015-12v-i>: HTTP status code is not handled or not allowed 2026-01-25 04:21:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrs175501l-cast-iron-right-angle-worm-gear-reducer-501-ratio>: HTTP status code is not handled or not allowed 2026-01-25 04:21:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-prong-fasteners-2-sheet-capacity-2-3-4l-base-50-box>: HTTP status code is not handled or not allowed 2026-01-25 04:21:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrf133601h56c-cast-iron-right-angle-worm-gear-reducer-501-ratio-56c-frame>: HTTP status code is not handled or not allowed 2026-01-25 04:21:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adhesive-name-badge-labels-2-1-3-x-3-3-8-blue-border-400-box>: HTTP status code is not handled or not allowed 2026-01-25 04:21:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aprilia-rsv-mille-r-nera-1000cc-motorcycle-replacement-battery-2004-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 04:21:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-bands-size-8-7-8-x-1-16-natural-1-lb-box>: HTTP status code is not handled or not allowed 2026-01-25 04:21:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-staples-28-sheet-capacity-1-4-leg-length-210-per-strip-5000-box>: HTTP status code is not handled or not allowed 2026-01-25 04:21:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrf237301h56c-cast-iron-right-angle-worm-gear-reducer-301-ratio-56c-frame>: HTTP status code is not handled or not allowed 2026-01-25 04:21:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrf262601l56c-cast-iron-right-angle-worm-gear-reducer-601-ratio-56c-frame>: HTTP status code is not handled or not allowed 2026-01-25 04:21:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flags-1-wide-yellow-50-flags-dispenser-2-dispensers-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:21:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/business-stapler-20-sheet-210-staple-capacity-black>: HTTP status code is not handled or not allowed 2026-01-25 04:21:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mz-zschopau-green-star-500cc-motorcycle-replacement-battery-1966-2008-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 04:21:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-head-tap-bolt-silicon-bronze-12-13-x-1-14-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 04:21:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-vis-hooded-jacket-zipper-fluorescent-yellowgreenblack-5xl>: HTTP status code is not handled or not allowed 2026-01-25 04:21:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waterproof-utility-cases-medium-case-12l-x-9w-x-4-1-2h-yellow>: HTTP status code is not handled or not allowed 2026-01-25 04:21:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hurricane-industrial-concrete-remover-gallon-bottle-4-case-h1128cs>: HTTP status code is not handled or not allowed 2026-01-25 04:21:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/bags_cases/computer_bags_briefcases already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:21:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honda-gl1800hp-gold-wing-audio-comfort-1800cc-motorcycle-battery-2006-2017-12v>: HTTP status code is not handled or not allowed 2026-01-25 04:21:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aprilia-etx-350-electric-start-350cc-motorcycle-replacement-battery-1985-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 04:21:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c0001-container-placard-label-holder-cstb1-3-x-5-white>: HTTP status code is not handled or not allowed 2026-01-25 04:21:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harley-davidson-dyna-35th-anniversary-fxdi35-1450cc-motorcycle-battery-2006-12v>: HTTP status code is not handled or not allowed 2026-01-25 04:21:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-head-lag-screw-carbon-steel-12-x-5-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 04:21:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-head-lag-screw-carbon-steel-12-x-6-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 04:21:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/bags_cases/computer_bags_briefcases landed on page that is not a product page. 2026-01-25 04:21:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/closed-add-on-cab73-2442-6-42-inch-w-x-24-inch-d-x-73-inch-h-6-shelves-20-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:21:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modern-equipment-meco-54vr-4-wheel-warehouse-hand-truck-mold-on-rubber-1000-lb>: HTTP status code is not handled or not allowed 2026-01-25 04:21:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/american-ironhorse-bandera-1640cc-motorcycle-replacement-battery-2008-2009-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 04:21:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-head-led-emergency-unit-adjustable-optics-and-ni-cad-battery>: HTTP status code is not handled or not allowed 2026-01-25 04:21:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kawasaki-jt1200-stx-12f-1200cc-personal-watercraft-battery-2002-2007-12v>: HTTP status code is not handled or not allowed 2026-01-25 04:21:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m3r-compound-action-snips-934>: HTTP status code is not handled or not allowed 2026-01-25 04:22:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/diversey-gum-remover-cherry-65oz-aerosol-12-case-dvo5628817ct>: HTTP status code is not handled or not allowed 2026-01-25 04:22:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbi-sala-nano-lok-edge-twin-leg-personal-self-retracting-lifeline-aluminum-comfort-hook-7l>: HTTP status code is not handled or not allowed 2026-01-25 04:22:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/diversey-conc-gen-purp-clean-w-hydro-peroxide-gal-btl-4cs-dvo4998841>: HTTP status code is not handled or not allowed 2026-01-25 04:22:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbi-sala174-exofit-nex-x300-comfort-positioning-safety-harness-backhip-d-ring-2xl>: HTTP status code is not handled or not allowed 2026-01-25 04:22:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polaris-virage-700cc-personal-watercraft-replacement-battery-2000-2004-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 04:22:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbi-sala-nano-lok-xl-self-retracting-lifeline-dyneema-webbing--locking-snap-hook-20l>: HTTP status code is not handled or not allowed 2026-01-25 04:22:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2411015002-heritage-sink-drain-w-1-14-trap-polished-chrome>: HTTP status code is not handled or not allowed 2026-01-25 04:22:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 63429 pages (at 96 pages/min), scraped 31930 items (at 37 items/min) 2026-01-25 04:22:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-in-2-way-general-purpose-npt-valve-body-2-5-cv-vt2222>: HTTP status code is not handled or not allowed 2026-01-25 04:22:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-all-models-1500cc-personal-watercraft-replacement-battery-1988-1993-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 04:22:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-ucf201-8-flange-mount-bearing-4-bolt-flange-set-screw-locking-bore-12-7-mm>: HTTP status code is not handled or not allowed 2026-01-25 04:22:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quantum-overhead-track-bracket-double-sided-24w-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 04:22:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-row-spherical-roller-bearing-22312exw33c3-60mm-bore-130mm-od>: HTTP status code is not handled or not allowed 2026-01-25 04:22:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quantum-track-connector-with-hardware-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-25 04:22:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-row-spherical-roller-bearing-22314exw33c3-70mm-bore-150mm-od>: HTTP status code is not handled or not allowed 2026-01-25 04:22:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metatarsal-knee-boots-chevron-outsole-steel-toe-size-12-16h-brick-redcream>: HTTP status code is not handled or not allowed 2026-01-25 04:22:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/imc-heartway-rumba-sfp4f-mobility-scooter-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 04:22:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xtreme-power-xa-750-xa750-electric-scooter-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 04:22:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-ucfl211-35-flange-mount-bearing-2-bolt-flange-set-screw-locking-bore-55-5625-mm>: HTTP status code is not handled or not allowed 2026-01-25 04:22:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-ucfl201-8s-40mm-flange-mount-bearing-2-bolt-flange-set-screw-locking-bore-12-7-mm>: HTTP status code is not handled or not allowed 2026-01-25 04:22:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-row-spherical-roller-bearing-23938ew33-190mm-bore-260mm-od>: HTTP status code is not handled or not allowed 2026-01-25 04:22:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quick-disconnect-disposal-safety-can-w-ss-fittings-for-38-tubing-5-gal-cap-white>: HTTP status code is not handled or not allowed 2026-01-25 04:22:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quantum-partition-wall-hanging-bracket-90176-angle-chrome-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-25 04:22:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pike-pole-10042-112-x-16-anodized-aluminum-handle-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:22:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/imc-heartway-rumba-s-hp4-mobility-scooter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 04:22:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-row-cylindrical-roller-bearing-nu221myc3-105mm-bore-190mm-od>: HTTP status code is not handled or not allowed 2026-01-25 04:22:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dalton-epower-bariatric-heavy-duty-pc1422f-mobility-scooter-battery-55ah-12v>: HTTP status code is not handled or not allowed 2026-01-25 04:22:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quantum-partition-store-long-basket-divider-5h-white>: HTTP status code is not handled or not allowed 2026-01-25 04:22:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/odin-pz-safety-glasses-polarized-smoke-lens-matte-gray-frame>: HTTP status code is not handled or not allowed 2026-01-25 04:22:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-row-cylindrical-roller-bearing-n218-90mm-bore-160mm-od>: HTTP status code is not handled or not allowed 2026-01-25 04:22:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-row-cylindrical-roller-bearing-nu309mc3-45mm-bore-100mm-od>: HTTP status code is not handled or not allowed 2026-01-25 04:22:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8091-noncertified-long-sleeve-tshirt-lime-s>: HTTP status code is not handled or not allowed 2026-01-25 04:22:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8091-noncertified-long-sleeve-tshirt-orange-l>: HTTP status code is not handled or not allowed 2026-01-25 04:22:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/cart_truck_accessories/vehicle_tugger_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:22:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/klip-it-open-add-on-oak85-1836-5-36-inch-w-x-18-inch-d-x-85-inch-h-5-shelves-20-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:22:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/klip-it-closed-starter-csk85-1836-7-36-inch-w-x-18-inch-d-x-85-inch-h-7-shelves-20-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:22:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-in-lead-free-brass-cap-fnpt-125-psi-import>: HTTP status code is not handled or not allowed 2026-01-25 04:22:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-chrome-esd-wire-shelving-unit-starter-60w-x-12d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 04:22:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-in-x-4-1-2-in-stainless-steel-pipe-nipple-16168-psi-sch-40-domestic>: HTTP status code is not handled or not allowed 2026-01-25 04:22:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/piaggio-vespa-mp3-400-400cc-scooter-and-moped-replacement-battery-2008-2014-12v>: HTTP status code is not handled or not allowed 2026-01-25 04:22:50 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/cart_truck_accessories/vehicle_tugger_accessories landed on page that is not a product page. 2026-01-25 04:22:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-bolt-closed-starter-cst75-1236-8-36-inch-w-x-12-inch-d-x-75-inch-h-8-shelves-20-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:22:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/citybug-helper-electricscooter-electric-scooter-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 04:22:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss62052rs-deep-groove-ball-bearing-stainless-steel-double-sealed-od-52mm-bore-25mmmetric>: HTTP status code is not handled or not allowed 2026-01-25 04:22:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-bolt-open-starter-ost75-1842-5x-42-inch-w-x-18-inch-d-x-75-inch-h-5-shelves-18-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:22:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/berm-builder-10-connecting-collar-8643>: HTTP status code is not handled or not allowed 2026-01-25 04:22:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pride-mobility-victory-10-lx-with-ctssuspension-mobility-scooter-battery-35ah-12v>: HTTP status code is not handled or not allowed 2026-01-25 04:22:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modern-equipment-meco-pt3048-8-steel-platform-truck-30x48-8-rubber-2000-lb>: HTTP status code is not handled or not allowed 2026-01-25 04:23:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-chrome-wire-shelving-unit-add-on-36w-x-14d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 04:23:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hxt-30-48-gallon-expansion-tank-hydronic-1-2-in-mnpt>: HTTP status code is not handled or not allowed 2026-01-25 04:23:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-chrome-wire-shelving-unit-add-on-48w-x-14d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 04:23:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-chrome-wire-shelving-unit-add-on-60w-x-14d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 04:23:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-literature-mailers-18-x-14-x-4-200lb-test-ect-32-kraft-25-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:23:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/120lb-cable-ties-natural-16-100-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:23:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/things-to-do-today-pad-8-1-2-x-11-white-100-sheets-pad>: HTTP status code is not handled or not allowed 2026-01-25 04:23:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 63522 pages (at 93 pages/min), scraped 31977 items (at 47 items/min) 2026-01-25 04:23:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pre-inked-message-stamp-completed-1-2-x-1-5-8-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:23:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yamaha-rs90r-rs-vector-900cc-snowmobile-replacement-battery-2008-2009-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 04:23:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-boxes-9-x-6-x-2-200lb-test-ect-32-25-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:23:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/job-invoice-unit-set-2-part-carbonless-8-1-2-x-25-7-16-100-sets-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:23:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-x-10-block-out-labels-kraft-500-per-roll>: HTTP status code is not handled or not allowed 2026-01-25 04:23:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-4-certified-line-regulator-325-7al-1-1-4-up-to-1250000-btu>: HTTP status code is not handled or not allowed 2026-01-25 04:23:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cool-shield-insulated-box-liners-14-x-10-x-10-25-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:23:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-summit-800-x-159-800cc-snowmobile-replacement-battery-2005-2007-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 04:23:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-ski-doo-expedition-tuv-600-h-o-sdi-594cc-snowmobile-battery-2007-12v>: HTTP status code is not handled or not allowed 2026-01-25 04:23:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modern-equipment-meco-bst-ph-bar-storage-truck-84-l-phenolic-wheels-5600-lb>: HTTP status code is not handled or not allowed 2026-01-25 04:23:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-poly-green-wire-shelving-unit-starter-48w-x-14d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 04:23:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-skandic-wide-track-lc-497cc-snowmobile-replacement-battery-1997-2000-12v-i>: HTTP status code is not handled or not allowed 2026-01-25 04:23:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-fuse-for-36-evaporative-cooler-model-600581-292227>: HTTP status code is not handled or not allowed 2026-01-25 04:23:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-mx-z-x-600-ho-sdi-594cc-snowmobile-replacement-battery-2007-2008-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 04:23:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chemical-resistant-gloves-smooth-grip-butyl-13-mil-thick-size-10-black>: HTTP status code is not handled or not allowed 2026-01-25 04:23:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/door-frame-with-drywall-afterset-curries-hinge-location-left-hand-30w-x-84h>: HTTP status code is not handled or not allowed 2026-01-25 04:23:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hollow-steel-vision-light-door-mortise-prep-hollow-hinge-18-ga-36w-x-84h>: HTTP status code is not handled or not allowed 2026-01-25 04:23:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-plastic-locker-11-911212600-single-tier-12-x-12-x-60-1-wide-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:23:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safetypro-335-danger-keep-out-belt-barrier-40-post-35-yellowblack-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-25 04:23:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bluff-mobilepro43-yard-ramp-with-hinged-lip--adj-legs-20000-lb-cap-36l-x-84w-x-48h>: HTTP status code is not handled or not allowed 2026-01-25 04:23:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-plastic-locker-11-921212600-double-tier-12-x-12-x-30-1-wide-cream>: HTTP status code is not handled or not allowed 2026-01-25 04:23:35 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-25 04:23:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/87-in-h-heavy-duty-patio-heater-cover-hvd-cvr-s-for-36-in-reflector-silver>: HTTP status code is not handled or not allowed 2026-01-25 04:23:36 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:32 2026-01-25 04:23:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/87-in-h-heavy-duty-patio-heater-cover-hvd-sgtcv-s-for-36-in-square-glass-tube-silver>: HTTP status code is not handled or not allowed 2026-01-25 04:23:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bombardier-grand-touring-le-1200-1200cc-snowmobile-replacement-battery-2010-12v>: HTTP status code is not handled or not allowed 2026-01-25 04:23:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25dtc-4-two-channel-pole-wo-receptacles-2-8-removable-communications-covers-105l>: HTTP status code is not handled or not allowed 2026-01-25 04:23:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/al5217-internal-elbow-7l>: HTTP status code is not handled or not allowed 2026-01-25 04:23:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bg-3-1-gallon-bucket-grid-509373000>: HTTP status code is not handled or not allowed 2026-01-25 04:23:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caution-anti-slip-stair-tread-black-yellow-6-x-24>: HTTP status code is not handled or not allowed 2026-01-25 04:23:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kawasaki-kaf620-mule-3010-4x4-620cc-utv-replacement-battery-2007-2008-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 04:23:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/short-block-for-engine-models-ch740>: HTTP status code is not handled or not allowed 2026-01-25 04:23:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-x-3-inventory-rectangle-labels-in-5-fluorescent-colors-5000-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:23:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4008-double-sided-foam-tape-2-x-5-yds-1-8-thick-natural->: HTTP status code is not handled or not allowed 2026-01-25 04:23:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fao-muslin-triangular-bandage-40-x-40-x-56-1-bag>: HTTP status code is not handled or not allowed 2026-01-25 04:23:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-541ls-lf-brass-sillcock-valve-with-lockshield-12-sweat>: HTTP status code is not handled or not allowed 2026-01-25 04:23:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-408nl-non-rising-stem-gate-valve-no-lead-brass-1-14-threaded>: HTTP status code is not handled or not allowed 2026-01-25 04:23:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f133-3-4hp-115-208-230v-1800rpm-1ph-tefc-56c-fr>: HTTP status code is not handled or not allowed 2026-01-25 04:23:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-337ab-butterfly-valve-with-buna-seals--10-position-handle-10-wafer>: HTTP status code is not handled or not allowed 2026-01-25 04:23:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rescue-fanny-pack-black>: HTTP status code is not handled or not allowed 2026-01-25 04:23:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hg648-1-3hp-115-208-230v-1725rpm-1ph-tenv-48-fr>: HTTP status code is not handled or not allowed 2026-01-25 04:23:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/explosion-proof-motor-g876-56c17g5327-1hp-115-208-230v-1800rpm-1ph-epfc>: HTTP status code is not handled or not allowed 2026-01-25 04:24:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/severe-duty-motor-e601-182ttts6501-3hp-460v-3600rpm-3ph-182t-fr-tenv>: HTTP status code is not handled or not allowed 2026-01-25 04:24:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/locking-security-mailbox-4350blk-black-usps-approved>: HTTP status code is not handled or not allowed 2026-01-25 04:24:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waterhog-diamondcord-3-8-entrance-charcoal-cord-3-x-4>: HTTP status code is not handled or not allowed 2026-01-25 04:24:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dittmar-741314-power-lifter-iv-medical-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 04:24:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inverter-duty-motor-y598-256thfna8038-20hp-230-460v-1800rpm-3ph-256tc-tefc>: HTTP status code is not handled or not allowed 2026-01-25 04:24:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u-channel-sign-post-8l-112-lbsft-galvanized-post-holes-half-way-down-post>: HTTP status code is not handled or not allowed 2026-01-25 04:24:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 63621 pages (at 99 pages/min), scraped 32030 items (at 53 items/min) 2026-01-25 04:24:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/epact-efficiency-motor-e706-10hp-3600rpm-208-230-460v-3ph-213t-fr-dp>: HTTP status code is not handled or not allowed 2026-01-25 04:24:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rg200w-range-gas-4-burners-241-cu-ft-spark-start-white-24-x-20-x-43>: HTTP status code is not handled or not allowed 2026-01-25 04:24:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g573-056c17f5323-1hp-3-4hp-1800rpm-115-208-230v-1ph-56hc-fr-tefc>: HTTP status code is not handled or not allowed 2026-01-25 04:24:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-motor-r301-63t17fh5326-1-4-18hp-1800rpm-230-460v-3ph-63-fr-tefc>: HTTP status code is not handled or not allowed 2026-01-25 04:24:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/tables/laboratory_tables already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:24:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vision-hp12-116w-sealed-lead-acid-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 04:24:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/picker-inernational-l-explorer-portable-x-ray-medical-battery-26ah-12v>: HTTP status code is not handled or not allowed 2026-01-25 04:24:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pipe-mounting-kit-for-fx-series-heaters-fx5fx6-16-frame-size>: HTTP status code is not handled or not allowed 2026-01-25 04:24:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xb-series-explosion-proof-convection-double-unit-heater-cw-tstat-1500w-240v-1-phase>: HTTP status code is not handled or not allowed 2026-01-25 04:24:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/easy-task-floor-finish-clean-bouquet-scent-32-oz-capacity-bottle-12carton>: HTTP status code is not handled or not allowed 2026-01-25 04:24:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fan-blower-motor-hg711-5kh32hn5620t-1-3hp-1725rpm-115v-1ph-48y-fr-open>: HTTP status code is not handled or not allowed 2026-01-25 04:24:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nps-mobile-science-cabinet-with-drawers-shelf-chemical-resistant-top-48l-x-28w-x-36-14h-oak>: HTTP status code is not handled or not allowed 2026-01-25 04:24:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/tables/laboratory_tables landed on page that is not a product page. 2026-01-25 04:24:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fan-blower-motor-x021-48a11o1353-3-4hp-1075rpm-277v-1ph-48y-fr-opao>: HTTP status code is not handled or not allowed 2026-01-25 04:24:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nps-mobile-science-cabinet-with-shelves-sink-chemical-resistant-top-48l-x-28w-x-36-14h-oak>: HTTP status code is not handled or not allowed 2026-01-25 04:24:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/amstron-ap-12180ev-sealed-lead-acid-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 04:24:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-nexelon-blue-wire-shelving-unit-add-on-48w-x-14d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 04:24:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wst1-3672-36-lp-72w-x-36d-fixed-height-welded-steel-workbench>: HTTP status code is not handled or not allowed 2026-01-25 04:24:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wst2-3048-36-pb-48w-x-30d-fixed-height-workbench-lower-shelf-pegboard-panel>: HTTP status code is not handled or not allowed 2026-01-25 04:24:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-nexelon-blue-wire-shelving-unit-starter-54w-x-24d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 04:24:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mints-wint-o-green-50-oz>: HTTP status code is not handled or not allowed 2026-01-25 04:24:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gogreen-power-gg-13825-25-extension-cord-orange-green>: HTTP status code is not handled or not allowed 2026-01-25 04:24:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/on-the-go-mix-sticks-iced-tea-16-oz-30box>: HTTP status code is not handled or not allowed 2026-01-25 04:24:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gogreen-power-gg-13002ms-3-outlet-power-strip-2-1-2-ft-cord-white>: HTTP status code is not handled or not allowed 2026-01-25 04:24:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ritar-rt12200-rt-12200-sealed-lead-acid-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 04:24:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/batterymart-sla-7-f1-sealed-lead-acid-replacement-battery-7ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 04:24:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-gallon-1-door-self-close-compac-acid-cabinet-23-1-4w-x-18d-x-35h-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:24:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/horizontal-12-cylinder-aluminum-storage-cabinet-60w-x-32d-x-49-1-2h>: HTTP status code is not handled or not allowed 2026-01-25 04:24:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/letter-size-thermal-laminating-pouches-3-mil-11-2-5-x-8-9-10-200-per-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:24:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-nexelate-silver-epoxy-wire-shelving-unit-add-on-72w-x-30d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 04:24:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/51355-6-drawer-hospital-medical-procedure-cart-black-15-x-17-x-46-whiteblack>: HTTP status code is not handled or not allowed 2026-01-25 04:24:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-shelf-nexelate-silver-epoxy-wire-shelving-unit-starter-24w-x-18d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 04:24:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caster-and-leveler-in-one-unit-wmlc-700fs-1540-lb-load-rating-plate-mounted>: HTTP status code is not handled or not allowed 2026-01-25 04:24:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fortress-1000fs-voyager-wheelchair-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 04:24:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-nexelate-silver-epoxy-wire-shelving-unit-starter-48w-x-30d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 04:24:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/imc-heartway-frontier-pf-1-wheelchair-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 04:24:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adapter-39004375-12-male-pipe-x-12-female-pipe-swivel>: HTTP status code is not handled or not allowed 2026-01-25 04:24:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cubbie-file-storage-6-binder-capacity-27-5-8w-x-14-1-2d-x-15-5-8h-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:25:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/permobil-entra-miniflex-wheelchair-replacement-battery-75ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 04:25:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/work-benches/bench_tops_accessories/anti_static_esd_control_devices already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:25:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/werner-4-type-1aa-fiberglass-podium-ladder-w-casters-375-lb-cap-pd7304-4c>: HTTP status code is not handled or not allowed 2026-01-25 04:25:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/casters-for-18-deep-medical-gas-cabinets>: HTTP status code is not handled or not allowed 2026-01-25 04:25:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/wire_shelving/poly_zbrite_wire_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:25:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-four-wing-fan-blade-galvanized-steel-props-60-inch-dia-cw-40-pitch>: HTTP status code is not handled or not allowed 2026-01-25 04:25:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-174-slim-jim-174-1883460-plastic-step-on-container-front-step-18-gallon-beige>: HTTP status code is not handled or not allowed 2026-01-25 04:25:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/amigo-mobility-fd-series-wheelchair-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 04:25:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-shelf-poly-z-brite-wire-shelving-unit-add-on-30w-x-21d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 04:25:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-inlet-blower-wheel-15-1-2-inch-dia-cw-or-ccw-11-1-8-inch-w-1-7-16-inch-bore>: HTTP status code is not handled or not allowed 2026-01-25 04:25:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 63715 pages (at 94 pages/min), scraped 32079 items (at 49 items/min) 2026-01-25 04:25:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classic-rolling-case-156-16-x-5-1-2-x-15-black>: HTTP status code is not handled or not allowed 2026-01-25 04:25:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-galvanized-steel-solid-shelving-unit-add-on-42w-x-18d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 04:25:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-poly-z-brite-wire-shelving-unit-starter-30w-x-14d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 04:25:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/work-benches/bench_tops_accessories/anti_static_esd_control_devices landed on page that is not a product page. 2026-01-25 04:25:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/country-clipper-2505koj-zetron-series-lawn-and-garden-battery-22ah-12v>: HTTP status code is not handled or not allowed 2026-01-25 04:25:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/shelving/wire_shelving/poly_zbrite_wire_shelves landed on page that is not a product page. 2026-01-25 04:25:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/country-clipper-2505kot-zetron-series-lawn-and-garden-battery-22ah-12v>: HTTP status code is not handled or not allowed 2026-01-25 04:25:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ryobi-mower-bmm2400-lawn-and-garden-replacement-battery-18ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 04:25:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-shelf-poly-z-brite-wire-shelving-unit-starter-36w-x-21d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 04:25:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sleeve-bearings-with-insulator-oiler-5-8-inch-shaft-diameter>: HTTP status code is not handled or not allowed 2026-01-25 04:25:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-fence-guide-for-psz-slm-portable-safety-zone>: HTTP status code is not handled or not allowed 2026-01-25 04:25:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pride-mobility-go-go-sc40lr-wheelchair-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 04:25:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tnr-pp-universal-rolling-sign-holder>: HTTP status code is not handled or not allowed 2026-01-25 04:25:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-galvanized-steel-solid-shelving-unit-add-on-72w-x-18d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 04:25:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:25:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arctic-insulated-bin-lid-1000c-50197-58l-x-46w-x-3h-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:25:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sure-lites-6-117p-emergency-light-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 04:25:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-inlet-blower-wheel-6-1-4-inch-dia-ccw-3450-rpm-5-8-inch-bore-2-11-16-inch-w-steel>: HTTP status code is not handled or not allowed 2026-01-25 04:25:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-inlet-blower-wheel-5-1-4-inch-dia-ccw-3450-rpm-1-2-inch-bore-2-15-16-inch-w-galvanized>: HTTP status code is not handled or not allowed 2026-01-25 04:25:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mt-5210-lcr-meter>: HTTP status code is not handled or not allowed 2026-01-25 04:25:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-28mm-hand-held-label-applicator>: HTTP status code is not handled or not allowed 2026-01-25 04:25:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dyna-ray-dr7375sg-emergency-light-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 04:25:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hw-6514b-14-pc-combination-wrench-set-metric>: HTTP status code is not handled or not allowed 2026-01-25 04:25:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manual-bag-sealercutter-with-wide-neck-opening>: HTTP status code is not handled or not allowed 2026-01-25 04:25:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dm0412-clinical-centrifuge-91302341>: HTTP status code is not handled or not allowed 2026-01-25 04:25:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portalac-pe17option-emergency-light-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 04:25:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-25 04:25:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shark-wash-down-duty-5018ep3ess56cfl-05-hp-1800-rpm-208-230-460-volts-tefc-3-ph>: HTTP status code is not handled or not allowed 2026-01-25 04:25:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-lm130i-12v-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 04:25:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-wide-boltless-shelving-add-on-60-in-w-x-36-in-d-x-60-in-h-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 04:25:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-flange-shelf-18-inch-l-x-48-inch-w-18-gauge-425-lb-capacity-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:25:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lightalarms-660-0004-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 04:25:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-wide-boltless-shelving-add-on-48-in-w-x-36-in-d-x-84-in-h>: HTTP status code is not handled or not allowed 2026-01-25 04:25:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/office_furniture/literature_mail_sorters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:25:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2522-8bs3-1-drop-in-stainless-steel-3-hole-single-bowl-kitchen-sink>: HTTP status code is not handled or not allowed 2026-01-25 04:25:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transport-storage-cabinet-all-welded-36w-x-24d-x-78h-putty>: HTTP status code is not handled or not allowed 2026-01-25 04:25:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/medical_chart_file_holders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:25:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wm-cascade-free-standing-14-pocket-magazine-rack-mahogany>: HTTP status code is not handled or not allowed 2026-01-25 04:25:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sonnenschein-4245139800-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 04:25:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/wall_corner_bumper_guards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:25:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/appliances/refrigerators_freezers/refrigerators/reach_in_refrigerators already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:25:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elite-counter-height-storage-cabinet-all-welded-36w-x-18d-x-36h-putty>: HTTP status code is not handled or not allowed 2026-01-25 04:25:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elite-bookcase-assembled-36w-x-12d-x-36h-dove-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:25:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wirecraftersrackback-wire-mesh-pallet-rack-enclosure-hinged-door-120x144-797138>: HTTP status code is not handled or not allowed 2026-01-25 04:25:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portalac-lac-txl12072-emergency-light-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 04:25:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/office_furniture/literature_mail_sorters landed on page that is not a product page. 2026-01-25 04:25:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ballistic-bullet-resistant-flotation-vest-level-iii-a-foam-one-size-black>: HTTP status code is not handled or not allowed 2026-01-25 04:25:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/medical_chart_file_holders>: HTTP status code is not handled or not allowed 2026-01-25 04:25:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/wall_corner_bumper_guards landed on page that is not a product page. 2026-01-25 04:25:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/appliances/refrigerators_freezers/refrigerators/reach_in_refrigerators>: HTTP status code is not handled or not allowed 2026-01-25 04:25:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/peg-perego-john-deere-ground-force-tractor-ride-on-toys-battery-12ah-12v>: HTTP status code is not handled or not allowed 2026-01-25 04:26:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/peg-perego-igmc0010us-ride-on-toys-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 04:26:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ps-70-f-charcoal-gray-privacy-screen>: HTTP status code is not handled or not allowed 2026-01-25 04:26:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bombardier-outlander-660-efi-660cc-atv-replacement-battery-2006-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 04:26:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yamaha-yfm250-raptor-se-custom-250cc-atv-replacement-battery-2008-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 04:26:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/five-shelf-heavy-duty-boltless-shelving-add-on-36in-w-x-12in-d-x-96in-h>: HTTP status code is not handled or not allowed 2026-01-25 04:26:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wine-bottle-rack78-bottle-48-x-14-x-54-797147>: HTTP status code is not handled or not allowed 2026-01-25 04:26:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tennis-tutor-r-plus-tennis-ball-machine-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 04:26:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 63809 pages (at 94 pages/min), scraped 32120 items (at 41 items/min) 2026-01-25 04:26:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/match-mate-rookie-tennis-ball-machine-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 04:26:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qvtrk-2-216est-single-unit-track-assembly-kit-2-tracks-216>: HTTP status code is not handled or not allowed 2026-01-25 04:26:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/loading-dock-containment-netting-4-feet-x-26-feet-OHDB426>: HTTP status code is not handled or not allowed 2026-01-25 04:26:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mt182436-2k195-machine-table-with-one-shelf-24w-x-18d-x-36h>: HTTP status code is not handled or not allowed 2026-01-25 04:26:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mtd243630-2k195-machine-table-with-2-shelves-and-a-drawer-36w-x-24d-x-30h>: HTTP status code is not handled or not allowed 2026-01-25 04:26:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/imported-universal-right-angle-irons-ground-finish-10-inch-x-12-inch-x-16-inch>: HTTP status code is not handled or not allowed 2026-01-25 04:26:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/k2008-unf-carbon-eyebolt-plain-pattern-3-4-16-2-shank>: HTTP status code is not handled or not allowed 2026-01-25 04:26:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/footwear_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:26:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m22-metric-eyebolt-shoulder-pattern-m22-x-250>: HTTP status code is not handled or not allowed 2026-01-25 04:26:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbi-sala-nano-lok-twin-leg-hot-work-self-retracting-lifeline-web--steel-rebar-lock-hook-6l>: HTTP status code is not handled or not allowed 2026-01-25 04:26:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/905-case-w-foam-12-1-2l-x-10w-x-6h-black>: HTTP status code is not handled or not allowed 2026-01-25 04:26:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/big-dog-pitbull-1750cc-motorcycle-replacement-battery-1998-2008-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 04:26:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/footwear_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:26:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foam-inserts-3-part-for-925-case>: HTTP status code is not handled or not allowed 2026-01-25 04:26:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aprilia-tuono-50-50cc-motorcycle-replacement-battery-2004-2005-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 04:26:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honda-sh50-scoopy-50cc-motorcycle-replacement-battery-1996-2001-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 04:26:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/930-case-w-padded-divider-19-13-16l-x-16w-x7-5-8h-silver>: HTTP status code is not handled or not allowed 2026-01-25 04:26:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/footwear_accessories landed on page that is not a product page. 2026-01-25 04:26:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/footwear_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:26:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bsm-inc-ada-ss-sink-4-station-with-electronic-faucet-floor-mount-96l-x-20w-x-6d-sink>: HTTP status code is not handled or not allowed 2026-01-25 04:26:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bsm-inc-ada-ss-sink-4-station-with-electronic-faucet-round-tubes-96l-x-20w-x-6d-sink>: HTTP status code is not handled or not allowed 2026-01-25 04:26:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/ink_toner/ribbons_solid_ink already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:26:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/footwear_accessories>: HTTP status code is not handled or not allowed 2026-01-25 04:26:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bsm-inc-ada-ss-hand-sink-1-station-with-electronic-faucet-wall-mount-19-38l-x-17w-x-5d-sink>: HTTP status code is not handled or not allowed 2026-01-25 04:26:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/we-wvhs125-18-460-405tp-db-165-vhs-motor-125hp-1800rpm-405tp-460v-wpi>: HTTP status code is not handled or not allowed 2026-01-25 04:26:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/plumbing_valves/ball_valves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:26:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/right-response-bleeding-control-kit-1-24-persons-15-pieces-fabric-case>: HTTP status code is not handled or not allowed 2026-01-25 04:26:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slide-away-portable-safe-10123-s-fblk-2-gun-cap-12-w-x-10-d-x-4-h-black>: HTTP status code is not handled or not allowed 2026-01-25 04:26:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/footwear_accessories>: HTTP status code is not handled or not allowed 2026-01-25 04:26:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bsm-inc-ada-ss-sink-2-station-with-electronic-faucet-6h-backsplash-60l-x-20w-x-6d-sink>: HTTP status code is not handled or not allowed 2026-01-25 04:26:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/laboratory_heating_cooling/cryogenic_storage already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:26:42 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/ink_toner/ribbons_solid_ink landed on page that is not a product page. 2026-01-25 04:26:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sola-350-cruiser-electric-scooter-replacement-battery-14ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 04:26:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bsm-inc-ada-ss-sink-4-station-with-electronic-faucet-wall-mount-132l-x-20w-x-6d-sink>: HTTP status code is not handled or not allowed 2026-01-25 04:26:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/plumbing/plumbing_valves/ball_valves>: HTTP status code is not handled or not allowed 2026-01-25 04:26:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/currie-phatflyer-electric-scooter-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 04:26:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/504-0232-00-deepwell-microplate-foam-insert-2>: HTTP status code is not handled or not allowed 2026-01-25 04:26:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rad2go-silver-fox-mobility-scooter-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 04:26:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/laboratory_heating_cooling/cryogenic_storage>: HTTP status code is not handled or not allowed 2026-01-25 04:26:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pacesaver-eclipse-mobility-scooter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 04:26:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/summit-swc1966-2-zone-wine-cellar-wglass-door-160-bottle-capacity-23-58-wide>: HTTP status code is not handled or not allowed 2026-01-25 04:26:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/navigator-5153-gt-insulated-overboots-cleated-outsole-l-15h-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:26:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-stp-stl-all-direct-safet-roll-lad-wt-actu-16-serr-stp-wa-ad-062414g>: HTTP status code is not handled or not allowed 2026-01-25 04:26:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd-welded-workbench-48-x-24-open-base--louvered-panel-steel-square-edge>: HTTP status code is not handled or not allowed 2026-01-25 04:26:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ballymore-drivable-mini-scissor-lift-19-platform-500-lb-capac-dmsl-19>: HTTP status code is not handled or not allowed 2026-01-25 04:26:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-stp-stl-all-direct-safet-roll-lad-wt-actu-24-perf-stp-wa-ad-103214p>: HTTP status code is not handled or not allowed 2026-01-25 04:26:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/avigo-off-road-350w-electric-scooter-replacement-battery-7ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 04:27:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11-stp-stl-all-direct-safet-roll-lad-wt-actu-24-perf-stp-wa-ad113214p>: HTTP status code is not handled or not allowed 2026-01-25 04:27:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zap-3-pro-flex-36v-electric-scooter-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 04:27:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-stp-stl-safet-roll-lad-wt-actu-stp-24-perf-stp-wa123214p>: HTTP status code is not handled or not allowed 2026-01-25 04:27:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss14n10a-commercial-kuhl-windowwall-air-conditioner-13500-btu-108-eer>: HTTP status code is not handled or not allowed 2026-01-25 04:27:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/personal-door-organizer-pdo36-for-mbf7236-gun-safes-28-1-2-w-x-61-1-2-h-black>: HTTP status code is not handled or not allowed 2026-01-25 04:27:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vestil-all-ttb-tow-bar-option-all-terrain-pallet-jacks>: HTTP status code is not handled or not allowed 2026-01-25 04:27:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rear-replacement-blade-red-gum-3-18x316x36-716-13-holes-vf-81216>: HTTP status code is not handled or not allowed 2026-01-25 04:27:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/front-replacement-blade-red-gum-2-516x18x34-11-holes-6-slots-vf-81215>: HTTP status code is not handled or not allowed 2026-01-25 04:27:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/z1321-34x16-pb-automatic-draining-wall-hydrant-exposed-ecolotrol-anti-siphon-34-x-16>: HTTP status code is not handled or not allowed 2026-01-25 04:27:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/z1315-34x8-non-freeze-wall-hydrant-wbronze-casing-exposed-34-x-8>: HTTP status code is not handled or not allowed 2026-01-25 04:27:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lc36-large-capacity-36w-security-drawer-ul-approved-bullet-resistant-faceplate-insert>: HTTP status code is not handled or not allowed 2026-01-25 04:27:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 63901 pages (at 92 pages/min), scraped 32160 items (at 40 items/min) 2026-01-25 04:27:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blue-ration-3600-kcal-20-box-dx3600f>: HTTP status code is not handled or not allowed 2026-01-25 04:27:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rescueme-personal-locator-beacon-yellow-1-case-os730s-01261m>: HTTP status code is not handled or not allowed 2026-01-25 04:27:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/signal-mirror-w-lanyard-1-case-dx1502m>: HTTP status code is not handled or not allowed 2026-01-25 04:27:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ema03i-solas-inflate-radar-reflector-for-liferafts-24-dia-1cs-ema03im>: HTTP status code is not handled or not allowed 2026-01-25 04:27:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/em230br-passive-radar-reflector-24x9-3-4-1-case-em230brm>: HTTP status code is not handled or not allowed 2026-01-25 04:27:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aprilia-gulliver-aclc-50cc-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 04:27:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2du1859-cantilever-rack-double-sided-upright-56d-x-18h>: HTTP status code is not handled or not allowed 2026-01-25 04:27:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/econo-strip-no-tape-heavy-duty>: HTTP status code is not handled or not allowed 2026-01-25 04:27:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arctic-cat-zl-800-785cc-snowmobile-replacement-battery-2002-2003-12v-i>: HTTP status code is not handled or not allowed 2026-01-25 04:27:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelate-36w-x-30d-x-69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 04:27:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xdu1884-cantilever-rack-double-sided-upright-84d-x-18h>: HTTP status code is not handled or not allowed 2026-01-25 04:27:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelate-60w-x-14d-x-60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 04:27:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-pole-feet-pole-1-12-id>: HTTP status code is not handled or not allowed 2026-01-25 04:27:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polaris-dragon-rmk-intl-700cc-snowmobile-replacement-battery-2007-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 04:27:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelate-72w-x-14d-x-80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 04:27:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pla-3d-printer-pla-max-filament-175mm-075-kg-black>: HTTP status code is not handled or not allowed 2026-01-25 04:27:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-summit-burton-e-tec-800r-800cc-snowmobile-battery-2016-2017-12v>: HTTP status code is not handled or not allowed 2026-01-25 04:27:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelate-72w-x-30d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 04:27:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-loop-locking-straps-5>: HTTP status code is not handled or not allowed 2026-01-25 04:27:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelate-48w-x-36d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 04:27:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-renegade-backcountry-800r-800cc-snowmobile-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 04:27:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metal-hinged-snap-ring-34>: HTTP status code is not handled or not allowed 2026-01-25 04:27:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-kit-for-electric-pallet-truck-gwk-pe4000xl-lw-fits-crown-model-pe-4000>: HTTP status code is not handled or not allowed 2026-01-25 04:27:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-kit-for-electric-pallet-truck-gwk-wp2000-lw-fits-crown-model-wp-2000>: HTTP status code is not handled or not allowed 2026-01-25 04:27:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelate-30w-x-18d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 04:27:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/VTT-1896-96-W-x-18-D-T-430-18ga-Stainless-Steel-Workbench-w-Galvanized-Legs>: HTTP status code is not handled or not allowed 2026-01-25 04:27:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelate-48w-x-30d-x-69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 04:27:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cushion-tire-lift-lock-combo-ctll-c>: HTTP status code is not handled or not allowed 2026-01-25 04:27:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/excavator-lock-keyed-differently-exl-kd>: HTTP status code is not handled or not allowed 2026-01-25 04:27:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-3l260-v-belt-3-8-x-26>: HTTP status code is not handled or not allowed 2026-01-25 04:27:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ba-fans-black-jack-portable-fan-72-inch>: HTTP status code is not handled or not allowed 2026-01-25 04:27:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-black-epoxy-60w-x-21d-x-60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 04:27:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/fans_blowers/ceiling_fans/high_volume_low_speed_hvls_fans already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:27:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-a51-4l530-v-belt-1-2-x-53>: HTTP status code is not handled or not allowed 2026-01-25 04:27:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-black-epoxy-72w-x-18d-x-60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 04:27:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wood-adjustable-literature-organizer-36-compartment>: HTTP status code is not handled or not allowed 2026-01-25 04:27:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-black-epoxy-24w-x-14d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 04:27:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8210hl-s-mesh-hi-vis-safety-vest-class-2-economy-single-size-3xl-lime>: HTTP status code is not handled or not allowed 2026-01-25 04:27:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8210hl-s-mesh-hi-vis-safety-vest-class-2-economy-single-size-s-lime>: HTTP status code is not handled or not allowed 2026-01-25 04:27:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelite-48w-x-24d-x-80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 04:27:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-black-epoxy-24w-x-24d-x-60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 04:27:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/fans_blowers/ceiling_fans/high_volume_low_speed_hvls_fans>: HTTP status code is not handled or not allowed 2026-01-25 04:27:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-b136-v-belt-5-8-x-139>: HTTP status code is not handled or not allowed 2026-01-25 04:27:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelite-42w-x-21d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 04:28:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-b94-5l970-v-belt-5-8-x-97>: HTTP status code is not handled or not allowed 2026-01-25 04:28:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelite-54w-x-24d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 04:28:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pump-motor-vhecp4106t-3-phase-20-hp-230-460-volts-3540-rpm-60-hz-tefc-256hp>: HTTP status code is not handled or not allowed 2026-01-25 04:28:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pump-motor-jl1205a-1-phase-033-hp-115-230-volts-3450-rpm-60-hz-open-56j>: HTTP status code is not handled or not allowed 2026-01-25 04:28:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-lockout-padlock-keyed-alike-1-1-2-aluminum-steel-yellow-6-pk>: HTTP status code is not handled or not allowed 2026-01-25 04:28:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-c195-v-belt-7-8-x-199>: HTTP status code is not handled or not allowed 2026-01-25 04:28:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esd-3-8-thick-anti-fatigue-floor-mat-3-x-60-full-roll-brown-fm236720bn>: HTTP status code is not handled or not allowed 2026-01-25 04:28:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/locks_lockout_devices/lockout_tagout_devices/lockout_padlocks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:28:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/as-40x-1000x-all-metal-all-optical-glass-lens-student-biological-field-microscope-w-led-light>: HTTP status code is not handled or not allowed 2026-01-25 04:28:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelite-54w-x-21d-x-69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 04:28:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 63999 pages (at 98 pages/min), scraped 32206 items (at 46 items/min) 2026-01-25 04:28:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/97085-glass-county-fair-drinking-jar-165-oz-12-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:28:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/connexion-l-desk-with-electric-height-adjustable-table-slate-and-sandstone>: HTTP status code is not handled or not allowed 2026-01-25 04:28:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelite-36w-x-21d-x-80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 04:28:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-phase-motor-cem3554t-5-15-hp-1760-rpm-145tc-frame-c-face-mount-tefc-575-volts>: HTTP status code is not handled or not allowed 2026-01-25 04:28:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-black-epoxy-24w-x-21d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 04:28:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/locks_lockout_devices/lockout_tagout_devices/lockout_padlocks>: HTTP status code is not handled or not allowed 2026-01-25 04:28:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-motor-230-460-v-60-hp-1775-rpm-3-ph-364t-opsb>: HTTP status code is not handled or not allowed 2026-01-25 04:28:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/light-duty-plastic-wheel-615p-of-6x150-1-3-8-offset-hub-1-2-bore-diamond-tread>: HTTP status code is not handled or not allowed 2026-01-25 04:28:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/k284-front-max-atv-tire-8010-2fm-i-22x800-10-2ply>: HTTP status code is not handled or not allowed 2026-01-25 04:28:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evolution-deluxe-6161-full-brim-hard-hat-hdpe-shell-6-pt-suspension-ratchet-adjustment-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:28:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kilimanjaro-full-brim-hard-hat-hdpe-shell-4-pt-nylon-webbing-cradle-ratchet-adjustment-navy>: HTTP status code is not handled or not allowed 2026-01-25 04:28:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dynamic-rocky-industrial-climbing-helmet-polycarbonate-abs-shell-ratchet-adjustment-light-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:28:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/severe-duty-motor-ecp84110t-4-3-ph-40-hp-460-v-1770-rpm-tefc-324t-frame>: HTTP status code is not handled or not allowed 2026-01-25 04:28:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vornado-miniature-vortexer-green-cup-head>: HTTP status code is not handled or not allowed 2026-01-25 04:28:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1004812-00u0014-gondola-shelving-48w-x-12d-black>: HTTP status code is not handled or not allowed 2026-01-25 04:28:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/key-storage-box-push-button-767-c-with-wall-mount-14-x-6>: HTTP status code is not handled or not allowed 2026-01-25 04:28:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4910-6-wind-towerdesktop-fan-3-speed-110v-platinum>: HTTP status code is not handled or not allowed 2026-01-25 04:28:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-poly-green-60wx14dx60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 04:28:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hvac-motor-em3546t-5g-3-ph-1-hp-575-v-1800-rpm-tefc-143t-frame>: HTTP status code is not handled or not allowed 2026-01-25 04:28:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hvac-motor-em3313t-g-3-ph-10-hp-230-460-v-1770-rpm-opsb-215t-frame>: HTTP status code is not handled or not allowed 2026-01-25 04:28:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/angle-rotor-for-z446-series-4-x-500ml>: HTTP status code is not handled or not allowed 2026-01-25 04:28:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/severe-duty-motor-ecp83584t-5-3-ph-15-hp-575-v-1760-rpm-tefc-145t-frame>: HTTP status code is not handled or not allowed 2026-01-25 04:28:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-round-black-cafeteria-table-with-8-stools-oak-mdf-core-top-black-stools>: HTTP status code is not handled or not allowed 2026-01-25 04:28:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pistol-safe-wmc-4-231208-ws25-4-tier-12-w-x-8-d-x-23-h-8-pistol-cap-key-lock-stealth>: HTTP status code is not handled or not allowed 2026-01-25 04:28:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-poly-green-54wx14dx80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 04:28:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wd-ibc-stair-closed-tread-with-open-riser-36-wide-9-clearance>: HTTP status code is not handled or not allowed 2026-01-25 04:28:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dc-metric-motor-vp3585d-2-hp-1750-rpm-tefc-d90d-frame>: HTTP status code is not handled or not allowed 2026-01-25 04:28:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lifting-magnet-motor-dmg2315-15-hp-1750-rpm-dpfg-259at-frame>: HTTP status code is not handled or not allowed 2026-01-25 04:28:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-motor-230-460-v-025-hp-1140-rpm-3-ph-48-tenv>: HTTP status code is not handled or not allowed 2026-01-25 04:28:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machined-chrome-steel-needle-roller-bearing-w-inner-ring-30-x-47-x-23mm>: HTTP status code is not handled or not allowed 2026-01-25 04:28:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/threshold-ramp-th2432-24l-x-32w-600-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 04:28:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shrouded-padlock-no-40dpf>: HTTP status code is not handled or not allowed 2026-01-25 04:28:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-poly-green-42w-x-21d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 04:28:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drt-l90-lifebuoy-light-solas-med-orange-20-30-inch-liferings>: HTTP status code is not handled or not allowed 2026-01-25 04:28:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wesco-non-telescoping-electric-high-lift-pallet-truck-2200-lbs-27-forks-987675>: HTTP status code is not handled or not allowed 2026-01-25 04:28:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-gallon-waste-convenience-center-gray-8001742>: HTTP status code is not handled or not allowed 2026-01-25 04:28:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-poly-green-54wx18dx69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 04:28:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-poly-green-72w-x-18d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 04:28:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/predator-gloves-impact-2-cutpro-arx-aramid-shell-hpt-palm-coated-yellow-gray-m>: HTTP status code is not handled or not allowed 2026-01-25 04:28:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelon-36w-x-14d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 04:28:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-ac-motor-c-face-kit-field-conversion-36-486gld-odp-36-182-4c-frame>: HTTP status code is not handled or not allowed 2026-01-25 04:28:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelon-54w-x-14d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 04:28:57 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/gloves_sleeves/cut_resistant_gloves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:28:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electric-pump-1-5-hp-3-gallon-4-way-3-position-solenoid-locking-valve-10-pendant>: HTTP status code is not handled or not allowed 2026-01-25 04:28:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-ac-motor-c-face-kit-field-conversion-35-8764-odp-35-56c-143-5tc-frame>: HTTP status code is not handled or not allowed 2026-01-25 04:29:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/awn-h23-6cop-standing-seam-awning-6-8-w-x-3d-x-2h-copper>: HTTP status code is not handled or not allowed 2026-01-25 04:29:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-nexelon-30w-x-24d-x-69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 04:29:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelon-30w-x-18d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 04:29:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-8482-bottles-cans-recycling-lid-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:29:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-12-foot-rectangle-cafeteria-table-with-12-stools-gray-mdf-core-top-blue-stools>: HTTP status code is not handled or not allowed 2026-01-25 04:29:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelon-42w-x-14d-x-69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 04:29:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/gloves_sleeves/cut_resistant_gloves>: HTTP status code is not handled or not allowed 2026-01-25 04:29:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2517-x-1-11-16-1-11-16-x-24-2517-series-tapered-locking-steel-bushing-1-11-16-bore>: HTTP status code is not handled or not allowed 2026-01-25 04:29:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/halo-go-rechargeable-led-magnifier-table-lamp-3-1-2-dia-lens-white>: HTTP status code is not handled or not allowed 2026-01-25 04:29:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electra-led-floor-lamp-satin-black>: HTTP status code is not handled or not allowed 2026-01-25 04:29:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 64098 pages (at 99 pages/min), scraped 32251 items (at 45 items/min) 2026-01-25 04:29:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drip-covers-for-field-conversion-fcd39-39-254-256>: HTTP status code is not handled or not allowed 2026-01-25 04:29:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/constant-velocity-blower-cooling-conversion-kit-blwm16-f-3-ph-230-380-460v-404tc-405tc>: HTTP status code is not handled or not allowed 2026-01-25 04:29:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alarm-clock-black>: HTTP status code is not handled or not allowed 2026-01-25 04:29:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelon-72w-x-18d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 04:29:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waring-wct702-toaster-chrome-2-slice-120v>: HTTP status code is not handled or not allowed 2026-01-25 04:29:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precision-series-174-782329-imprinted-360-stainless-steel-receptacle-with-open-lid-45-gallon>: HTTP status code is not handled or not allowed 2026-01-25 04:29:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/celltreat-scientific-products-150l-pipette-mla-dtipper-silver>: HTTP status code is not handled or not allowed 2026-01-25 04:29:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mounted-gear-storage-rack-locker-rrwm-2-20-two-20-sections-red>: HTTP status code is not handled or not allowed 2026-01-25 04:29:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/celltreat-scientific-products-200l-pipette-mla-dtipper-lg-tip-red>: HTTP status code is not handled or not allowed 2026-01-25 04:29:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90002-grill-scraper-4-count>: HTTP status code is not handled or not allowed 2026-01-25 04:29:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90044-grill-brush-hand-held-12-count>: HTTP status code is not handled or not allowed 2026-01-25 04:29:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/we-mc440-19375-vhs-alternate-coupling-bore-size-19375-frame-444tp-or-445tp>: HTTP status code is not handled or not allowed 2026-01-25 04:29:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/r035c-5-air-sander-with-hook--loop-316-orbit-dia-13000-rpm>: HTTP status code is not handled or not allowed 2026-01-25 04:29:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beverage-napkins-1-ply-9-1-2-x-9-1-2-4000-carton-white>: HTTP status code is not handled or not allowed 2026-01-25 04:29:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/celltreat-scientific-products-5l-pipette-mla-micro-dtipper-silver>: HTTP status code is not handled or not allowed 2026-01-25 04:29:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-french-roast-coffee-decaffeinated-arabica-bean-25-oz-18-carton>: HTTP status code is not handled or not allowed 2026-01-25 04:29:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/recycled-lunch-napkins-1-ply-quad-fold-12-1-2-x-11-1-4-400-pack-white>: HTTP status code is not handled or not allowed 2026-01-25 04:29:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/office-partition-blue-24w-x-42h-277660BL>: HTTP status code is not handled or not allowed 2026-01-25 04:29:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/celltreat-scientific-products-10ml-wobblenot-serological-pipet-paperplastic-bag-sterile-200-pa>: HTTP status code is not handled or not allowed 2026-01-25 04:29:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jurni-flip-top-training-table-with-casters-60l-x-24w-x-29h-asian-night>: HTTP status code is not handled or not allowed 2026-01-25 04:29:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/topoge-series-180-handhole-gasket-412-x-612-x-114-black-rubber-elliptical-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:29:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rcl-306-wobble-head-coil>: HTTP status code is not handled or not allowed 2026-01-25 04:29:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-cal-diamond-plate-metallic-pvc-flooring-beige-2-5mm-x-4-x-25>: HTTP status code is not handled or not allowed 2026-01-25 04:29:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/topoge-series-180-handhole-gasket-314-x-434-x-12-black-rubber-obround-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:29:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s60-water-based-epoxy-maintenance-coating-1-gallon-can-satin-activator>: HTTP status code is not handled or not allowed 2026-01-25 04:29:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/web-winch-side-mount-low-profile>: HTTP status code is not handled or not allowed 2026-01-25 04:29:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/topoge-series-2000-handhole-gasket-312-x-5-x-916-graphite-obround-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:29:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-e-a-cam-strap-assembly-with-spring-actuated-fitting-20>: HTTP status code is not handled or not allowed 2026-01-25 04:29:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/topoge-series-180-manway-gasket-1012-x-1412-x-134-black-rubber-elliptical-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:29:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/topoge-series-180-handhole-gasket-212-x-334-x-12-black-rubber-obround-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:29:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wetline-kit-smwlk70admcw-70-gallon-clockwise-rotation-direct-wetline-kit>: HTTP status code is not handled or not allowed 2026-01-25 04:29:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dxs2153-standard-anvil-2-piece-set-12--34-drive>: HTTP status code is not handled or not allowed 2026-01-25 04:29:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dxs2153-10-extended-anvil-attachment-12-drive>: HTTP status code is not handled or not allowed 2026-01-25 04:29:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cl-focus-ii-disc-26-walk-behind-midsize-scrubber-disc-05398a>: HTTP status code is not handled or not allowed 2026-01-25 04:29:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/for-counterclockwise-rotation-direct-mount>: HTTP status code is not handled or not allowed 2026-01-25 04:29:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/fall_protection/harnesses already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:29:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hospitality-base-cabinet-with-2-doors-36w-x-25d-x-39h-mahogany>: HTTP status code is not handled or not allowed 2026-01-25 04:29:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jurni-multi-purpose-table-with-t-legs--casters-60l-x-24w-x-29h-asian-night>: HTTP status code is not handled or not allowed 2026-01-25 04:29:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jurni-multi-purpose-table-with-t-legs--casters-72l-x-24w-x-29h-asian-night>: HTTP status code is not handled or not allowed 2026-01-25 04:29:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metro-mycart-two-shelf-utility-cart-with-chrome-plated-posts-28x23-shelves-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:29:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/battery-charger-for-roam-battery>: HTTP status code is not handled or not allowed 2026-01-25 04:29:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mechanix-wear-original174-tactical-gloves-synthetic-leather-wtrekdry153-covert-xl>: HTTP status code is not handled or not allowed 2026-01-25 04:29:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mechanix-wear-mpact174-tactical-gloves-synthetic-leatherd30174-padding-multicam-large>: HTTP status code is not handled or not allowed 2026-01-25 04:29:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/combination-nipple-hcn075-3-4-inch-75-npt-male-thread-75-inch-hose-id-3125-inchl>: HTTP status code is not handled or not allowed 2026-01-25 04:29:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-heated-vest-battery-for-300-hv100-5v-black>: HTTP status code is not handled or not allowed 2026-01-25 04:29:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/worldwide-fm12-18-56-farm-duty-motor-12hp-1800rpm-56-115230v-tefc>: HTTP status code is not handled or not allowed 2026-01-25 04:29:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12v-dc-motor-m3300-bi-rotational>: HTTP status code is not handled or not allowed 2026-01-25 04:29:56 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/fall_protection/harnesses landed on page that is not a product page. 2026-01-25 04:29:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/modular_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:29:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-element-u1l6fe-25-micron>: HTTP status code is not handled or not allowed 2026-01-25 04:29:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wsmr5-91-shaft-mount-reducer-size-5-91-ratio-2-1516-tapered-bore>: HTTP status code is not handled or not allowed 2026-01-25 04:30:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2wtbk-158-tapered-bushing-kit-1-58-fits-reducer-styles-smr2wsmr2>: HTTP status code is not handled or not allowed 2026-01-25 04:30:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-bolt-clamp-hc150-2-1-8-inch-nominal-dia-186-inch-min-dia-217-inch-max-dia>: HTTP status code is not handled or not allowed 2026-01-25 04:30:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wing-type-quick-detach-hydraulic-coupler-qdwc12-3-4-inch-npt-coupler-18-flow-gpm>: HTTP status code is not handled or not allowed 2026-01-25 04:30:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9oz-30-sateen-cotton-jacket-navy-3xl>: HTTP status code is not handled or not allowed 2026-01-25 04:30:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/storage_panels/tool_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:30:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/floor_mats/modular_mats>: HTTP status code is not handled or not allowed 2026-01-25 04:30:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9oz-30-sateen-cotton-jacket-royal-blue-2xl>: HTTP status code is not handled or not allowed 2026-01-25 04:30:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reservoir-accessory-tfa005715-filler-strainer-breather-cap-assy>: HTTP status code is not handled or not allowed 2026-01-25 04:30:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 64195 pages (at 97 pages/min), scraped 32292 items (at 41 items/min) 2026-01-25 04:30:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-head-plug-h3159x4-1-4-inch-male-pipe-thread>: HTTP status code is not handled or not allowed 2026-01-25 04:30:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sa-whiffle-typical-3-double-48-spectrumblue>: HTTP status code is not handled or not allowed 2026-01-25 04:30:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/modular_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:30:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/storage_panels/louvered_panels_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:30:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-foam-mattress-7028x21-for-7028-7030-9200-bassinets-12w-x-25-3-4l>: HTTP status code is not handled or not allowed 2026-01-25 04:30:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-8-industrial-grade-particleboard-decking-for-hallowell-shelving-48x24>: HTTP status code is not handled or not allowed 2026-01-25 04:30:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:30:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:30:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/storage_panels/tool_storage_racks>: HTTP status code is not handled or not allowed 2026-01-25 04:30:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4000-1-1-2-ball-valve-flanged-1-1-2-cast-iron>: HTTP status code is not handled or not allowed 2026-01-25 04:30:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/worldwide-electric-winl67-5-1-145tc-helical-inline-speed-reducer-145tc-input-flange-5-1-ratio>: HTTP status code is not handled or not allowed 2026-01-25 04:30:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/casual-duck-jacket-with-reflective-tape-2xl-orange>: HTTP status code is not handled or not allowed 2026-01-25 04:30:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:30:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:30:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/floor_mats/modular_mats>: HTTP status code is not handled or not allowed 2026-01-25 04:30:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/storage_panels/louvered_panels_accessories>: HTTP status code is not handled or not allowed 2026-01-25 04:30:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats>: HTTP status code is not handled or not allowed 2026-01-25 04:30:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts>: HTTP status code is not handled or not allowed 2026-01-25 04:30:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/wall_corner_bumper_guards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:30:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-long-sleeve-button-down-work-shirt-3xl-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:30:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ds-147213b-shrink-wrap-14w-x-213l-7mil-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:30:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 04:30:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats>: HTTP status code is not handled or not allowed 2026-01-25 04:30:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:30:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:30:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4165427-78h-su-jumbo-cabt-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:30:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/snap-button-down-long-sleeve-work-shirt-l-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:30:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/laptop_tablet_stands already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:30:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/desk_accessories/document_holders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:30:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-visibility-athletic-performance-t-shirt-flame-resistant-6-oz-l-tall-yellow>: HTTP status code is not handled or not allowed 2026-01-25 04:30:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sit-stand-mobile-laptop-workstation-glass>: HTTP status code is not handled or not allowed 2026-01-25 04:30:34 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/wall_corner_bumper_guards landed on page that is not a product page. 2026-01-25 04:30:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-locker-accessory-individual-slope-top-15w-x-21d-x-7h-729-parchment>: HTTP status code is not handled or not allowed 2026-01-25 04:30:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-locker-accessory-individual-slope-top-18w-x-24d-x-8h-729-parchment>: HTTP status code is not handled or not allowed 2026-01-25 04:30:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/underbed-truck-bx-black-steel-standard-6-2-cu-ft-536-5-02>: HTTP status code is not handled or not allowed 2026-01-25 04:30:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:30:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/laptop_tablet_stands already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:30:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/shirts>: HTTP status code is not handled or not allowed 2026-01-25 04:30:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats>: HTTP status code is not handled or not allowed 2026-01-25 04:30:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/work-benches/bench_tops_accessories/workbench_tool_balancers_holders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:30:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/laptop_tablet_stands>: HTTP status code is not handled or not allowed 2026-01-25 04:30:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/basic_supplies/desk_accessories/document_holders>: HTTP status code is not handled or not allowed 2026-01-25 04:30:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:30:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/truck_boxes_utility_boxes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:30:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-side-truck-bx-black-alum-driver-side-11-1-cu-ft-cap-390-5-02>: HTTP status code is not handled or not allowed 2026-01-25 04:30:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-29w-x-36l-black>: HTTP status code is not handled or not allowed 2026-01-25 04:30:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-28w-x-30l-black>: HTTP status code is not handled or not allowed 2026-01-25 04:30:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 04:30:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:30:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/laptop_tablet_stands landed on page that is not a product page. 2026-01-25 04:30:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/work-benches/bench_tops_accessories/workbench_tool_balancers_holders>: HTTP status code is not handled or not allowed 2026-01-25 04:30:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-42w-x-32l-navy>: HTTP status code is not handled or not allowed 2026-01-25 04:30:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 04:30:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/truck_boxes_utility_boxes>: HTTP status code is not handled or not allowed 2026-01-25 04:30:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:30:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:30:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-28w-x-30l-green>: HTTP status code is not handled or not allowed 2026-01-25 04:30:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rclo-sm-4bt-portion-cup-and-lid-organizer-4-sections-countertop>: HTTP status code is not handled or not allowed 2026-01-25 04:30:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:30:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carson-gn-88-helpinghands-soldering-craft-magnifier>: HTTP status code is not handled or not allowed 2026-01-25 04:30:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:30:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/activity-tables-48-l-x-48-w-standard-height-round-gray-granite>: HTTP status code is not handled or not allowed 2026-01-25 04:30:56 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 04:30:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:30:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dewalt-dw235g-12-vsr-drill-0-850-rpm-534210>: HTTP status code is not handled or not allowed 2026-01-25 04:30:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/md-vertical-unit-heater-v347s01-247000-btu-4820-cfm-115v>: HTTP status code is not handled or not allowed 2026-01-25 04:30:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:31:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 04:31:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 04:31:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:31:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 04:31:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classic-pro-ctl-bandsaw-blade-11-long-x-1-wide-46-tpi-x-0035-thick>: HTTP status code is not handled or not allowed 2026-01-25 04:31:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-outlet-tri-tap-rubber-adapter-gg-03418beb-beige>: HTTP status code is not handled or not allowed 2026-01-25 04:31:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 04:31:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 04:31:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cardboard-single-wall-corrugated-box-26-x-20-x-8-200lb-test-ect-32-kraft-15-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:31:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 04:31:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 64271 pages (at 76 pages/min), scraped 32315 items (at 23 items/min) 2026-01-25 04:31:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cardboard-single-wall-corrugated-box-16-x-12-x-16-200lb-test-ect-32-kraft-25-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:31:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manchester-non-locking-stucco-column-mailbox-in-slate-gray-color-column-only>: HTTP status code is not handled or not allowed 2026-01-25 04:31:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 04:31:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-double-wall-cardboard-corrugated-box-20-x-20-x-10-275-lb-ect-48-10-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:31:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-inch-drive-impact-socket-20-mm-6-point-1-5-8-inch-long>: HTTP status code is not handled or not allowed 2026-01-25 04:31:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-54w-x-33l-black>: HTTP status code is not handled or not allowed 2026-01-25 04:31:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/334-piece-master-advanced-maintenance-set>: HTTP status code is not handled or not allowed 2026-01-25 04:31:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:31:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/satin-xl-combination-wrench-2-9-16-inch-12-point>: HTTP status code is not handled or not allowed 2026-01-25 04:31:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/j3016-oh-needle-bearing-drawn-cup-caged-oil-hole-bore-47625mm>: HTTP status code is not handled or not allowed 2026-01-25 04:31:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:31:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-52w-x-29l-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:31:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:31:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jumbo-chenille-stems-6mm-x-12-inchl-assorted-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:31:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/arts_crafts_supplies/paint_paint_brushes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:31:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b188-oh-needle-bearing-drawn-cup-full-complement-oil-hole-bore-28575mm>: HTTP status code is not handled or not allowed 2026-01-25 04:31:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hk1416-2rs-needle-bearing-drawn-cup-caged-metric-2-seals-bore-13995mm>: HTTP status code is not handled or not allowed 2026-01-25 04:31:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/arts_crafts_supplies/craft_basics already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:31:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 04:31:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washable-markers-classpack-fine-line-10-assorted-colors-200-box>: HTTP status code is not handled or not allowed 2026-01-25 04:31:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 04:31:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vem31112-75hp-1800rpm-56c-frame-3ph-208-230-460v-odp-c-face-footless-premium-efficiency>: HTTP status code is not handled or not allowed 2026-01-25 04:31:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vem31120-15hp-3600rpm-56c-frame-3ph-230-460v-odp-c-face-footless-premium-efficiency>: HTTP status code is not handled or not allowed 2026-01-25 04:31:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:31:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-work-coveralls-40-regular-red>: HTTP status code is not handled or not allowed 2026-01-25 04:31:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/arts_crafts_supplies/paint_paint_brushes>: HTTP status code is not handled or not allowed 2026-01-25 04:31:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/educational_materials_books/instructional_materials/flash_cards_educational_games already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:31:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/artista-ii-washable-tempera-paint-nontoxic-16-oz-green>: HTTP status code is not handled or not allowed 2026-01-25 04:31:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/math-division-0-12-flash-cards-3-inch-x-6-inch-91-cards-box>: HTTP status code is not handled or not allowed 2026-01-25 04:31:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-44w-x-34l-black>: HTTP status code is not handled or not allowed 2026-01-25 04:31:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 04:31:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-56w-x-33l-black>: HTTP status code is not handled or not allowed 2026-01-25 04:31:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:31:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/psm-60-spiralmixer-102qt-capacity-twinmotor-2speed-2-3-4hp-208v>: HTTP status code is not handled or not allowed 2026-01-25 04:31:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/post-medical-15-gallon-leak-tight-sharps-container-with-locking-screw-cap-translucent-22-cs>: HTTP status code is not handled or not allowed 2026-01-25 04:31:39 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/arts_crafts_supplies/craft_basics landed on page that is not a product page. 2026-01-25 04:31:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 04:31:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-36w-x-30l-black>: HTTP status code is not handled or not allowed 2026-01-25 04:31:42 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/educational_materials_books/instructional_materials/flash_cards_educational_games landed on page that is not a product page. 2026-01-25 04:31:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hanging-feeder-with-hardware-hg-3-3>: HTTP status code is not handled or not allowed 2026-01-25 04:31:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-50w-x-31l-tan>: HTTP status code is not handled or not allowed 2026-01-25 04:31:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-12-gauge-pegboard-shelf-cabinet-36w-x-24d-x-78h>: HTTP status code is not handled or not allowed 2026-01-25 04:31:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 04:31:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:31:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:31:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:31:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-58w-x-31l-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:31:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-ghs2012alv2-oxidizer-ghs-label-vinyl-2-x-2-250-roll>: HTTP status code is not handled or not allowed 2026-01-25 04:31:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:31:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wedge-locking-washer-carbon-steel-zinc-flake-coated-3-4-pkg-of-4>: HTTP status code is not handled or not allowed 2026-01-25 04:31:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-work-coverall-with-leg-zippers-54-regular-navy>: HTTP status code is not handled or not allowed 2026-01-25 04:31:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:31:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-kingportable-reel-rack-24-28-dia-coilsprecuation-blue-798017>: HTTP status code is not handled or not allowed 2026-01-25 04:31:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/31604-scalloped-slicer-high-carbon-steel-stamped-white-handle-10-l>: HTTP status code is not handled or not allowed 2026-01-25 04:31:57 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 04:31:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 04:31:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15271-paring-knife-high-carbon-steel-stamped-3-1-4-l>: HTTP status code is not handled or not allowed 2026-01-25 04:31:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:32:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-36w-x-32l-dark-navy>: HTTP status code is not handled or not allowed 2026-01-25 04:32:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 04:32:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 04:32:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:32:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/r75i-tas-135-2303-rotary-screw-air-compressor-3-phase-230-volts-10hp-120-gal>: HTTP status code is not handled or not allowed 2026-01-25 04:32:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 04:32:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-work-coveralls-44-regular-red>: HTTP status code is not handled or not allowed 2026-01-25 04:32:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-mesh-baskets-15x10x8-price-each-for-qty-1-2>: HTTP status code is not handled or not allowed 2026-01-25 04:32:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:32:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-8482-thermoplastic-32-gallon-mesh-receptacle-w-flat-lid-black>: HTTP status code is not handled or not allowed 2026-01-25 04:32:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 04:32:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:32:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-work-coverall-with-leg-zippers-50-tall-navy>: HTTP status code is not handled or not allowed 2026-01-25 04:32:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 64352 pages (at 81 pages/min), scraped 32342 items (at 27 items/min) 2026-01-25 04:32:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 04:32:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-work-coverall-with-leg-zippers-58-tall-navy>: HTTP status code is not handled or not allowed 2026-01-25 04:32:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/utility-jeans-basket-weave-flame-resistant-8-oz-36w-x-32l-green>: HTTP status code is not handled or not allowed 2026-01-25 04:32:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:32:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-door-magnetic-weatherstrip-01636-brown-36-inch-x-81-inch>: HTTP status code is not handled or not allowed 2026-01-25 04:32:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 04:32:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-density-foam-tape-open-cell-02055-gray-3-16-inch-x-3-8-inch-x-17-foot>: HTTP status code is not handled or not allowed 2026-01-25 04:32:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 04:32:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/en-7-ss-drop-forged-eye-nut-5-8-11-style-a-304-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 04:32:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-work-pants-reflective-material-flame-resistant-38w-x-32l-navy>: HTTP status code is not handled or not allowed 2026-01-25 04:32:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-x-1-5-8-phillips-bugle-head-drywall-screw-steel-full-thread-coarse-pkg-of-1-lb>: HTTP status code is not handled or not allowed 2026-01-25 04:32:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-x-2-phillips-bugle-head-drywall-screw-steel-full-thread-coarse-pkg-of-1-lb>: HTTP status code is not handled or not allowed 2026-01-25 04:32:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 04:32:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-westex-42w-x-32l-tan>: HTTP status code is not handled or not allowed 2026-01-25 04:32:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wallbase-dry-back-75960-120-footl-x-2-1-2-inchw-almond>: HTTP status code is not handled or not allowed 2026-01-25 04:32:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zinsser-b-i-n-shellac-base-primer-spray-white-13-oz-can-1008>: HTTP status code is not handled or not allowed 2026-01-25 04:32:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-wire-shelving-add-on-24-inch-w-x-24-inch-d-x-54-inch-h>: HTTP status code is not handled or not allowed 2026-01-25 04:32:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-wire-shelving-30-inch-w-x-18-inch-d-x-54-inch-h>: HTTP status code is not handled or not allowed 2026-01-25 04:32:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-work-pants-reflective-material-flame-resistant-42w-x-unhemmed-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:32:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fa500-3cc-cross-cut-industrial-shredder-120-130-sheet>: HTTP status code is not handled or not allowed 2026-01-25 04:32:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shredder-oil-16oz-pint-bottles-12-case>: HTTP status code is not handled or not allowed 2026-01-25 04:32:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/assy-es-s2442-ramp-trim-kit-fits-24-x-42-mat>: HTTP status code is not handled or not allowed 2026-01-25 04:32:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cargo-pants-with-double-reinforced-knees-flame-resistant-50w-x-unhemmed-black>: HTTP status code is not handled or not allowed 2026-01-25 04:32:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:32:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/x-o-rust-oil-base-dtm-enamel-gloss-finish-orange-gallon-802702>: HTTP status code is not handled or not allowed 2026-01-25 04:32:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:32:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-60w-x-14d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 04:32:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:32:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cargo-pants-with-double-reinforced-knees-flame-resistant-38w-x-32l-black>: HTTP status code is not handled or not allowed 2026-01-25 04:32:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:32:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-9a-3ph-remote-start-terminals-start-stop-100-250v-5-7-7-6a>: HTTP status code is not handled or not allowed 2026-01-25 04:32:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/start-right-interior-latex-vapor-barrier-primer-gallon-775836>: HTTP status code is not handled or not allowed 2026-01-25 04:32:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-9a-1ph-direct-online-start-stop-100-250v-3-1-4-2a>: HTTP status code is not handled or not allowed 2026-01-25 04:32:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 04:32:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 04:32:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-9a-1ph-remote-start-reset-button-24-60v-2-3-3-1a>: HTTP status code is not handled or not allowed 2026-01-25 04:32:46 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 04:32:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manitowoc-ice-rns-12a-ice-maker-water-disp-countertop-nugget-style-air-cooled-self-contained>: HTTP status code is not handled or not allowed 2026-01-25 04:32:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 04:32:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-door-for-welded-shelv-15-in-42-in-8502-3-01>: HTTP status code is not handled or not allowed 2026-01-25 04:32:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eco-premier-carpet-tile-22157314000-geometric-18l-x-18w-x-14h-grey-ash-12pk>: HTTP status code is not handled or not allowed 2026-01-25 04:32:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eco-premier-carpet-tile-22187514000-diamond-18l-x-18w-x-14h-chestnut-brown-12pk>: HTTP status code is not handled or not allowed 2026-01-25 04:32:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shrink-film-roll-16-in-x-3500-ft-75-gauge>: HTTP status code is not handled or not allowed 2026-01-25 04:32:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-12a-1ph-remote-start-terminals-start-stop-100-250v-4-2-5-7a>: HTTP status code is not handled or not allowed 2026-01-25 04:32:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wesco-174-plastic-box-truck-20-bushel-red-272526-5-casters>: HTTP status code is not handled or not allowed 2026-01-25 04:32:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_storage_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:32:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hvac-mech-van-pack-mid-roof-ford-trans-148-wb-600-8213l>: HTTP status code is not handled or not allowed 2026-01-25 04:32:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lab-tables-30-w-x-72-l-x-36-h-fusion-maple>: HTTP status code is not handled or not allowed 2026-01-25 04:32:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sc-fcc-1-speedyfoam-coil-cleaner-concentrate-non-acidic-alkaline-1-gallon>: HTTP status code is not handled or not allowed 2026-01-25 04:32:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-16a-1ph-direct-online-start-stop-100-250v-5-7-7-6a>: HTTP status code is not handled or not allowed 2026-01-25 04:33:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deep-groove-ball-bearings-metric-6012-zz-2-metal-shields-light-duty-60mm-bore-95mm-od>: HTTP status code is not handled or not allowed 2026-01-25 04:33:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deep-groove-ball-bearings-metric-6011-2rs-2-rubber-seals-light-duty-55mm-bore-90mm-od>: HTTP status code is not handled or not allowed 2026-01-25 04:33:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-row-angular-contact-bearings-5302-2rs-2-rubber-seals-heavy-duty-15mm-bore-42mm-od>: HTTP status code is not handled or not allowed 2026-01-25 04:33:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:33:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-row-angular-contact-bearings-5306-2rs-2-rubber-seals-heavy-duty-30mm-bore-72mm-od>: HTTP status code is not handled or not allowed 2026-01-25 04:33:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deep-groove-ball-bearings-metric-6008-open-light-duty-40mm-bore-68mm-od>: HTTP status code is not handled or not allowed 2026-01-25 04:33:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-row-angular-contact-bearings-5208-zz-2-metal-shields-medium-duty-40mm-bore-80mm-od>: HTTP status code is not handled or not allowed 2026-01-25 04:33:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-26a-3ph-start-stop-24-60v-13-16a>: HTTP status code is not handled or not allowed 2026-01-25 04:33:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 64441 pages (at 89 pages/min), scraped 32380 items (at 38 items/min) 2026-01-25 04:33:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-26a-3ph-remote-start-terminals-reset-button-250-500v-10-13a>: HTTP status code is not handled or not allowed 2026-01-25 04:33:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-funnel-cake-mold-with-base-plate>: HTTP status code is not handled or not allowed 2026-01-25 04:33:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-26a-1ph-remote-start-start-stop-24-60v-16-20a>: HTTP status code is not handled or not allowed 2026-01-25 04:33:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rhino-4200-label-printer>: HTTP status code is not handled or not allowed 2026-01-25 04:33:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-38a-3ph-remote-start-terminals-start-stop-250-500v-24-29a>: HTTP status code is not handled or not allowed 2026-01-25 04:33:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-38a-3ph-direct-online-reset-button-100-250v-16-20a>: HTTP status code is not handled or not allowed 2026-01-25 04:33:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-38a-3ph-direct-online-reset-button-100-250v-20-24a>: HTTP status code is not handled or not allowed 2026-01-25 04:33:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_cabinets_drawers?token=null already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:33:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-38a-3ph-remote-start-terminals-start-stop-250-500v-35-40a>: HTTP status code is not handled or not allowed 2026-01-25 04:33:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/489026-carriage-bolt-1-4-20-x-2-1-2-round-head-steel-zinc-grade-a-ft-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 04:33:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/313070-heavy-hex-nut-3-8-16-steel-hot-dip-galvanized-unc-astm-a194-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 04:33:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/812050-nylon-insert-lock-nut-m10-x-1-50-steel-zinc-unc-class-8-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 04:33:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-38a-1ph-remote-start-start-stop-100-250v-35-40a>: HTTP status code is not handled or not allowed 2026-01-25 04:33:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/d383rb-osha-sign-danger-confined-space-do-not-enter-10-x-14-whiteredblack>: HTTP status code is not handled or not allowed 2026-01-25 04:33:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v6c-extension-spring-05-od-x-00625-wire-dia-x-4-oal-546-lbs-in-mbhd>: HTTP status code is not handled or not allowed 2026-01-25 04:33:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/d38rb-osha-sign-danger-flammable-liquids-10-x-14-whiteredblack>: HTTP status code is not handled or not allowed 2026-01-25 04:33:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hms14l-rtk-labels-write-on-nfpa-4-x-4-redyellowwhiteblue-psp>: HTTP status code is not handled or not allowed 2026-01-25 04:33:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-52a-3ph-separate-coil-voltage-start-stop-100-250v-25-33a>: HTTP status code is not handled or not allowed 2026-01-25 04:33:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_cabinets_drawers?token=null>: HTTP status code is not handled or not allowed 2026-01-25 04:33:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waterproof-maintenance-tarp-28432-5l-x-5w->: HTTP status code is not handled or not allowed 2026-01-25 04:33:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/d18pb-osha-sign-danger-diesel-no-smoking-10-x-14-whiteredblack>: HTTP status code is not handled or not allowed 2026-01-25 04:33:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precision-compression-spring-024-od-x-0022-x-0562-music-wire-pkg-of-5>: HTTP status code is not handled or not allowed 2026-01-25 04:33:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/travira-60-square-dining-table-tekwood-natural>: HTTP status code is not handled or not allowed 2026-01-25 04:33:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m81pb-sign-emergency-eye-wash-10-x-14-whitegreen>: HTTP status code is not handled or not allowed 2026-01-25 04:33:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bluff-174-20cr8472-forkli-container-ramp-84-x-72-20-000-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 04:33:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-65a-3ph-remote-start-reset-button-24-60v-44-53a>: HTTP status code is not handled or not allowed 2026-01-25 04:33:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c359r-osha-sign-caution-low-overhead-clearance-7-x-10-yellowblack>: HTTP status code is not handled or not allowed 2026-01-25 04:33:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tm228j-traffic-sign-shipping-receiving-18-x-24-whiteblack>: HTTP status code is not handled or not allowed 2026-01-25 04:33:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vsm-resin-fiber-disc-149145-ceramic-5-x-7-8-80-grit>: HTTP status code is not handled or not allowed 2026-01-25 04:33:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rpt1g-tags-danger-do-not-operate-6-x-3-whiteredblack-25pk>: HTTP status code is not handled or not allowed 2026-01-25 04:33:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hibiscus-with-fluted-vase-silk-flower-arrangement>: HTTP status code is not handled or not allowed 2026-01-25 04:33:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-80a-1ph-reset-button-24-60v-57-68a>: HTTP status code is not handled or not allowed 2026-01-25 04:33:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vsm-abrasive-belt-214927-zirconia-alumina-37-x-60-150-grit>: HTTP status code is not handled or not allowed 2026-01-25 04:33:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vsm-abrasive-belt-282010-ceramic-2-x-132-100-grit>: HTTP status code is not handled or not allowed 2026-01-25 04:33:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vsm-abrasive-belt-282831-ceramic-6-x-48-60-grit>: HTTP status code is not handled or not allowed 2026-01-25 04:33:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vsm-abrasive-belt-291426-ceramic-4-x-36-40-grit>: HTTP status code is not handled or not allowed 2026-01-25 04:33:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vsm-abrasive-belt-307822-ceramic-3-4-x-12-50-grit>: HTTP status code is not handled or not allowed 2026-01-25 04:33:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vsm-abrasive-belt-57480-aluminum-oxide-4-x-24-50-grit>: HTTP status code is not handled or not allowed 2026-01-25 04:33:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/star-single-wire-surveillance-earpiece-kenwood-hyt-relm-radios>: HTTP status code is not handled or not allowed 2026-01-25 04:33:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vsm-abrasive-belt-97194-zirconia-alumina-37-x-60-40-grit>: HTTP status code is not handled or not allowed 2026-01-25 04:33:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paraclipse-60-day-capture-cartridges-insect-inn-ultra-i-ii-12-cs-73836>: HTTP status code is not handled or not allowed 2026-01-25 04:33:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/folding-mezzanine-ladder-96h>: HTTP status code is not handled or not allowed 2026-01-25 04:33:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/diemaster-2-bi-metal-bandsaw-coil-100-long-05w-14-18-tpi-025-thick>: HTTP status code is not handled or not allowed 2026-01-25 04:33:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-sae-deep-impact-socket-set-2641-19-piece-alloy-steel>: HTTP status code is not handled or not allowed 2026-01-25 04:34:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pioneer-mod-3mo180-bn-single-lever-bathroom-faucet-with-pop-up-pvd-brushed-nickel>: HTTP status code is not handled or not allowed 2026-01-25 04:34:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/54-ibc-stairway-stair36i-8-54>: HTTP status code is not handled or not allowed 2026-01-25 04:34:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/63-ibc-stairway-stair36i-9-63>: HTTP status code is not handled or not allowed 2026-01-25 04:34:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fao-16-unit-first-aid-kit-25-person-ansi-compliant-class-a-plastic-case>: HTTP status code is not handled or not allowed 2026-01-25 04:34:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/019151m-19-inch-series-2000-tool-box-with-2-3-tray>: HTTP status code is not handled or not allowed 2026-01-25 04:34:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/121-ibc-stairway-stair36i-18-121>: HTTP status code is not handled or not allowed 2026-01-25 04:34:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/133-osha-stairway-stair36o-15-133>: HTTP status code is not handled or not allowed 2026-01-25 04:34:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-60-inch-w-x-21-inch-d-x-62-inch-h-2000-lbs-cap-3-open-1-solid-shelf-dunnage-stand>: HTTP status code is not handled or not allowed 2026-01-25 04:34:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-72-inch-w-x-21-inch-d-x-86-inch-h-2000-lbs-cap-3-open-1-solid-shelf-dunnage-stand>: HTTP status code is not handled or not allowed 2026-01-25 04:34:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 64539 pages (at 98 pages/min), scraped 32426 items (at 46 items/min) 2026-01-25 04:34:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keyless-rigid-coupling-c600e312-steel-3125d-x-6693d-318l-shaft>: HTTP status code is not handled or not allowed 2026-01-25 04:34:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pillow-block-bronze-bearing-pbpsbr100-selfaligning-pressed-steel-housing-1-bore>: HTTP status code is not handled or not allowed 2026-01-25 04:34:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/winding-arbor-wa3162-steel-0032w-slot-1364od>: HTTP status code is not handled or not allowed 2026-01-25 04:34:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pillow-block-ball-bearing-pbdcbl075-rigid-die-cast-zinc-housing-34-bore>: HTTP status code is not handled or not allowed 2026-01-25 04:34:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-sided-self-supporting-tank-ls-750-99-dia-x-33h-750-gallon-cap-orange>: HTTP status code is not handled or not allowed 2026-01-25 04:34:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-sided-self-supporting-tank-ls-3000-188-dia-x-33h-3000-gallon-cap-yellow>: HTTP status code is not handled or not allowed 2026-01-25 04:34:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-sided-self-supporting-tank-ls-3000-188-dia-x-33h-3000-gallon-cap-green>: HTTP status code is not handled or not allowed 2026-01-25 04:34:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/toilet-safety-rail-rtl12087-white>: HTTP status code is not handled or not allowed 2026-01-25 04:34:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-24-inch-w-x-18-inch-d-x-62-inch-h-2000-lbs-cap-4-solid-shelves>: HTTP status code is not handled or not allowed 2026-01-25 04:34:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-24-inch-w-x-18-inch-d-x-74-inch-h-2000-lbs-cap-4-open-grid-shelves>: HTTP status code is not handled or not allowed 2026-01-25 04:34:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cb101002-head-flushing-with-hose-and-nozzle-30-x-30>: HTTP status code is not handled or not allowed 2026-01-25 04:34:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-30-inch-w-x-21-inch-d-x-62-inch-h-2000-lbs-cap-5-solid-shelves>: HTTP status code is not handled or not allowed 2026-01-25 04:34:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eyesaver-service-sink-faucet-with-thermostatic-mixing-valve-sef-9000-tw-chrome-yellow>: HTTP status code is not handled or not allowed 2026-01-25 04:34:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-54-inch-w-x-21-inch-d-x-50-inch-h-2000-lbs-cap-4-solid-shelves-dunnage-stand>: HTTP status code is not handled or not allowed 2026-01-25 04:34:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-24-inch-w-x-21-inch-d-x-86-inch-h-2000-lbs-cap-4-solid-shelves>: HTTP status code is not handled or not allowed 2026-01-25 04:34:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quick-adjust-wire-shelving-36-x-24-x-74-poly-z-brite>: HTTP status code is not handled or not allowed 2026-01-25 04:34:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-60-inch-w-x-24-inch-d-x-74-inch-h-2000-lbs-cap-5-solid-shelves-dunnage-stand>: HTTP status code is not handled or not allowed 2026-01-25 04:34:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-stage-gas-powered-air-compressor-30-gallon-8-hp-horizontal>: HTTP status code is not handled or not allowed 2026-01-25 04:34:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/boot-ceiling-box-register-vent-boot-w-adj-hangers-10-inch-x-10-inch-x-10-inch>: HTTP status code is not handled or not allowed 2026-01-25 04:34:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kni-high-leverage-diagonal-cutter-w-multi-component-casings>: HTTP status code is not handled or not allowed 2026-01-25 04:34:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/burglary-file-safe-cabinet-2-hr-factory-fire-rating-digital-lock-22w-x-22d-x-40h>: HTTP status code is not handled or not allowed 2026-01-25 04:34:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24x36x79-metromax-cart>: HTTP status code is not handled or not allowed 2026-01-25 04:34:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/shelf_trucks/plastic_shelf_trucks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:34:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chair-sleeves-1-2-size-27-1-2-x-24-225-box-3-box-case>: HTTP status code is not handled or not allowed 2026-01-25 04:34:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-modular-fr-lubricator-wend-ports-c70352w-gauge-metal-bowl-wsight-manual>: HTTP status code is not handled or not allowed 2026-01-25 04:34:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/florence-4b-horizontal-mailbox-140064pla-33-38-h-6-parcel-front-loading-aluminum-usps>: HTTP status code is not handled or not allowed 2026-01-25 04:34:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hammer-1364-12-long-w-oak-handle-interchangeable-tips>: HTTP status code is not handled or not allowed 2026-01-25 04:34:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/centering-punch-41-5-8-90-5-8-tip>: HTTP status code is not handled or not allowed 2026-01-25 04:34:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-phillips-power-bit-12031x-3-1-2l-x-3-16-number1-tip-1-4-hexagon-drive>: HTTP status code is not handled or not allowed 2026-01-25 04:34:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:34:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/carts-trucks/shelf_trucks/plastic_shelf_trucks>: HTTP status code is not handled or not allowed 2026-01-25 04:34:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-grade-square-recess-power-bit-13054-number2-tip-1-15-16-long>: HTTP status code is not handled or not allowed 2026-01-25 04:34:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/butterfly-wound-closure-sterile-12l-x-2-34w-2400-pcs>: HTTP status code is not handled or not allowed 2026-01-25 04:34:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transfer-sling-large>: HTTP status code is not handled or not allowed 2026-01-25 04:34:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medina-series-quarter-round-corner-2-shelf-bookcase-mocha>: HTTP status code is not handled or not allowed 2026-01-25 04:34:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/air_compressors/portable_electric_air_compressors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:34:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u-sling-w-head-support-medium>: HTTP status code is not handled or not allowed 2026-01-25 04:34:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tms309j-traffic-sign-reserved-parking-connecticut-24-x-12-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:34:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tm176k-traffic-sign-detour-ahead-sign-30-x-30-orange>: HTTP status code is not handled or not allowed 2026-01-25 04:34:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-25 04:34:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sheer-plastic-adhesive-sterile-bandage-1l-x-3w-2400-pcs>: HTTP status code is not handled or not allowed 2026-01-25 04:34:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cubbie-lid-13-1-2l-x-8-5-8w-x-1-4h-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:34:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tm098h-traffic-sign-no-standing-anytime-18-x-12-white>: HTTP status code is not handled or not allowed 2026-01-25 04:34:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-w-x-30-d-steel-square-edge-workbench-adjustable-height-black>: HTTP status code is not handled or not allowed 2026-01-25 04:34:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/air_compressors/portable_electric_air_compressors>: HTTP status code is not handled or not allowed 2026-01-25 04:34:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-end-truck-w-removable-shelf-16-x-60>: HTTP status code is not handled or not allowed 2026-01-25 04:35:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/parking_traffic_signs/parking_traffic_signs_1 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:35:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tma55j-traffic-sign-tow-away-zone-6-x-12-white>: HTTP status code is not handled or not allowed 2026-01-25 04:35:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tmas10j-traffic-sign-violators-will-be-towed-6-x-12-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:35:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/indirect-fired-forced-air-heater-hs2000id-205000-btu>: HTTP status code is not handled or not allowed 2026-01-25 04:35:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-bariatric-walker-medical>: HTTP status code is not handled or not allowed 2026-01-25 04:35:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-l552-10-in-type-2-5hp-left-tilt-w-52-in-fence-system>: HTTP status code is not handled or not allowed 2026-01-25 04:35:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/calming-cream-62-oz>: HTTP status code is not handled or not allowed 2026-01-25 04:35:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/massage-roll-6-5-x-16-cm-2-6-x-6-3-blue-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 04:35:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rt206048-2-aluminum-retractable-shelving-48w-x-20d-x-50h>: HTTP status code is not handled or not allowed 2026-01-25 04:35:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/wall_mount_belts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:35:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/amscope-inspection-zoom-stereo-microscope-3-5x-90x-magnification-w-10mp-usb-camera>: HTTP status code is not handled or not allowed 2026-01-25 04:35:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 64632 pages (at 93 pages/min), scraped 32467 items (at 41 items/min) 2026-01-25 04:35:10 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/parking_traffic_signs/parking_traffic_signs_1 landed on page that is not a product page. 2026-01-25 04:35:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anatomical-model-classic-skull-3-part-painted>: HTTP status code is not handled or not allowed 2026-01-25 04:35:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/square-table-30-with-two-chairs-silvermocha>: HTTP status code is not handled or not allowed 2026-01-25 04:35:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/21in-replacement-roller-galvanized-steel-pack-of-3>: HTTP status code is not handled or not allowed 2026-01-25 04:35:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mc-digital-vacuum-gauge-with-bluetooth-wireless-technology>: HTTP status code is not handled or not allowed 2026-01-25 04:35:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anatomical-chart-hand-wrist-laminated>: HTTP status code is not handled or not allowed 2026-01-25 04:35:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/wall_mount_belts>: HTTP status code is not handled or not allowed 2026-01-25 04:35:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cd-rewritable-media-ver95156-cd-rw-media-12x-speed-700-mb-capcity>: HTTP status code is not handled or not allowed 2026-01-25 04:35:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shell-type-needle-roller-bearing-metric-closed-end-14mm-bore-20mm-od-12mm-width>: HTTP status code is not handled or not allowed 2026-01-25 04:35:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/76213-drill-cup-brush-shank-1-4-fine-1-3-4-crimped>: HTTP status code is not handled or not allowed 2026-01-25 04:35:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbk-athletic-work-shoe-knit-black-dark-grey-4m>: HTTP status code is not handled or not allowed 2026-01-25 04:35:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inner-ring-for-shell-type-needle-roller-bearing-metric-25mm-bore-30mm-od-205mm-width>: HTTP status code is not handled or not allowed 2026-01-25 04:35:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:35:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fl-casual-work-three-quarter-smooth-leather-cognac-10-5d>: HTTP status code is not handled or not allowed 2026-01-25 04:35:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gf-ups15-42f-circulator-water-pump-59896180-cast-iron-220v-125-hp-3-speeds>: HTTP status code is not handled or not allowed 2026-01-25 04:35:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/75205-mounted-grinding-point-metal-a32>: HTTP status code is not handled or not allowed 2026-01-25 04:35:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss316-66006x03-34x38-class-150-hex-bushing-stainless-steel-316>: HTTP status code is not handled or not allowed 2026-01-25 04:35:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss316-66004x03-12x38-class-150-hex-bushing-stainless-steel-316>: HTTP status code is not handled or not allowed 2026-01-25 04:35:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fl-slip-on-boat-shoe-smooth-leather-cognac-13eee>: HTTP status code is not handled or not allowed 2026-01-25 04:35:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rp-works-two-eye-tie-work-sneaker-breathable-knit-textile-leather-brown-tan-8-5m>: HTTP status code is not handled or not allowed 2026-01-25 04:35:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-25 04:35:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:35:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rain-tent-for-m-scope>: HTTP status code is not handled or not allowed 2026-01-25 04:35:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wheelchair-training-stools-set-of-5>: HTTP status code is not handled or not allowed 2026-01-25 04:35:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-round-pedestal-table-with-crisp-linen-top-round-silver-base-bistro-height>: HTTP status code is not handled or not allowed 2026-01-25 04:35:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/valve-knob-1-1-4-inch-d-off-pilot-on-for-keating-004803>: HTTP status code is not handled or not allowed 2026-01-25 04:35:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ntep-floor-scale-gbk60am-60-lb-capacity-0-01-lb-readability>: HTTP status code is not handled or not allowed 2026-01-25 04:35:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ml-a1106-anodized-aluminum-safety-padlock-1-1-2-inchw-1-1-2-inchtall-shackle-keyed-alike-silver>: HTTP status code is not handled or not allowed 2026-01-25 04:35:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-25 04:35:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zone-defense-red-nylon-shell-cut-resistant-gloves-wblack-polyurethane-palm-coat-medium>: HTTP status code is not handled or not allowed 2026-01-25 04:35:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ironcat-insulated-slightly-select-cowhide-welding-gloves-blue-large-all-leather>: HTTP status code is not handled or not allowed 2026-01-25 04:35:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u1282-3a-me-black-tie-locker-triple-tier-12x18x24-3-doors-assembled-black>: HTTP status code is not handled or not allowed 2026-01-25 04:35:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c8052-mk-master-key-for-cell-phone-tablet-locker-with-keyed-locks-ready-to-assemble>: HTTP status code is not handled or not allowed 2026-01-25 04:35:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/400-series-solid-door-storage-cabinet-36wx18dx72h-midnight-ebony-assembled>: HTTP status code is not handled or not allowed 2026-01-25 04:35:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u1282-2mr-black-tie-locker-double-tier-12x18x36-2-doors-unassembled-black-red>: HTTP status code is not handled or not allowed 2026-01-25 04:35:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inner-ring-for-machined-type-needle-roller-bearing-metric-25mm-bore-30mm-od-255mm-width-B1032734>: HTTP status code is not handled or not allowed 2026-01-25 04:35:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-chrome-3-tier-wire-shelving-add-on-unit-24w-x-24d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 04:35:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/indexing-plunger-w-knob-zinc-5-0-24-0n-pressure-m8-1-25-thread-5-5mm-pin>: HTTP status code is not handled or not allowed 2026-01-25 04:35:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oralite-48-inch-x-48-inch-super-bright-vivid-orange-roll-up-sign-worker-symbol>: HTTP status code is not handled or not allowed 2026-01-25 04:35:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/89dh-storagemaster-piano-box-w-junk-trunk-thermosteel-steel-tan>: HTTP status code is not handled or not allowed 2026-01-25 04:35:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transformer-tr75va002-75va-120-24v-dual-hub-foot-mount-circuit-breaker>: HTTP status code is not handled or not allowed 2026-01-25 04:35:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transformer-tr40va013-40va-208-240-277-480-120vac-1-hub-foot-mount-circuit-breaker>: HTTP status code is not handled or not allowed 2026-01-25 04:35:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/tool_storage/jobsite_boxes_storage already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:36:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mounting-track-mt4-4-4w-x-4l>: HTTP status code is not handled or not allowed 2026-01-25 04:36:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sliding-door-tandem-roller-assembly-1-1-8-steel-ball-bearing>: HTTP status code is not handled or not allowed 2026-01-25 04:36:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/bulk_container_trucks_tilt_trucks/basket_trucks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:36:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/diamond-drill-bit-dw5572-1-4-diameter-for-tile>: HTTP status code is not handled or not allowed 2026-01-25 04:36:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metal-stainless-cutting-wheel-dw8425-type-27-5-diameter-12200-rpm-25-pk>: HTTP status code is not handled or not allowed 2026-01-25 04:36:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbwp-12-bushel-ups-fedex-able-vinyl-basket-all-swivel-casters-red>: HTTP status code is not handled or not allowed 2026-01-25 04:36:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tt-low-profile-common-point-ground-cord-cp2522-15l>: HTTP status code is not handled or not allowed 2026-01-25 04:36:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tt-esd-rubber-matting-mt2472-24x72x0080-royal-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:36:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/tool_storage/jobsite_boxes_storage>: HTTP status code is not handled or not allowed 2026-01-25 04:36:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 64724 pages (at 92 pages/min), scraped 32511 items (at 44 items/min) 2026-01-25 04:36:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/horizon-mfg-over-the-seat-binder-rack-3006-13-1-4l-9w-11-3-4h>: HTTP status code is not handled or not allowed 2026-01-25 04:36:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/17-utility-sanitation-hook-rack-4032-pvc-coated-five-2-hooks-red>: HTTP status code is not handled or not allowed 2026-01-25 04:36:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/carts-trucks/bulk_container_trucks_tilt_trucks/basket_trucks>: HTTP status code is not handled or not allowed 2026-01-25 04:36:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/toggle-switch-125-277v-20a-black-silver-for-winston-ps1529>: HTTP status code is not handled or not allowed 2026-01-25 04:36:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-top-connector-global-industrial-153-gantry-cranes-1-pc>: HTTP status code is not handled or not allowed 2026-01-25 04:36:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glass-globe-3-1-4-dia-x-6-3-4-for-chg-l10-x006>: HTTP status code is not handled or not allowed 2026-01-25 04:36:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ajc-mk-es5-12sa-12v-5ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-25 04:36:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/parts-drawer-cabinet-20-drawers-bench-style-14-3-4w-x-6d-x-7-3-8h>: HTTP status code is not handled or not allowed 2026-01-25 04:36:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/furniture_parts_accessories/chair_parts_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:36:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poopy-doo-diaper-disposal-bag-dispenser-400-bag-capacity-pd-dsp-06-wh>: HTTP status code is not handled or not allowed 2026-01-25 04:36:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cha-procedure-accessory-package-1>: HTTP status code is not handled or not allowed 2026-01-25 04:36:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-x-2-v-groove-iron-wheel-with-roller-bearing-for-1-2-axle-800-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-25 04:36:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-gallon-huntington-receptacle-w-flat-top-lid-plastic-liner-brown>: HTTP status code is not handled or not allowed 2026-01-25 04:36:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ansi-grade-2-hd-electronic-lockset-interchangeable-core-brushed-steel>: HTTP status code is not handled or not allowed 2026-01-25 04:36:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/75-series-10-1-8height-3w-8-phenolic-wheel-with-cam-wheel-brake-swivel-caster>: HTTP status code is not handled or not allowed 2026-01-25 04:36:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blower-motor-1-2-to-1-3-hp-1050-rpm-115v>: HTTP status code is not handled or not allowed 2026-01-25 04:36:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-6-step-tilt-roll-ladder-expanded-metal-tr6x>: HTTP status code is not handled or not allowed 2026-01-25 04:36:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-2-step-tilt-roll-ladder-perforated-hl2nsp>: HTTP status code is not handled or not allowed 2026-01-25 04:36:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/furniture_parts_accessories/chair_parts_accessories landed on page that is not a product page. 2026-01-25 04:36:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-5-step-tilt-roll-ladder-serrated-grating-tr5g>: HTTP status code is not handled or not allowed 2026-01-25 04:36:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/contactor-40-amp-240v-2-pole>: HTTP status code is not handled or not allowed 2026-01-25 04:36:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dti-frock-serged-seams-elastic-wrists-white-sm-30-qty>: HTTP status code is not handled or not allowed 2026-01-25 04:36:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lnr-xt-75-waste-oil-heater-6-inch-wall-chimney-10-gallon-tank-75000-btu>: HTTP status code is not handled or not allowed 2026-01-25 04:36:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/122030-horizontal-wall-mount-sign-holder-w-adhesive-tape-8-5-x-5-5-10-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:36:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lnr-xt-300-waste-oil-heater-300000-btu>: HTTP status code is not handled or not allowed 2026-01-25 04:36:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/532ff-floor-mount-scrub-sink-with-faucet>: HTTP status code is not handled or not allowed 2026-01-25 04:36:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/525fl-wall-mount-sink-with-faucet>: HTTP status code is not handled or not allowed 2026-01-25 04:36:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wern-proform-f3-climbing-construction-harness-w-tongue-buckle-legs-s>: HTTP status code is not handled or not allowed 2026-01-25 04:36:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/225596-96-slot-mascara-cosmetic-tray-for-pegboard-slatwall-10-25-x-2-25-acrylic>: HTTP status code is not handled or not allowed 2026-01-25 04:36:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wern-100-feet-2-man-rope-horizontal-lifeline-w-d-bolt-ratchet>: HTTP status code is not handled or not allowed 2026-01-25 04:36:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/300651-adjustable-t-bar-necklace-display-13-x-25-metal>: HTTP status code is not handled or not allowed 2026-01-25 04:36:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-quart-sharps-retrieval-program>: HTTP status code is not handled or not allowed 2026-01-25 04:36:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mail-boss-epoch-wall-mount-locking-drop-box-7415-11-14w-x-4-34d-x-16-14h-granite>: HTTP status code is not handled or not allowed 2026-01-25 04:36:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wern-positioning-belt-for-harness-orange-xxl>: HTTP status code is not handled or not allowed 2026-01-25 04:36:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wern-cross-arm-strap-25-feet-l-web-o-ring-d-ring>: HTTP status code is not handled or not allowed 2026-01-25 04:36:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/703389-org-4-sided-interlocking-pegboard-display-8-x-60-orange-opaque>: HTTP status code is not handled or not allowed 2026-01-25 04:36:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/701465-clr-4-sided-interlocking-pegboard-floor-display-w-wheels-14-x-60-clear>: HTTP status code is not handled or not allowed 2026-01-25 04:36:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-industrial-workstation-fan-3-speed-silver>: HTTP status code is not handled or not allowed 2026-01-25 04:36:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/600073-wht-1-5-high-gridwall-extrusion-6-long-white>: HTTP status code is not handled or not allowed 2026-01-25 04:36:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-109l-commercial-series-single-pantry-faucet-12-spout-low-lead>: HTTP status code is not handled or not allowed 2026-01-25 04:36:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hs-2017-20-x-17-drop-in-hand-sink>: HTTP status code is not handled or not allowed 2026-01-25 04:36:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/800002-w-2-opaque-plastic-pegboard-locking-hook-50-pack-white>: HTTP status code is not handled or not allowed 2026-01-25 04:36:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/workman-winch-w-synthetic-rope-65-ft>: HTTP status code is not handled or not allowed 2026-01-25 04:37:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/771360-yel-pegboard-wall-panel-13-5-x-60-yellow-opaque>: HTTP status code is not handled or not allowed 2026-01-25 04:37:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/igg-precision-combination-squares-w-18-inch-blade-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-25 04:37:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hygrade-blue-poly-tarp-mt1020-5-mil-10l-x-20w>: HTTP status code is not handled or not allowed 2026-01-25 04:37:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/igg-digital-micrometer-w-alloy-gold-anodized-friction-thimble-pack-of-3>: HTTP status code is not handled or not allowed 2026-01-25 04:37:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hygrade-heavy-duty-super-cover-poly-tarp-sth4060-10-mil-silverblack-40l-x-60w>: HTTP status code is not handled or not allowed 2026-01-25 04:37:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rock-utility-tray-for-landscape-cart>: HTTP status code is not handled or not allowed 2026-01-25 04:37:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 64822 pages (at 98 pages/min), scraped 32560 items (at 49 items/min) 2026-01-25 04:37:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-hammer-dust-extraction-dwh050k-hole-cleaning>: HTTP status code is not handled or not allowed 2026-01-25 04:37:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inplant-office-vinyl-cov-hdbd-12x12-2-wall-3-lgts-4-dplx-1-door-white>: HTTP status code is not handled or not allowed 2026-01-25 04:37:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aware-wear-s721-non-ansi-female-vest-61914-pink-3x-large>: HTTP status code is not handled or not allowed 2026-01-25 04:37:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inplant-office-vinyl-cov-drywall-12x24-4-wall-4-lgt-5-dplx-1-dr-white>: HTTP status code is not handled or not allowed 2026-01-25 04:37:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inplant-office-vinyl-cov-drywall-8x10-4-wall-2-lgt-2-dplx-1-dr-white>: HTTP status code is not handled or not allowed 2026-01-25 04:37:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-green-wire-shelf-30-x-18>: HTTP status code is not handled or not allowed 2026-01-25 04:37:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fao-standard-bleeding-control-wall-station-single-kit-up-to-24-persons>: HTTP status code is not handled or not allowed 2026-01-25 04:37:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-plastic-barrier-chain-3-4-inch-x-25-feet-l-army-tan>: HTTP status code is not handled or not allowed 2026-01-25 04:37:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esd-closed-bin-box-12w-x-12d-x-4-1-2h>: HTTP status code is not handled or not allowed 2026-01-25 04:37:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastek-esd-open-bin-box-8w-x-18d-x-4h>: HTTP status code is not handled or not allowed 2026-01-25 04:37:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-step-aluminum-rolling-ladder-24inw-grip-tread-21ind-top-step-32in-handrails>: HTTP status code is not handled or not allowed 2026-01-25 04:37:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-step-aluminum-rolling-ladder-24inw-ribbed-tread-28ind-top-step-32in-handrails>: HTTP status code is not handled or not allowed 2026-01-25 04:37:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/protective-pak-static-dissipative-foam-inside-16-3-4l-x-9-3-4w-x-1-4h>: HTTP status code is not handled or not allowed 2026-01-25 04:37:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-series-swivel-plate-caster-204456944-brk4-polyurethane-with-brake-4dia-275-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 04:37:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/no-wire-test-tube-half-rack-187480016-polypropylene-16mm-30-places-white>: HTTP status code is not handled or not allowed 2026-01-25 04:37:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wash-bottle-117320004-ldpe-1000ml-capacity-distilled-water-4-pk-blue-cap-wide-mouth>: HTTP status code is not handled or not allowed 2026-01-25 04:37:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/octagon-gable-vent-functional-gvoc22x22f-22w-x-22h-x-218d>: HTTP status code is not handled or not allowed 2026-01-25 04:37:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fitness-c521u-cycle-40l-x-22w-x-56h>: HTTP status code is not handled or not allowed 2026-01-25 04:37:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-exercise-putty-red-soft-6-ounce>: HTTP status code is not handled or not allowed 2026-01-25 04:37:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/small-grape-bunch-center-onlay-onl07x09x01gr-718w-x-914h-x-118d>: HTTP status code is not handled or not allowed 2026-01-25 04:37:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-exercise-putty-red-soft-5-pound>: HTTP status code is not handled or not allowed 2026-01-25 04:37:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/springtime-stair-bracket-sb11x07spl-11w-x-7h-x-58d>: HTTP status code is not handled or not allowed 2026-01-25 04:37:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bd-urn-finial-fin06x13ur-634od-x-13h-x-61516>: HTTP status code is not handled or not allowed 2026-01-25 04:37:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-5-tools-4-x-5-lb-putties-medium-yellow-red-green-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:37:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/balance-paws-2-advanced-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 04:37:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/designer-wood-locker-34365-four-tier-3-wide-12w-x-15d-x-18h-gray-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 04:37:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/designer-wood-locker-36361-six-tier-3-wide-12w-x-21d-x-12h-cherry-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 04:37:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/designer-wood-locker-36361-six-tier-3-wide-12w-x-21d-x-12h-maple-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 04:37:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/designer-wood-locker-36365-six-tier-3-wide-12w-x-15d-x-12h-maple-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 04:37:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/lockers/standard_lockers/wood_standard_lockers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:37:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-unit-storage-shelving-chrome-72w-x-18d-x-74h-14-ft>: HTTP status code is not handled or not allowed 2026-01-25 04:37:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cl-ma10-12e-12in-upright-automatic-scrubber-w-accessories-107408161>: HTTP status code is not handled or not allowed 2026-01-25 04:37:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foam-roller-upright-storage-rack-24l-x-34w-x-30h>: HTTP status code is not handled or not allowed 2026-01-25 04:37:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-pe-foam-roller-half-round-6-dia-x-12l>: HTTP status code is not handled or not allowed 2026-01-25 04:37:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hand-exerciser-black-x-heavy>: HTTP status code is not handled or not allowed 2026-01-25 04:37:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hand-exercise-web-latex-free-7-diameter-yellow-x-light>: HTTP status code is not handled or not allowed 2026-01-25 04:37:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hand-exerciser-green-moderate-10-3773>: HTTP status code is not handled or not allowed 2026-01-25 04:37:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/lockers/standard_lockers/wood_standard_lockers>: HTTP status code is not handled or not allowed 2026-01-25 04:37:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-blower-9539-12-12-dia-1hp-2180-cfm>: HTTP status code is not handled or not allowed 2026-01-25 04:37:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-metal-venturi-blower-9518-08-100-psig-5435>: HTTP status code is not handled or not allowed 2026-01-25 04:37:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/surface-conditioning-disc-3-inch-dia-ts-alum-oxide-vfn-grit>: HTTP status code is not handled or not allowed 2026-01-25 04:37:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-bellow-pump-for-inflatable-exercise-balls-and-rolls>: HTTP status code is not handled or not allowed 2026-01-25 04:37:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/concentrate-liquid-laundry-detergent-mountain-breeze-5-gallon-pail-dia-06354>: HTTP status code is not handled or not allowed 2026-01-25 04:37:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-8482-48-18-outlet-aluminum-power-strip-with-15-cord>: HTTP status code is not handled or not allowed 2026-01-25 04:37:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1000-lb-12-span-501-series-steel-wall-mounted-jib-crane-cantilever-design-with-trolley>: HTTP status code is not handled or not allowed 2026-01-25 04:38:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/13-stp-steel-safe-roll-lad-wgt-act-lock-24-w-exp-stp-org-wa134014x-o>: HTTP status code is not handled or not allowed 2026-01-25 04:38:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11-stp-steel-safe-roll-lad-wgt-act-lock-24-w-exp-stp-org-wa113214x-o>: HTTP status code is not handled or not allowed 2026-01-25 04:38:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/modular_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:38:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergodeck-general-purpose-kit-7-8in-thick-66-x-42-charcoal>: HTTP status code is not handled or not allowed 2026-01-25 04:38:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/17100-202-typical-floor-mount-single-adjustable-concealed-arms-lavatory-carrier>: HTTP status code is not handled or not allowed 2026-01-25 04:38:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mat-pak-4-x-8-black-smooth-on-1-side-amcp4s1>: HTTP status code is not handled or not allowed 2026-01-25 04:38:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-tech-heavy-duty-outrigger-pad-36-x-36-x-1-thick-pad36361>: HTTP status code is not handled or not allowed 2026-01-25 04:38:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conntek-1f515520-1-power-adapter-cord-with-5-15p-male-plug-to-5-15-20r-female-connector>: HTTP status code is not handled or not allowed 2026-01-25 04:38:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6328-25ft-30a-generator-cord-w-nema-l14-30-ends>: HTTP status code is not handled or not allowed 2026-01-25 04:38:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 64918 pages (at 96 pages/min), scraped 32604 items (at 44 items/min) 2026-01-25 04:38:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3in-x-1-2in-high-capacity-foam-corner-roller-12-case-3fc-050>: HTTP status code is not handled or not allowed 2026-01-25 04:38:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50913-hood-or-wall-mount-pipette-storage-bin-w-magnet-mount-5-w-x-27-h-x-10-d>: HTTP status code is not handled or not allowed 2026-01-25 04:38:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1000-lb-steel-davit-crane-with-110-ac-electric-winch>: HTTP status code is not handled or not allowed 2026-01-25 04:38:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-trolley-bumper-kit-1-2-3-ton-model-33337905>: HTTP status code is not handled or not allowed 2026-01-25 04:38:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/907002-50h-50-ton-hand-hydraulic-press>: HTTP status code is not handled or not allowed 2026-01-25 04:38:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/901006-1-3-ton-single-leverage-press>: HTTP status code is not handled or not allowed 2026-01-25 04:38:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-watercraft-cleaning-kit-sw81100>: HTTP status code is not handled or not allowed 2026-01-25 04:38:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10in-chenille-microfiber-replacement-bonnet-for-sw19140-blue-sw19175>: HTTP status code is not handled or not allowed 2026-01-25 04:38:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-174-black-trash-bags-40-to-45-gallon-1-0-mil-100-case>: HTTP status code is not handled or not allowed 2026-01-25 04:38:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/liquid-bag-filter-nylon-mesh-7-3-50-dia-x-32-l-150-micron-steel-ring-pkg-qty-50>: HTTP status code is not handled or not allowed 2026-01-25 04:38:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/liquid-bag-filter-nylon-mesh-8-3-8-dia-x-18-l-1000-micron-steel-ring-pkg-qty-50>: HTTP status code is not handled or not allowed 2026-01-25 04:38:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/floor_mats/modular_mats landed on page that is not a product page. 2026-01-25 04:38:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sewn-liquid-bag-filter-polyester-multifilament-4-1-8-x-14-200-micron-steel-ring-pkg-50>: HTTP status code is not handled or not allowed 2026-01-25 04:38:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-16-x-1-security-cap-screw-button-socket-head-18-8-stainless-steel-pkg-of-25>: HTTP status code is not handled or not allowed 2026-01-25 04:38:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/welded-liquid-bag-filter-polyester-felt-4-1-8-x-14-1-micron-plastic-flange-pkg-50>: HTTP status code is not handled or not allowed 2026-01-25 04:38:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/liquid-bag-filter-polyester-felt-7-3-50-dia-x-32-l-100-micron-steel-ring-pkg-qty-50>: HTTP status code is not handled or not allowed 2026-01-25 04:38:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10a-x-2-security-sheet-metal-screw-pan-trilobular-head-410-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 04:38:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8a-x-1-2-tri-wing-sheet-metal-screw-pan-head-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 04:38:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/liquid-bag-filter-poly-p-felt-8-3-8-dia-x-36-l-75-micron-steel-ring-pkg-qty-50>: HTTP status code is not handled or not allowed 2026-01-25 04:38:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/liquid-bag-filter-polyester-felt-8-3-8-dia-x-36-l-5-micron-steel-ring-pkg-qty-50>: HTTP status code is not handled or not allowed 2026-01-25 04:38:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4mm-balldriver-power-bit-10860>: HTTP status code is not handled or not allowed 2026-01-25 04:38:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-x-1-1-2-security-sheet-metal-screw-flat-phillips-head-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 04:38:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-32-tamperproof-hex-bit>: HTTP status code is not handled or not allowed 2026-01-25 04:38:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/office-partition-panel-with-partial-window-and-raceway-48-1-4w-x-46h-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:38:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m6-x-1-0-x-16mm-spanner-machine-screw-pan-head-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 04:38:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tamperproof-spanner-nut-bit-1-4-hex-shank>: HTTP status code is not handled or not allowed 2026-01-25 04:38:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-tamperproof-spanner-bit-5-16-hex-shank>: HTTP status code is not handled or not allowed 2026-01-25 04:38:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a99bb-200c-temperature-sensor>: HTTP status code is not handled or not allowed 2026-01-25 04:38:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelving-60w-x-14d-x-14h-1-shelf-add-on-chrome>: HTTP status code is not handled or not allowed 2026-01-25 04:38:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelving-60w-x-14d-x-63h-4-shelf-starter-chrome>: HTTP status code is not handled or not allowed 2026-01-25 04:38:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nilfisk-gm-80-variable-speed-control>: HTTP status code is not handled or not allowed 2026-01-25 04:38:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelf-level-72w-x-14d-chrome>: HTTP status code is not handled or not allowed 2026-01-25 04:38:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelving-72w-x-18d-x-54h-3-shelf-add-on-chrome>: HTTP status code is not handled or not allowed 2026-01-25 04:38:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/needle-valve-10mm-tube-x-1-4-metal-release-collet-flowin-screw-adjustment>: HTTP status code is not handled or not allowed 2026-01-25 04:38:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultradrain-circular-seal-12inches-diameter>: HTTP status code is not handled or not allowed 2026-01-25 04:38:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50700n-reducer-8mm-stud-x-4mm-tube>: HTTP status code is not handled or not allowed 2026-01-25 04:38:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelf-level-30w-x-24d-chrome>: HTTP status code is not handled or not allowed 2026-01-25 04:38:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/battery-connector-din-a320-female-connector-20-socket-contacts-4-aux-handle>: HTTP status code is not handled or not allowed 2026-01-25 04:38:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/digital-stirring-hotplate-7-7-ceramic-top-100-120v>: HTTP status code is not handled or not allowed 2026-01-25 04:38:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-epoxy-wire-shelving-189269gz-54w-x-24d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 04:38:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10ml-15ml-paddles-for-tube-revolver-88881001-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:38:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hershey-s-nuggets-chocolate-candy-assortment-15-6-oz-gr>: HTTP status code is not handled or not allowed 2026-01-25 04:39:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-96-w-x-36-d-x-84-h-add-on-3-shelf-levels-wood-deck-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:39:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usb-headset-981000014-noise-cancelling-mic-8-cord-black-silver>: HTTP status code is not handled or not allowed 2026-01-25 04:39:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kahlo-36-square-multipurpose-breakroom-tapered-leg-table-white-chrome>: HTTP status code is not handled or not allowed 2026-01-25 04:39:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/concrete-anchor-516-x-2-14-hex-washer-head-large-dia-pkg-of-15-24292>: HTTP status code is not handled or not allowed 2026-01-25 04:39:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-value-max-locker-12-w-x-15-d-x-78-h-729-parchment-triple-tier-1-wide-assembled>: HTTP status code is not handled or not allowed 2026-01-25 04:39:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/legacy-71-double-full-pedestal-l-desk-with-35-return-ash-grey>: HTTP status code is not handled or not allowed 2026-01-25 04:39:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/legacy-classic-6-shelf-71-high-bookcase-ash-grey>: HTTP status code is not handled or not allowed 2026-01-25 04:39:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/highlighters_markers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:39:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-hi-tech-metal-bin-shelving-36-w-x-12-d-x-87-h-725-hallowell-gry-starter-unit-21-bins>: HTTP status code is not handled or not allowed 2026-01-25 04:39:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-hi-tech-metal-bin-shelving-36-w-x-12-d-x-87-h-725-hallowell-gray-add-on-unit-38-bins>: HTTP status code is not handled or not allowed 2026-01-25 04:39:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 65015 pages (at 97 pages/min), scraped 32651 items (at 47 items/min) 2026-01-25 04:39:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-72-w-x-36-d-x-72-h-starter-3-shelf-levels-steel-deck-sand>: HTTP status code is not handled or not allowed 2026-01-25 04:39:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mseries-digital-scalecertified>: HTTP status code is not handled or not allowed 2026-01-25 04:39:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1901331-locknut-1-4-inch-for-bunn-o-matic>: HTTP status code is not handled or not allowed 2026-01-25 04:39:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2221395-pad-rubber-foot-for-waring-products>: HTTP status code is not handled or not allowed 2026-01-25 04:39:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/westcott-19016-kleencut-shears-leftright-hand-6-black>: HTTP status code is not handled or not allowed 2026-01-25 04:39:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arctic-express-dual-10-shipper-storage-system-10-liters>: HTTP status code is not handled or not allowed 2026-01-25 04:39:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/command-17081-general-purpose-hooks-designer-holds-3-lbs-white-2pack>: HTTP status code is not handled or not allowed 2026-01-25 04:39:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-72-w-x-36-d-x-120-h-add-on-3-shelf-levels-wood-deck-sand>: HTTP status code is not handled or not allowed 2026-01-25 04:39:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/highlighters_markers>: HTTP status code is not handled or not allowed 2026-01-25 04:39:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roval-surface-mounted-shelf-6-x-30>: HTTP status code is not handled or not allowed 2026-01-25 04:39:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/rack_accessories/bulk_storage_rack_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:39:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:39:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kimberly-clark-1980-scott-scottfold-towels9-25-x-12-25white175-towelspack>: HTTP status code is not handled or not allowed 2026-01-25 04:39:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lockout-tagout-hasps-5501-1-opening-red>: HTTP status code is not handled or not allowed 2026-01-25 04:39:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precision-high-performance-incubator-gravity-convection-11-2-cu-ft>: HTTP status code is not handled or not allowed 2026-01-25 04:39:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-48-w-x-36-d-x-84-h-add-on-3-shelf-levels-wire-deck-medium-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:39:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/441625-model-b-50-60-coil-kit24-vac>: HTTP status code is not handled or not allowed 2026-01-25 04:39:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-shelf-level-for-bulk-storage-rack-72-w-x-48-d-wood-deck-medium-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:39:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/utility_racks/bulk_storage_racks>: HTTP status code is not handled or not allowed 2026-01-25 04:39:29 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/utility_racks/rack_accessories/bulk_storage_rack_shelves landed on page that is not a product page. 2026-01-25 04:39:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/champion-sports-tc9set-indooroutdoor-flexible-cone-set-vinyl-assorted-colors-6set>: HTTP status code is not handled or not allowed 2026-01-25 04:39:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-ton-magic-lift-service-jack-c-25057>: HTTP status code is not handled or not allowed 2026-01-25 04:39:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-40-14w-mobile-cabinet-5-drawers-63-compart-bright-blue-keyed-alike>: HTTP status code is not handled or not allowed 2026-01-25 04:39:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-28-14w-mobile-cabinet-7-drawer-72-compart-light-gray-individual-lock>: HTTP status code is not handled or not allowed 2026-01-25 04:39:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/essentials-series-mahogany-laminate-bookshelf-355x148x36>: HTTP status code is not handled or not allowed 2026-01-25 04:39:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-40-14w-drawer-cabinet-4-drawer-24-compart-classic-blue-master-keyed>: HTTP status code is not handled or not allowed 2026-01-25 04:39:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/450-lb-cap-toolbox-creeper-seat-92450>: HTTP status code is not handled or not allowed 2026-01-25 04:39:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/y1-01-75-payloader-4-wheel-two-passenger-48v-burden-carrier>: HTTP status code is not handled or not allowed 2026-01-25 04:39:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sb-65-106-80-a-pvc-shrink-bags-6-1-2-x-10-1-2-x-80-gauge-clear-500-case>: HTTP status code is not handled or not allowed 2026-01-25 04:39:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/physicianscare-90089-antacid-calcium-carbonate-medication-50-doses>: HTTP status code is not handled or not allowed 2026-01-25 04:39:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-band-loop-10-long-yellow>: HTTP status code is not handled or not allowed 2026-01-25 04:39:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/expander-3-piece-set-red-green-blue-10-0056>: HTTP status code is not handled or not allowed 2026-01-25 04:39:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/expander-red-light-10-0042>: HTTP status code is not handled or not allowed 2026-01-25 04:39:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-ab14b-14-single-arm-shelf-bracket-black-epoxy>: HTTP status code is not handled or not allowed 2026-01-25 04:39:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-free-exercise-tubing-blue-25-roll-bag>: HTTP status code is not handled or not allowed 2026-01-25 04:39:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-40-14w-cabinet-14-drawer-222-compart-bright-blue-keyed-alike>: HTTP status code is not handled or not allowed 2026-01-25 04:39:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uv-black-standard-cable-ties-502212-100-141-100-pcs>: HTTP status code is not handled or not allowed 2026-01-25 04:39:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010523-motor-assembly-for-southbend-range>: HTTP status code is not handled or not allowed 2026-01-25 04:39:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/w-755aa-30-automatic-double-electromagnetic-impulse-sealer-with-5mm-seal-width>: HTTP status code is not handled or not allowed 2026-01-25 04:39:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brazed-seam-ring-terminal-copper-lugs-107202-025-6-gauge-1-2-stud>: HTTP status code is not handled or not allowed 2026-01-25 04:39:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/electrochemistry_meters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:39:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a211-benchtop-ph-meter-durable-ross-kit-stara2116>: HTTP status code is not handled or not allowed 2026-01-25 04:39:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-powder-pre-cut-exercise-band-blue-48-l-strip-1-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:39:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-amp-mini-blade-fuses-509125-025-tan-25-pcs>: HTTP status code is not handled or not allowed 2026-01-25 04:39:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ironwood-3-compartment-solid-door-wood-storage-cabinet-cactus-star-color>: HTTP status code is not handled or not allowed 2026-01-25 04:39:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/wiring_devices/wire_clips_clamps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:39:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/321154-flange-washerfor-3-12-inch-sink-opening>: HTTP status code is not handled or not allowed 2026-01-25 04:40:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/expandable-sleeving-505303-2010-1-2-10-ft>: HTTP status code is not handled or not allowed 2026-01-25 04:40:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/wiring_devices/fuses/fuse_holders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:40:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-band-red-48-l-band-box-of-40>: HTTP status code is not handled or not allowed 2026-01-25 04:40:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/electrochemistry_meters landed on page that is not a product page. 2026-01-25 04:40:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leviton-2537-slimline-base-single-pin-standard-fluorescent-lampholder>: HTTP status code is not handled or not allowed 2026-01-25 04:40:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-station-accessory-exercise-handles-with-d-ring-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 04:40:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/321539-o-ring-1-3-8-inch-id-x-1-8-inch-width-for-hubbell-the-electric-heater-co>: HTTP status code is not handled or not allowed 2026-01-25 04:40:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/thresholds_bottom_sweeps already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:40:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/wiring_devices/wire_clips_clamps>: HTTP status code is not handled or not allowed 2026-01-25 04:40:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 65104 pages (at 89 pages/min), scraped 32690 items (at 39 items/min) 2026-01-25 04:40:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/261360-chopper-plate-for-univex>: HTTP status code is not handled or not allowed 2026-01-25 04:40:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/wiring_devices/fuses/fuse_holders>: HTTP status code is not handled or not allowed 2026-01-25 04:40:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hager-full-surface-reinforcing-pivot>: HTTP status code is not handled or not allowed 2026-01-25 04:40:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-strapping-12w-x-7200l-x-0026-thick-8-x-8-core-black>: HTTP status code is not handled or not allowed 2026-01-25 04:40:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/huot-toolscoot-hsk-63a-cnc-holders>: HTTP status code is not handled or not allowed 2026-01-25 04:40:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/262337-locknut1-inch-zinc>: HTTP status code is not handled or not allowed 2026-01-25 04:40:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/door_window_hardware/thresholds_bottom_sweeps landed on page that is not a product page. 2026-01-25 04:40:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/69716-1-5-stainless-steel-scraper-yellow>: HTTP status code is not handled or not allowed 2026-01-25 04:40:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/69594-5-mini-bench-scraper-red>: HTTP status code is not handled or not allowed 2026-01-25 04:40:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-cylinder-truck-4-wheel-10-1-2-pneumatic-wheels-1000-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-25 04:40:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/98012-1-color-coding-rubber-band-x5-green>: HTTP status code is not handled or not allowed 2026-01-25 04:40:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/261384-union-for-garland-range>: HTTP status code is not handled or not allowed 2026-01-25 04:40:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-surface-mounted-multi-roll-tissue-dispenser-w-shelf>: HTTP status code is not handled or not allowed 2026-01-25 04:40:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/70376-tank-brush-stiff-yellow>: HTTP status code is not handled or not allowed 2026-01-25 04:40:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/53515-stainless-steel-flex-rod-handle>: HTTP status code is not handled or not allowed 2026-01-25 04:40:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:40:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-classicseries-heavy-duty-shower-curtain-rod-36-w>: HTTP status code is not handled or not allowed 2026-01-25 04:40:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scotch-brite-exl-pro-deburring-wheel-8-inch-x-3-inch-x-1-inch-silicon-carbide-fin>: HTTP status code is not handled or not allowed 2026-01-25 04:40:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-trimline-surface-mounted-ada-dryer-115v-ss>: HTTP status code is not handled or not allowed 2026-01-25 04:40:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bishamon-pallet-truck-pallet-jack-5500-lb-capacity-21-x-48>: HTTP status code is not handled or not allowed 2026-01-25 04:40:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-shelf-liner-48w-x-18d-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:40:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-shelf-liner-48w-x-12d-white>: HTTP status code is not handled or not allowed 2026-01-25 04:40:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/458966-bench-brush-medium-brown>: HTTP status code is not handled or not allowed 2026-01-25 04:40:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots landed on page that is not a product page. 2026-01-25 04:40:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rotating-pedestal-grill-with-3-1-2-dia-post-280-sq-cooking-surface>: HTTP status code is not handled or not allowed 2026-01-25 04:40:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-1-1-2-dia-straight-peened-grab-bar-18-w>: HTTP status code is not handled or not allowed 2026-01-25 04:40:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/53725-hook-brush-medium-white>: HTTP status code is not handled or not allowed 2026-01-25 04:40:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/52-bollard-flex-with-sign-post-concrete-installation-white-cover-yellow-tapes>: HTTP status code is not handled or not allowed 2026-01-25 04:40:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/panel-sheet-mover-truck-with-carpet-padded-steel-deck-1200-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 04:40:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/bollards/safety_bollards_posts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:40:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-x-18-x-72-phenolic-locker-z-style-graphite-grafix>: HTTP status code is not handled or not allowed 2026-01-25 04:40:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-polyester-filter-20-w-x-24-h-x-2-d>: HTTP status code is not handled or not allowed 2026-01-25 04:40:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/bollards/safety_bollards_posts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:40:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-w-x-30-d-production-workbench-shop-top-square-edge-with-drawers-shelf-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:40:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/29343-6-mini-handle-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:40:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medium-duty-low-lint-wipes-12-x-16-1-2-250-case-m-pr822>: HTTP status code is not handled or not allowed 2026-01-25 04:40:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medium-duty-low-lint-wipes-12-x-16-1-2-250-case-m-pr922>: HTTP status code is not handled or not allowed 2026-01-25 04:40:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/296288-59-ultra-hygiene-handle-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:40:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-polyester-filter-12-w-x-20-h-x-1-d>: HTTP status code is not handled or not allowed 2026-01-25 04:40:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/er70s-6-mild-steel-mig-welding-wire-045-dia-x-33-lb-spool>: HTTP status code is not handled or not allowed 2026-01-25 04:40:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/innovera-compatible-remanufactured-ce390a-90a-toner-10000-page-yield-black>: HTTP status code is not handled or not allowed 2026-01-25 04:40:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/wire_cable/electrical_wire_cable already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:40:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/bollards/safety_bollards_posts>: HTTP status code is not handled or not allowed 2026-01-25 04:40:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-3-sjoow-black-250-ft>: HTTP status code is not handled or not allowed 2026-01-25 04:40:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/bollards/safety_bollards_posts landed on page that is not a product page. 2026-01-25 04:40:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stronghold-ventilated-divider-cabinet-72-x-24-x-78>: HTTP status code is not handled or not allowed 2026-01-25 04:40:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-utility-tray-orange-26-l-x-12-1-2-w-x-4-1-2-h>: HTTP status code is not handled or not allowed 2026-01-25 04:40:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2462006-handle-housing-power-unit-for-rf-hunter-co-filtration-systems>: HTTP status code is not handled or not allowed 2026-01-25 04:40:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/airtight-watertight-access-door-18-x-18>: HTTP status code is not handled or not allowed 2026-01-25 04:41:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009297-knob-for-moffat>: HTTP status code is not handled or not allowed 2026-01-25 04:41:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8754-replacement-liner-for-13-by-43-modular-model-berm-textured-tan>: HTTP status code is not handled or not allowed 2026-01-25 04:41:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/29759-62-113-telescopic-aluminum-handle-black>: HTTP status code is not handled or not allowed 2026-01-25 04:41:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ductless-heat-pump-high-wall-indoor-unit-30000-btu-20-seer-208230v>: HTTP status code is not handled or not allowed 2026-01-25 04:41:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1031154-gasket-drain-neoprene-for-ultrafryer>: HTTP status code is not handled or not allowed 2026-01-25 04:41:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/walk-through-access-door-30-x-36>: HTTP status code is not handled or not allowed 2026-01-25 04:41:04 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/wire_cable/electrical_wire_cable landed on page that is not a product page. 2026-01-25 04:41:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/264609-grease-filter-stainless-steel-16-x-16-x-2>: HTTP status code is not handled or not allowed 2026-01-25 04:41:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/-kuul-comfort-evaporative-replacement-media-set-for-jetstream-230-12w-x-6d-x-23h>: HTTP status code is not handled or not allowed 2026-01-25 04:41:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/outdoor-snow-ice-melting-heated-stair-mat-11-x-36-120-volts>: HTTP status code is not handled or not allowed 2026-01-25 04:41:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/outdoor-snow-ice-melting-heated-stair-mat-11-x-48-120-volts>: HTTP status code is not handled or not allowed 2026-01-25 04:41:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 65201 pages (at 97 pages/min), scraped 32731 items (at 41 items/min) 2026-01-25 04:41:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carson-optical-lt-30-linentest-magnifier>: HTTP status code is not handled or not allowed 2026-01-25 04:41:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-rack-96x42x120-starter-unit-green-orange-4000-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-25 04:41:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dinex-turnbury-insulated-pedestal-base-mug-8-oz-48-cs-dark-blue-dx300050>: HTTP status code is not handled or not allowed 2026-01-25 04:41:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/original-wrist-and-ankle-weight-1-lb-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:41:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dinex-retro-traycover-w-straight-edge-round-corner-12-3-4-x-16-5-8-1000-cs-dx5331i>: HTTP status code is not handled or not allowed 2026-01-25 04:41:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/original-wrist-and-ankle-weight-4-lb-turquoise>: HTTP status code is not handled or not allowed 2026-01-25 04:41:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nugget-ice-machine-340lb>: HTTP status code is not handled or not allowed 2026-01-25 04:41:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metal-indoor-outdoor-stackable-chair-copper>: HTTP status code is not handled or not allowed 2026-01-25 04:41:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-shop-table-with-maple-top-96-x-36-x-34>: HTTP status code is not handled or not allowed 2026-01-25 04:41:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storability-15-wire-basket-with-lock-on-brackets>: HTTP status code is not handled or not allowed 2026-01-25 04:41:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foremans-cabinet-desk-48-x-28-x-54>: HTTP status code is not handled or not allowed 2026-01-25 04:41:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2901319-gmax-40v-25ah-battery>: HTTP status code is not handled or not allowed 2026-01-25 04:41:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forkli-able-shipping-container-with-lid-42-x-42-2>: HTTP status code is not handled or not allowed 2026-01-25 04:41:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conntek-20411-100-100-sjtw-12-3-outdoor-extension-cord-with-3-outlet-nema-5-15p-r>: HTTP status code is not handled or not allowed 2026-01-25 04:41:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forkli-able-shipping-container-with-lid-51-x-19-1>: HTTP status code is not handled or not allowed 2026-01-25 04:41:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/650-10-1-digital-neonatal-dual-range-scale-0-10-lb-x-0-05-oz-10-33-lb-x-0-1-oz>: HTTP status code is not handled or not allowed 2026-01-25 04:41:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-sectional-stackbin-unit-37-w-8-d-x-4-1-2-h-blue-8-compartments>: HTTP status code is not handled or not allowed 2026-01-25 04:41:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aerospec-ss-nsf-wall-shelf-29>: HTTP status code is not handled or not allowed 2026-01-25 04:41:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-compartment-steel-sectional-unit-37-w-x-24-d-x-33-h-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:41:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/americana-climbing-wind-safety-helmet-type-2-4-point-mega-ratchet-suspension-purple>: HTTP status code is not handled or not allowed 2026-01-25 04:41:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industries-tube-2-x-050-x-12-polished-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 04:41:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-5-quart-sauce-pan-black-handle-1>: HTTP status code is not handled or not allowed 2026-01-25 04:41:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industries-perpendicular-collar-for-15-tubing-satin-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 04:41:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/28-1090-lever-waste-drain-body-bushing>: HTTP status code is not handled or not allowed 2026-01-25 04:41:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/positioning-wedge-blue-22l-x-20w-x-4h>: HTTP status code is not handled or not allowed 2026-01-25 04:41:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1901296-nut-hex-brass-1-2-14-thread-for-bunn-o-matic>: HTTP status code is not handled or not allowed 2026-01-25 04:41:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bin-nestable-4-1-8-w-x-11-5-8-d-x-4-h-yellow-1>: HTTP status code is not handled or not allowed 2026-01-25 04:41:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cube-ice-maker-undercounter-air-cooled-approx-238-lb-production-full-size-cubes-full-size-cube>: HTTP status code is not handled or not allowed 2026-01-25 04:41:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cnc02016-machine-screw>: HTTP status code is not handled or not allowed 2026-01-25 04:41:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ripstop-insulated-safety-bib-overall-xl-black>: HTTP status code is not handled or not allowed 2026-01-25 04:41:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brc08-nylon-insert-lock-nut>: HTTP status code is not handled or not allowed 2026-01-25 04:41:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/anchors_hangers/wedge_anchors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:41:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelite-vented-plastic-mat-shelf-truck-42l-x-21w-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 04:41:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/78-x-8-stud-wedge-anchor-steel-zinc-pkg-of-5>: HTTP status code is not handled or not allowed 2026-01-25 04:41:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelite-vented-plastic-mat-shelf-truck-54l-x-18w-x-80h-polyurethane-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 04:41:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/full-size-perforated-pan-4-inch>: HTTP status code is not handled or not allowed 2026-01-25 04:41:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/anchors_hangers/wedge_anchors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:41:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-x-2-34-stud-wedge-anchor-steel-zinc-pkg-of-50>: HTTP status code is not handled or not allowed 2026-01-25 04:41:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industries-12-foot-rail-kit-2-tube-satin-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 04:41:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/fasteners/anchors_hangers/wedge_anchors>: HTTP status code is not handled or not allowed 2026-01-25 04:42:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/relaxed-fit-flex-twill-cargo-pants-32l-inseam-x-30-waist-navy>: HTTP status code is not handled or not allowed 2026-01-25 04:42:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-security-laminated-padlocks-with-master-keyed-system-3lf>: HTTP status code is not handled or not allowed 2026-01-25 04:42:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pop-blind-rivet-1-8-x-4-6-button-head-up-to-3-8-grip-stainless-steel-steel-pkg-of-500>: HTTP status code is not handled or not allowed 2026-01-25 04:42:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stacking-bin-16-1-2-x-10-7-8-x-5-yellow>: HTTP status code is not handled or not allowed 2026-01-25 04:42:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/fasteners/anchors_hangers/wedge_anchors>: HTTP status code is not handled or not allowed 2026-01-25 04:42:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insulated-safety-coverall-4xl-orange>: HTTP status code is not handled or not allowed 2026-01-25 04:42:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/321111-bdb-1340a-belt-drive-bench-lathe-w-collet-closer-2-hp>: HTTP status code is not handled or not allowed 2026-01-25 04:42:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-grip-blind-rivet-3-16-x-3-6-dome-head-188-375-grip-range-aluminum-steel-250-pk>: HTTP status code is not handled or not allowed 2026-01-25 04:42:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 65297 pages (at 96 pages/min), scraped 32781 items (at 50 items/min) 2026-01-25 04:42:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/54-h-nexel-chrome-wire-shelving-24-w-x-24-d>: HTTP status code is not handled or not allowed 2026-01-25 04:42:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-hand-knob-w-ss-screw-1-4-20-thread-1-1-8-inch-thread-length-1-inch-knob-dia>: HTTP status code is not handled or not allowed 2026-01-25 04:42:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ckt-48-cruiser-bench-checkweighing-scale-100-lb-x-0-005-lb>: HTTP status code is not handled or not allowed 2026-01-25 04:42:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-manila-reinforced-shelf-folder-9-high-front-letter-100box>: HTTP status code is not handled or not allowed 2026-01-25 04:42:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1071049-faucet-inlet-1-2-inch-nps-m-1-2slp-for-fisher-manufacturing>: HTTP status code is not handled or not allowed 2026-01-25 04:42:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/74-h-nexel-chrome-wire-shelving-add-on-24-w-x-14-d>: HTTP status code is not handled or not allowed 2026-01-25 04:42:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smead-1-31-open-accordion-expanding-file-31-pockets-letter-leather-like-redrope>: HTTP status code is not handled or not allowed 2026-01-25 04:42:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-80mm-bore-x-320mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 04:42:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fox-40-whistle-green-10-421-gre>: HTTP status code is not handled or not allowed 2026-01-25 04:42:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clamps-wilton14742>: HTTP status code is not handled or not allowed 2026-01-25 04:42:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fireproof-4-drawer-vertical-file-cabinet-legal-31-9-16-lx20-13-16-wx52-3-4-h-parchment>: HTTP status code is not handled or not allowed 2026-01-25 04:42:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/86-h-nexel-chrome-wire-shelving-42-w-x-24-d>: HTTP status code is not handled or not allowed 2026-01-25 04:42:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8ft-long-box-pleat-skirting-8h-stage-black>: HTTP status code is not handled or not allowed 2026-01-25 04:42:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smead-slash-pocket-folders-letter-11-point-bluegreenmanilaredyellow-25pack>: HTTP status code is not handled or not allowed 2026-01-25 04:42:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1201176-caster-boot-2-inch-f-1-inch-od-tube>: HTTP status code is not handled or not allowed 2026-01-25 04:42:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/74-h-nexel-chrome-wire-shelving-add-on-60-w-x-18-d>: HTTP status code is not handled or not allowed 2026-01-25 04:42:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stage-pie-unit-carpet-36w-x-8h-stage-units-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:42:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-ripstop-4-in-1-safety-jacket-5xl-fluorescent-orange>: HTTP status code is not handled or not allowed 2026-01-25 04:42:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-ripstop-fleece-lined-safety-jacket-xs-fluorescent-green>: HTTP status code is not handled or not allowed 2026-01-25 04:42:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010552-pilot-orifice-30-lp-for-garland-range>: HTTP status code is not handled or not allowed 2026-01-25 04:42:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mp1-det-clv-mount-kit-2-bore-nfpa-cylinder>: HTTP status code is not handled or not allowed 2026-01-25 04:42:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/86-h-nexel-chrome-wire-shelving-60-w-x-24-d>: HTTP status code is not handled or not allowed 2026-01-25 04:42:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/341594-warmer-element-120v-700w-for-intermetro-industries>: HTTP status code is not handled or not allowed 2026-01-25 04:42:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1331191-bumper-corner-strip-bumper>: HTTP status code is not handled or not allowed 2026-01-25 04:42:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-chrome-wire-shelf-cart-48x24-2-shelves-800-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 04:42:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/warming-gel-32-oz-bottle>: HTTP status code is not handled or not allowed 2026-01-25 04:42:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compress-neck-7-x-22>: HTTP status code is not handled or not allowed 2026-01-25 04:42:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-utility-cart-36x24-3-shelves-800-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 04:42:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-ripstop-fleece-lined-safety-jacket-xs-fluorescent-orange>: HTTP status code is not handled or not allowed 2026-01-25 04:42:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cep001-r-custom-molded-earplugs-nrr-26db-red-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 04:42:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paraffin-bath-refill-36-lb-blocks-wintergreen-fragrance>: HTTP status code is not handled or not allowed 2026-01-25 04:42:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-solid-galvanized-shelf-cart-36x24-2-shelves-800-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 04:42:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-100mm-bore-x-360mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 04:42:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-solid-galvanized-shelf-cart-36x18-3-shelves-800-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 04:42:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/63-post-kit>: HTTP status code is not handled or not allowed 2026-01-25 04:42:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2171101-pump-bottle-28mm-top-1-4-oz-for-server-products>: HTTP status code is not handled or not allowed 2026-01-25 04:42:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-50mm-bore-x-90mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 04:42:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/utility-cart-for-heating-unit-e1-21l-x-16w-x-30h>: HTTP status code is not handled or not allowed 2026-01-25 04:43:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blue-vinyl-reusable-cold-pack-quarter-size-5-x-7>: HTTP status code is not handled or not allowed 2026-01-25 04:43:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-ram-closet-hose-case>: HTTP status code is not handled or not allowed 2026-01-25 04:43:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/physical_therapy/hot_cold_packs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:43:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-8>: HTTP status code is not handled or not allowed 2026-01-25 04:43:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/physical_therapy/pain_relief_gel already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:43:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-9>: HTTP status code is not handled or not allowed 2026-01-25 04:43:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/28-magnetic-burning-guide>: HTTP status code is not handled or not allowed 2026-01-25 04:43:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insulated-cold-gear-jacket-4xl-blackfluorescent-yellow-green>: HTTP status code is not handled or not allowed 2026-01-25 04:43:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 65391 pages (at 94 pages/min), scraped 32831 items (at 50 items/min) 2026-01-25 04:43:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-control-85968-04-32-knob-adj14-tube-x-10-32-unf-male-threadcomposite-body>: HTTP status code is not handled or not allowed 2026-01-25 04:43:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12mm-id-x-16mm-od-x-025mm-die-button-shim-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-25 04:43:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/physical_therapy/hot_cold_packs landed on page that is not a product page. 2026-01-25 04:43:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-run-tee-89222-08-04-12-tube-x-14-swift-fit-univ-thread-metallic-collet>: HTTP status code is not handled or not allowed 2026-01-25 04:43:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/physical_therapy/pain_relief_gel>: HTTP status code is not handled or not allowed 2026-01-25 04:43:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eagle-faucet-cans-brass-5-gallons>: HTTP status code is not handled or not allowed 2026-01-25 04:43:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-wide-span-rack-96x24x60-gray-with-3-shelves-laminated-deck-1100-lb-cap-per-level>: HTTP status code is not handled or not allowed 2026-01-25 04:43:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0030-inch-plastic-color-coded-shim-5-inch-x-20-inch-flat-sheet-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 04:43:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-40mm-bore-x-220mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 04:43:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/38-inch-id-x-58-inch-od-x-00015-inch-steel-arbor-shim-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 04:43:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-18-inch-id-x-1-58-inch-od-x-0062-inch-steel-arbor-spacer-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 04:43:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1831222-sprocket-1-2-8-pack-vct-2010-for-roundup-food-equipment>: HTTP status code is not handled or not allowed 2026-01-25 04:43:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disc-pad-ts-and-tsm-14212-hard-2-inch-1-4-20-internal>: HTTP status code is not handled or not allowed 2026-01-25 04:43:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/moto-vest-4-pockets-cottonpolyester-m-black>: HTTP status code is not handled or not allowed 2026-01-25 04:43:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-bulk-rack-shelf-wood-deck-48x18-gray-1>: HTTP status code is not handled or not allowed 2026-01-25 04:43:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-branch-tee-60215-8-18-8mm-tube-x-18-male-bspp-thread-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 04:43:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hs-blend-and-finish-disc-6-inch-x-1-2-inch-med-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-25 04:43:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/70024-6-7-x-6-7-mobile-skylight-protection-system-polyester-galvanized-steel-black>: HTTP status code is not handled or not allowed 2026-01-25 04:43:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-strength-disc-12-inch-x-1-1-4-inch-vfn-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-25 04:43:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48in-rectangular-metal-mesh-picnic-table-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:43:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latch-arm-assembly-for-globalindustrial-2-3-cu-yd-dumping-hopper-w-bump-release>: HTTP status code is not handled or not allowed 2026-01-25 04:43:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pockets-metal-base-black>: HTTP status code is not handled or not allowed 2026-01-25 04:43:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulkhead-union-89050-53-532-tube-metallic-collet>: HTTP status code is not handled or not allowed 2026-01-25 04:43:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thru-wall-package-drop-with-tilt-out-door-dvwm0062s-copper-vein>: HTTP status code is not handled or not allowed 2026-01-25 04:43:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/light-meter-with-bluetooth-50-connectivity-4-aaa-batteries-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:43:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2221216-container-assembly-plastic-ns-for-waring-products>: HTTP status code is not handled or not allowed 2026-01-25 04:43:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/parcel-protector-vault-mailbox-and-parcel-drop-dvu0050-rear-access-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:43:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36x18x86-stainless-steel-solid-shelving>: HTTP status code is not handled or not allowed 2026-01-25 04:43:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012219-hose-fitting-for-cleveland-range>: HTTP status code is not handled or not allowed 2026-01-25 04:43:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-aquatic-exercise-kit-jogger-belt-ankle-cuffs-hand-bars-large-red>: HTTP status code is not handled or not allowed 2026-01-25 04:43:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/definite_purpose_electric_motors/agricultural_motors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:43:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sting-relief-wipes-hydrocortisone-packs-for-smartcompliance-cabinet-30box>: HTTP status code is not handled or not allowed 2026-01-25 04:43:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lyon-locker-six-tier-12x18x12-18-door-assembled-putty>: HTTP status code is not handled or not allowed 2026-01-25 04:43:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eagle-e-a-3-8x50-b-hybri-flex-hybrid-polymer-air-hose>: HTTP status code is not handled or not allowed 2026-01-25 04:43:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/341708-booster-element-208v-6000w-for-hubbell-the-electric-heater-co>: HTTP status code is not handled or not allowed 2026-01-25 04:43:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m6-x-1-0-nylon-insert-lock-nut-304-stainless-steel-din-985-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 04:43:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30007-25-plastic-barrier-chain-hdpe-1-5-x25-6-38mm-gold>: HTTP status code is not handled or not allowed 2026-01-25 04:44:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/definite_purpose_electric_motors/agricultural_motors>: HTTP status code is not handled or not allowed 2026-01-25 04:44:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30324-10-s-hook-acetal-copolymer-1-5-blue-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 04:44:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4mm-tube-fitting-kit-50860n-4-60-assorted-fittings-durham-slide-drawer>: HTTP status code is not handled or not allowed 2026-01-25 04:44:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-cassette-for-queuepro-250-safetypro-250-stanchions-11l-yellow-black-belt>: HTTP status code is not handled or not allowed 2026-01-25 04:44:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/51004-25-heavy-duty-plastic-barrier-chain-hdpe-2-x25-54mm-green>: HTTP status code is not handled or not allowed 2026-01-25 04:44:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/281472-tray-bracket-grease>: HTTP status code is not handled or not allowed 2026-01-25 04:44:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-48-3-door-indoor-wood-enclosed-letterboard-walnut-frame>: HTTP status code is not handled or not allowed 2026-01-25 04:44:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 65488 pages (at 97 pages/min), scraped 32885 items (at 54 items/min) 2026-01-25 04:44:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-upper-tape-head-carton-sealing-head-pullback-spring-for-412560>: HTTP status code is not handled or not allowed 2026-01-25 04:44:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-48-tefc-1ph-1725rpm-c4c17fc9cc>: HTTP status code is not handled or not allowed 2026-01-25 04:44:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-tecgen-flame-resistant-work-shirt-m-tall-navy>: HTTP status code is not handled or not allowed 2026-01-25 04:44:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-lid-32-gallon-round-trash-container-2>: HTTP status code is not handled or not allowed 2026-01-25 04:44:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lite-insulated-gas-cylinder-heater-pbl420-420-lb-capacity-90f-fixed>: HTTP status code is not handled or not allowed 2026-01-25 04:44:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-3-8-reversible-pistol-grip-drill>: HTTP status code is not handled or not allowed 2026-01-25 04:44:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-unicoil-hose-bender-for-34-inch-id-109-inch-max-od-hose>: HTTP status code is not handled or not allowed 2026-01-25 04:44:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/221101-kason-10538000004-latch-with-strike-for-atlas-metal-industries>: HTTP status code is not handled or not allowed 2026-01-25 04:44:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-flame-resistant-dh-air153-work-shirt-xl-regular-khaki>: HTTP status code is not handled or not allowed 2026-01-25 04:44:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-partition-flexible-nine-panel-5>: HTTP status code is not handled or not allowed 2026-01-25 04:44:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/room_dividers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:44:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ballymore-battery-operated-power-lift-stacker-ballypal22ag63>: HTTP status code is not handled or not allowed 2026-01-25 04:44:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dart-concorde-foam-plate-compartmented-9-dia-we-125pack-500carton>: HTTP status code is not handled or not allowed 2026-01-25 04:44:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1751107-hanger-basket-retro-kit-sg14-for-magikitchen-products>: HTTP status code is not handled or not allowed 2026-01-25 04:44:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-flame-resistant-dh-air153-work-shirt-l-short-navy>: HTTP status code is not handled or not allowed 2026-01-25 04:44:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/263911-orifice-kit>: HTTP status code is not handled or not allowed 2026-01-25 04:44:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/threaded-insert-installation-kit-for-hard-wood-brass-m5-0-8-ez-400-m5>: HTTP status code is not handled or not allowed 2026-01-25 04:44:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/partitions_dividers/room_dividers landed on page that is not a product page. 2026-01-25 04:44:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/red-plastic-box-truck-20-bushel>: HTTP status code is not handled or not allowed 2026-01-25 04:44:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-full-vision-show-case-white>: HTTP status code is not handled or not allowed 2026-01-25 04:44:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-stacking-bin-clear-7>: HTTP status code is not handled or not allowed 2026-01-25 04:44:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-bushel-canvas-basket-truck-bulk-truck>: HTTP status code is not handled or not allowed 2026-01-25 04:44:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-flame-resistant-14th-zip-fleece-sweatshirt-s-regular-black>: HTTP status code is not handled or not allowed 2026-01-25 04:44:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/341921-heater-strip-240v500w>: HTTP status code is not handled or not allowed 2026-01-25 04:44:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/humidifiers_dehumidifiers/dehumidifiers_desiccators already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:44:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/342138-quartz-heater-60v-450w-for-star-manufacturing>: HTTP status code is not handled or not allowed 2026-01-25 04:44:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chicago-pneumatic-3-8-reversible-pistol-drill-cp9790-0-37-hp-3-8-chuck>: HTTP status code is not handled or not allowed 2026-01-25 04:44:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/type-r-class-3-heavy-duty-engineer-vest-green-5xl-sv55-3zgd-5x>: HTTP status code is not handled or not allowed 2026-01-25 04:44:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/type-r-class-2-heavy-duty-engineer-vest-orange-3xl-sv55-2zod-3x>: HTTP status code is not handled or not allowed 2026-01-25 04:44:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chicago-pneumatic-1-8-pencil-grinder-cp9104q-60000-rpm>: HTTP status code is not handled or not allowed 2026-01-25 04:44:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18awg-4c-stranded-control-cable-plenum-cmp-1000-ft-box-white>: HTTP status code is not handled or not allowed 2026-01-25 04:44:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1171433-strainer-floor-sink-dome>: HTTP status code is not handled or not allowed 2026-01-25 04:44:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kleen-plug-use-to-keep-beer-taps-clean-overnight-1>: HTTP status code is not handled or not allowed 2026-01-25 04:44:50 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/humidifiers_dehumidifiers/dehumidifiers_desiccators landed on page that is not a product page. 2026-01-25 04:44:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/instant-reusable-hot-compress-neck-11-x-17-12-pk>: HTTP status code is not handled or not allowed 2026-01-25 04:44:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rwg52m-kamori-leather-gloves-w-aramid-liner-cut-a4-1-pair-white-m>: HTTP status code is not handled or not allowed 2026-01-25 04:44:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-50mm-bore-x-900mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 04:44:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dishwashing-glove-large-yellow>: HTTP status code is not handled or not allowed 2026-01-25 04:44:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/needle-valve-89978-04-04-knob-adj-14-tube-x-14-swift-fit-metallic-collet>: HTTP status code is not handled or not allowed 2026-01-25 04:44:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waffle-weave-bar-towel-18x18>: HTTP status code is not handled or not allowed 2026-01-25 04:44:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-wide-span-rack-72x24x60-gray-with-3-shelves-laminated-deck-900-lb-cap-per-level>: HTTP status code is not handled or not allowed 2026-01-25 04:45:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kolorcut-cutting-board-12x18x1-2-red>: HTTP status code is not handled or not allowed 2026-01-25 04:45:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galvanized-garbage-can-20-gallon-commercial-duty>: HTTP status code is not handled or not allowed 2026-01-25 04:45:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insulated-cold-gear-coverall-4xl-black>: HTTP status code is not handled or not allowed 2026-01-25 04:45:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-extra-shelving-36x12x96-with-5-shelves-wire-deck-2>: HTTP status code is not handled or not allowed 2026-01-25 04:45:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-wide-span-rack-96x48x84-with-3-shelves-wood-deck-1100-lb-capacity-per-level>: HTTP status code is not handled or not allowed 2026-01-25 04:45:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-control-55901-5-18-screw-adj-5mm-tube-x-18-swift-fit-composite-body>: HTTP status code is not handled or not allowed 2026-01-25 04:45:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-40mm-bore-x-50mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 04:45:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-40mm-bore-x-40mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 04:45:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ecos-liquid-laundry-detergent-free-clear-2x-5-gallon-pail>: HTTP status code is not handled or not allowed 2026-01-25 04:45:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 65584 pages (at 96 pages/min), scraped 32932 items (at 47 items/min) 2026-01-25 04:45:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-red-stainless-steel-temperature-sensor>: HTTP status code is not handled or not allowed 2026-01-25 04:45:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-6432-double-acting-magnetic-cylinder-25mm-bore-x-80mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 04:45:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-6432-single-acting-magnetic-cylinder-12mm-bore-x-25mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 04:45:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beanie-watch-cap-xl-brown>: HTTP status code is not handled or not allowed 2026-01-25 04:45:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compression-sleeve-medium>: HTTP status code is not handled or not allowed 2026-01-25 04:45:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/hats_headwear already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:45:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-4>: HTTP status code is not handled or not allowed 2026-01-25 04:45:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/printed-portion-control-bags-friday-print-in-green-ink-7-x-65-05-mil>: HTTP status code is not handled or not allowed 2026-01-25 04:45:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-32mm-bore-x-800mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 04:45:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-density-silverware-bags-flat-packed-with-flip-top-closure-10-x-325-05-mil-lldpe>: HTTP status code is not handled or not allowed 2026-01-25 04:45:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/moto-vest-4-pockets-cottonpolyester-l-brown>: HTTP status code is not handled or not allowed 2026-01-25 04:45:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reusable-hot-cold-pack-5-x-10-case-of-12>: HTTP status code is not handled or not allowed 2026-01-25 04:45:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/hats_headwear landed on page that is not a product page. 2026-01-25 04:45:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-z-brite-wire-shelving-add-on-48-w-x-24-d-x-86-h>: HTTP status code is not handled or not allowed 2026-01-25 04:45:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pain-relief-gel-32-oz-pump-bottle>: HTTP status code is not handled or not allowed 2026-01-25 04:45:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-wide-span-rack-60x48x60-gray-with-3-shelves-laminated-deck-1200-lb-cap-per-level>: HTTP status code is not handled or not allowed 2026-01-25 04:45:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-6432-single-acting-magnetic-cylinder-8mm-bore-x-50mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 04:45:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-50mm-bore-x-15mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 04:45:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1111298-tee-swivel-ez-install-chrome-for-ts-brass-bronze-works>: HTTP status code is not handled or not allowed 2026-01-25 04:45:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-50mm-bore-x-270mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 04:45:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0285-inch-id-x-0374-inch-od-x-0032-inch-die-button-shim-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-25 04:45:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/male-y-55320-6-18-6mm-tube-x-18-swift-fit-univ-thread-composite-body>: HTTP status code is not handled or not allowed 2026-01-25 04:45:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-1066-stainless-steel-8-32-machine-hex-nut-100-box>: HTTP status code is not handled or not allowed 2026-01-25 04:45:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/needle-valve-55925-5-14-knob-adj-5mm-tube-x-14-swift-fit-composite-body>: HTTP status code is not handled or not allowed 2026-01-25 04:45:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insulated-cold-gear-jacket-2xl-blackfluorescent-yellow-green>: HTTP status code is not handled or not allowed 2026-01-25 04:45:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/diablo-11073-8-self-retracting-lifeline-leading-edge-galvanized-aluminum-rebar-hook>: HTTP status code is not handled or not allowed 2026-01-25 04:45:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metal-finishing-wheel-6-inch-x-1-inch-x-1-inch-crs-5a-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-25 04:45:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rotary-pump-drip-tray-with-hinged-cover>: HTTP status code is not handled or not allowed 2026-01-25 04:45:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-extra-heavy-duty-boltless-shelving-add-on-48-in-w-x-12-in-d-x-72-in-h>: HTTP status code is not handled or not allowed 2026-01-25 04:45:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cyclone-21046-harness-quick-connect-chest-legs-m-l>: HTTP status code is not handled or not allowed 2026-01-25 04:45:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-bulk-rack-shelf-wire-deck-72x24-gray-1>: HTTP status code is not handled or not allowed 2026-01-25 04:45:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0005-inch-brass-shim-stock-6-inch-x-100-inch-roll>: HTTP status code is not handled or not allowed 2026-01-25 04:45:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/281972-ptfe-release-sheet-pk-4-for-marshall-air-products>: HTTP status code is not handled or not allowed 2026-01-25 04:45:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/264028-kason-10065009048-pilaster-keyhole-for-kason>: HTTP status code is not handled or not allowed 2026-01-25 04:45:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-branch-tee-55210-14-38-14mm-tube-x-38-swift-fit-composite-body>: HTTP status code is not handled or not allowed 2026-01-25 04:45:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/maxima-ultra-36-automatic-barrier-arm-opener-for-10-16-12l-booms-120v>: HTTP status code is not handled or not allowed 2026-01-25 04:45:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-18-inch-square-keystock-plain-finish-12-inch-length>: HTTP status code is not handled or not allowed 2026-01-25 04:45:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-extra-shelving-48x24x60-gray-with-4-shelves-1200-lb-cap-per-shelf>: HTTP status code is not handled or not allowed 2026-01-25 04:45:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-20-ga-36-wx24-dx85-h-open-clip-style-6-shelf>: HTTP status code is not handled or not allowed 2026-01-25 04:46:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-20-ga-36-wx18-dx97-h-open-clip-style-4-shelf>: HTTP status code is not handled or not allowed 2026-01-25 04:46:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/moisture-detector-3-cr2032-batteries-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:46:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2151367-block-pusher-1-4-inch-dice>: HTTP status code is not handled or not allowed 2026-01-25 04:46:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-56-tefc-1ph-1725rpm-m6c17fk80ee-2>: HTTP status code is not handled or not allowed 2026-01-25 04:46:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-function-environmental-meter-9v-battery-black>: HTTP status code is not handled or not allowed 2026-01-25 04:46:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/streamlight-45670-portable-scene-light-120v-ac-12v-dc-yellow>: HTTP status code is not handled or not allowed 2026-01-25 04:46:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/streamlight-68751-dualie-3aa-without-batteries-mailer-yellow>: HTTP status code is not handled or not allowed 2026-01-25 04:46:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/streamlight-85175-cr123a-lithium-batteries-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:46:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 65681 pages (at 97 pages/min), scraped 32984 items (at 52 items/min) 2026-01-25 04:46:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1681423-screen-filter-11-1-8-inch-x19-1-8-inch-for-frymaster>: HTTP status code is not handled or not allowed 2026-01-25 04:46:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36x18x63-stainless-steel-solid-shelving>: HTTP status code is not handled or not allowed 2026-01-25 04:46:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/drum_lifters_dumpers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:46:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/46in-round-metal-mesh-picnic-table-red>: HTTP status code is not handled or not allowed 2026-01-25 04:46:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-aquatic-exercise-kit-jogger-belt-hand-bars-large-red>: HTTP status code is not handled or not allowed 2026-01-25 04:46:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m4-split-lock-washer-304-stainless-steel-din-127-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 04:46:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30024-25-plastic-barrier-chain-hdpe-1-5-x25-6-38mm-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:46:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-3916-10-inch-h-x-16-inch-w-x-2-inch-t-aluminum-hood-filter-ridged-baffles>: HTTP status code is not handled or not allowed 2026-01-25 04:46:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50024-25-plastic-barrier-chain-hdpe-2-x25-8-51mm-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:46:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/carts-trucks/drum_lifters_dumpers>: HTTP status code is not handled or not allowed 2026-01-25 04:46:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/suv-tailgate-salt-spreader-residential-use>: HTTP status code is not handled or not allowed 2026-01-25 04:46:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd56sn-916-inch-band-heavy-duty-4-pc-hose-clamp-2-116-inch-4-inch-clamping-diam-10-pcs>: HTTP status code is not handled or not allowed 2026-01-25 04:46:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m4-x-0-7-x-12mm-machine-screw-phillips-flat-head-304-stainless-steel-din-965-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 04:46:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inchmetric-10-inch-trupunchpunch-and-die-set>: HTTP status code is not handled or not allowed 2026-01-25 04:46:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1201026-caster-stem-3-1-2-inch-gry>: HTTP status code is not handled or not allowed 2026-01-25 04:46:29 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-25 04:46:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hog-heaven-fashion-mat-seven-eighths-3x5-cocoa-brown>: HTTP status code is not handled or not allowed 2026-01-25 04:46:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b24hsp-300-series-ss-hose-clamp-1-116-inch-2-inch-clamping-diameter-box-of-10>: HTTP status code is not handled or not allowed 2026-01-25 04:46:30 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:33 2026-01-25 04:46:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-tecgen-flame-resistant-work-shirt-l-long-tall-tan>: HTTP status code is not handled or not allowed 2026-01-25 04:46:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m5-x-0-8-x-10mm-machine-screw-phillips-flat-head-304-stainless-steel-din-965-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 04:46:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qr104h-quick-release-part-ss-hose-clamp-1-34-inch-7-inch-clamping-diam-10-pcs>: HTTP status code is not handled or not allowed 2026-01-25 04:46:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qr56h-quick-release-part-ss-hose-clamp-1-inch-4-inch-clamping-diam-10-pcs>: HTTP status code is not handled or not allowed 2026-01-25 04:46:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-tecgen-flame-resistant-work-shirt-5xl-regular-tan>: HTTP status code is not handled or not allowed 2026-01-25 04:46:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m5-x-0-8-x-20mm-socket-head-cap-screw-304-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 04:46:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eco-products-bluestripe-recycled-content-clear-plastic-cold-drink-cups-12oz-clear-1000ctn>: HTTP status code is not handled or not allowed 2026-01-25 04:46:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/comfy-elbow-orthoses-pediatric-medium-with-one-cover>: HTTP status code is not handled or not allowed 2026-01-25 04:46:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd12s-916-inch-band-heavy-duty-3-pc-hose-clamp-916-inch-1-14-inch-clampingdiam-10-pcs>: HTTP status code is not handled or not allowed 2026-01-25 04:46:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/solo-cup-company-bare-paper-dinnerware-85-plate-greentan-250carton>: HTTP status code is not handled or not allowed 2026-01-25 04:46:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-tecgen-flame-resistant-work-shirt-5xl-tall-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:46:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-hot-box-bulk-material-heat-oven-hb54-1200-54-cu-ft>: HTTP status code is not handled or not allowed 2026-01-25 04:46:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-56-tefc-3ph-1725-1425rpm-c6t17fc114dd>: HTTP status code is not handled or not allowed 2026-01-25 04:46:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6121216-screw-sm-th-ss-100>: HTTP status code is not handled or not allowed 2026-01-25 04:46:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sentinel-conversion-kit-8-plates-hoisting-rings-box-rack-hardware>: HTTP status code is not handled or not allowed 2026-01-25 04:46:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/penetrator-cart-quick-brake-10-l-x-10-w-x-10-h>: HTTP status code is not handled or not allowed 2026-01-25 04:46:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-56-tefc-1ph-1725rpm-c6c17fc7mm>: HTTP status code is not handled or not allowed 2026-01-25 04:46:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-inch-x-1-inch-x-0002-inch-red-plastic-color-coded-slotted-shim-pack-of-20>: HTTP status code is not handled or not allowed 2026-01-25 04:46:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1171535-filter-water-system-cfs02-for-cuno-inc>: HTTP status code is not handled or not allowed 2026-01-25 04:46:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-flame-resistant-dh-air153-work-shirt-2xl-short-khaki>: HTTP status code is not handled or not allowed 2026-01-25 04:46:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metal-bookcase-five-shelf-34-1-2-w-x-12-5-8-d-x-71-h-putty>: HTTP status code is not handled or not allowed 2026-01-25 04:46:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/solo-cup-company-plastic-souffle-portion-cups-2oz-translucent-250bag>: HTTP status code is not handled or not allowed 2026-01-25 04:46:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metal-bookcase-three-shelf-34-1-2-w-x-12-5-8-d-x-41-h-putty>: HTTP status code is not handled or not allowed 2026-01-25 04:46:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thread-repair-kit-for-metal-standard-wall-m6-1-0-x-3-8-16-ez-650-6>: HTTP status code is not handled or not allowed 2026-01-25 04:47:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011239-gasket-door-22-1-2-inch-x-31-1-2-inch-for-beverage-air>: HTTP status code is not handled or not allowed 2026-01-25 04:47:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-hang-type-magnetic-sweeper-with-forkli-hanger-84-w>: HTTP status code is not handled or not allowed 2026-01-25 04:47:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/irwin-marathon-14080>: HTTP status code is not handled or not allowed 2026-01-25 04:47:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/size-b-3-inch-x-3-inch-stainless-steel-slotted-shim-260-piece-full-assortment>: HTTP status code is not handled or not allowed 2026-01-25 04:47:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s9623-95w-led-br30-reflector-medium-base-105-beam-bpread-5000k-dimmable>: HTTP status code is not handled or not allowed 2026-01-25 04:47:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s9177-08w-led-t6-indicator-bulb-candelabra-base-clear-2700k>: HTTP status code is not handled or not allowed 2026-01-25 04:47:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hydronic-in-wall-heater-h612-6-8-as-fs-gw-w-aqua-fan-switch-120v-6000-7600-btu>: HTTP status code is not handled or not allowed 2026-01-25 04:47:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009585-door-gasket-for-true-manufacturing>: HTTP status code is not handled or not allowed 2026-01-25 04:47:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 65779 pages (at 98 pages/min), scraped 33033 items (at 49 items/min) 2026-01-25 04:47:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-display-countertop-4-tier-black>: HTTP status code is not handled or not allowed 2026-01-25 04:47:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spku-lift-universal-forklift-spill-kit>: HTTP status code is not handled or not allowed 2026-01-25 04:47:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/respiratory_protection/reusable_face_masks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:47:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1200va-smart-pro-rack-tower-digital-lcd-8-outlet>: HTTP status code is not handled or not allowed 2026-01-25 04:47:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-double-air-actuated-valve-pilot-spring-centered-pressure-center-g1-8-ports>: HTTP status code is not handled or not allowed 2026-01-25 04:47:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/twin-glazed-porcelain-socket-w-flange-bushing-cap-and-9-in-leads>: HTTP status code is not handled or not allowed 2026-01-25 04:47:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-carabiner-1-2>: HTTP status code is not handled or not allowed 2026-01-25 04:47:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010417-basket-hanger>: HTTP status code is not handled or not allowed 2026-01-25 04:47:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-panel-divider-6-ft-high-x-9-ft-5-inch-long-fabric-black>: HTTP status code is not handled or not allowed 2026-01-25 04:47:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/respiratory_protection/reusable_face_masks>: HTTP status code is not handled or not allowed 2026-01-25 04:47:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-2120-sandpaper-120-grit-fits-22-44-model-sanders>: HTTP status code is not handled or not allowed 2026-01-25 04:47:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1331758-lidfood-pan-w-hndl-full-clear-for-carter-hoffmann>: HTTP status code is not handled or not allowed 2026-01-25 04:47:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shop-stool-with-manual-height-adjustment-2>: HTTP status code is not handled or not allowed 2026-01-25 04:47:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/575931-sanding-sleeves-2x5-1-2-60-grit-4-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:47:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-horizontallamp-support-clip>: HTTP status code is not handled or not allowed 2026-01-25 04:47:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-knob>: HTTP status code is not handled or not allowed 2026-01-25 04:47:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lab-pack-open-head-poly-drum-plastic-lever-lock-10-gallon-capacity-black>: HTTP status code is not handled or not allowed 2026-01-25 04:47:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/13-panel-divider-8-ft-high-x-24-ft-1-inch-long-fabric-black>: HTTP status code is not handled or not allowed 2026-01-25 04:47:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lab-pack-open-head-poly-drum-metal-lever-lock-20-gallon-capacity-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:47:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dyn12-06l-dynamo-industrial-led-emergency-unit-wet-rated-6w-led-lamps>: HTTP status code is not handled or not allowed 2026-01-25 04:47:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/partitions_dividers/room_dividers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:47:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-adjustable-height-steel-gantry-crane-9-10w-x-7-6-12h-6000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 04:47:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-ft-high-dry-erase-overlay>: HTTP status code is not handled or not allowed 2026-01-25 04:47:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/augusta-steel-outdoor-trash-can-with-flat-top-lid-36-gallon-capacity-black>: HTTP status code is not handled or not allowed 2026-01-25 04:47:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/46-round-bar-height-table-with-4-seats-in-ground-mount-diamond-pattern-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:47:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/guest_chairs/guest_reception_chairs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:47:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/74-1203-magnetic-door-gasket-28-1-16-inch-x-61-3-8-inch>: HTTP status code is not handled or not allowed 2026-01-25 04:47:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ptfe-single-turn-backup-ring-dash-121-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 04:47:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/definite_purpose_electric_motors/washdown_motors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:47:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/partitions_dividers/room_dividers>: HTTP status code is not handled or not allowed 2026-01-25 04:47:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keyless-glazed-porcelain-ceiling-receptacle-2-screw-terminal>: HTTP status code is not handled or not allowed 2026-01-25 04:47:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/denali-38-12-square-recycled-picnic-table-with-4-seats-in-ground-mount-brown>: HTTP status code is not handled or not allowed 2026-01-25 04:47:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/push-to-connect-tube-fitting-nylon-plastic-bulkhead-straight-connector-6mm-tube-od>: HTTP status code is not handled or not allowed 2026-01-25 04:47:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washdown-motor-56-tefc-1ph-3450rpm-c6c34wk4mm>: HTTP status code is not handled or not allowed 2026-01-25 04:47:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/guest_chairs/guest_reception_chairs landed on page that is not a product page. 2026-01-25 04:47:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1291011-filter-mesh-16x20x1-inch-galvanized>: HTTP status code is not handled or not allowed 2026-01-25 04:47:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ptfe-syringe-filter-0-22um-30mm-bulk-packed-non-sterile-100-case>: HTTP status code is not handled or not allowed 2026-01-25 04:47:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylon-syringe-filter-0-22um-30mm-bulk-packed-non-sterile-100-case>: HTTP status code is not handled or not allowed 2026-01-25 04:47:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-square-plate-washer-i-d-3-8-thick-steel-hot-dip-galvanized-grade-2-pkg-of-10>: HTTP status code is not handled or not allowed 2026-01-25 04:47:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-ft-extension-cord-16-2-spt-2-white>: HTTP status code is not handled or not allowed 2026-01-25 04:47:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2541033-shelf-19-3-4-inch-x-19-3-4-inch-for-glasspro>: HTTP status code is not handled or not allowed 2026-01-25 04:47:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/definite_purpose_electric_motors/washdown_motors landed on page that is not a product page. 2026-01-25 04:47:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-16-split-lock-washer-21-64-i-d-3-32-thick-steel-galvanized-grade-2-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 04:47:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012340-grease-filter-stainless-steel-16-x-16>: HTTP status code is not handled or not allowed 2026-01-25 04:47:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traex-plastic-condiment-dispenser-w-standard-lid-4743-01-brown-pint-insert-size>: HTTP status code is not handled or not allowed 2026-01-25 04:47:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-round-table-with-2-imme-armless-barstools-light-gray-seatriver-cherry-top>: HTTP status code is not handled or not allowed 2026-01-25 04:48:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-square-table-with-4-imme-armless-caster-chairs-navy-seatmaple-top>: HTTP status code is not handled or not allowed 2026-01-25 04:48:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/back-panel-36-x-97>: HTTP status code is not handled or not allowed 2026-01-25 04:48:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-1-1-2-hp-thru-bolt-115-230v>: HTTP status code is not handled or not allowed 2026-01-25 04:48:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-hp-single-phase-energy-efficient-spa-motor>: HTTP status code is not handled or not allowed 2026-01-25 04:48:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traex-kondi-keeper-dispenser-insert-4801>: HTTP status code is not handled or not allowed 2026-01-25 04:48:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-shelf-level-boltless-wood-deck-1>: HTTP status code is not handled or not allowed 2026-01-25 04:48:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-spoon-46995-21-long-18-gauge>: HTTP status code is not handled or not allowed 2026-01-25 04:48:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/back-end-stops-60-x-36>: HTTP status code is not handled or not allowed 2026-01-25 04:48:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traex-dripcut-sugar-packet-holders-wr-1009-chrome-plated-w-o-handles>: HTTP status code is not handled or not allowed 2026-01-25 04:48:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 65872 pages (at 93 pages/min), scraped 33075 items (at 42 items/min) 2026-01-25 04:48:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-x-24-adjustable-height-shop-stand-1>: HTTP status code is not handled or not allowed 2026-01-25 04:48:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silicone-rubber-cord-4mm-cross-section-10-ft-length>: HTTP status code is not handled or not allowed 2026-01-25 04:48:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8000r-random-case-sealer-with-accuglide-v-taping-head-side-belt-drive-2w-tape>: HTTP status code is not handled or not allowed 2026-01-25 04:48:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/322175-strip-curtain-clear-flex-40-inch-x-84-inch-8-inch>: HTTP status code is not handled or not allowed 2026-01-25 04:48:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-8-drawer-standard-width-cabinet-blue-no-lock>: HTTP status code is not handled or not allowed 2026-01-25 04:48:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-9-drawer-standard-width-cabinet-blue-master-keyed-2>: HTTP status code is not handled or not allowed 2026-01-25 04:48:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traex-bar-keep-storage-jar-3664-2-quarts>: HTTP status code is not handled or not allowed 2026-01-25 04:48:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mixer-flat-beater-xmix0703-for-7-quart-mixer>: HTTP status code is not handled or not allowed 2026-01-25 04:48:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/304ss-straight-adapters-tubeto-fnptfor-quick-clamp-fittings-for-34-tubeodx-34-npt-female>: HTTP status code is not handled or not allowed 2026-01-25 04:48:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clean-day-liquid-hand-soap-lemon-12-5-oz-pump-6-carton-651321>: HTTP status code is not handled or not allowed 2026-01-25 04:48:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-round-table-with-4-imme-armless-barstools-white-seatstudio-teak-top>: HTTP status code is not handled or not allowed 2026-01-25 04:48:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traex-squeeze-dispensers-2912-13-standard-12-oz>: HTTP status code is not handled or not allowed 2026-01-25 04:48:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rear-caster-10001941>: HTTP status code is not handled or not allowed 2026-01-25 04:48:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/distribution-container-with-hinged-lid-21-1-2x15-1-4x9-1-2>: HTTP status code is not handled or not allowed 2026-01-25 04:48:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-width-wall-mounted-hanger-style-coat-rack-sandstone>: HTTP status code is not handled or not allowed 2026-01-25 04:48:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/squeeze-dishers-47377-black-1-7-8-bowl-diameter>: HTTP status code is not handled or not allowed 2026-01-25 04:48:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/lathes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:48:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/circular-wire-flight-caddy-wr-1026-06-holds-6-glasses-9-x-11>: HTTP status code is not handled or not allowed 2026-01-25 04:48:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/suzuki-vanvan-200-200cc-motorcycle-replacement-battery-2017>: HTTP status code is not handled or not allowed 2026-01-25 04:48:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-container-without-lid-23-1-2x15-1-2x12>: HTTP status code is not handled or not allowed 2026-01-25 04:48:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-starter-rack-48x36x96-with-3-levels-wood-deck-1500lb-cap-per-shelf-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:48:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-starter-rack-72x24x96-with-3-level-steel-deck-1000lb-cap-per-shelf-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:48:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lathes-jet321439>: HTTP status code is not handled or not allowed 2026-01-25 04:48:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012604-sr-burner-w-nat-orifice-for-garland-range>: HTTP status code is not handled or not allowed 2026-01-25 04:48:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/machining/machinery/lathes>: HTTP status code is not handled or not allowed 2026-01-25 04:48:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/stacking_chairs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:48:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2171206-kit-pump-repair-f-217-1126-for-server-products>: HTTP status code is not handled or not allowed 2026-01-25 04:48:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/690082-jtm-1-vertical-milling-machine-2-hp-230v-3-phase>: HTTP status code is not handled or not allowed 2026-01-25 04:48:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cayenne-model-1001-food-warmer-71001-700-watt-5-8-amp>: HTTP status code is not handled or not allowed 2026-01-25 04:48:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-square-table-with-4-imme-armless-caster-chairs-white-seatriver-cherry-top>: HTTP status code is not handled or not allowed 2026-01-25 04:48:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-add-on-rack-60x36x96-with-3-levels-steel-deck-1300lb-cap-per-level-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:48:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/577004-j-41002-2w-x-42l-bench-belt-8-disc-sander-3-4-hp>: HTTP status code is not handled or not allowed 2026-01-25 04:48:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/seating/task_desk_chairs_1/stacking_chairs>: HTTP status code is not handled or not allowed 2026-01-25 04:48:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjust-stepmate-stand-stainless-steel-ahwl2436ss>: HTTP status code is not handled or not allowed 2026-01-25 04:48:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modern-chair-mid-back-leather-guest-chair-black>: HTTP status code is not handled or not allowed 2026-01-25 04:48:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adj-workmate-stand-w-ergo-mat-ahtl2436>: HTTP status code is not handled or not allowed 2026-01-25 04:48:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/sinks/hand_sinks_wash_stations already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:48:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-skid-carousel-turntable-rotating-ring-2000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 04:48:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-level-96x24-high-capacity-rack-wood-deck-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:48:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/complete-packing-table-with-butcher-block-square-edge-top-70-x-36>: HTTP status code is not handled or not allowed 2026-01-25 04:48:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mega-wing-brush-sections-set-for-72-mega-broom-forklift-broom-sweepers-set-of-11>: HTTP status code is not handled or not allowed 2026-01-25 04:48:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/privacy-netting-wreinforced-grommets-6-x-50-black-pn-30056>: HTTP status code is not handled or not allowed 2026-01-25 04:48:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-fighter-2-hour-digital-fire-water-resistant-safe-19-48-cu-ft-off-white-steel>: HTTP status code is not handled or not allowed 2026-01-25 04:48:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/plumbing/sinks/hand_sinks_wash_stations>: HTTP status code is not handled or not allowed 2026-01-25 04:48:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wear-ever-fry-pan-with-powercoat-2-interior-s4007-8-gauge-4-3-4-bottom-diameter>: HTTP status code is not handled or not allowed 2026-01-25 04:48:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rail-kit-two-one-procart-40-l-x-19-1-2-w>: HTTP status code is not handled or not allowed 2026-01-25 04:48:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-1756-mesh-filter-20-inch-x-20-inch-x-2-inch>: HTTP status code is not handled or not allowed 2026-01-25 04:48:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tap-holder-w-clutch-1-8-inch-no-0-6-m3-m3-15-at5506>: HTTP status code is not handled or not allowed 2026-01-25 04:49:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/norton-66253044535-gemini-bench-and-pedestal-wheel-8-inch-x-1-inch-x-1-inch-120-grit-silicon-carbide>: HTTP status code is not handled or not allowed 2026-01-25 04:49:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traex-cutlery-box-52654-polyethylene-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:49:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/norton-66243510645-gemini-small-diameter-cut-off-wheel-3-inch-x-1-16-inch-x-1-4-inch-36-grit-alum-oxide-type-1>: HTTP status code is not handled or not allowed 2026-01-25 04:49:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-water-resistant-key-lock-storage-cabinet-44-h-10-cu-ft-putty-steel>: HTTP status code is not handled or not allowed 2026-01-25 04:49:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1421652-handle-floor-brushfiberglass>: HTTP status code is not handled or not allowed 2026-01-25 04:49:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-cable-tie-30-175-lb-25-pk>: HTTP status code is not handled or not allowed 2026-01-25 04:49:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chill-its-multi-band-face-cover-neck-gaiter-gray-heather>: HTTP status code is not handled or not allowed 2026-01-25 04:49:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/371-box-sealing-tape-2-1316-x-110-yds-18-mil-clear>: HTTP status code is not handled or not allowed 2026-01-25 04:49:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 65970 pages (at 98 pages/min), scraped 33121 items (at 46 items/min) 2026-01-25 04:49:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-body-saw-with-24-32-tooth-blades-14-npt>: HTTP status code is not handled or not allowed 2026-01-25 04:49:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/norton-66252842027-gemini-cutting-wheel-4-1-2-inch-x-3-32-inch-x-5-8-11-inch-30-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-25 04:49:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-12-belt-sander>: HTTP status code is not handled or not allowed 2026-01-25 04:49:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011658-contactor-208-240v-for-lincoln-manufacturing>: HTTP status code is not handled or not allowed 2026-01-25 04:49:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/item-pf39030001-charlie-pendant-control-station>: HTTP status code is not handled or not allowed 2026-01-25 04:49:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-dock-board-with-steel-curbs-13>: HTTP status code is not handled or not allowed 2026-01-25 04:49:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rw30-es1y-general-purpose-rain-bib-pants-hi-viz-lime-4xl>: HTTP status code is not handled or not allowed 2026-01-25 04:49:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cordless-d-handle-jig-saw-tool-only>: HTTP status code is not handled or not allowed 2026-01-25 04:49:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/cookware_bakeware/chafing_dishes_fuel already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:49:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acry-fab-vertical-lid-holder-fml-3v-3-column-19-3-8-x-6-1-2-x-5-3-8>: HTTP status code is not handled or not allowed 2026-01-25 04:49:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ignition-20-reactiv-task-chair-mid-back-17-14-21-34h-seat-fern>: HTTP status code is not handled or not allowed 2026-01-25 04:49:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acry-fab-vertical-lid-holder-fml-4v-4-column-25-1-2-x-6-1-2-x-5-3-8>: HTTP status code is not handled or not allowed 2026-01-25 04:49:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/eye_protection/eyewear_cleaning_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:49:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fp91-deflector-disposable-foam-earplugs-33db-corded-light-green>: HTTP status code is not handled or not allowed 2026-01-25 04:49:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drafting-table-36-long-24-wide-36-high-1-piece-top-large-drawer>: HTTP status code is not handled or not allowed 2026-01-25 04:49:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gpx-sporty-cd-and-radio-boombox>: HTTP status code is not handled or not allowed 2026-01-25 04:49:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/graphic-facility-signs-emergency-eye-wash-vinyl-7x7>: HTTP status code is not handled or not allowed 2026-01-25 04:49:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-use-indoor-outdoor-small-yellow-box-2-lights-2-int-sensors-15-cord>: HTTP status code is not handled or not allowed 2026-01-25 04:49:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/cookware_bakeware/chafing_dishes_fuel landed on page that is not a product page. 2026-01-25 04:49:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-x-72-table-with-6-imme-armless-caster-chairs-sky-blue-seatstudio-teak-top>: HTTP status code is not handled or not allowed 2026-01-25 04:49:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cipher153-task-chair-mid-back-15-20h-seat-basalt>: HTTP status code is not handled or not allowed 2026-01-25 04:49:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-rack-tear-drop-beam-pair-108-l-1>: HTTP status code is not handled or not allowed 2026-01-25 04:49:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/exit_devices already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:49:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/eye_protection/eyewear_cleaning_accessories>: HTTP status code is not handled or not allowed 2026-01-25 04:49:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-step-angled-ladder-l65>: HTTP status code is not handled or not allowed 2026-01-25 04:49:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/al-stratus-cone-diffuser-with-10-neck-white-str-c-10w>: HTTP status code is not handled or not allowed 2026-01-25 04:49:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ptp211yl-tough-mark-strip>: HTTP status code is not handled or not allowed 2026-01-25 04:49:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_hinges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:49:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ptp206wt-tough-mark-strip>: HTTP status code is not handled or not allowed 2026-01-25 04:49:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vpl901lensled-sign-projector-lens-only>: HTTP status code is not handled or not allowed 2026-01-25 04:49:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/fasteners/door_window_hardware/exit_devices>: HTTP status code is not handled or not allowed 2026-01-25 04:49:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/staples-ns18-1-18-gauge-1-1000-bx>: HTTP status code is not handled or not allowed 2026-01-25 04:49:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1711271-ring-egg-10-inch-dia-omelet-for-prince-castle>: HTTP status code is not handled or not allowed 2026-01-25 04:49:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2561145-shelf-ref-16-1-2-inch-x24-3-8-inch-x6-inch-for-silver-king>: HTTP status code is not handled or not allowed 2026-01-25 04:49:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tac812-5s-red-tag-center>: HTTP status code is not handled or not allowed 2026-01-25 04:49:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gml110rd-gauge-marking-decal>: HTTP status code is not handled or not allowed 2026-01-25 04:49:47 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_hinges landed on page that is not a product page. 2026-01-25 04:49:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-stainless-steel-lug-cart-capacity-3-lug>: HTTP status code is not handled or not allowed 2026-01-25 04:49:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-w-x-20-d-x-10-h-stackbox-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:49:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2271129-hp-bw-blower-motor-for-henny-penny>: HTTP status code is not handled or not allowed 2026-01-25 04:49:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6x9-2-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 04:49:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vertical-storage-folding-chair-dolly-35-chairs-capacity>: HTTP status code is not handled or not allowed 2026-01-25 04:49:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m3-series-half-solid-door-refrigerator-2-door>: HTTP status code is not handled or not allowed 2026-01-25 04:49:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kee-klamp-rail-support-1>: HTTP status code is not handled or not allowed 2026-01-25 04:49:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/packaging-economy-magnetic-dispenser-18in-size>: HTTP status code is not handled or not allowed 2026-01-25 04:49:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/psr527-slip-gard-marking-kit>: HTTP status code is not handled or not allowed 2026-01-25 04:49:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1331553-cart-oil-short-shuttle-115-inch-h-for-worchester-industrial-products>: HTTP status code is not handled or not allowed 2026-01-25 04:50:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onix-plus-safety-glasses-anti-fog-shade-50-ir-green-lensblack-frame>: HTTP status code is not handled or not allowed 2026-01-25 04:50:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kee-klamp-medium-flange-4>: HTTP status code is not handled or not allowed 2026-01-25 04:50:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-deluxe-series-worktop-refrigerator-48-long-2-door>: HTTP status code is not handled or not allowed 2026-01-25 04:50:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1041129-organizer-4-lid-w-straw-hldr>: HTTP status code is not handled or not allowed 2026-01-25 04:50:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10194978-harness-back-d-ring-tongue-buckle-leg-straps-extra-large>: HTTP status code is not handled or not allowed 2026-01-25 04:50:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m3-series-solid-door-refrigerator-2-door>: HTTP status code is not handled or not allowed 2026-01-25 04:50:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16x6x6-corrugated-mailers>: HTTP status code is not handled or not allowed 2026-01-25 04:50:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-starter-72in-w-x-24in-d-x-72in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-25 04:50:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 66063 pages (at 93 pages/min), scraped 33163 items (at 42 items/min) 2026-01-25 04:50:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coast-21531-model-fl75r-rechargeable-dual-color-530-lumen-headlamp-kit>: HTTP status code is not handled or not allowed 2026-01-25 04:50:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cantilever-rack-single-sided-upright-12>: HTTP status code is not handled or not allowed 2026-01-25 04:50:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10194872-harness-back-hip-d-rings-quick-connect-leg-straps-extra-small>: HTTP status code is not handled or not allowed 2026-01-25 04:50:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-1-2-x4-1-2-x72-telescoping-outer-boxes>: HTTP status code is not handled or not allowed 2026-01-25 04:50:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blickman-paul-instrument-medical-supply-cabinet-35-5-8-x-16-x-79-1-4-glass-shelves>: HTTP status code is not handled or not allowed 2026-01-25 04:50:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spillmagic-sm106-spillmagic-3-lb-bag-resealable>: HTTP status code is not handled or not allowed 2026-01-25 04:50:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cantilever-rack-horizontal-brace-set-9>: HTTP status code is not handled or not allowed 2026-01-25 04:50:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10194968-harness-back-shoulder-d-rings-quick-connect-leg-straps-extra-small>: HTTP status code is not handled or not allowed 2026-01-25 04:50:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/atlantic-olefin-entrance-carpet-mat-3-ft-x-6-ft-dark-toast>: HTTP status code is not handled or not allowed 2026-01-25 04:50:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-inch-carbon-steel-main-ball-with-1-4-inch-stud-in-a-carbon-steel-housing>: HTTP status code is not handled or not allowed 2026-01-25 04:50:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/weblok-assembly-867025-4-synthetic-to-attachment-3-4-18750-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-25 04:50:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-eye-rigging-hook-without-latch-10000-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-25 04:50:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-1191-on-off-lighted-rocker-switch-20a-250v>: HTTP status code is not handled or not allowed 2026-01-25 04:50:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/bathroom_partition_replacement_hardware already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:50:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galvanized-shelf-truck-36x24x69-1200-pound-capacity>: HTTP status code is not handled or not allowed 2026-01-25 04:50:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-21-x-60-blue-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:50:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ratchet-lever-hoist-3-ton-capacity-20-lift>: HTTP status code is not handled or not allowed 2026-01-25 04:50:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-plastic-chain-red>: HTTP status code is not handled or not allowed 2026-01-25 04:50:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galaxy-12-gal-box-carpet-extractor-220-psi-pump-185-inch-waterlift-without-heat-2000ix-hr>: HTTP status code is not handled or not allowed 2026-01-25 04:50:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dynamic-2-1-2-fluid-glycerine-filled-pressure-gage-panel-flange-100-psi>: HTTP status code is not handled or not allowed 2026-01-25 04:50:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/bathroom_partition_replacement_hardware>: HTTP status code is not handled or not allowed 2026-01-25 04:50:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/541179-oven-control-for-garland-range>: HTTP status code is not handled or not allowed 2026-01-25 04:50:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dynamic-2-1-2-fluid-glycerine-filled-pressure-gage-stem-300-psi>: HTTP status code is not handled or not allowed 2026-01-25 04:50:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-8-stick-feed-staples>: HTTP status code is not handled or not allowed 2026-01-25 04:50:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-add-on-42in-w-x-30in-d-x-96in-h-no-deck>: HTTP status code is not handled or not allowed 2026-01-25 04:50:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-14-x-48-light-grey-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:50:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-18-x-72-black-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:50:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-utility-basket-mail-cart-200-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 04:50:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-21-x-48-light-blue-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:50:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/120-x-30-extra-long-bench-shop-top-1>: HTTP status code is not handled or not allowed 2026-01-25 04:50:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergomat-complete-smooth-2-x-4>: HTTP status code is not handled or not allowed 2026-01-25 04:50:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chapin-200-lb-capacity-stainless-steel-professional-rock-salt-drop-spreader>: HTTP status code is not handled or not allowed 2026-01-25 04:50:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-30-x-48-grey-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:50:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durastripe-32-octagone-sign-stop-watch-for-forklifts>: HTTP status code is not handled or not allowed 2026-01-25 04:50:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-triangle-24-x-24-blue-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:50:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24x24x2-white-easy-fold-mailers>: HTTP status code is not handled or not allowed 2026-01-25 04:50:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durastripe-20-round-sign-hearing-protection-required>: HTTP status code is not handled or not allowed 2026-01-25 04:50:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wide-span-rack-60x36x96-with-3-shelves-wire-deck-1200-lb-capacity-per-level>: HTTP status code is not handled or not allowed 2026-01-25 04:50:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-wire-deck-shelf-60x48>: HTTP status code is not handled or not allowed 2026-01-25 04:51:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3x6-2-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 04:51:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/262914-rotor-shredder-for-insinkerator-disposer>: HTTP status code is not handled or not allowed 2026-01-25 04:51:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durastripe-12x9-rectangle-3-mph-speed-limit>: HTTP status code is not handled or not allowed 2026-01-25 04:51:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durastripe-16-round-sign-no-pedestrian-traffic>: HTTP status code is not handled or not allowed 2026-01-25 04:51:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-dbl-solenoid-vlv-ext-pilot-pres-cntr-spr-cntred-g1-8-24v-dc-2w-coil-led-conn>: HTTP status code is not handled or not allowed 2026-01-25 04:51:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durastripe-20-round-sign-no-cell-phones-no-text>: HTTP status code is not handled or not allowed 2026-01-25 04:51:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-starter-72inw-x-48ind-x-60inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 04:51:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-heavy-duty-boltless-shelving-starter-60inw-x-24ind-x-72inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 04:51:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/shelving_accessories/shelving_enclosure_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:51:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-dbl-solenoid-valve-pilot-spring-closed-cntr-1-4-nptf-24v-dc-3w-coil-black-conn>: HTTP status code is not handled or not allowed 2026-01-25 04:51:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 66160 pages (at 97 pages/min), scraped 33213 items (at 50 items/min) 2026-01-25 04:51:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-starter-60inw-x-24ind-x-96inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 04:51:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-high-capacity-bulk-rack-with-wood-decking-72x48x72-add-on>: HTTP status code is not handled or not allowed 2026-01-25 04:51:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-starter-60inw-x-48ind-x-96inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 04:51:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-starter-96inw-x-24ind-x-96inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 04:51:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-starter-96inw-x-36ind-x-84inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 04:51:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-high-capacity-bulk-rack-with-steel-decking-96x24x72-starter>: HTTP status code is not handled or not allowed 2026-01-25 04:51:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/shelving/shelving_accessories/shelving_enclosure_kits>: HTTP status code is not handled or not allowed 2026-01-25 04:51:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-titanium-non-stick-straight-scissors>: HTTP status code is not handled or not allowed 2026-01-25 04:51:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kit2sc11xl10-12-cal-arc-flash-kit-w-short-coat-bib-overall-in-ultrasoft-xl-sz-10>: HTTP status code is not handled or not allowed 2026-01-25 04:51:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kit4sc40ng2x-40-cal-compliance-arc-flash-kit-short-coat-bib-overall-2xl-no-gloves>: HTTP status code is not handled or not allowed 2026-01-25 04:51:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sc-drawer-layout-24-plastic-boxes-2-h>: HTTP status code is not handled or not allowed 2026-01-25 04:51:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-add-on-60inw-x-24ind-x-72inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 04:51:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-3-awg-thhn-black-white-green-aluminum-mc>: HTTP status code is not handled or not allowed 2026-01-25 04:51:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-2-single-solenoid-pilot-valve-spring-return-1-4-24v-dc-3w-coil-led-connection>: HTTP status code is not handled or not allowed 2026-01-25 04:51:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-heavy-duty-boltless-shelving-add-on-60inw-x-48ind-x-72inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 04:51:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sc-drawer-layout-11-compartments-3-h>: HTTP status code is not handled or not allowed 2026-01-25 04:51:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-12-x-72-blue-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:51:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-add-on-48inw-x-24ind-x-96inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 04:51:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-long-sleeve-t-shirt-3xl-navy-c541nnbls3xl>: HTTP status code is not handled or not allowed 2026-01-25 04:51:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-liners-12-x-72-white-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 04:51:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-16-14-x-4-1-2-hex-head-cap-screw-steel-zinc-yellow-unc-grade-8-usa-25-pk-454244>: HTTP status code is not handled or not allowed 2026-01-25 04:51:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2561154-pad-touch-membrane-for-silver-king>: HTTP status code is not handled or not allowed 2026-01-25 04:51:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8015213-solenoid-valve-inlet-120v-2gpm-for-wilbur-curtis-co>: HTTP status code is not handled or not allowed 2026-01-25 04:51:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metromax-4-48-w-x-24-d-x-54-h-mobile-unit>: HTTP status code is not handled or not allowed 2026-01-25 04:51:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-high-capacity-bulk-rack-with-wire-decking-72x48x120-add-on>: HTTP status code is not handled or not allowed 2026-01-25 04:51:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-1-2-npt-double-solenoid-valve-pilot-24v-ac-5va-coil-black-connection>: HTTP status code is not handled or not allowed 2026-01-25 04:51:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-g-1-4-double-solenoid-valve-pilot-220v-ac-5va-coil-led-connection>: HTTP status code is not handled or not allowed 2026-01-25 04:51:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/k93218-filter-regulator-1-in-npt>: HTTP status code is not handled or not allowed 2026-01-25 04:51:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ws20-waveseal-360-constant-tension-hose-clamp-103-156-dia-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 04:51:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/roof_pipe_heating_cables already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:51:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11-1-8-x-8-3-4-x-4-kraft-literature-mailers>: HTTP status code is not handled or not allowed 2026-01-25 04:51:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-pressure-cntr-g1-2-dbl-solenoid-valve-pilot-spr-cntred-24v-ac-5va-coil-led-conn>: HTTP status code is not handled or not allowed 2026-01-25 04:51:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-computer-workstation-1>: HTTP status code is not handled or not allowed 2026-01-25 04:51:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pbi-glove-with-thermobest-cuff-large-brown-g51pclw13714>: HTTP status code is not handled or not allowed 2026-01-25 04:51:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2801790-adaptor-ac-hd-400-lb-scale>: HTTP status code is not handled or not allowed 2026-01-25 04:51:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-wire-shelf-basket-48-w-x-24-d-x-63-h>: HTTP status code is not handled or not allowed 2026-01-25 04:51:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/bar_restaurant_equipment/bartending_supplies/bar_mats_shelf_liners already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:51:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/roof_pipe_heating_cables>: HTTP status code is not handled or not allowed 2026-01-25 04:51:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kit2cv08ng-lg-8-cal-cm2-arc-flash-kit-with-fr-coverall-lg-no-gloves>: HTTP status code is not handled or not allowed 2026-01-25 04:51:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-hose-female-male-reducer-adapter-369-2521521-2-1-2-nh-x-1-1-2-nh-brass>: HTTP status code is not handled or not allowed 2026-01-25 04:51:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-pressure-cntr-g1-4-dbl-solenoid-valve-pilot-spr-cntred-24v-ac-5va-coil-led-conn>: HTTP status code is not handled or not allowed 2026-01-25 04:51:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-16-x-12-x-900-feet-upsable-perforated-air-foam-roll>: HTTP status code is not handled or not allowed 2026-01-25 04:52:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commercial-wedge-anchor-1-2-13-x-3-3-4-steel-zinc-pkg-of-50-279110>: HTTP status code is not handled or not allowed 2026-01-25 04:52:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/structural-plastic-shelving-36x16x51>: HTTP status code is not handled or not allowed 2026-01-25 04:52:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8002715-circuit-breaker-for-blodgett-oven>: HTTP status code is not handled or not allowed 2026-01-25 04:52:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/structural-plastic-shelving-66x24x72-adjustable>: HTTP status code is not handled or not allowed 2026-01-25 04:52:04 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/bar_restaurant_equipment/bartending_supplies/bar_mats_shelf_liners landed on page that is not a product page. 2026-01-25 04:52:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deep-cut-band-saw-w-case-6232-6n>: HTTP status code is not handled or not allowed 2026-01-25 04:52:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-hose-storz-cap-685-40z-4-black-silver>: HTTP status code is not handled or not allowed 2026-01-25 04:52:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-add-on-60inw-x-48ind-x-84inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 04:52:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/r-series-maple-cutting-boad-20-x-15-x-1-1-2>: HTTP status code is not handled or not allowed 2026-01-25 04:52:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 66255 pages (at 95 pages/min), scraped 33258 items (at 45 items/min) 2026-01-25 04:52:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-industrial-fog-nozzle-520-1521-1-1-2-nh-brass>: HTTP status code is not handled or not allowed 2026-01-25 04:52:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-stacking-bin-16-1-2-x-18-x-11-red>: HTTP status code is not handled or not allowed 2026-01-25 04:52:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-hose-common-spanner-wrench-876-25-2-1-2-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 04:52:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010588-control-valve-sit-lp-for-garland-range>: HTTP status code is not handled or not allowed 2026-01-25 04:52:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/popcorn-jumbo-bags-5-25x12>: HTTP status code is not handled or not allowed 2026-01-25 04:52:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hotdog-roller-grills-10-dogs>: HTTP status code is not handled or not allowed 2026-01-25 04:52:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-ft-x12-ft-aluminum-frame-vinyl-tackboard-navy>: HTTP status code is not handled or not allowed 2026-01-25 04:52:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-hose-reel-cover-138-2-26-x-7-1-2-10-oz-red-vinyl>: HTTP status code is not handled or not allowed 2026-01-25 04:52:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamper-cap-10-oz-vinyl-10-bushel-yellow>: HTTP status code is not handled or not allowed 2026-01-25 04:52:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25-drawcord-nylon-blue-straight>: HTTP status code is not handled or not allowed 2026-01-25 04:52:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-dbl-solenoid-vlv-ext-pilot-closed-cntr-spr-cntred-1-8-nptf-24v-dc-3w-coil-black>: HTTP status code is not handled or not allowed 2026-01-25 04:52:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-dbl-solenoid-vlv-ext-pilot-closed-cntr-spr-cntred-g1-8-220v-ac-5va-coil-led>: HTTP status code is not handled or not allowed 2026-01-25 04:52:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/open-sided-wire-exchange-truck-five-wire-shelves-800-lb-cap-2>: HTTP status code is not handled or not allowed 2026-01-25 04:52:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-24w-7-step-steel-rolling-ladder-14d-top-step-lock-style-b-wcal-osha-handrail-ca>: HTTP status code is not handled or not allowed 2026-01-25 04:52:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/computer-desk-with-cpu-caddy-44w-x-24d-x-30h-dark-honey>: HTTP status code is not handled or not allowed 2026-01-25 04:52:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-work-shirt-m-tan-tcg01120216>: HTTP status code is not handled or not allowed 2026-01-25 04:52:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-starter-36inw-x-12ind-x-96inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 04:52:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-starter-60inw-x-36ind-x-84inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 04:52:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h-d-shelving-poly-z-c-o-18>: HTTP status code is not handled or not allowed 2026-01-25 04:52:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010040-release-sheet-clip-fornew-grill-for-taylor-freezer>: HTTP status code is not handled or not allowed 2026-01-25 04:52:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-58-double-shaft-blower-230v-1625rpm-13hp>: HTTP status code is not handled or not allowed 2026-01-25 04:52:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-closed-single-solenoid-valve-pilot-spr-return-g-1-8-24v-ac-5va-coil-blk-connection>: HTTP status code is not handled or not allowed 2026-01-25 04:52:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/haws-swingaway-eyewash-designed-to-be-mounted-on-right-side-of-sink>: HTTP status code is not handled or not allowed 2026-01-25 04:52:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/attix-33-01-ic-8-gallon-wet-dry-vacuum-w-auto-filter-cleaning>: HTTP status code is not handled or not allowed 2026-01-25 04:52:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-work-shirt-2xl-ln-gray-tcg01150226>: HTTP status code is not handled or not allowed 2026-01-25 04:52:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4c-horizontal-mailbox-n1029416-56-1-2h-single-column-9-doors-1-parcel-usps>: HTTP status code is not handled or not allowed 2026-01-25 04:52:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/haws-combination-shower-softflo-eyewash-ss>: HTTP status code is not handled or not allowed 2026-01-25 04:52:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1331503-lid-milk-dispenser-2-qt-for-carlisle-foodservice-products>: HTTP status code is not handled or not allowed 2026-01-25 04:52:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-starter-96inw-x-48ind-x-84inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 04:52:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/newport-single-residential-mailbox-post-n1023734-black>: HTTP status code is not handled or not allowed 2026-01-25 04:52:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-kevlar-knit-sleeve-with-thumbhole-yellow-s00kn0318>: HTTP status code is not handled or not allowed 2026-01-25 04:52:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/photo_presentation_paper already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:52:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/strongknit-hi-vis-long-sleeve-fr-t-shirt-type-r-class-3-m-yellow>: HTTP status code is not handled or not allowed 2026-01-25 04:52:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-18-ga-48-wx30-dx97-h-open-clip-style-6-shelf>: HTTP status code is not handled or not allowed 2026-01-25 04:52:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-capacity-33-lbs-cylinders-heavy-duty-steel-gas-cylinder-cage-44w-x-30d-x-70h-white>: HTTP status code is not handled or not allowed 2026-01-25 04:52:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-heavy-duty-boltless-shelving-starter-96inw-x-36ind-x-72inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 04:52:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-locker-accessory-closed-side-base-15d-x-6h-729-parchment>: HTTP status code is not handled or not allowed 2026-01-25 04:52:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8463-containment-berm-foam-wall-12-feet-by-35-feet-22-ounce-pvc>: HTTP status code is not handled or not allowed 2026-01-25 04:52:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c-thru-standard-welded-steel-wardrobe-cabinet-wardrobe-cabinet-36w-x-18d-x-72h-putty>: HTTP status code is not handled or not allowed 2026-01-25 04:52:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/paper_stationery/photo_presentation_paper>: HTTP status code is not handled or not allowed 2026-01-25 04:52:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-dbl-solenoid-valve-pilot-spring-closed-cntr-g1-4-110v-ac-5va-coil-black-conn>: HTTP status code is not handled or not allowed 2026-01-25 04:52:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-steel-table-2-shelves-48x30-1200-lb-capacity-unassembled-2>: HTTP status code is not handled or not allowed 2026-01-25 04:52:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m4-x-0-7-x-20mm-security-machine-screw-button-torx-head-18-8-stainless-steel-ft-unc-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 04:53:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-cal-flame-resistant-coveralls-2xl-ln-tan-tcg02120874>: HTTP status code is not handled or not allowed 2026-01-25 04:53:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-32-x-3-8-security-machine-screw-flat-spanner-head-18-8-stainless-steel-ft-unc-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 04:53:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quick-adjust-wire-shelf-truck-72x18x69-1200-pound-capacity-with-brakes>: HTTP status code is not handled or not allowed 2026-01-25 04:53:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m4-x-0-7-x-40mm-security-machine-screw-flat-torx-head-18-8-stainless-steel-ft-unc-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 04:53:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-2-single-solenoid-valve-ext-pilot-spr-return-1-8-nptf-110v-ac-5va-coil-black-conn>: HTTP status code is not handled or not allowed 2026-01-25 04:53:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quick-adjust-shelf-36x18-with-4-hooks>: HTTP status code is not handled or not allowed 2026-01-25 04:53:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/laser-ink-jet-insert-letter-pref-2-in-4-in-500-per-package>: HTTP status code is not handled or not allowed 2026-01-25 04:53:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pegboard-pack-2-panels-red-metal-32-x-32-x-3-4>: HTTP status code is not handled or not allowed 2026-01-25 04:53:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-shelf-truck-36x24x80-1200-pound-capacity-with-brakes>: HTTP status code is not handled or not allowed 2026-01-25 04:53:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 66352 pages (at 97 pages/min), scraped 33305 items (at 47 items/min) 2026-01-25 04:53:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securit-chalk-markers-rain-smear-proof-small-tip-white>: HTTP status code is not handled or not allowed 2026-01-25 04:53:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inertia-womens-safety-work-boots-steel-toe-7h-size-105-matte-black>: HTTP status code is not handled or not allowed 2026-01-25 04:53:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-4086-bento-5-light-chandelier-satin-white-polished-chrome-25-75w-x-24-75h>: HTTP status code is not handled or not allowed 2026-01-25 04:53:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spice-womens-safety-work-boots-steel-toe-7h-size-9-jet-black>: HTTP status code is not handled or not allowed 2026-01-25 04:53:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-32-x-5-8-security-machine-screw-button-hex-socket-head-alloy-steel-black-oxide-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 04:53:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/easy-adjust-boltless-shelf-truck-with-wood-shelves-14>: HTTP status code is not handled or not allowed 2026-01-25 04:53:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dial-tire-gauge-0-to-15-psi-s-931>: HTTP status code is not handled or not allowed 2026-01-25 04:53:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/funnel-1-pt-capacity-5-1-4-high-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 04:53:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-187-ballerina-5-light-chandelier-alabaster-bell-textured-white-24w-x-18h>: HTTP status code is not handled or not allowed 2026-01-25 04:53:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-king-sk3000-pallet-rack-96w-x-36d-x-120h-4900-lbs-cap-per-level-add-on-unit>: HTTP status code is not handled or not allowed 2026-01-25 04:53:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14ab-x-3-4-security-sheet-metal-screw-button-torx-head-302hq-stainless-steel-usa-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 04:53:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/faultless-stainless-steel-swivel-plate-caster-5-tpr-wheel>: HTTP status code is not handled or not allowed 2026-01-25 04:53:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/sign_light_combination_units already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:53:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009821-large-plastic-drawermp-554-for-henny-penny>: HTTP status code is not handled or not allowed 2026-01-25 04:53:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/water-pitcher-67-oz-contemporary-satin-finish>: HTTP status code is not handled or not allowed 2026-01-25 04:53:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8ab-x-1-2-security-sheet-metal-screw-flat-torx-head-302hq-stainless-steel-ft-usa-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 04:53:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6690-cooling-arm-sleeves-blue-2xl-12186>: HTTP status code is not handled or not allowed 2026-01-25 04:53:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/transformers_uninterrupted_power_supplies/uninterrupted_power_supply_backup_systems already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:53:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-x-1-2-security-sheet-metal-screw-round-one-way-head-18-8-stainless-steel-ft-usa-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 04:53:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-4644-decker-4-light-vanity-fixt-clear-frosted-inner-brushed-nickel-29-75w-x-10-25h>: HTTP status code is not handled or not allowed 2026-01-25 04:53:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/lighted_emergency_exit_signs/sign_light_combination_units landed on page that is not a product page. 2026-01-25 04:53:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-lcd-10-30-flat-screen-vesa-mount-global-workbenches>: HTTP status code is not handled or not allowed 2026-01-25 04:53:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbc94-2u-replacement-battery-cartridge-for-select-tripp-lite-and-other-major-ups-brand>: HTTP status code is not handled or not allowed 2026-01-25 04:53:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/76-625-1-light-surface-mount-industrial-style-frosted-old-bronze-4-25w-x-9-375h>: HTTP status code is not handled or not allowed 2026-01-25 04:53:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/transformers_uninterrupted_power_supplies/uninterrupted_power_supply_backup_systems already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:53:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-2-inch-a-aluminum-spec-cam-and-groove-adapter-x-female-npt>: HTTP status code is not handled or not allowed 2026-01-25 04:53:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-24-x-1-security-machine-screw-flat-torx-plus-head-18-8-stainless-steel-ft-unc-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 04:53:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fr-mesh-vest-c2-lime-4xl-5xl>: HTTP status code is not handled or not allowed 2026-01-25 04:53:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/transformers_uninterrupted_power_supplies/uninterrupted_power_supply_backup_systems landed on page that is not a product page. 2026-01-25 04:53:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnetic-glass-whiteboard-48-x-36-black>: HTTP status code is not handled or not allowed 2026-01-25 04:53:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wide-span-rack-96x24x96-with-3-shelves-wood-deck-1100-lb-capacity-per-level>: HTTP status code is not handled or not allowed 2026-01-25 04:53:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-20-x-1-1-4-security-machine-screw-button-torx-head-302hq-stainless-steel-ft-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 04:53:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/transformers_uninterrupted_power_supplies/uninterrupted_power_supply_backup_systems>: HTTP status code is not handled or not allowed 2026-01-25 04:53:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iron-eagle-174-overall-gold-knee-patch-pockets-loto-straps-xl>: HTTP status code is not handled or not allowed 2026-01-25 04:53:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/265863-repair-kit-cam-shaft>: HTTP status code is not handled or not allowed 2026-01-25 04:53:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/valence-womens-safety-work-boots-steel-toe-7h-size-7w-vegan-midnight-black>: HTTP status code is not handled or not allowed 2026-01-25 04:53:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/300-mm-exp-pipe-brush-50-mm-diameter-9-989-052-0>: HTTP status code is not handled or not allowed 2026-01-25 04:53:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/floor_machine_replacement_parts/vacuum_hoses already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:53:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-t-shirt-long-sleeve-1-pocket-fl-lime-small>: HTTP status code is not handled or not allowed 2026-01-25 04:53:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/321143-s-s-container-for-waring-products>: HTTP status code is not handled or not allowed 2026-01-25 04:53:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/floor_machine_replacement_parts/vacuum_hoses already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:53:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/valence-womens-safety-work-boots-steel-toe-7h-size-75-chestnut-brown>: HTTP status code is not handled or not allowed 2026-01-25 04:53:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-shelving-48x18x84-with-5-shelves-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 04:53:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/decorative_lighting/flush_mount_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:53:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/decorative_lighting/flush_mount_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:54:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gravity-womens-safety-work-boots-steel-toe-4h-size-85w-cognac-brown>: HTTP status code is not handled or not allowed 2026-01-25 04:54:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/surface-mount-flexible-stake-36-in-h-orange>: HTTP status code is not handled or not allowed 2026-01-25 04:54:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/valence-womens-safety-work-boots-steel-toe-7h-size-12-vegan-midnight-black>: HTTP status code is not handled or not allowed 2026-01-25 04:54:04 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/floor_machine_replacement_parts/vacuum_hoses landed on page that is not a product page. 2026-01-25 04:54:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machinery-roller-dolly-rigid-plates-adjustable-connector-bar-4400-cap>: HTTP status code is not handled or not allowed 2026-01-25 04:54:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-support-crossbar-48ind-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-25 04:54:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-stem-caster-5-dia-nomadic-grip-ring-with-top-lock-1-3-8-h-stem>: HTTP status code is not handled or not allowed 2026-01-25 04:54:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/decorative_lighting/flush_mount_lights>: HTTP status code is not handled or not allowed 2026-01-25 04:54:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lux-ssetpbk-om-class-2-black-bottom-t-shirt-w-pocket-orange-m>: HTTP status code is not handled or not allowed 2026-01-25 04:54:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 66439 pages (at 87 pages/min), scraped 33345 items (at 40 items/min) 2026-01-25 04:54:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/decorative_lighting/flush_mount_lights>: HTTP status code is not handled or not allowed 2026-01-25 04:54:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deep-cleaning-ribbed-4-foot-wide-cut-length-mat-charcoal-1>: HTTP status code is not handled or not allowed 2026-01-25 04:54:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/air_conditioners_chillers/wall_air_conditioners already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:54:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/floor_machine_replacement_parts/vacuum_hoses landed on page that is not a product page. 2026-01-25 04:54:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-206-2-light-ceiling-flush-mount-alabaster-glass-old-bronze-13-125w-x-5-375h>: HTTP status code is not handled or not allowed 2026-01-25 04:54:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-high-letter-starter-empty-black>: HTTP status code is not handled or not allowed 2026-01-25 04:54:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/decorative_lighting/flush_mount_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:54:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011082-brush-kit-taylor-for-taylor-freezer>: HTTP status code is not handled or not allowed 2026-01-25 04:54:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8407901-control-board-j>: HTTP status code is not handled or not allowed 2026-01-25 04:54:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-boltless-shelving-starter-48inw-x-12ind-x-96inh-no-deck>: HTTP status code is not handled or not allowed 2026-01-25 04:54:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-357-empire-6-light-chandelier-alabaster-bell-polished-brass-26w-x-14h>: HTTP status code is not handled or not allowed 2026-01-25 04:54:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drink-mat-21679-mlb-cleveland-indians-3-1-4-x-24-x-1>: HTTP status code is not handled or not allowed 2026-01-25 04:54:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hubbel-constant-pull-12>: HTTP status code is not handled or not allowed 2026-01-25 04:54:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drink-mat-20514-nfl-los-angeles-chargers-3-1-4-x-24-x-1>: HTTP status code is not handled or not allowed 2026-01-25 04:54:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/decorative_lighting/flush_mount_lights>: HTTP status code is not handled or not allowed 2026-01-25 04:54:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-boltless-shelving-add-on-36inw-x-24ind-x-72inh-no-deck>: HTTP status code is not handled or not allowed 2026-01-25 04:54:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/air_conditioners_chillers/wall_air_conditioners landed on page that is not a product page. 2026-01-25 04:54:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-8482-10-x-12-super-8-oz-tarp-brown>: HTTP status code is not handled or not allowed 2026-01-25 04:54:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010994-switch-pressure-for-henny-penny>: HTTP status code is not handled or not allowed 2026-01-25 04:54:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-boltless-shelving-starter-48inw-x-12ind-x-96inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 04:54:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-boltless-shelving-starter-48inw-x-18ind-x-96inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 04:54:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/621101-infrared-thermometerduplicate-for-raypak>: HTTP status code is not handled or not allowed 2026-01-25 04:54:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bus-smoking-shelter-flat-roof-with-three-sided-open-front-6-5-w-x-3-8-d-x-7-h-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:54:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-male-x-1-4-male-coupling-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-25 04:54:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ego-st1502sa-580cfm-blower-kit-5-0ah-battery-210w-charger>: HTTP status code is not handled or not allowed 2026-01-25 04:54:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-flow-brass-6-ball-1-4-x-1-4-female-coupler-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 04:54:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-boltless-shelving-add-on-36inw-x-18ind-x-84inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 04:54:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22-gray-extreme-stripping-pad-5-per-case>: HTTP status code is not handled or not allowed 2026-01-25 04:54:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-high-back-clean-room-vinyl-chair-with-aluminum-base-black>: HTTP status code is not handled or not allowed 2026-01-25 04:54:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/262845-shaft-5-1-2-for-hobart>: HTTP status code is not handled or not allowed 2026-01-25 04:54:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-boltless-shelving-starter-48inw-x-24ind-x-84inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 04:54:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009580-evaporator-coil-assembly-for-true-manufacturing>: HTTP status code is not handled or not allowed 2026-01-25 04:54:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-shelf-single-rivet-laminate-deck-36inw-x-12ind-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:54:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/outdoor-4-multipurpose-led-striplight-4900l-40w-40k-0-10v-dimming-120-277v>: HTTP status code is not handled or not allowed 2026-01-25 04:54:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/humidifiers_dehumidifiers/dehumidifiers_desiccators already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:54:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/outdoor-led-bullet-flood-light-20w-3000k-wide-dist-120v-dark-bronze>: HTTP status code is not handled or not allowed 2026-01-25 04:54:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-boltless-shelving-starter-36inw-x-24ind-x-60inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 04:54:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/342277-wire-heater-45-ohm-100>: HTTP status code is not handled or not allowed 2026-01-25 04:54:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lester-series-36v-115vac-60hz-portable-charger-40-amps>: HTTP status code is not handled or not allowed 2026-01-25 04:54:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sb-app-connector-10-12-wire-gauge-50-amp-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:54:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-4-step-36-wx-36-l-platform-w-o-handrails>: HTTP status code is not handled or not allowed 2026-01-25 04:54:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-cantilever-heavy-duty-shelf-18-wx36-l>: HTTP status code is not handled or not allowed 2026-01-25 04:54:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/configurable-multi-platform-access-system-30-x90-platform-w-serrated-tread-mpasp90>: HTTP status code is not handled or not allowed 2026-01-25 04:54:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/humidifiers_dehumidifiers/dehumidifiers_desiccators landed on page that is not a product page. 2026-01-25 04:54:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-75-250-psi-191-lbs-hose-assemblies>: HTTP status code is not handled or not allowed 2026-01-25 04:54:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/configurable-multi-platform-access-system-handrail-for-6-step-stairway-mpashr7>: HTTP status code is not handled or not allowed 2026-01-25 04:54:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hydraulic-1-2-50-ft-2000-psi-twin-hydraulic-with-hose>: HTTP status code is not handled or not allowed 2026-01-25 04:54:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-boltless-shelving-starter-36inw-x-12ind-x-96inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 04:54:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-handtruck-fbal18-10fpn-curved-handle-pneumatic-wheels-600-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 04:55:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-boltless-shelving-starter-36inw-x-24ind-x-96inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 04:55:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-24-w-6-step-steel-rolling-ladder-10-d-top-step-kdsr106246>: HTTP status code is not handled or not allowed 2026-01-25 04:55:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/digital-clock>: HTTP status code is not handled or not allowed 2026-01-25 04:55:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-boltless-shelving-add-on-36inw-x-24ind-x-60inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 04:55:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-boltless-shelving-add-on-36inw-x-12ind-x-72inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 04:55:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-boltless-shelving-add-on-36inw-x-24ind-x-72inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 04:55:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-emergency-exit-combo-w-field-selectable-red-green-leds-white-remote-capacity-120-277v>: HTTP status code is not handled or not allowed 2026-01-25 04:55:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 66536 pages (at 97 pages/min), scraped 33389 items (at 44 items/min) 2026-01-25 04:55:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-lift-beam-low-headroom-multiple-length-4000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 04:55:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-boltless-shelving-add-on-48inw-x-24ind-x-84inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 04:55:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-cabinet-with-removable-bins-10>: HTTP status code is not handled or not allowed 2026-01-25 04:55:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-cabinet-with-removable-bins-17>: HTTP status code is not handled or not allowed 2026-01-25 04:55:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1681437-computer-fpph50-fv-eurolook-for-frymaster>: HTTP status code is not handled or not allowed 2026-01-25 04:55:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safe-cell-phone-safe-48-w-x-30-d-x-71-h-digital-lock-black>: HTTP status code is not handled or not allowed 2026-01-25 04:55:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/461510-t-stat-sp-kit>: HTTP status code is not handled or not allowed 2026-01-25 04:55:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-diamond-series-laboratory-and-medical-solid-door-refrigerator-36-cu-ft>: HTTP status code is not handled or not allowed 2026-01-25 04:55:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-3-shelf-boltless-shelving-add-on-36inw-x-18ind-x-48inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 04:55:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-ft-rectangular-outdoor-picnic-table-with-backrests-expanded-metal-black>: HTTP status code is not handled or not allowed 2026-01-25 04:55:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-stage-portable-drum-dumper-dump-height-48-1500-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-25 04:55:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-3-shelf-boltless-shelving-add-on-36inw-x-12ind-x-72inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 04:55:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20t4-g9-cl-20w-halogen-w-double-loop-base>: HTTP status code is not handled or not allowed 2026-01-25 04:55:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/49752-green-grinding-wheel-4-1-2-inch-x-1-4-inch-x-7-8-inch-t27-36-grit-zirconia>: HTTP status code is not handled or not allowed 2026-01-25 04:55:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-wall-corrugated-sheets-24l-x-24w-kraft-pack-of-5>: HTTP status code is not handled or not allowed 2026-01-25 04:55:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caged-walk-through-fixed-access-ladder-gray-1>: HTTP status code is not handled or not allowed 2026-01-25 04:55:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/156-keys-lock-box-with-combination-lock-11-w-x-3-1-4-d-x-14-1-2-h-black>: HTTP status code is not handled or not allowed 2026-01-25 04:55:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20mr11-nfl-dc-20w-halogen-w-dc-bay-base>: HTTP status code is not handled or not allowed 2026-01-25 04:55:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pump-seal-kit-for-rol-lift-5500-capacity-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 04:55:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/62-1006-rinse-thermometer-100-220-f-panel-mount>: HTTP status code is not handled or not allowed 2026-01-25 04:55:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dispense-rite-surface-mount-gravity-feed-cup-dispenser-32-oz>: HTTP status code is not handled or not allowed 2026-01-25 04:55:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zipt88d2-8-channel-hd-tvi-dvr-and-8-dome-camera-kit-2tb>: HTTP status code is not handled or not allowed 2026-01-25 04:55:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dispense-rite-5-section-vertical-wire-rack-cup-dispenser>: HTTP status code is not handled or not allowed 2026-01-25 04:55:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dispense-rite-in-counter-one-size-cup-dispenser-2-spring-17-in-l>: HTTP status code is not handled or not allowed 2026-01-25 04:55:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8898-strapping-tape-1-x-60-yds-46-mil-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:55:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/letter-size-fireproof-file-cabinet-18-w-x-25-d-x-53-h-black>: HTTP status code is not handled or not allowed 2026-01-25 04:55:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-mailers-9l-x-4w-x-4h-kraft-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-25 04:55:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50t4-cl-24v-50w-halogen-w-bi-pin-base>: HTTP status code is not handled or not allowed 2026-01-25 04:55:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eu-vacuum-hepa-filter-for-eu-gd930>: HTTP status code is not handled or not allowed 2026-01-25 04:55:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rain-bonnet-lid-24-gallon-green>: HTTP status code is not handled or not allowed 2026-01-25 04:55:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-seat-replacement-belt-kit-royal-blue-b008>: HTTP status code is not handled or not allowed 2026-01-25 04:55:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/homak-bk04004193-rs-pro-series-41w-x-24d-x-39h-9-drawer-black-roller-cabinet-with-lift-latch>: HTTP status code is not handled or not allowed 2026-01-25 04:55:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hydraulic-long-chassis-service-jack-5-ton-air>: HTTP status code is not handled or not allowed 2026-01-25 04:55:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/apc-replacement-battery-rbc32>: HTTP status code is not handled or not allowed 2026-01-25 04:55:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/homak-rd02041062-pro-ii-series-40-1-2-w-x-24-1-4-d-x-21-3-8-h-6-drawer-red-tool-chest>: HTTP status code is not handled or not allowed 2026-01-25 04:55:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rover-jump-over-natural-pbark-450n>: HTTP status code is not handled or not allowed 2026-01-25 04:55:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f8t5-d-8w-fluorescent-w-minature-bi-pin-base-daylight>: HTTP status code is not handled or not allowed 2026-01-25 04:55:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/transformers_uninterrupted_power_supplies/power_conditioners already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:55:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vented-bump-cap-4-point-suspension-hi-viz-yellow-19117>: HTTP status code is not handled or not allowed 2026-01-25 04:56:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rotary-pump-high-volume-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 04:56:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/svp208-safety-glasses-pink-frame-clear-lens>: HTTP status code is not handled or not allowed 2026-01-25 04:56:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-wall-corrugated-boxes-w-hand-holes-20l-x-18w-x-12h-kraft-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 04:56:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/266162-foot-s-s-f-2-inch-od-rd>: HTTP status code is not handled or not allowed 2026-01-25 04:56:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergency-cubicle-shower-covered-abs-eye-face-wash-sd32k45g>: HTTP status code is not handled or not allowed 2026-01-25 04:56:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/combination_metalworking_machines already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:56:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/transformers_uninterrupted_power_supplies/power_conditioners landed on page that is not a product page. 2026-01-25 04:56:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hydraulic-bottle-jack-20-ton-super-duty-low-profile-air>: HTTP status code is not handled or not allowed 2026-01-25 04:56:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 66634 pages (at 98 pages/min), scraped 33442 items (at 53 items/min) 2026-01-25 04:56:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/621165-vacuum-gauge-for-accu-temp>: HTTP status code is not handled or not allowed 2026-01-25 04:56:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/air_compressors/two_stage_reciprocating_air_compressors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:56:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/db3p-m612-polypropylene-mag-drive-pump-1-8hp-115v-1-phase-15-gpm>: HTTP status code is not handled or not allowed 2026-01-25 04:56:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/db5-5p-m622-polypropylene-mag-drive-pump-1-2hp-115v-1-phase-30-gpm>: HTTP status code is not handled or not allowed 2026-01-25 04:56:14 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/compact_fluorescent_light_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:56:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/long-cardboard-corrugated-boxes-50l-x-12w-x-12h-kraft-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 04:56:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cfs9w-835-9w-compact-fluorescent-w-g23-base-neutral-white>: HTTP status code is not handled or not allowed 2026-01-25 04:56:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/combination_metalworking_machines>: HTTP status code is not handled or not allowed 2026-01-25 04:56:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foot-pedal-siphon-system-201714yw>: HTTP status code is not handled or not allowed 2026-01-25 04:56:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/air_compressors/two_stage_reciprocating_air_compressors>: HTTP status code is not handled or not allowed 2026-01-25 04:56:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tote-box-dividable-31>: HTTP status code is not handled or not allowed 2026-01-25 04:56:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rlph1ns-long-sleeve-pullover-hoodie-with-uv-protection-m-hi-vis-orange>: HTTP status code is not handled or not allowed 2026-01-25 04:56:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cut-resistant-gloves-micro-foam-nitrile-coated-ansi-a1-s-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:56:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/compact_fluorescent_light_bulbs landed on page that is not a product page. 2026-01-25 04:56:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-153-deluxe-thermoplastic-32-gal-perforated-recycling-receptacle-w-rain-bonnet-lid-base-bl>: HTTP status code is not handled or not allowed 2026-01-25 04:56:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rlph1ns-long-sleeve-pullover-hoodie-with-uv-protection-l-hi-vis-orange>: HTTP status code is not handled or not allowed 2026-01-25 04:56:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/power_tools/drills_drivers_impact_wrenches/hammer_drills_rotary_hammers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:56:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/511515-pilot-assembly-nat-for-frymaster>: HTTP status code is not handled or not allowed 2026-01-25 04:56:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20v-max-cordless-circular-saw-7-1-4-5500-rpm-brushless-rubber-over-mold-grip>: HTTP status code is not handled or not allowed 2026-01-25 04:56:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tote-box-accessory-long-divider-20>: HTTP status code is not handled or not allowed 2026-01-25 04:56:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cut-resistant-gloves-micro-foam-nitrile-coated-ansi-a4-m-graypurple>: HTTP status code is not handled or not allowed 2026-01-25 04:56:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tote-box-accessory-long-divider-24>: HTTP status code is not handled or not allowed 2026-01-25 04:56:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dsc-214m-flush-access-door-for-all-surf-lock-8wx8h-mp88l>: HTTP status code is not handled or not allowed 2026-01-25 04:56:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tote-box-accessory-long-divider-26>: HTTP status code is not handled or not allowed 2026-01-25 04:56:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/strain-relief-cord-connectors-13>: HTTP status code is not handled or not allowed 2026-01-25 04:56:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/strain-relief-cord-connectors-14>: HTTP status code is not handled or not allowed 2026-01-25 04:56:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/power_tools/drills_drivers_impact_wrenches/hammer_drills_rotary_hammers landed on page that is not a product page. 2026-01-25 04:56:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7801-7800-series-premium-silicone-half-mask-respirator-small>: HTTP status code is not handled or not allowed 2026-01-25 04:56:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/incandescent_light_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:56:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/full-face-visor-kit-black>: HTTP status code is not handled or not allowed 2026-01-25 04:56:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/cable_management/cable_clamps_clips_hardware already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:56:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-strip-door-curtain-12-w-x-7-h>: HTTP status code is not handled or not allowed 2026-01-25 04:56:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uhmw-polyethylene-plastic-sheet-1-2-thick-x-24-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 04:56:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/complete-kit-with-pro-120-tool>: HTTP status code is not handled or not allowed 2026-01-25 04:56:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-sheet-1-8-thick-x-12-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 04:56:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011684-sensor-for-master-bilt-products>: HTTP status code is not handled or not allowed 2026-01-25 04:56:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scratch-resistant-strip-door-curtain-7-w-x-12-h>: HTTP status code is not handled or not allowed 2026-01-25 04:56:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-hc-narrow-section-molded-notch-vbelt-gates-3vx500>: HTTP status code is not handled or not allowed 2026-01-25 04:56:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-drum-econo-spill-shell-51l-x-47w-x-5h-36-gallon-capacity>: HTTP status code is not handled or not allowed 2026-01-25 04:56:45 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/incandescent_light_bulbs landed on page that is not a product page. 2026-01-25 04:56:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fast-pack-spill-kit-universal-5-gallon-capacity-yellow-1300-ye>: HTTP status code is not handled or not allowed 2026-01-25 04:56:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/cable_management/cable_clamps_clips_hardware>: HTTP status code is not handled or not allowed 2026-01-25 04:56:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibco-silent-pneumatic-turbine-vibrator-mhiss-19>: HTTP status code is not handled or not allowed 2026-01-25 04:56:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/511607-pilot-burner-nat>: HTTP status code is not handled or not allowed 2026-01-25 04:56:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009337-water-inlet-valve-120v-for-wilbur-curtis-co>: HTTP status code is not handled or not allowed 2026-01-25 04:56:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grill-protective-cover-for-indu-bistrogrill-island-57l-x-28w-x-36h>: HTTP status code is not handled or not allowed 2026-01-25 04:56:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fleetrunner-microv-serpentine-drive-belt-gates-k080994hd>: HTTP status code is not handled or not allowed 2026-01-25 04:56:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/embassy-71-executive-desk-kit-dark-chocolate>: HTTP status code is not handled or not allowed 2026-01-25 04:56:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibco-adjustable-speed-force-electric-vibrator-scr-60>: HTTP status code is not handled or not allowed 2026-01-25 04:56:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uhmw-polyethylene-plastic-sheet-1-8-thick-x-48-wide-x-96-long>: HTTP status code is not handled or not allowed 2026-01-25 04:56:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/250-1-4-dia-6x19-improved-plow-steel-bright-wire-rope-independent-wire-rope-center>: HTTP status code is not handled or not allowed 2026-01-25 04:56:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flex-enriched-foam-alcohol-plus-hand-sanitizer-e3-1000ml>: HTTP status code is not handled or not allowed 2026-01-25 04:56:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/250-5-32-diameter-7x19-galvanized-aircraft-cable>: HTTP status code is not handled or not allowed 2026-01-25 04:56:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:56:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uhmw-polyethylene-plastic-bar-1-thick-x-1-1-2-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 04:57:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/500-3-8-diameter-7x19-galvanized-aircraft-cable>: HTTP status code is not handled or not allowed 2026-01-25 04:57:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uhmw-polyethylene-plastic-sheet-3-8-thick-x-6-wide-x-6-long>: HTTP status code is not handled or not allowed 2026-01-25 04:57:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/381369-cord-set>: HTTP status code is not handled or not allowed 2026-01-25 04:57:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/skid-steer-grab-bucket-72-wide>: HTTP status code is not handled or not allowed 2026-01-25 04:57:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pld-u-bend-fluorescent-bulb-g24q1-base-13w-780-lumens-5000k-super-white-pk-of-10>: HTTP status code is not handled or not allowed 2026-01-25 04:57:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/fluorescent_linear_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:57:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pld-u-bend-fluorescent-bulb-g24q3-base-26w-1560-lumens-2700k-warm-white-pk-of-10>: HTTP status code is not handled or not allowed 2026-01-25 04:57:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/fluorescent_linear_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:57:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/offset-band-breaker-aviation-sni-p>: HTTP status code is not handled or not allowed 2026-01-25 04:57:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 66723 pages (at 89 pages/min), scraped 33475 items (at 33 items/min) 2026-01-25 04:57:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-electrical-tape-34-x-60-red-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 04:57:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/fluorescent_linear_bulbs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:57:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-1-8-thick-x-1-1-2-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 04:57:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-rod-2-diameter-x-1-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 04:57:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-welded-steel-container-green-3>: HTTP status code is not handled or not allowed 2026-01-25 04:57:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-container-forkli-able-with-hopper-front-5>: HTTP status code is not handled or not allowed 2026-01-25 04:57:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009507-orifice-1-4-18-mpt-56>: HTTP status code is not handled or not allowed 2026-01-25 04:57:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classic-paper-plates-9-14-inches-white-round-3-compartments-125pack>: HTTP status code is not handled or not allowed 2026-01-25 04:57:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/fluorescent_linear_bulbs landed on page that is not a product page. 2026-01-25 04:57:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/fluorescent_linear_bulbs>: HTTP status code is not handled or not allowed 2026-01-25 04:57:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5031098-float-switch-for-hoshizaki-of-america>: HTTP status code is not handled or not allowed 2026-01-25 04:57:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/840-style-woven-wire-slide-door-5w-x-10h-10-5-1-4-overall-height>: HTTP status code is not handled or not allowed 2026-01-25 04:57:19 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 04:57:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/light_bulbs_lamps/fluorescent_linear_bulbs>: HTTP status code is not handled or not allowed 2026-01-25 04:57:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/761182-cartridge-water-filtr-ic614-for-selecto-scientific-filters>: HTTP status code is not handled or not allowed 2026-01-25 04:57:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/foodservice_disposables/disposable_lids already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:57:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milton-s-767w-v-style-high-flow-coupler-3-8-mnpt>: HTTP status code is not handled or not allowed 2026-01-25 04:57:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-g1sc-043-ss>: HTTP status code is not handled or not allowed 2026-01-25 04:57:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/881332-filter-drier-liquid-line-for-emerson-radio-corp>: HTTP status code is not handled or not allowed 2026-01-25 04:57:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stop-turn-and-taillight-socket-intermotor-s2612>: HTTP status code is not handled or not allowed 2026-01-25 04:57:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/variable-reel-dispenser>: HTTP status code is not handled or not allowed 2026-01-25 04:57:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-w-ash-38-gallon-9935bl>: HTTP status code is not handled or not allowed 2026-01-25 04:57:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:57:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60x30-shop-top-square-edge-mobile-power-apron-production-bench-tan-32313637>: HTTP status code is not handled or not allowed 2026-01-25 04:57:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/composite-reel-spring-rewind-with-3-8-hose-2>: HTTP status code is not handled or not allowed 2026-01-25 04:57:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:57:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/foodservice_disposables/disposable_lids>: HTTP status code is not handled or not allowed 2026-01-25 04:57:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-inch-w-polypropylene-push-broom-with-stiff-center-and-fine-border-bristles-and-60-inch-l-handle>: HTTP status code is not handled or not allowed 2026-01-25 04:57:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0423rblk3xl-womens-softshell-quilted-jacket-black-3xl>: HTTP status code is not handled or not allowed 2026-01-25 04:57:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-par16-recessed-spotlight-bulb-e26-base-7w-550-lumens-50k-super-white-pk-of-6>: HTTP status code is not handled or not allowed 2026-01-25 04:57:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/spill_control_supplies/chemical_sorbents_spill_kits/spill_kits_stations already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:57:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/56-1148-1-2-inch-fpt-water-pressure-regulator-valve-10-to-125-psi-range>: HTTP status code is not handled or not allowed 2026-01-25 04:57:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents>: HTTP status code is not handled or not allowed 2026-01-25 04:57:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grip-24-w-10-step-steel-rolling-ladder-10-d-top-step>: HTTP status code is not handled or not allowed 2026-01-25 04:57:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1200-k-1-2-kevlarcomposite-xtreme-power-twin-clutch-impact-wrench>: HTTP status code is not handled or not allowed 2026-01-25 04:57:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-par30-short-neck-colored-recessed-light-bulb-e26-base-8w-blue-pack-of-3>: HTTP status code is not handled or not allowed 2026-01-25 04:57:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/56-1349-50-psi-brass-steam-safety-relief-valve-3-4-inch-npt-805-lb-hour>: HTTP status code is not handled or not allowed 2026-01-25 04:57:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-x-24-adjustable-height-5-8-steel-work-platform>: HTTP status code is not handled or not allowed 2026-01-25 04:57:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/boards_easels/easels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:57:44 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/spill_control_supplies/absorbents landed on page that is not a product page. 2026-01-25 04:57:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/suture-transport-cart-18-bins-22-diameter-star-base-31-41-height>: HTTP status code is not handled or not allowed 2026-01-25 04:57:44 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/spill_control_supplies/chemical_sorbents_spill_kits/spill_kits_stations landed on page that is not a product page. 2026-01-25 04:57:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vapor-tight-industrial-jar-fixture-surface-mount-100w-120v-metallic-finish>: HTTP status code is not handled or not allowed 2026-01-25 04:57:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2531407-wire-hi-temp-14ga-392f-10-white>: HTTP status code is not handled or not allowed 2026-01-25 04:57:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72x36-esd-square-edge-power-apron-production-bench-blue-32313830>: HTTP status code is not handled or not allowed 2026-01-25 04:57:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-series-chair-caster-with-hard-rubber-wheel-stem-type-c>: HTTP status code is not handled or not allowed 2026-01-25 04:57:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-gsc-218-ss>: HTTP status code is not handled or not allowed 2026-01-25 04:57:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/faultless-swivel-plate-caster-5-mold-on-rubber-wheel>: HTTP status code is not handled or not allowed 2026-01-25 04:57:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-round-outdoor-post-top-light-fixture-75w-9750-lumens-15-dia-bronze>: HTTP status code is not handled or not allowed 2026-01-25 04:57:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/boards_easels/easels>: HTTP status code is not handled or not allowed 2026-01-25 04:57:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-round-regressed-downlight-fixture-18w-1360-lumens-4-12dia-white>: HTTP status code is not handled or not allowed 2026-01-25 04:57:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/std050334-10-hp-oil-less-scroll-comp-80-gal-horiz-116-psi-3-phase-230v-refrigerated-dryer>: HTTP status code is not handled or not allowed 2026-01-25 04:57:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-x-30-steel-square-edge-with-drawer>: HTTP status code is not handled or not allowed 2026-01-25 04:57:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/seq20073-20-hp-oil-less-scroll-compressor-tankless-116-psi-3-phase-230v>: HTTP status code is not handled or not allowed 2026-01-25 04:57:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-signs-caution-blank-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 04:57:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-signs-caution-blank-rigid-plastic-10-h-x-14-w>: HTTP status code is not handled or not allowed 2026-01-25 04:57:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commercial-grade-35-25-in-round-black-indoor-outdoor-steel-patio-table>: HTTP status code is not handled or not allowed 2026-01-25 04:58:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bearkat-bkh25-safety-glasses-bk1-magnifier-25-strength-clear-lens>: HTTP status code is not handled or not allowed 2026-01-25 04:58:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sts075442-75-hp-oil-less-scroll-compressor-120-gallon-horizontal-116-psi-3-phase-460v>: HTTP status code is not handled or not allowed 2026-01-25 04:58:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5031020-valve-control-head-pres-190psi-for-hoshizaki-of-america>: HTTP status code is not handled or not allowed 2026-01-25 04:58:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bearkat-bk113-safety-glasses-bk1-light-blue-lens-clear-frame>: HTTP status code is not handled or not allowed 2026-01-25 04:58:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sts050234hp-5hp-oil-less-scroll-comp-60-gal-horiz-145-psi-3-ph-208-230v-refrigerated-dryer>: HTTP status code is not handled or not allowed 2026-01-25 04:58:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bearkat-bk110af-safety-glasses-black-temple-clear-lens-anti-fog>: HTTP status code is not handled or not allowed 2026-01-25 04:58:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cylinder-sngl-acting-3-bore-3-stroke-dbl-side-lug-mount>: HTTP status code is not handled or not allowed 2026-01-25 04:58:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/venture-patio-umbrella-with-silver-pole-132w-sunbrella-1a-melon-fabric>: HTTP status code is not handled or not allowed 2026-01-25 04:58:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ses030832-3-hp-oil-less-scroll-compressor-13-gallon-horizontal-116-psi-3-phase-230v>: HTTP status code is not handled or not allowed 2026-01-25 04:58:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h500-harness-construction-comfort-tongue-buckle-side-d-ring-l-xl>: HTTP status code is not handled or not allowed 2026-01-25 04:58:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aah-48rbmag3-c-storage-cabinet-all-welded-48-wx24-dx78-h-charcoal>: HTTP status code is not handled or not allowed 2026-01-25 04:58:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 66814 pages (at 91 pages/min), scraped 33504 items (at 29 items/min) 2026-01-25 04:58:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/made-in-usa-o-1-oil-hardening-drill-rod-10mm>: HTTP status code is not handled or not allowed 2026-01-25 04:58:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-vertical-file-legal-size-4-drawer-black-18w-x-26-1-2d-x-52h>: HTTP status code is not handled or not allowed 2026-01-25 04:58:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aah-48rbmag3-ts-storage-cabinet-all-welded-48-wx24-dx78-h-tropic-sand>: HTTP status code is not handled or not allowed 2026-01-25 04:58:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-handle-and-parts-for-capture-shield-cs-hk>: HTTP status code is not handled or not allowed 2026-01-25 04:58:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-g2sc-112-b>: HTTP status code is not handled or not allowed 2026-01-25 04:58:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/decorative_lighting/vanity_bath_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:58:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/newport-patio-umbrella-with-silver-pole-108w-sunbrella-1a-wheat-fabric>: HTTP status code is not handled or not allowed 2026-01-25 04:58:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-step-24-wx72-d-stainless-steel-rolling-safety-ladder-serrated-grating-w-cal-osha-handrail-cal-ss073228g>: HTTP status code is not handled or not allowed 2026-01-25 04:58:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tub-cart-black-32-x-18-2-shelves-2-outrigger-utility-cart-bins-150-lbs-shelf-cap>: HTTP status code is not handled or not allowed 2026-01-25 04:58:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conntek-y1450520s-3-50-to-15-20-amp-generator-y-adapter-with-nema-14-50p-to-5-15-20r-yellow>: HTTP status code is not handled or not allowed 2026-01-25 04:58:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-deck-truck-60-l-x-30-w-1200-lb-capacity-with-ergo-handle>: HTTP status code is not handled or not allowed 2026-01-25 04:58:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-deck-truck-60-l-x-30-w-2000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 04:58:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/decorative_lighting/vanity_bath_lights>: HTTP status code is not handled or not allowed 2026-01-25 04:58:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-24-w-15-step-steel-rolling-ladder-14-d-top-step-w-handrail-lock-step-orange-w-cal-osha-handrail-cal-fs154014p-o>: HTTP status code is not handled or not allowed 2026-01-25 04:58:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/graphic-signs-danger-flammable-vinyl-7-w-x-10-h>: HTTP status code is not handled or not allowed 2026-01-25 04:58:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-w-x-20-h-x-1-d-pleated-merv-13-leed-compliant-air-filter-global-industrial>: HTTP status code is not handled or not allowed 2026-01-25 04:58:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/retail_fixtures_displays_equipment/displays_display_racks/gondola_shelving already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:58:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss-hose-braid-w-male-npt-nipple-2-5-x-18>: HTTP status code is not handled or not allowed 2026-01-25 04:58:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-w-x-12-h-x-1-d-pleated-merv-8-standard-capacity-air-filter-wire-back-global-industrial>: HTTP status code is not handled or not allowed 2026-01-25 04:58:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-in-silver-epoxy-wall-double-bracket>: HTTP status code is not handled or not allowed 2026-01-25 04:58:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nanofiber-filter-for-attx-19-xc>: HTTP status code is not handled or not allowed 2026-01-25 04:58:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-bariatric-folding-bedside-commode-seat>: HTTP status code is not handled or not allowed 2026-01-25 04:58:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-w-x-20-h-x-4-d-pleated-merv-8-standard-capacity-air-filter-global-industrial>: HTTP status code is not handled or not allowed 2026-01-25 04:58:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/retail_fixtures_displays_equipment/displays_display_racks/gondola_shelving>: HTTP status code is not handled or not allowed 2026-01-25 04:58:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/backstroke-flags-white>: HTTP status code is not handled or not allowed 2026-01-25 04:58:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-w-x-16-h-x-1-d-pleated-merv-13-leed-compliant-air-filter-global-industrial>: HTTP status code is not handled or not allowed 2026-01-25 04:58:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-folding-security-gate-5-1-2-x-8>: HTTP status code is not handled or not allowed 2026-01-25 04:58:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lightweight-walking-forearm-crutches-adult>: HTTP status code is not handled or not allowed 2026-01-25 04:58:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25-w-x-16-h-x-4-d-pleated-merv-11-high-capacity-air-filter-global-industrial>: HTTP status code is not handled or not allowed 2026-01-25 04:58:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/prominence-heavy-duty-floor-cleaner-citrus-scent-5-l-bottle>: HTTP status code is not handled or not allowed 2026-01-25 04:58:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-wheelchair-oxygen-inche-inch-cylinder-holder-with-iv-pole>: HTTP status code is not handled or not allowed 2026-01-25 04:58:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-w-x-24-h-x-12-d-rigid-cell-merv-13-air-filter-box-fiberglass-global-industrial>: HTTP status code is not handled or not allowed 2026-01-25 04:58:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-wheel-chock-with-rubber-pad-polyethylene-18l-x-15w-x-9-12h>: HTTP status code is not handled or not allowed 2026-01-25 04:58:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polymer-pilaster-w-shoe-18w-x-82h-olive>: HTTP status code is not handled or not allowed 2026-01-25 04:58:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-oxygen-cylinder-rack-use-with-12-inchd-e-inch-style-cylinders>: HTTP status code is not handled or not allowed 2026-01-25 04:58:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/verishield-1035100-vs-100-series-passive-earmuffs-over-the-head-black-nrr-24>: HTTP status code is not handled or not allowed 2026-01-25 04:58:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sk12dl-x03-over-the-ear-style-1-wire-surveillance-kit-earpiece>: HTTP status code is not handled or not allowed 2026-01-25 04:59:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/valencia-series-hutch-with-doors-64-34wx15dx35-12h-espresso>: HTTP status code is not handled or not allowed 2026-01-25 04:59:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s201037-20-poly-web-sling-endless-17000-21200-42400-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-25 04:59:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-squeegee-set-urethane-for-nobles-tennant-9015735>: HTTP status code is not handled or not allowed 2026-01-25 04:59:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-squeegee-front-for-nilfisk-advance-vf81215>: HTTP status code is not handled or not allowed 2026-01-25 04:59:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/breakdown-odor-eliminator-fresh-scent-1-gallon-bottle-4case>: HTTP status code is not handled or not allowed 2026-01-25 04:59:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mota-series-big-and-tall-stool-black>: HTTP status code is not handled or not allowed 2026-01-25 04:59:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-aligning-steel-closed-linear-bearing-wresin-retainer-sw6gr-38id-0875l>: HTTP status code is not handled or not allowed 2026-01-25 04:59:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/dollies/open_deck_dollies already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:59:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacment-skirt-for-nilfisk-advance-56412107>: HTTP status code is not handled or not allowed 2026-01-25 04:59:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 66908 pages (at 94 pages/min), scraped 33555 items (at 51 items/min) 2026-01-25 04:59:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wireless-manual-muscle-tester-and-rom-inclinometer>: HTTP status code is not handled or not allowed 2026-01-25 04:59:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-squeegee-front-for-ipc-eagle-mpvr48380>: HTTP status code is not handled or not allowed 2026-01-25 04:59:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plain-magnetic-roll-stock-100l-x-2-12h-x-003-thick-black>: HTTP status code is not handled or not allowed 2026-01-25 04:59:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-pilaster-w-shoe-4w-x-82h-sand-beach>: HTTP status code is not handled or not allowed 2026-01-25 04:59:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-pilaster-w-shoe-22w-x-82h-royal-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:59:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1802-21-64-cobalt-heavy-duty-straw-135-split-point-jobber-length-drill>: HTTP status code is not handled or not allowed 2026-01-25 04:59:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clip-label-34-12l-x-1-14h-gray-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-25 04:59:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wireless-push-pull-force-gauge>: HTTP status code is not handled or not allowed 2026-01-25 04:59:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-pilaster-w-shoe-22w-x-82h-satin>: HTTP status code is not handled or not allowed 2026-01-25 04:59:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-inch-flush-bridge-in-espresso-executive-modular-furniture>: HTTP status code is not handled or not allowed 2026-01-25 04:59:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/infinity-8482-locker-3-tier-12-x-15-x-24-3-door-blue>: HTTP status code is not handled or not allowed 2026-01-25 04:59:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1823-5-16-heavy-duty-steam-oxide-power-wood-bits-drill>: HTTP status code is not handled or not allowed 2026-01-25 04:59:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dcs7485t1-flexvolt-60v-max-table-saw-kit-8-14-1-battery>: HTTP status code is not handled or not allowed 2026-01-25 04:59:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacment-squeegee-set-for-factory-cat-22-770l-factory-cat-22-754l-factory-cat-22-755l>: HTTP status code is not handled or not allowed 2026-01-25 04:59:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-pilaster-w-shoe-12w-x-82h-charcoal>: HTTP status code is not handled or not allowed 2026-01-25 04:59:26 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/dollies/open_deck_dollies landed on page that is not a product page. 2026-01-25 04:59:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1823-7-16-heavy-duty-steam-oxide-power-wood-bits-drill>: HTTP status code is not handled or not allowed 2026-01-25 04:59:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1823-9-16-heavy-duty-steam-oxide-power-wood-bits-drill>: HTTP status code is not handled or not allowed 2026-01-25 04:59:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b151058-6-single-leg-chain-sling-w-grab-hook>: HTTP status code is not handled or not allowed 2026-01-25 04:59:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/digital-door-lock-2200-surface-rim-mount-jet-black>: HTTP status code is not handled or not allowed 2026-01-25 04:59:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b151068-16-single-leg-chain-sling-w-sling-grab-hook>: HTTP status code is not handled or not allowed 2026-01-25 04:59:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b151078-4-single-leg-chain-sling-w-sling-hook>: HTTP status code is not handled or not allowed 2026-01-25 04:59:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48x48-slatwall-panel-without-inserts-ca>: HTTP status code is not handled or not allowed 2026-01-25 04:59:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-tier-2-door-digital-locker-12inwx18indx78inh-tan-unassembled-2>: HTTP status code is not handled or not allowed 2026-01-25 04:59:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/sockets_bits/wood_drill_bits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:59:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-industry-standard-clamping-couplings-w-keyway-climax-2iscc087-087skw>: HTTP status code is not handled or not allowed 2026-01-25 04:59:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b151059-8-single-leg-chain-sling-w-grab-hook>: HTTP status code is not handled or not allowed 2026-01-25 04:59:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/din6335-tp-star-knob-50197-m10-x-15-10nj03k>: HTTP status code is not handled or not allowed 2026-01-25 04:59:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-squeegee-side-for-nobles-tennant-1031011>: HTTP status code is not handled or not allowed 2026-01-25 04:59:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacment-skirt-for-nobles-tennant-1011376>: HTTP status code is not handled or not allowed 2026-01-25 04:59:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cushion-wrap-anti-static-24in-x-500ft-x-3-16in>: HTTP status code is not handled or not allowed 2026-01-25 04:59:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-locker-double-tier-3-wide-12-3-4w-x-18d-x-36-1-2h-gray-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 04:59:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/sockets_bits/wood_drill_bits>: HTTP status code is not handled or not allowed 2026-01-25 04:59:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-climax-2c-100-z>: HTTP status code is not handled or not allowed 2026-01-25 04:59:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mcr-safety-9669m-economy-pu-coated-work-gloves-13-gauge-black-medium-12-pairs>: HTTP status code is not handled or not allowed 2026-01-25 04:59:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/three-sided-frame-18x60-nexelon>: HTTP status code is not handled or not allowed 2026-01-25 04:59:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-climax-2c-012-z>: HTTP status code is not handled or not allowed 2026-01-25 04:59:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dunnage-shelf-24x24-nexelon>: HTTP status code is not handled or not allowed 2026-01-25 04:59:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1899-5-16-hss-general-purpose-steam-oxide-118-point-jobber-length-drill>: HTTP status code is not handled or not allowed 2026-01-25 04:59:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-rectangular-strip-carbide-blank-18x14x1-stb48a>: HTTP status code is not handled or not allowed 2026-01-25 04:59:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polymer-partition-panel-wo-brackets-5912w-folkstone-gray>: HTTP status code is not handled or not allowed 2026-01-25 04:59:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/tables/laboratory_tables already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:59:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-rectangular-strip-carbide-blank-18x38x12-stb412l>: HTTP status code is not handled or not allowed 2026-01-25 04:59:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacment-squeegee-set-for-nilfisk-advance-56325101-nilfisk-advance-56393395>: HTTP status code is not handled or not allowed 2026-01-25 04:59:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/polymer_bathroom_partitions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 04:59:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-industry-standard-clamping-couplings-climax-2iscc-137-137>: HTTP status code is not handled or not allowed 2026-01-25 04:59:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-climax-2c-262-z>: HTTP status code is not handled or not allowed 2026-01-25 04:59:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/polymer_bathroom_partitions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:00:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polymer-inward-swing-partition-door-26w-white>: HTTP status code is not handled or not allowed 2026-01-25 05:00:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-industry-standard-clamping-couplings-w-keyway-climax-2iscc100-075skw>: HTTP status code is not handled or not allowed 2026-01-25 05:00:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/tables/laboratory_tables>: HTTP status code is not handled or not allowed 2026-01-25 05:00:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-g-10-fr-4-garolite-sheet-1-16-thick-x-24-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 05:00:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/polymer_bathroom_partitions>: HTTP status code is not handled or not allowed 2026-01-25 05:00:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slotted-cap-with-staz-on-suspension-red>: HTTP status code is not handled or not allowed 2026-01-25 05:00:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/polymer_bathroom_partitions>: HTTP status code is not handled or not allowed 2026-01-25 05:00:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 67002 pages (at 94 pages/min), scraped 33598 items (at 43 items/min) 2026-01-25 05:00:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-flame-resistant-heavyweight-zip-up-hoodie-xl-hi-vis-yellow>: HTTP status code is not handled or not allowed 2026-01-25 05:00:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plb-r-oval-platter-baskets>: HTTP status code is not handled or not allowed 2026-01-25 05:00:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-carbon-fiber-tube-twill-weave-7-8-id-x-1-od-x-1-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 05:00:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/easy-trap-duster-8-x-125-ft-white-1-250-sheet-rollcarton>: HTTP status code is not handled or not allowed 2026-01-25 05:00:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-safety-coveralls-4xl-navy>: HTTP status code is not handled or not allowed 2026-01-25 05:00:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scp-15-sauce-cup-1-1-2-oz-stainless-steel-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 05:00:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/shelf_trucks/aluminum_shelf_trucks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:00:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tuffcomfort-standing-desk-anti-fatigue-mat-34-thick-2w-x-6l-coal-black>: HTTP status code is not handled or not allowed 2026-01-25 05:00:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/abc-industries-ceiling-panel-kit-belt-x-plain-end-16-dia-x-8l-x-6-pitch-white>: HTTP status code is not handled or not allowed 2026-01-25 05:00:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibrant-color-sponges-47-x-3-x-06-assorted-colors-4pack>: HTTP status code is not handled or not allowed 2026-01-25 05:00:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slotted-cap-with-yellow-green-stripe-hi-viz-orange>: HTTP status code is not handled or not allowed 2026-01-25 05:00:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-x-12-x-6-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-25 05:00:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v-tec-edge-personal-fall-limiter-single-leg-36cs-swivel-snap-hook-8l>: HTTP status code is not handled or not allowed 2026-01-25 05:00:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stack-pack-2-drawer-tool-box-22l-x-15-18w-x-16-516h-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:00:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/45-inch-portable-orange-delineator-post-2-ea-3-high-reflective>: HTTP status code is not handled or not allowed 2026-01-25 05:00:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-climax-1c-106-s>: HTTP status code is not handled or not allowed 2026-01-25 05:00:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/carts-trucks/shelf_trucks/aluminum_shelf_trucks>: HTTP status code is not handled or not allowed 2026-01-25 05:00:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-auto-ride-on-floor-scrubber-34-cleaning-path-three-170-amp-batteries>: HTTP status code is not handled or not allowed 2026-01-25 05:00:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rear-squeegee-blade-32-auto-ride-on-floor-scrubber>: HTTP status code is not handled or not allowed 2026-01-25 05:00:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-makrofol-de1-1-g-g-pc-film-0-010-x-24-x-48-100-sheet-case>: HTTP status code is not handled or not allowed 2026-01-25 05:00:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/phenolic-black-core-inward-swing-partition-door-w-hardware-24w-silver-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:00:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-laminate-pilaster-w-shoe-24w-x-82h-almond>: HTTP status code is not handled or not allowed 2026-01-25 05:00:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-sweatband-for-v-gard-h2-safety-helmet-black>: HTTP status code is not handled or not allowed 2026-01-25 05:00:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waterproof-patch-and-seal-tape-4-x-10-feet>: HTTP status code is not handled or not allowed 2026-01-25 05:00:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sd-5-paneled-sherbet-dish-5-oz-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 05:00:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-laminate-outward-swing-door-w-hardware-24w-dove-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:00:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/indoor-outdoor-sidebar-table-with-tablet-holder>: HTTP status code is not handled or not allowed 2026-01-25 05:00:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-construction-adhesive-tube-white-9-ounces>: HTTP status code is not handled or not allowed 2026-01-25 05:00:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-laminate-outward-swing-door-w-hardware-26w-black>: HTTP status code is not handled or not allowed 2026-01-25 05:00:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hot-glue-sticks-4-full-30-per-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:00:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ps-5-utility-scoop-5-oz-clear-polycarbonate>: HTTP status code is not handled or not allowed 2026-01-25 05:00:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-canvas-ce-phenolic-sheet-0-375-x-48-000-x-96-000>: HTTP status code is not handled or not allowed 2026-01-25 05:00:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-canvas-ce-phenolic-sheet-0-500-x-48-000-x-96-000>: HTTP status code is not handled or not allowed 2026-01-25 05:00:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-fep-sheet-0-375-thick-12-x-12>: HTTP status code is not handled or not allowed 2026-01-25 05:00:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/phenolic_bathroom_partitions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:00:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/psb-08y-squeeze-bottles-8-oz>: HTTP status code is not handled or not allowed 2026-01-25 05:00:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/phenolic-black-core-pilaster-w-shoe-24w-x-82h-folkstone-celesta>: HTTP status code is not handled or not allowed 2026-01-25 05:00:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rfm-2k-fluted-ramekins-2-oz-melamine-black>: HTTP status code is not handled or not allowed 2026-01-25 05:00:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pmc-5k-single-menu-cover-8-1-2w-5-1-2h-plastic-w-vinyl-binding-single-page-black>: HTTP status code is not handled or not allowed 2026-01-25 05:00:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-laminate-outward-swing-door-w-hardware-36w-almond>: HTTP status code is not handled or not allowed 2026-01-25 05:00:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-canvas-ce-phenolic-sheet-0-625-thick-48-x-96>: HTTP status code is not handled or not allowed 2026-01-25 05:00:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/plastic_laminate_bathroom_partitions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:00:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelf-vented-sides-18wx18dx78h-red-all-welded>: HTTP status code is not handled or not allowed 2026-01-25 05:00:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/phenolic-black-core-pilaster-w-shoe-10w-x-82h-graphite-grafix>: HTTP status code is not handled or not allowed 2026-01-25 05:00:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/phenolic_bathroom_partitions>: HTTP status code is not handled or not allowed 2026-01-25 05:00:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-x-36-x-36-24-22-20-multi-depth-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-25 05:00:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11-1-4-x-8-3-4-x-12-regular-slotted-container-1>: HTTP status code is not handled or not allowed 2026-01-25 05:00:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-canvas-ce-phenolic-sheet-0-047-x-36-000-x-48-000>: HTTP status code is not handled or not allowed 2026-01-25 05:00:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pmcp-100-measuring-cup-w-red-blue-markings-1-qt-clear-plastic>: HTTP status code is not handled or not allowed 2026-01-25 05:00:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/integra-office-chair-with-adj-arms-high-back-articulating-tilt-18-23h-seat-black>: HTTP status code is not handled or not allowed 2026-01-25 05:00:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dxcmv5076055-5-hp-two-stage-piston-compressor-60-gallon-vertical-175-psi-1-phase-230v>: HTTP status code is not handled or not allowed 2026-01-25 05:00:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-two-piece-industry-standard-clamping-couplings-w-keyway-climax-2miscc-12-12skw>: HTTP status code is not handled or not allowed 2026-01-25 05:00:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/plastic_laminate_bathroom_partitions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:00:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ptp-09a-pebbled-tumblers-95-oz-11h>: HTTP status code is not handled or not allowed 2026-01-25 05:00:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-climax-2c-143-z>: HTTP status code is not handled or not allowed 2026-01-25 05:00:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-soil-dishtable-right-to-left-96-long-16304-ss>: HTTP status code is not handled or not allowed 2026-01-25 05:00:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-soil-dishtable-right-to-left-crossrails-107-long-16304-ss>: HTTP status code is not handled or not allowed 2026-01-25 05:00:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/plastic_laminate_bathroom_partitions landed on page that is not a product page. 2026-01-25 05:01:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/equipment-stand-w-adjustable-undershelf-24-x-30-24-high-ss-top-galvanized-legs-undershelf>: HTTP status code is not handled or not allowed 2026-01-25 05:01:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-x-12-red-mailing-tubes-with-caps-case>: HTTP status code is not handled or not allowed 2026-01-25 05:01:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/plastic_laminate_bathroom_partitions>: HTTP status code is not handled or not allowed 2026-01-25 05:01:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/work-table-30-wide-top-24-long-adjustable-undershelf-ss-frame-shelf-16-gauge>: HTTP status code is not handled or not allowed 2026-01-25 05:01:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-x-12-white-mailing-tubes-with-caps-case>: HTTP status code is not handled or not allowed 2026-01-25 05:01:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-threaded-clamping-collar-recessed-screw-climax-tc-175-16>: HTTP status code is not handled or not allowed 2026-01-25 05:01:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commercial-kitchen-100-silicone-sealant-9-8-oz-white-7079808656>: HTTP status code is not handled or not allowed 2026-01-25 05:01:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 67095 pages (at 93 pages/min), scraped 33630 items (at 32 items/min) 2026-01-25 05:01:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-paper-phenolic-sheet-0-062-thick-48-x-96>: HTTP status code is not handled or not allowed 2026-01-25 05:01:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stretch-wrap-with-extended-core-handle-cast-80-gauge-20w-x-1000l-clear-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-25 05:01:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/atch-811-table-card-holder-acrylic>: HTTP status code is not handled or not allowed 2026-01-25 05:01:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelf-security-box-24wx24dx78h-blue-all-welded>: HTTP status code is not handled or not allowed 2026-01-25 05:01:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wsecurity-bxfootlockervented24x18x76blall-welded>: HTTP status code is not handled or not allowed 2026-01-25 05:01:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qf-clamp-with-pin-12-dia-galvanized-steel>: HTTP status code is not handled or not allowed 2026-01-25 05:01:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-coupling-w-keyway-climax-rc-062-kw>: HTTP status code is not handled or not allowed 2026-01-25 05:01:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-couplings-recessed-screw-w-keyway-cc-series-climax-2cc-087-087-kw>: HTTP status code is not handled or not allowed 2026-01-25 05:01:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-couplings-recessed-screw-w-keyway-cc-series-climax-2cc-100-100-kw>: HTTP status code is not handled or not allowed 2026-01-25 05:01:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/king-economy-9-section-with-t-hinge-privacy-screen>: HTTP status code is not handled or not allowed 2026-01-25 05:01:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-one-piece-industry-standard-clamping-couplings-w-keyway-climax-miscc-08-08-kw>: HTTP status code is not handled or not allowed 2026-01-25 05:01:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-nova-3-hygiene-kit-size-a-10>: HTTP status code is not handled or not allowed 2026-01-25 05:01:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-poly-bag-rolls-10x16-2-mil-1000-rl>: HTTP status code is not handled or not allowed 2026-01-25 05:01:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0016-03-winston-dinner-spoon>: HTTP status code is not handled or not allowed 2026-01-25 05:01:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergodyne-skullerz-8961-single-reflective-hard-hat-safety-helmet-sticker-kit-green>: HTTP status code is not handled or not allowed 2026-01-25 05:01:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0030-21-shangarila-large-bowl-serving-spoon>: HTTP status code is not handled or not allowed 2026-01-25 05:01:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergodyne-proflex-7002-eco-recycled-pu-coated-gloves-2xl-green-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 05:01:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/floor_machine_accessories_supplies/vacuum_attachments_tools already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:01:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-sided-bulk-truck-slat-sides-2-shlvs-lip-3600-lbs-cap-30-48-poly-wheels>: HTTP status code is not handled or not allowed 2026-01-25 05:01:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wsecurity-bxfootlockervented18x18x72grall-welded>: HTTP status code is not handled or not allowed 2026-01-25 05:01:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/floor_machine_accessories_supplies/replacement_scrubber_squeegees already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:01:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/as-5-utility-scoop-5-oz-aluminum-sure-grip-handle>: HTTP status code is not handled or not allowed 2026-01-25 05:01:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-threaded-clamping-collar-recessed-screw-climax-tc-075-16>: HTTP status code is not handled or not allowed 2026-01-25 05:01:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0345r-iron-tuff-low-bib-overall-short-navy-3xl>: HTTP status code is not handled or not allowed 2026-01-25 05:01:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slimline-rectangular-trash-can-lid-with-can-document-cutout-1623-gal-cap-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:01:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-door-cell-phone-locker-cp12-091572-dbk4-wide-whasp36w-x-15d-x-82-34h-black>: HTTP status code is not handled or not allowed 2026-01-25 05:01:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/floor_machine_accessories_supplies/vacuum_attachments_tools>: HTTP status code is not handled or not allowed 2026-01-25 05:01:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-grate-guard-with-overflows-sediment-high-flow-24l-x-24w-x-4h>: HTTP status code is not handled or not allowed 2026-01-25 05:01:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0021-01-continental-teaspoon>: HTTP status code is not handled or not allowed 2026-01-25 05:01:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/floor_machines_vacuums/floor_machine_accessories_supplies/replacement_scrubber_squeegees>: HTTP status code is not handled or not allowed 2026-01-25 05:01:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tb450b-adjustable-hydraulic-gate-closer>: HTTP status code is not handled or not allowed 2026-01-25 05:01:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelf-security-box-18wx18dx72h-blue-all-welded>: HTTP status code is not handled or not allowed 2026-01-25 05:01:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cbgr-1520-cutting-board-15l-20w-1-2h-green>: HTTP status code is not handled or not allowed 2026-01-25 05:01:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0358r-iron-tuff-siberian-jacket-regular-navy-4xl>: HTTP status code is not handled or not allowed 2026-01-25 05:01:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelf-security-box-18wx18dx78h-gray-all-welded>: HTTP status code is not handled or not allowed 2026-01-25 05:01:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/max-lift-standard-scissor-lift-table-foot-operated-4000-lb-capacity-64l-x-25w>: HTTP status code is not handled or not allowed 2026-01-25 05:01:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0344r-iron-tuff-minus-50-suit-regular-navy-2xl>: HTTP status code is not handled or not allowed 2026-01-25 05:01:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ghs-skull-crossbones-pictogram-label-2-x-2-500-roll>: HTTP status code is not handled or not allowed 2026-01-25 05:01:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=jtm-4vs-mill-with-3-axis-acu-rite-g-2-millpwr-cnc-with-air-powered-draw-bar already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:01:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/azar-600612-blk-c-channel-12-w-x-1-4-d-x-6-h>: HTTP status code is not handled or not allowed 2026-01-25 05:01:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-gx4-mounting-bracket>: HTTP status code is not handled or not allowed 2026-01-25 05:01:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-threaded-clamping-collar-recessed-screw-climax-tc-100-14-s>: HTTP status code is not handled or not allowed 2026-01-25 05:01:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-blast-suit-small>: HTTP status code is not handled or not allowed 2026-01-25 05:01:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/organic-waste-trash-cart-w-casters-64-gallon-organic-green-acg64-00lim>: HTTP status code is not handled or not allowed 2026-01-25 05:01:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-climax-1c-037-z>: HTTP status code is not handled or not allowed 2026-01-25 05:01:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/maxicut-ultra-micro-foam-nitrile-coated-palm-fingers-gloves-ansi-a4-xl-gray-6-dozencase>: HTTP status code is not handled or not allowed 2026-01-25 05:01:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-coupling-climax-rc-137>: HTTP status code is not handled or not allowed 2026-01-25 05:02:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coveralls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:02:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/azar-300211-slv-vertical-horizontal-optic-snap-frame-17-h>: HTTP status code is not handled or not allowed 2026-01-25 05:02:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/asfb-16-utility-scoop-w-flat-bottom-16-oz-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 05:02:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bash-soft-face-sledge-hammer-4-lb-head-12l-unbreakable-steel-core-handle>: HTTP status code is not handled or not allowed 2026-01-25 05:02:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-cu-yd-yellow-self-dumping-forkli-hopper>: HTTP status code is not handled or not allowed 2026-01-25 05:02:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=jtm-4vs-mill-with-3-axis-acu-rite-g-2-millpwr-cnc-with-air-powered-draw-bar landed on page that is not a product page. 2026-01-25 05:02:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0034-051-stanford-dinner-fork>: HTTP status code is not handled or not allowed 2026-01-25 05:02:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bw-dk8-carving-knife>: HTTP status code is not handled or not allowed 2026-01-25 05:02:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0630r-utility-jacket-regular-black-small>: HTTP status code is not handled or not allowed 2026-01-25 05:02:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 67189 pages (at 94 pages/min), scraped 33672 items (at 42 items/min) 2026-01-25 05:02:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coveralls>: HTTP status code is not handled or not allowed 2026-01-25 05:02:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dxs2153-metric-deep-impact-socket-16mm-drive-black>: HTTP status code is not handled or not allowed 2026-01-25 05:02:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/personnel-locker-double-tier-74x24x78-6-doors-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:02:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ut-16ht-utility-tong-coiled-spring-coiled-spring-16l-extra-heavyweight-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 05:02:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-l-x-30-w-14-gauge-stainless-workbench-w-undershelf-center-sink-2>: HTTP status code is not handled or not allowed 2026-01-25 05:02:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0442-chillbreaker-parka-regular-navy-medium>: HTTP status code is not handled or not allowed 2026-01-25 05:02:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0361-iron-tuff-winter-seal-jacket-regular-navy-small>: HTTP status code is not handled or not allowed 2026-01-25 05:02:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/azar-700775-clr-two-sided-pegboard-floor-display-w-adj-studio-base-17-w-x-17-d-x-70-h>: HTTP status code is not handled or not allowed 2026-01-25 05:02:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/respiratory_protection/welding_face_masks_respirators already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:02:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/maxicut-ultra-micro-foam-nitrile-coated-palm-fingers-gloves-ansi-a6-m-black-6-dozencase>: HTTP status code is not handled or not allowed 2026-01-25 05:02:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-z4-side-window-blockers-grey>: HTTP status code is not handled or not allowed 2026-01-25 05:02:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/four-inch-brass-interlocking-stencil-numbers-fifteen-piece-set>: HTTP status code is not handled or not allowed 2026-01-25 05:02:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rackable-plastic-pallet-black-48-x-40-closed-deck-6-runner-fork-cap-4400-lbs>: HTTP status code is not handled or not allowed 2026-01-25 05:02:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rackable-plastic-pallet-blue-48-x-40-open-deck-fda-grade-6-runner-fork-cap-4400-lbs>: HTTP status code is not handled or not allowed 2026-01-25 05:02:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0490-insulated-softshell-jacket-regular-black-charcoal-xl>: HTTP status code is not handled or not allowed 2026-01-25 05:02:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0491-non-insulated-softshell-jacket-regular-black-medium>: HTTP status code is not handled or not allowed 2026-01-25 05:02:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/respiratory_protection/welding_face_masks_respirators already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:02:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-electric-auto-floor-scrubber-18-cleaning-path-corded>: HTTP status code is not handled or not allowed 2026-01-25 05:02:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/respiratory_protection/welding_face_masks_respirators>: HTTP status code is not handled or not allowed 2026-01-25 05:02:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/narrow-column-protectors-4-to-6-inch-column-opening-yellow>: HTTP status code is not handled or not allowed 2026-01-25 05:02:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-px5-spark-arrestor>: HTTP status code is not handled or not allowed 2026-01-25 05:02:32 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=jtm-4vs-milling-machine-with-acu-rite-203-dro-x-axis-powerfeed-3-hp-230460v already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:02:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-z4-fr-face-seal>: HTTP status code is not handled or not allowed 2026-01-25 05:02:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0525-cooler-wear-jacket-regular-navy-5xl>: HTTP status code is not handled or not allowed 2026-01-25 05:02:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/respiratory_protection/welding_face_masks_respirators>: HTTP status code is not handled or not allowed 2026-01-25 05:02:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/maxflow-ansi-life-vest-mesh-uscgtc-type-iii-m-orange>: HTTP status code is not handled or not allowed 2026-01-25 05:02:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bstop4-08wm-bleed-stop-active-shooter-wall-mount-system-multi-use-multi-injury>: HTTP status code is not handled or not allowed 2026-01-25 05:02:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/twp-42-turner-4l-blade-8w-white-polypropylene-handle>: HTTP status code is not handled or not allowed 2026-01-25 05:02:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-30-plastic-top-pedestal-workbench-w-4-drawers-cabinet>: HTTP status code is not handled or not allowed 2026-01-25 05:02:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-2-wheel-trash-cart-w-casters-96-gallon-blue-acc96-00blu>: HTTP status code is not handled or not allowed 2026-01-25 05:02:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=jtm-4vs-milling-machine-with-acu-rite-203-dro-x-axis-powerfeed-3-hp-230460v>: HTTP status code is not handled or not allowed 2026-01-25 05:02:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plc-10046-sterile-natural-pp-cornell-hinged-vials-3-dram-11ml-case-of-500>: HTTP status code is not handled or not allowed 2026-01-25 05:02:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/respiratory_protection/powered_air_purifying_respirator_systems already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:02:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0302-stretch-liner-black-one-size>: HTTP status code is not handled or not allowed 2026-01-25 05:02:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gla-05558-amber-glass-screw-thread-sample-vials-only-1-dram-4ml-case-of-2016>: HTTP status code is not handled or not allowed 2026-01-25 05:02:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-by-20-stage-package-16-inches-height-grey-carpet-black-shirred-pleat-skirting>: HTTP status code is not handled or not allowed 2026-01-25 05:02:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-inch-x-5-yards-self-adhesive-bandage-non-latex-kelly-green-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-25 05:02:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s-1142-mini-piggyback-filterregulator-polycarbonate-bowl-14-npt-150-psi-20-micron>: HTTP status code is not handled or not allowed 2026-01-25 05:02:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cantilever-rack-inclined-arm-8>: HTTP status code is not handled or not allowed 2026-01-25 05:02:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cantilever-rack-straight-arm-with-2-lip-34>: HTTP status code is not handled or not allowed 2026-01-25 05:02:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-mb-4x2-mandrel-tube-bender-without-table-3ph-220v>: HTTP status code is not handled or not allowed 2026-01-25 05:02:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/respiratory_protection/powered_air_purifying_respirator_systems landed on page that is not a product page. 2026-01-25 05:02:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-bp-11213cnc-156-2-axis-programmable-hydraulic-press-brake-3ph-220v>: HTTP status code is not handled or not allowed 2026-01-25 05:02:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1451111-fisher-conveyor-chain-hc-68474>: HTTP status code is not handled or not allowed 2026-01-25 05:02:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modesto-32-in-tall-planter-white>: HTTP status code is not handled or not allowed 2026-01-25 05:02:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1110-piggyback-filterregulator-metal-bowl-38-npt-200-psi-40-micron>: HTTP status code is not handled or not allowed 2026-01-25 05:02:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1308110-universal-nylon-markers-b2036>: HTTP status code is not handled or not allowed 2026-01-25 05:02:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-green-3-tier-wire-shelving-add-on-unit-54-w-x-14-d-x-34-h>: HTTP status code is not handled or not allowed 2026-01-25 05:03:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ift-1d-dolly-w-cargo-strap-for-ift-1-polypropylene-green>: HTTP status code is not handled or not allowed 2026-01-25 05:03:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g-105c-mushroom-tops-for-g-105>: HTTP status code is not handled or not allowed 2026-01-25 05:03:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:03:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16in-step-on-dustpan-silver-2126>: HTTP status code is not handled or not allowed 2026-01-25 05:03:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/residential_decorative_lighting/recessed_lighting already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:03:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-cabinet-110>: HTTP status code is not handled or not allowed 2026-01-25 05:03:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-sheet-with-acrylic-adhesive-70a-38-thick-x-36-wide-x-36-long>: HTTP status code is not handled or not allowed 2026-01-25 05:03:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-br-16e-36-36-throat-depth-bead-roller-12-hp-110v>: HTTP status code is not handled or not allowed 2026-01-25 05:03:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1301805-universal-curb-guard>: HTTP status code is not handled or not allowed 2026-01-25 05:03:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:03:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-rdb-150-as-hydraulic-rotary-draw-tube-pipe-bender-with-auto-stop-110v>: HTTP status code is not handled or not allowed 2026-01-25 05:03:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 67282 pages (at 93 pages/min), scraped 33713 items (at 41 items/min) 2026-01-25 05:03:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-tier-shelving-starter-unit-solid-galvanized-steel-42w-x-18d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 05:03:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gh-26120zh-large-spindle-bore-lathe-10-hp-3ph-230460v>: HTTP status code is not handled or not allowed 2026-01-25 05:03:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips>: HTTP status code is not handled or not allowed 2026-01-25 05:03:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/residential_decorative_lighting already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:03:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mstf-8s-fine-single-mesh-8-diameter-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 05:03:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-green-2-tier-wire-shelving-starter-unit-48-w-x-12-d-x-34-h>: HTTP status code is not handled or not allowed 2026-01-25 05:03:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/241-punch-assembly-for-50t-55t-60t-ironworkers-2013-older>: HTTP status code is not handled or not allowed 2026-01-25 05:03:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls>: HTTP status code is not handled or not allowed 2026-01-25 05:03:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pdt-15-decanter-w-lid-51-oz-polycarbonate>: HTTP status code is not handled or not allowed 2026-01-25 05:03:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pecc-m-square-cover-for-pesc-6-8-and-pcsc-6c-8c-white>: HTTP status code is not handled or not allowed 2026-01-25 05:03:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-tier-shelving-starter-unit-solid-galvanized-steel-42w-x-24d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 05:03:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-tier-shelving-add-on-unit-solid-galvanized-steel-54w-x-18d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 05:03:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/indoor_lighting_fixtures/residential_decorative_lighting landed on page that is not a product page. 2026-01-25 05:03:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:03:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-tier-shelving-starter-unit-solid-galvanized-steel-54w-x-18d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 05:03:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-tier-shelving-starter-unit-solid-galvanized-steel-24w-x-18d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 05:03:25 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/hose_cord_reels/power_cord_reels already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:03:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bl-240r-bar-liner-red>: HTTP status code is not handled or not allowed 2026-01-25 05:03:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/57901-sanding-belt-1a-x-4-inch-x-24-inch-36-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-25 05:03:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0007-03-regency-dinner-spoon>: HTTP status code is not handled or not allowed 2026-01-25 05:03:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-recessed-screw-climax-h2c-225>: HTTP status code is not handled or not allowed 2026-01-25 05:03:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-43-welding-positioner-4400-lb-capacity-1-ph-220v>: HTTP status code is not handled or not allowed 2026-01-25 05:03:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fiberglass-tray-26x18x1-1-8-gray-1>: HTTP status code is not handled or not allowed 2026-01-25 05:03:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-25 05:03:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-climax-c-175-s>: HTTP status code is not handled or not allowed 2026-01-25 05:03:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/hose_cord_reels/power_cord_reels>: HTTP status code is not handled or not allowed 2026-01-25 05:03:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/l-322an-alloy-hook-swivel-w-latch-22t-wll>: HTTP status code is not handled or not allowed 2026-01-25 05:03:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ctk50-150-piece-mechanics-tool-set>: HTTP status code is not handled or not allowed 2026-01-25 05:03:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/37521-hook-and-loop-disc-5-inch-dia-320-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-25 05:03:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-roll-with-acrylic-adhesive-60a-14-thick-x-36-w-x-10-ft-l>: HTTP status code is not handled or not allowed 2026-01-25 05:03:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chain-double-clevis-link-1-2-9200-lbs-wll-s>: HTTP status code is not handled or not allowed 2026-01-25 05:03:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fiberglass-tote-lid-25-1-4x18-white-2>: HTTP status code is not handled or not allowed 2026-01-25 05:03:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-roll-with-acrylic-adhesive-50a-38-thick-x-36-w-x-10-ft-l>: HTTP status code is not handled or not allowed 2026-01-25 05:03:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g-209-galvanized-carbon-shackle-spa-3-16-1-3t-wll>: HTTP status code is not handled or not allowed 2026-01-25 05:03:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/air_curtains already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:03:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mounting-bracket-for-all-terrain-vise153>: HTTP status code is not handled or not allowed 2026-01-25 05:03:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dolly-for27-1-2-x-20-x-14-1-8-tote-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:03:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-piece-round-water-hyacinth-baskets-natural>: HTTP status code is not handled or not allowed 2026-01-25 05:03:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-row-national-rep-tip-and-roll-aluminum-bleacher-15-wide-single-footboard>: HTTP status code is not handled or not allowed 2026-01-25 05:03:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:03:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lightweight-set-screw-collar-l-climax-lc-037>: HTTP status code is not handled or not allowed 2026-01-25 05:03:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lightweight-set-screw-collar-l-climax-lc-050>: HTTP status code is not handled or not allowed 2026-01-25 05:03:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/bins_totes_containers/baskets_organizers/baskets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:03:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-stacking-bin-11-x-18-x-10-black>: HTTP status code is not handled or not allowed 2026-01-25 05:03:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-industry-standard-clamping-couplings-climax-iscc-100-100>: HTTP status code is not handled or not allowed 2026-01-25 05:03:54 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/air_curtains landed on page that is not a product page. 2026-01-25 05:03:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50146-3a-fiber-disc-4-1-2-inch-x-7-8-inch-100-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-25 05:03:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-climax-c-150>: HTTP status code is not handled or not allowed 2026-01-25 05:03:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:03:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g-209-galvanized-carbon-shackle-spa-1-1-2-17t-wll>: HTTP status code is not handled or not allowed 2026-01-25 05:03:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls>: HTTP status code is not handled or not allowed 2026-01-25 05:04:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n-7302-bi-fold-door-top-guide-wheel-58-inch-wheelpack-of-2>: HTTP status code is not handled or not allowed 2026-01-25 05:04:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ck-58-clamping-kit-w-tray-for-58-t-slot-pack-of-52>: HTTP status code is not handled or not allowed 2026-01-25 05:04:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s-4320-latch-kit-new-style-3t-carbon-5t-alloy-hook-2-31-l-x-0-83-w>: HTTP status code is not handled or not allowed 2026-01-25 05:04:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/bins_totes_containers/baskets_organizers/baskets landed on page that is not a product page. 2026-01-25 05:04:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leeson-sub-fhp-gearmotor-conduit-box-for-42-motor-frame>: HTTP status code is not handled or not allowed 2026-01-25 05:04:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/th-23088-concealed-casement-window-hinges-pair>: HTTP status code is not handled or not allowed 2026-01-25 05:04:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/din-6319-spherical-washers-steel-dished-type-blackened-m30-1-2-t-1-3-8-i-d>: HTTP status code is not handled or not allowed 2026-01-25 05:04:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/006-oring-epdm-18id-x-14od-70-duro-round-black-pkg-100>: HTTP status code is not handled or not allowed 2026-01-25 05:04:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-25 05:04:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:04:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/parts-bin-13-lb-capacity-7-38l-x-4-18w-x-3h>: HTTP status code is not handled or not allowed 2026-01-25 05:04:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/record-storage-rack-gray-96-wx-48-d-x-60-h-with-polyethylene-file-boxes>: HTTP status code is not handled or not allowed 2026-01-25 05:04:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 67373 pages (at 91 pages/min), scraped 33749 items (at 36 items/min) 2026-01-25 05:04:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leeson-single-phase-general-purpose-motor-5hp-3490rpm-184-dp-208-230v-60hz-1-15sf-rigid-c>: HTTP status code is not handled or not allowed 2026-01-25 05:04:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/capital-locker-single-tier-12x18x72-1-door-tan>: HTTP status code is not handled or not allowed 2026-01-25 05:04:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=magnetic-drill-single-phase-110v-md-5015 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:04:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/instrument-cart-w-hand-guard-flush-shelves-30-x-48>: HTTP status code is not handled or not allowed 2026-01-25 05:04:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-roll-with-high-temp-adhesive-40a-132-thick-x-36-wide-x-10-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 05:04:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wagon-truck-lip-deck-36-x-60-mold-on-rubber-wheels>: HTTP status code is not handled or not allowed 2026-01-25 05:04:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leeson-single-phase-general-purpose-motor-50hz-1hp-75kw-1425rpm-56h-ip541-0sf-rigid>: HTTP status code is not handled or not allowed 2026-01-25 05:04:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips landed on page that is not a product page. 2026-01-25 05:04:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leeson-3-phase-motor-1hp-1200rpm-145t-tefc-575v-60hz-cont-40c-1-15sf-rigid>: HTTP status code is not handled or not allowed 2026-01-25 05:04:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leeson-single-phase-general-purpose-motor-50hz-3-4hp-55kw-1425rpm-56-ip54manual-1-0sf-rigid>: HTTP status code is not handled or not allowed 2026-01-25 05:04:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-shelf-24-x-48>: HTTP status code is not handled or not allowed 2026-01-25 05:04:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-tarp-system-with-solid-vinyl-tarp-7-ft-x-18-ft-dtr7018s>: HTTP status code is not handled or not allowed 2026-01-25 05:04:21 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=magnetic-drill-single-phase-110v-md-5015 landed on page that is not a product page. 2026-01-25 05:04:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/front-base-15-w-x-6-h-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:04:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-tarp-system-with-mesh-tarp-7-ft-x-18-ft-dtr7018>: HTTP status code is not handled or not allowed 2026-01-25 05:04:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wow-ballpoint-retractable-pen-blue-ink-medium-dozen>: HTTP status code is not handled or not allowed 2026-01-25 05:04:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-shelf-30-x-48>: HTTP status code is not handled or not allowed 2026-01-25 05:04:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pem-aluminum-meeting-stile-w-nylon-insert-84-l-x-1-3-8-w-clear-anodized-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 05:04:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:04:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tackboard-black-fabric-for-71-hutch-black-fabric>: HTTP status code is not handled or not allowed 2026-01-25 05:04:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tyvek-400-sfr-coverall-elastic-wrists--ankle-serged-seams-3xl-white-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-25 05:04:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cnc-plasma-cutting-table-single-phase-110v-pt-22>: HTTP status code is not handled or not allowed 2026-01-25 05:04:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-strip-no-adhesive-70a-332-thick-x-2-wide-x-5-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 05:04:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-giant-stackable-storage-bin-16-1-2x18x11-red>: HTTP status code is not handled or not allowed 2026-01-25 05:04:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:04:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-rack-teardrop-backstop-120-inch-l-0-inch-offset>: HTTP status code is not handled or not allowed 2026-01-25 05:04:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets>: HTTP status code is not handled or not allowed 2026-01-25 05:04:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wagon-truck-steel-deck-8-x-250-rubber-wheel>: HTTP status code is not handled or not allowed 2026-01-25 05:04:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-mesh-partition-security-room-20x20x10-without-roof-3-sides>: HTTP status code is not handled or not allowed 2026-01-25 05:04:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:04:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foot-operated-corner-notcher-16-gauge-mild-steel-capacity-3-l-blade>: HTTP status code is not handled or not allowed 2026-01-25 05:04:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=manually-operated-corner-notcher-16-gauge-mild-steel-capacity-6-l-blade already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:04:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-general-security-laminated-padlocks-no-5lj>: HTTP status code is not handled or not allowed 2026-01-25 05:04:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets>: HTTP status code is not handled or not allowed 2026-01-25 05:04:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/notchers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:04:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caster-steer-trailer-30-x-60>: HTTP status code is not handled or not allowed 2026-01-25 05:04:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/32500-11-in-1-screwdrivernut-driver-w-cushion-grip>: HTTP status code is not handled or not allowed 2026-01-25 05:04:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/94-l-inside-corner-trim-in-rain-cloud-5-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:04:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:04:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1000-hot-melt-tape-3-x-55-yds-3-mil-clear-6pack>: HTTP status code is not handled or not allowed 2026-01-25 05:04:51 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-25 05:04:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-sheet-no-adhesive-60a-132-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 05:04:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=manually-operated-corner-notcher-16-gauge-mild-steel-capacity-6-l-blade landed on page that is not a product page. 2026-01-25 05:04:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-sheet-with-acrylic-adhesive-50a-12-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 05:04:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/machining/machinery/metal_cutting_forming_machines/notchers>: HTTP status code is not handled or not allowed 2026-01-25 05:04:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iteq235-circuit-breaker-type-qp-2-pole-35a>: HTTP status code is not handled or not allowed 2026-01-25 05:04:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mb920b-dipped-neoprene-snugleg-boots-brick-redbrown-size-11>: HTTP status code is not handled or not allowed 2026-01-25 05:04:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets>: HTTP status code is not handled or not allowed 2026-01-25 05:04:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manual-chain-hoist-20-foot-li-10-000-pound-capacity>: HTTP status code is not handled or not allowed 2026-01-25 05:05:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:05:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-mesh-partition-security-room-20x20x10-without-roof-4-sides-w-window>: HTTP status code is not handled or not allowed 2026-01-25 05:05:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-roll-no-adhesive-70a-12-thick-x-36-wide-x-10-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 05:05:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ited240-classified-circuit-breaker-type-qd-replacement-for-square-d-type-qo-2-pole-40a>: HTTP status code is not handled or not allowed 2026-01-25 05:05:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-part-carbonless-inventory-tag-pre-wired-4500-4999-8-6-14l-x-3-18w-500pack>: HTTP status code is not handled or not allowed 2026-01-25 05:05:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:05:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-shelving-open-shelving-36x18x75-17>: HTTP status code is not handled or not allowed 2026-01-25 05:05:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1350-wintertuff-ice-traction-stretch-overshoes-black-studded-outsole-xl>: HTTP status code is not handled or not allowed 2026-01-25 05:05:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 67461 pages (at 88 pages/min), scraped 33788 items (at 39 items/min) 2026-01-25 05:05:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets>: HTTP status code is not handled or not allowed 2026-01-25 05:05:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-shelving-open-shelving-36x12x39-24>: HTTP status code is not handled or not allowed 2026-01-25 05:05:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1200-weather-fashions-storm-rubber-overshoes-black-lg>: HTTP status code is not handled or not allowed 2026-01-25 05:05:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mid-size-aluminum-spray-nozzle-safety-orange>: HTTP status code is not handled or not allowed 2026-01-25 05:05:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o21007-eagle-plain-front-overall-yellow-xl>: HTTP status code is not handled or not allowed 2026-01-25 05:05:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-electric-horizontal-unit-heater-10kw-480v>: HTTP status code is not handled or not allowed 2026-01-25 05:05:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:05:16 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-25 05:05:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/capital-locker-double-tier-12x15x36-6-door-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:05:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carton-sealing-tape-inspected-3-x-110-yds-22-mil-redwhite-6pack>: HTTP status code is not handled or not allowed 2026-01-25 05:05:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluidity-03f-hung-diaphragm-solenoid-valve-2-2-nc-fkm-seal-1-nptf-25-mm-24v-dc>: HTTP status code is not handled or not allowed 2026-01-25 05:05:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/digital-timer-extra-loud-black>: HTTP status code is not handled or not allowed 2026-01-25 05:05:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-315-nipple-1-4-female-npt-asian-interchange>: HTTP status code is not handled or not allowed 2026-01-25 05:05:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluidity-02f-direct-acting-solenoid-valve-3-2-nc-nbr-seal-3-8-nptf-3-mm-12v-dc>: HTTP status code is not handled or not allowed 2026-01-25 05:05:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/900-hot-melt-tape-2-x-55-yds-25-mil-tan-6pack>: HTTP status code is not handled or not allowed 2026-01-25 05:05:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slate-design-rubber-tile-1969-x-1969-x-125-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:05:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-strip-no-adhesive-70a-316-thick-x-4-wide-x-5-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 05:05:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/p12008-magnaprene-plain-front-pants-green-4xl>: HTTP status code is not handled or not allowed 2026-01-25 05:05:31 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_strips landed on page that is not a product page. 2026-01-25 05:05:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o53122-comfortbrite-snap-fly-front-overall-fluorescent-lime-xl>: HTTP status code is not handled or not allowed 2026-01-25 05:05:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o53129-comfortbrite-snap-fly-front-overall-fluorescent-orange-2xl>: HTTP status code is not handled or not allowed 2026-01-25 05:05:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:05:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-ketron-peek-sheet-q-0-500-thick-x-24-w-x-48-l>: HTTP status code is not handled or not allowed 2026-01-25 05:05:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-af-blend-sheet-0-250-thick-x-24-w-x-24-l>: HTTP status code is not handled or not allowed 2026-01-25 05:05:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merit-alum-40-pipe-nipple-4-6-npt-male>: HTTP status code is not handled or not allowed 2026-01-25 05:05:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-delrin-homopolymer-sheet-3-000-thick-x-24-000-w-x-48-000-l>: HTTP status code is not handled or not allowed 2026-01-25 05:05:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:05:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mail-box-drop-safe-b5325dm-sr2sg40-26l-x-25w-x-53h-electronic-lock-936-cu-ft-black>: HTTP status code is not handled or not allowed 2026-01-25 05:05:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-delrin-homopolymer-sheet-0-125-thick-x-24-000-w-x-48-000-l>: HTTP status code is not handled or not allowed 2026-01-25 05:05:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-sheet-no-adhesive-50a-116-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 05:05:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-roll-no-adhesive-50a-38-thick-x-36-wide-x-10-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 05:05:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-25 05:05:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merit-ss-304-barstock-half-coupling-38-npt-female-plain>: HTTP status code is not handled or not allowed 2026-01-25 05:05:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-power-cold-gun-aircoolant-system-dual-outlet-2000-btu-hr>: HTTP status code is not handled or not allowed 2026-01-25 05:05:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/cable_management/cable_protectors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:05:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3m-990-mandrel048011-15012>: HTTP status code is not handled or not allowed 2026-01-25 05:05:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-in-super-air-knife-shim-set-alum>: HTTP status code is not handled or not allowed 2026-01-25 05:05:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets landed on page that is not a product page. 2026-01-25 05:05:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mb816b-metatarsal-steel-toe-boots-black-steel-midsole-size-6>: HTTP status code is not handled or not allowed 2026-01-25 05:05:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-cu-yd-self-dumping-heavy-duty-steel-hopper-with-bump-release-d-500-hd-6000-lb>: HTTP status code is not handled or not allowed 2026-01-25 05:05:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-sheet-no-adhesive-60a-132-thick-x-24-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 05:05:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-in-line-vac-only-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 05:05:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/abrasives/sanding_discs/flap_sanding_discs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:05:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-tag-pre-strung-5-4-34l-x-2-38w-fluorescent-green-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 05:05:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-clear-plastic-box-5-gallon-18x12x9>: HTTP status code is not handled or not allowed 2026-01-25 05:05:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/abrasives/sanding_discs/flap_sanding_discs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:05:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v73852-job-sight-class-2-surveyor-style-vest-fluorescent-lime-lxl>: HTTP status code is not handled or not allowed 2026-01-25 05:05:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-tag-pre-wired-3-3-34l-x-1-78w-fluorescent-orange-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 05:05:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slatwall-birch-shelves-3-4-x10-x47-3-4-finished-on-2-sides-and-3-edges>: HTTP status code is not handled or not allowed 2026-01-25 05:05:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/cable_management/cable_protectors landed on page that is not a product page. 2026-01-25 05:06:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-shelf-bench-rack-with-32-bins-8-wide-33x12x21-2>: HTTP status code is not handled or not allowed 2026-01-25 05:06:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-ft-dvi-d-cable-dual-link-male-to-male-black>: HTTP status code is not handled or not allowed 2026-01-25 05:06:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gas-caddy-hose-assembly>: HTTP status code is not handled or not allowed 2026-01-25 05:06:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/abrasives/sanding_discs/flap_sanding_discs landed on page that is not a product page. 2026-01-25 05:06:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/parts-bags-12w-x-22h-roll-500>: HTTP status code is not handled or not allowed 2026-01-25 05:06:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lan-station-utility-drawer-african-mahogany>: HTTP status code is not handled or not allowed 2026-01-25 05:06:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/abrasives/sanding_discs/flap_sanding_discs landed on page that is not a product page. 2026-01-25 05:06:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-shelf-steel-shelving-with-36-akrobins-36x18x39>: HTTP status code is not handled or not allowed 2026-01-25 05:06:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tyvek174-shipping-tag-pre-wired-2-34l-x-1-38w-blue-100pack>: HTTP status code is not handled or not allowed 2026-01-25 05:06:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 67554 pages (at 93 pages/min), scraped 33828 items (at 40 items/min) 2026-01-25 05:06:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-drawer-aluminum-tall-anesthesia-cart-push-button-lock-dark-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:06:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/j22208-iron-eagle-storm-fly-front-jacket-green-hood-snaps-3xl>: HTTP status code is not handled or not allowed 2026-01-25 05:06:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/13-shelf-steel-shelving-with-60-akro-mils-shelf-bins-4>: HTTP status code is not handled or not allowed 2026-01-25 05:06:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3x1-32x1-4-t1-a60-r-bf-cutoff-wheel>: HTTP status code is not handled or not allowed 2026-01-25 05:06:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1000-weather-fashions-commuter-rubber-overshoes-black-sm>: HTTP status code is not handled or not allowed 2026-01-25 05:06:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/filter-regulator-lubricator-kit>: HTTP status code is not handled or not allowed 2026-01-25 05:06:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/expandable-add-on-rack-48x12x84-gray-with-3-level-wood-deck-1500lb-cap-per-level>: HTTP status code is not handled or not allowed 2026-01-25 05:06:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/used-oil-storage-system-245-gallon>: HTTP status code is not handled or not allowed 2026-01-25 05:06:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/61500064490-belt-1-4w-x-18l-med-grit>: HTTP status code is not handled or not allowed 2026-01-25 05:06:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamper-bag-blue-vinyl-standard-pocket>: HTTP status code is not handled or not allowed 2026-01-25 05:06:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s61317-tuffenuff-3-pc-suit-gold-detachable-hood-3xl>: HTTP status code is not handled or not allowed 2026-01-25 05:06:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7w-120v-warm-white-flood-led-uplight-hartford-green-texture>: HTTP status code is not handled or not allowed 2026-01-25 05:06:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-tag-1-2-34l-x-1-38w-pink-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 05:06:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perm-liner-truck-18-bu-canvas-wood-base-2-rg-2-sw>: HTTP status code is not handled or not allowed 2026-01-25 05:06:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-tag-pre-strung-4-4-14l-x-2-18w-fluorescent-red-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 05:06:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-wall-corrugated-sheets-60l-x-48w-kraft-5pack>: HTTP status code is not handled or not allowed 2026-01-25 05:06:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perm-liner-truck-12-bu-canvas-wood-base-2-rg-2-sw>: HTTP status code is not handled or not allowed 2026-01-25 05:06:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perm-liner-truck-12-bu-blue-vinyl-wood-base-2-rg-2-sw>: HTTP status code is not handled or not allowed 2026-01-25 05:06:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:06:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/del-polypropylene-filter-class-m-19-7-10-l-x-19-7-10-w-x-23-3-5-h>: HTTP status code is not handled or not allowed 2026-01-25 05:06:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-machine-table-16-x-30-x-24>: HTTP status code is not handled or not allowed 2026-01-25 05:06:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/del-lance-w-flat-end-antistatic>: HTTP status code is not handled or not allowed 2026-01-25 05:06:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wide-span-rack-48x36x60-gray-with-3-shelves-laminated-deck-1200-lb-cap-per-level>: HTTP status code is not handled or not allowed 2026-01-25 05:06:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-tag-pre-strung-12-8l-x-4w-gray-500pack>: HTTP status code is not handled or not allowed 2026-01-25 05:06:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arrow-pneumatics-ar-10-a-non-cycling-refrigerated-air-dryer-10-cfm-1-ph-115v>: HTTP status code is not handled or not allowed 2026-01-25 05:06:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arrow-pneumatics-ar-20-a-non-cycling-refrigerated-air-dryer-20-cfm-1-ph-115v>: HTTP status code is not handled or not allowed 2026-01-25 05:06:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/del-vacuum-hose-yellow-50-l>: HTTP status code is not handled or not allowed 2026-01-25 05:06:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/rubber/rubber_rolls landed on page that is not a product page. 2026-01-25 05:06:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/del-vacuum-hose-yellow-20-l>: HTTP status code is not handled or not allowed 2026-01-25 05:06:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnetic-glass-dry-erase-board-36-x-24-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:06:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wide-span-rack-48x48x96-gray-with-3-shelves-laminated-deck-1200-lb-cap-per-level>: HTTP status code is not handled or not allowed 2026-01-25 05:06:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-his-sheet-0-093-thick-x-48-000-w-x-96-000-l>: HTTP status code is not handled or not allowed 2026-01-25 05:06:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/office_packing_tape/packaging_tape_carton_sealing_tape already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:06:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-shelf-with-laminated-deck-60x36-gray-2>: HTTP status code is not handled or not allowed 2026-01-25 05:06:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gss-saftey-8711-quilted-pants-class-e-lime-black-l-xl>: HTTP status code is not handled or not allowed 2026-01-25 05:06:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jewelry-boxes-6l-x-5w-x-1h-white-50pack>: HTTP status code is not handled or not allowed 2026-01-25 05:06:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-white-starboard-sheet-0-250-thick-x-54-000-w-x-96-000-l>: HTTP status code is not handled or not allowed 2026-01-25 05:06:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gpo-3-angle-2882-utr1494-0-250-thick-x-1-500-w-x-10-l>: HTTP status code is not handled or not allowed 2026-01-25 05:06:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-4-shelf-unit-60-x-60-x-13-1-2-9376-3-03>: HTTP status code is not handled or not allowed 2026-01-25 05:06:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-adjustable-4-shelf-unit-60-x-60-x-16-9366-3-03>: HTTP status code is not handled or not allowed 2026-01-25 05:06:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/welded-steel-shelving-5-shelves-24-x-48>: HTTP status code is not handled or not allowed 2026-01-25 05:06:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jewelry-boxes-3-12l-x-3-12w-x-1h-white-100pack>: HTTP status code is not handled or not allowed 2026-01-25 05:06:51 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/office_packing_tape/packaging_tape_carton_sealing_tape landed on page that is not a product page. 2026-01-25 05:06:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/346-flatback-tape-12-x-60-yds-167-mil-tan>: HTTP status code is not handled or not allowed 2026-01-25 05:06:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/complete-portable-frame-unit-with-tip-out-bins-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:06:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zepride-e-multi-use-liquid-cleaner-degreaser-55-gallon-drum>: HTTP status code is not handled or not allowed 2026-01-25 05:06:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6312530-nylon-plastic-threaded-plug-g-1-pipe-thread>: HTTP status code is not handled or not allowed 2026-01-25 05:06:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g5-145t-aff-7-5hp-oil-injected-rotary-screw-71-gal-tank-145psi-1ph-230v-w-dryer>: HTTP status code is not handled or not allowed 2026-01-25 05:06:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/great-lakes-tin-toledo-2-x-2-nail-up-tin-ceiling-tile-in-copper-t55-08>: HTTP status code is not handled or not allowed 2026-01-25 05:07:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aspect-3-x-6-peel-stick-glass-decorative-wall-tile-in-frost-8-pack-a50-63>: HTTP status code is not handled or not allowed 2026-01-25 05:07:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grade-cfp3-qualitative-cellulose-filter-paper-0-32-mm-thick-9-0-cm-dia-6-um-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 05:07:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/walls_ceilings/ceiling_tiles already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:07:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/53-natural-rubber-tape-2-x-110-yds-21-mil-clear-6pack>: HTTP status code is not handled or not allowed 2026-01-25 05:07:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-double-wall-corrugated-boxes-14l-x-14w-x-14h-kraft-15pack>: HTTP status code is not handled or not allowed 2026-01-25 05:07:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acrylic-safety-message-mirror-19-inch-x-16-inch-meet-the-person-sm301>: HTTP status code is not handled or not allowed 2026-01-25 05:07:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/libman-commercial-dual-surface-scrub-brush-handle>: HTTP status code is not handled or not allowed 2026-01-25 05:07:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/great-lakes-tin-jamestown-2-x-2-lay-in-tin-ceiling-tile-in-clear-y51-04>: HTTP status code is not handled or not allowed 2026-01-25 05:07:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-z-brite-wire-shelving-48wx21dx54h>: HTTP status code is not handled or not allowed 2026-01-25 05:07:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 67648 pages (at 94 pages/min), scraped 33868 items (at 40 items/min) 2026-01-25 05:07:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fresh-products-tidal-wave-urinal-screens-cotton-blossom-scent-6-box>: HTTP status code is not handled or not allowed 2026-01-25 05:07:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/libman-commercial-10-inch-nitty-gritty-roller-mop-refill>: HTTP status code is not handled or not allowed 2026-01-25 05:07:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/walls_ceilings/ceiling_tiles landed on page that is not a product page. 2026-01-25 05:07:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-surface-vinegar-cleaner-fresh-clean-scent-23-oz-spray-bottle-8-case>: HTTP status code is not handled or not allowed 2026-01-25 05:07:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7060-knob-adjusting-5512342>: HTTP status code is not handled or not allowed 2026-01-25 05:07:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ledge-48-l-x-4-h-wire-shelves-black-epoxy>: HTTP status code is not handled or not allowed 2026-01-25 05:07:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-strength-pine-cleaner-gallon-bottle-4-case>: HTTP status code is not handled or not allowed 2026-01-25 05:07:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/power_tools/concrete_cement_mixers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:07:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/maxithins-sanitary-pads-250-carton>: HTTP status code is not handled or not allowed 2026-01-25 05:07:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-z-brite-wire-shelving-30wx14dx86h>: HTTP status code is not handled or not allowed 2026-01-25 05:07:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-z-brite-wire-shelving-add-on-72wx14dx86h>: HTTP status code is not handled or not allowed 2026-01-25 05:07:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/free-and-gentle-laundry-detergent-32-loads-46-oz-bottle-6-bottles-case>: HTTP status code is not handled or not allowed 2026-01-25 05:07:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/probite-174-1-2-x-1-2-x-1-2-lead-free-brass-tee>: HTTP status code is not handled or not allowed 2026-01-25 05:07:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/panel-kit-60-w-workbench-with-18-w-louver-36-w-whiteboard-mounting-rail-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:07:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cover-for-optima-20-qt-stock-pot>: HTTP status code is not handled or not allowed 2026-01-25 05:07:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/power_tools/concrete_cement_mixers landed on page that is not a product page. 2026-01-25 05:07:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/non-ammoniated-glass-multi-surface-cleaner-pleasant-scent-128-oz-bottle>: HTTP status code is not handled or not allowed 2026-01-25 05:07:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/liquid-hand-soap-pumps-fresh-7-5-oz-bottle-6-case>: HTTP status code is not handled or not allowed 2026-01-25 05:07:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-standard-coalescing-filter>: HTTP status code is not handled or not allowed 2026-01-25 05:07:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/embassy-swj-5-wall-joiner-5-inch>: HTTP status code is not handled or not allowed 2026-01-25 05:07:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/497-diamond-plate-diamond-plate-ergonomic-mat-48-x-75-x-5-8-black>: HTTP status code is not handled or not allowed 2026-01-25 05:07:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-laundry-detergent-packs-powder-unscented-45-packets-pack-8-case>: HTTP status code is not handled or not allowed 2026-01-25 05:07:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/welding/arc_welding/tig_welding already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:07:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-clean-water-plastic-cap-641410-641411-floor-scrubbers>: HTTP status code is not handled or not allowed 2026-01-25 05:07:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stain-remover-and-color-booster-powder-original-49-2-oz-box-4-case>: HTTP status code is not handled or not allowed 2026-01-25 05:07:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/na-3s-submerged-arc-controller-for-cc-cv-applications-115-vac>: HTTP status code is not handled or not allowed 2026-01-25 05:07:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/overhead-door-track-protector-36-high>: HTTP status code is not handled or not allowed 2026-01-25 05:07:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-latex-bootshoe-covers-orange-xl-25-pairscase>: HTTP status code is not handled or not allowed 2026-01-25 05:07:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-midflow-particulate-filter-w-metal-bowl>: HTTP status code is not handled or not allowed 2026-01-25 05:07:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/welding/arc_welding/tig_welding>: HTTP status code is not handled or not allowed 2026-01-25 05:07:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-wool-soap-pad-4-pads-box-24-boxes-case>: HTTP status code is not handled or not allowed 2026-01-25 05:07:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-sintered-muffler>: HTTP status code is not handled or not allowed 2026-01-25 05:07:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chill-its-6615-high-performance-dew-rag-lime-one-size>: HTTP status code is not handled or not allowed 2026-01-25 05:07:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/470-marble-sof-tyle153-anti-fatigue-mat-12-thick-2w-x-up-to-75l-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:07:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wesco-folding-hand-cart-550-pound-capacity>: HTTP status code is not handled or not allowed 2026-01-25 05:07:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chill-its-6700-evaporative-cooling-bandana-tie-stars-stripes-one-size>: HTTP status code is not handled or not allowed 2026-01-25 05:07:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-midflow-regulator-with-gauge-t-handle-215-scfm>: HTTP status code is not handled or not allowed 2026-01-25 05:07:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:07:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/100-recycled-paper-towel-rolls-2-ply-11-x-5-4-140-towels-rl-24-rl-ct>: HTTP status code is not handled or not allowed 2026-01-25 05:07:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/table-lock-hndl--57--4421-5644931>: HTTP status code is not handled or not allowed 2026-01-25 05:07:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-x-1-7-8-drywall-screw-phillips-bugle-head-black-phosphate-tek-self-drilling-pt-100-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:07:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/100-recycled-bathroom-tissue-septic-safe-240-sheets-roll-12-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:07:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/410-airug-anti-fatigue-mat-38-thick-2w-x-60l-black>: HTTP status code is not handled or not allowed 2026-01-25 05:07:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8295-class-2-polo-shirt-lime-5-x-large>: HTTP status code is not handled or not allowed 2026-01-25 05:07:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8215ba-class-2-econo-breakaway-vest-orange-l-xl>: HTTP status code is not handled or not allowed 2026-01-25 05:07:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7020-screw-516x12-ref-162-5714261>: HTTP status code is not handled or not allowed 2026-01-25 05:07:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jrt-jumbo-bath-tissue-septic-safe-2-ply-white-3-3-x-500-ft-12-case>: HTTP status code is not handled or not allowed 2026-01-25 05:07:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions landed on page that is not a product page. 2026-01-25 05:08:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-straight-leg-dunnage-rack-60-lx24-wx8-h>: HTTP status code is not handled or not allowed 2026-01-25 05:08:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8215ba-class-2-econo-breakaway-vest-orange-2xl-3xl>: HTTP status code is not handled or not allowed 2026-01-25 05:08:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/water_heaters/gas_tankless_water_heaters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:08:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-mobile-dunnage-rack-42-lx20-wx8-h>: HTTP status code is not handled or not allowed 2026-01-25 05:08:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/turbo-boosting-de-icer-delivery-system>: HTTP status code is not handled or not allowed 2026-01-25 05:08:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8230z-class-2-two-tone-vest-orange-s-m>: HTTP status code is not handled or not allowed 2026-01-25 05:08:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foodsafe-cfswdnm3555t-e-ip69-2hp-3480rpm-3ph-60hz-145tc-3546m-tenv-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:08:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 67743 pages (at 95 pages/min), scraped 33912 items (at 44 items/min) 2026-01-25 05:08:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-l-deluxe-save-t-loc-strip-curtain-door-mounting-bracket-kit>: HTTP status code is not handled or not allowed 2026-01-25 05:08:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-seal-kit-for-manual-pallet-jack-truck-rl-t5bok-f-super-fits-model-t-and-e-series>: HTTP status code is not handled or not allowed 2026-01-25 05:08:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8205z-class-2-super-econo-vest-orange-2xl-3xl>: HTTP status code is not handled or not allowed 2026-01-25 05:08:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/plumbing/water_heaters/gas_tankless_water_heaters>: HTTP status code is not handled or not allowed 2026-01-25 05:08:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jet-jtm-949evs-mill-x-axis-powerfeed-air-powered-drawbar-690609>: HTTP status code is not handled or not allowed 2026-01-25 05:08:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-drawer-2-cabinet-workbench-plastic>: HTTP status code is not handled or not allowed 2026-01-25 05:08:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/belt-5l310-6077034>: HTTP status code is not handled or not allowed 2026-01-25 05:08:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dv25-dualrange-ac-voltage-detector-wflashlight-24-to-1000vac-100-to-1000vac>: HTTP status code is not handled or not allowed 2026-01-25 05:08:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-t-bar-3-shelf-rack-20-wx60-hx38-l>: HTTP status code is not handled or not allowed 2026-01-25 05:08:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ibg-8vsb-8-variable-speed-industrial-buffer-1hp-1-phase-115-230v>: HTTP status code is not handled or not allowed 2026-01-25 05:08:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/electrical_power_testing/multimeters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:08:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/drilling_hole_making/metalworking_drill_bits/drill_blanks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:08:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-h-laptop-wall-mount-unit-beige>: HTTP status code is not handled or not allowed 2026-01-25 05:08:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jet-jtm-1050evs2-230-mill-3-axis-acu-rite-200s-dro-quill-x-y-and-z-axis-powerfeeds-690617>: HTTP status code is not handled or not allowed 2026-01-25 05:08:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8225z-class-2-standard-vest-orange-4xl-5xl>: HTTP status code is not handled or not allowed 2026-01-25 05:08:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-mal-grit-grit-brush-stripping-w-np-9200-clutch-plate-black-813218np>: HTTP status code is not handled or not allowed 2026-01-25 05:08:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8320z-class-3-standard-vest-lime-l-xl>: HTTP status code is not handled or not allowed 2026-01-25 05:08:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8020hl-non-certified-standard-vest-lime-one-size>: HTTP status code is not handled or not allowed 2026-01-25 05:08:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:08:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/testing_inspection_instruments/electrical_power_testing/multimeters>: HTTP status code is not handled or not allowed 2026-01-25 05:08:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/machining/drilling_hole_making/metalworking_drill_bits/drill_blanks>: HTTP status code is not handled or not allowed 2026-01-25 05:08:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-4w-x-28l-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 05:08:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/long-sleeve-wicking-t-shirt-class-2-hi-vis-orange-m>: HTTP status code is not handled or not allowed 2026-01-25 05:08:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nut-hex-38-16-9056991>: HTTP status code is not handled or not allowed 2026-01-25 05:08:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8925-class-e-thermal-pants-lime-3-x-large>: HTTP status code is not handled or not allowed 2026-01-25 05:08:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cart-cover-10-gauge-vinyl-for-18-x-72-x-63-unit-zipper-closure-clear>: HTTP status code is not handled or not allowed 2026-01-25 05:08:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-rack-guard-with-rubber-bumper-5-5-wx3-75-dx36-h>: HTTP status code is not handled or not allowed 2026-01-25 05:08:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions>: HTTP status code is not handled or not allowed 2026-01-25 05:08:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blade-wheel-front--idler-5710421>: HTTP status code is not handled or not allowed 2026-01-25 05:08:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cart-cover-10-gauge-vinyl-for-18-x-36-x-63-unit-zipper-closure-clear>: HTTP status code is not handled or not allowed 2026-01-25 05:08:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-16w-x-40l-1-mil-clear-500pack>: HTTP status code is not handled or not allowed 2026-01-25 05:08:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/27-25-x-57-5mm-5-dram-glass-vial-w-24-400-green-thermoset-cap-vacuum-144pk>: HTTP status code is not handled or not allowed 2026-01-25 05:08:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speed-collection-premium-breathable-rain-pants-hi-viz-yellow-3xl>: HTTP status code is not handled or not allowed 2026-01-25 05:08:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/managers-desk-shell-dark-cherry-office-furniture-groupings-1>: HTTP status code is not handled or not allowed 2026-01-25 05:08:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-unit-with-two-enclosures-chrome-24w-x-48l-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 05:08:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-670-ambidextrous-single-strap-wrist-support-black-x-large>: HTTP status code is not handled or not allowed 2026-01-25 05:08:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acrylic-dispensing-bins-large-clear-7-14-w-x-12-12-d-x-11-h>: HTTP status code is not handled or not allowed 2026-01-25 05:08:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-one-table-of-contents-dividers-assorted-color-12-tab-months-letter-white>: HTTP status code is not handled or not allowed 2026-01-25 05:08:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-parka-hi-viz-yellow-3xl>: HTTP status code is not handled or not allowed 2026-01-25 05:08:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-28w-x-40l-2-mil-clear-250pack>: HTTP status code is not handled or not allowed 2026-01-25 05:08:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7ml-gen-purp-ldpe-transfer-pipets-standard-155mm-500-box-10-boxes-unit-5000pk>: HTTP status code is not handled or not allowed 2026-01-25 05:08:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/halo-heat-trace-shower-and-eye-face-wash-top-supply-s19-304gat>: HTTP status code is not handled or not allowed 2026-01-25 05:08:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_repair_maintenance/horns_horn_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:08:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/breathable-foul-weather-coat-hi-viz-orange-4xl>: HTTP status code is not handled or not allowed 2026-01-25 05:08:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bearing-9225-199>: HTTP status code is not handled or not allowed 2026-01-25 05:08:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-x-32mm-2ml-glass-big-mouth-screw-thread-vial-w-10-425-neck-finish-vial-only-1000pk>: HTTP status code is not handled or not allowed 2026-01-25 05:08:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rear-cover-jsg-0750-28-0028>: HTTP status code is not handled or not allowed 2026-01-25 05:08:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/128oz-clear-jug-w-38-400-black-phenolic-solid-pe-cap-vacuum-4pk>: HTTP status code is not handled or not allowed 2026-01-25 05:08:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/breathable-foul-weather-coat-hi-viz-yellow-xl>: HTTP status code is not handled or not allowed 2026-01-25 05:09:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dewalt-dwh303dh-onboard-dust-extractor-system-1-20v-max-sds-rotary-hammers-dch273p2-dch273b>: HTTP status code is not handled or not allowed 2026-01-25 05:09:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_repair_maintenance/horns_horn_accessories>: HTTP status code is not handled or not allowed 2026-01-25 05:09:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bin-nestable-6-5-8-w-x23-5-8-d-x-6-h-clear>: HTTP status code is not handled or not allowed 2026-01-25 05:09:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onoff-valve-jdp-20evs-20evs-c51>: HTTP status code is not handled or not allowed 2026-01-25 05:09:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-40w-x-48l-3-mil-clear-100pack>: HTTP status code is not handled or not allowed 2026-01-25 05:09:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/notebooks_pads already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:09:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/suction-disc-steel-rubber-6-w-x-5-d-x-6-h>: HTTP status code is not handled or not allowed 2026-01-25 05:09:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multipurpose-paper-98-brightness-20lb-812-x-11-bright-white-5000-shtsctn>: HTTP status code is not handled or not allowed 2026-01-25 05:09:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/miracool-hard-hat-pads-navy-24-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:09:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 67837 pages (at 94 pages/min), scraped 33953 items (at 41 items/min) 2026-01-25 05:09:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-h-upright-kit-tan>: HTTP status code is not handled or not allowed 2026-01-25 05:09:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-edge-writing-pad-jr-legal-rule-5-x-8-white-50sheet-dozen>: HTTP status code is not handled or not allowed 2026-01-25 05:09:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/separator-w-single-clamping-spindle-5-16-8-mm-to-2-3-8-60-mm-used-w-18-0-pulling-device>: HTTP status code is not handled or not allowed 2026-01-25 05:09:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/paper_stationery/notebooks_pads>: HTTP status code is not handled or not allowed 2026-01-25 05:09:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/separating-set-with-pulling-device-7-8-22-mm-to-4-1-2-115-mm-open>: HTTP status code is not handled or not allowed 2026-01-25 05:09:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-l-x-30-w-production-bench-plastic-safety-edge-blue-1>: HTTP status code is not handled or not allowed 2026-01-25 05:09:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sponge-cup-moistener-3-dia-clear>: HTTP status code is not handled or not allowed 2026-01-25 05:09:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/410kared-safety-padlock-1-12h-shackle-red>: HTTP status code is not handled or not allowed 2026-01-25 05:09:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3lhwht-laminated-steel-safety-padlock-1-916w-x-2h-shackle-white>: HTTP status code is not handled or not allowed 2026-01-25 05:09:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ball-joint-extractor-1-7-8-46-mm-opening-11-ton-capacity>: HTTP status code is not handled or not allowed 2026-01-25 05:09:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classic-cotton-high-visibility-t-shirt-orange-small>: HTTP status code is not handled or not allowed 2026-01-25 05:09:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/contemporary-melrose-white-leather-sofa-with-encasing-frame>: HTTP status code is not handled or not allowed 2026-01-25 05:09:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-spacer-jdp-20evs-20evs-h36>: HTTP status code is not handled or not allowed 2026-01-25 05:09:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-3w-x-7l-4-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 05:09:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/contemporary-bucket-seat-adjustable-height-barstool-vinyl-black>: HTTP status code is not handled or not allowed 2026-01-25 05:09:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s1107clr-aluminum-safety-padlock-1-12w-x-3h-shackle-silver>: HTTP status code is not handled or not allowed 2026-01-25 05:09:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/beds_mattresses/mattresses already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:09:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-5w-x-6l-4-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 05:09:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anchor-concrete-bolt-on-2104560>: HTTP status code is not handled or not allowed 2026-01-25 05:09:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/notebooks_pads already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:09:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-performance-faceshield-window-wide-view-4178cl>: HTTP status code is not handled or not allowed 2026-01-25 05:09:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/centers-28-1031-vul404145-1>: HTTP status code is not handled or not allowed 2026-01-25 05:09:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-premium-locker-single-tier-12x15x72-3-door-ready-assemble-parchment>: HTTP status code is not handled or not allowed 2026-01-25 05:09:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/410kaltred-safety-padlock-3h-shackle-red>: HTTP status code is not handled or not allowed 2026-01-25 05:09:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/beds_mattresses/mattresses>: HTTP status code is not handled or not allowed 2026-01-25 05:09:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-9w-x-9l-4-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 05:09:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/paper_stationery/notebooks_pads landed on page that is not a product page. 2026-01-25 05:09:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-premium-locker-single-tier-12x15x72-1-door-ready-assemble-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:09:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spring-jsm-403-40329-12>: HTTP status code is not handled or not allowed 2026-01-25 05:09:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3m-6391-respirator-6391>: HTTP status code is not handled or not allowed 2026-01-25 05:09:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-double-arm-fixed-wall-bracket-green>: HTTP status code is not handled or not allowed 2026-01-25 05:09:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnetrons-34-2124-tucngc-3015>: HTTP status code is not handled or not allowed 2026-01-25 05:09:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/23-poly-grout-bag>: HTTP status code is not handled or not allowed 2026-01-25 05:09:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-mortar-hoe-66-wood-handle>: HTTP status code is not handled or not allowed 2026-01-25 05:09:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-s1424gz-green-wire-shelf-24-wx14-d>: HTTP status code is not handled or not allowed 2026-01-25 05:09:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/funnels-68-374-ham952493205>: HTTP status code is not handled or not allowed 2026-01-25 05:09:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-plug-body-with-high-flow-profile-1-2-hose-barb-connection-used-with-esi-coupling>: HTTP status code is not handled or not allowed 2026-01-25 05:09:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-step-24in-w-steel-safety-angle-rolling-ladder-grip-strut-gray-kdec114242>: HTTP status code is not handled or not allowed 2026-01-25 05:09:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-18w-x-20l-6-mil-clear-250pack>: HTTP status code is not handled or not allowed 2026-01-25 05:09:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-unit-5-tier-poly-z-brite-18w-x-36l-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 05:09:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fix-plate-cos18-155>: HTTP status code is not handled or not allowed 2026-01-25 05:09:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s1-safety-quick-coupling-1-4-industrial-interchange-with-1-2-mnpt-connection>: HTTP status code is not handled or not allowed 2026-01-25 05:09:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ball-peen-hammers-101g>: HTTP status code is not handled or not allowed 2026-01-25 05:09:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-unit-5-tier-nexelon-21w-x-54l-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 05:09:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gear-box-cover-white-j-5710111>: HTTP status code is not handled or not allowed 2026-01-25 05:09:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/lathe_chucks_accessories/lathe_chucks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:09:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/penco-locker-single-tier-18x18x72-1-door-assembled-gray-2>: HTTP status code is not handled or not allowed 2026-01-25 05:09:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lamps-38-1673-henbl01-018>: HTTP status code is not handled or not allowed 2026-01-25 05:10:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hard-master-jaws-for-bison-scroll-chuck-6-inch-3-jaw-3-piece-set>: HTTP status code is not handled or not allowed 2026-01-25 05:10:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ceiling-kit-2-ton-2-5-ton-air-conditioners-1>: HTTP status code is not handled or not allowed 2026-01-25 05:10:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/end-mill-holder-r8-shank-1>: HTTP status code is not handled or not allowed 2026-01-25 05:10:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tongue-groove-pliers-440-bulk>: HTTP status code is not handled or not allowed 2026-01-25 05:10:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-step-steel-caged-walk-through-fixed-access-ladder-gray-wlfc1230>: HTTP status code is not handled or not allowed 2026-01-25 05:10:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gusseted-poly-bags-38w-x-36d-x-70h-3-mil-clear-50pack>: HTTP status code is not handled or not allowed 2026-01-25 05:10:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/machining/machinery_clamping_positioning/lathe_chucks_accessories/lathe_chucks>: HTTP status code is not handled or not allowed 2026-01-25 05:10:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-unit-5-tier-nexelon-21w-x-42l-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 05:10:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-l-x-30-w-stainless-steel-crossbraced-leg-table-with-galvanized-understructure>: HTTP status code is not handled or not allowed 2026-01-25 05:10:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 67933 pages (at 96 pages/min), scraped 33995 items (at 42 items/min) 2026-01-25 05:10:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/measurement_layout_tools/calipers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:10:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0005-x-030-horizontal-test-indicator-0-15-0>: HTTP status code is not handled or not allowed 2026-01-25 05:10:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/apwk724nfss-window-kit-700x2400-950x2650x31-ss>: HTTP status code is not handled or not allowed 2026-01-25 05:10:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-step-16in-w-20in-d-top-step-steel-rolling-ladder-grip-strut-tread-no-handrail-kdsr004162-d2>: HTTP status code is not handled or not allowed 2026-01-25 05:10:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-unit-5-tier-chrome-18w-x-48l-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 05:10:15 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-25 05:10:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-easy-access-18-bushel-plastic-box-truck-with-attached-cargo-netting>: HTTP status code is not handled or not allowed 2026-01-25 05:10:16 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:34 2026-01-25 05:10:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-6w-x-30l-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 05:10:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-od-polyurethane-tubing-black-color-100-roll-125-200-psi>: HTTP status code is not handled or not allowed 2026-01-25 05:10:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/measurement_layout_tools/calipers landed on page that is not a product page. 2026-01-25 05:10:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-24-inch-adjustable-height-steel-work-platform-5inh-to-8inh-mcos72245>: HTTP status code is not handled or not allowed 2026-01-25 05:10:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gear-pullers-4038>: HTTP status code is not handled or not allowed 2026-01-25 05:10:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-ventilated-locker-six-tier-12x15x12-6-door-assembled-2>: HTTP status code is not handled or not allowed 2026-01-25 05:10:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ck23-311-contactor-3-pole-460v>: HTTP status code is not handled or not allowed 2026-01-25 05:10:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-step-plastic-step-stand-20-w-x-28-1-2-d-x-33-1-2-h-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:10:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-gas-infrared-heater-092020ngs-straight-configuration-20-l-50000-btu>: HTTP status code is not handled or not allowed 2026-01-25 05:10:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-48x24x72-with-4-shelves-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 05:10:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shovels-52315>: HTTP status code is not handled or not allowed 2026-01-25 05:10:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gear-leverleft-ghd-2020pf-2401067>: HTTP status code is not handled or not allowed 2026-01-25 05:10:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wedge-pillow-to-support-and-elevate-neck-head-and-back-for-acid-reflux-or-feet-and-legs-to-reduce-back-pain-and-improve-circulation-with-removable-cover-7x24x24-inch-white>: HTTP status code is not handled or not allowed 2026-01-25 05:10:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-sprng-vbs-1610-2040>: HTTP status code is not handled or not allowed 2026-01-25 05:10:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hat-style-saddle-hanger-4-inch-diameter-galvanized>: HTTP status code is not handled or not allowed 2026-01-25 05:10:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clamp-together-90-deg-elbow-10-clr-3-inch-diameter-glv-22-gage>: HTTP status code is not handled or not allowed 2026-01-25 05:10:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/center-punches-p-1291>: HTTP status code is not handled or not allowed 2026-01-25 05:10:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-bulk-box-12-bushel-truck-steel-chassis-base-red>: HTTP status code is not handled or not allowed 2026-01-25 05:10:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-protective-underwear-size-l-for-waist-size-40-56-20-bag>: HTTP status code is not handled or not allowed 2026-01-25 05:10:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lamp-holder-bayonet-style-1820621>: HTTP status code is not handled or not allowed 2026-01-25 05:10:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/locking-link-with-studs-1820701>: HTTP status code is not handled or not allowed 2026-01-25 05:10:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mighty-products-mn-ms70-12100-70-shade-cloth-shade-tarp-12-x-100>: HTTP status code is not handled or not allowed 2026-01-25 05:10:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/minigrip-reclosable-lab-guard-uv-protection-bags-4w-x-6l-3-mil-amber-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 05:10:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-with-hang-hole-5w-x-8l-4-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 05:10:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-step-roll-fold-steel-rolling-ladder-perforated-tread>: HTTP status code is not handled or not allowed 2026-01-25 05:10:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/venture-ii-eyewear-orange-lens-with-black-frame>: HTTP status code is not handled or not allowed 2026-01-25 05:10:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/163-sjtw-8ft-3-outlet-heavy-duty-extension-cord-gg-15008-orange>: HTTP status code is not handled or not allowed 2026-01-25 05:10:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-oz-tube-super-lube-silicone-lubricating-grease-with-ptfe>: HTTP status code is not handled or not allowed 2026-01-25 05:10:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/45550-electric-oil-pump-115v-ac-motor-4-4-gpm>: HTTP status code is not handled or not allowed 2026-01-25 05:10:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tinted-shopping-bags-16w-x-6d-x-12h-green-tea-250pack>: HTTP status code is not handled or not allowed 2026-01-25 05:10:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-w-x-30-d-production-workbench-maple-butcher-block-safety-edge-with-drawers-shelf-tan>: HTTP status code is not handled or not allowed 2026-01-25 05:10:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/44031-professional-manual-rotary-barrel-pump-with-hose-discharge-spout-nbr-seals>: HTTP status code is not handled or not allowed 2026-01-25 05:10:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72in-over-head-light-and-fixture>: HTTP status code is not handled or not allowed 2026-01-25 05:10:58 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/power_tools/power_saws_blades/band_saws/band_saw_blades already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:10:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-value-pro-cotton-economy-wet-mop-w-1in-headband-fgv11800wh00>: HTTP status code is not handled or not allowed 2026-01-25 05:10:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/red-vinyl-braided-rope-59in-with-ends-for-portable-gold-post>: HTTP status code is not handled or not allowed 2026-01-25 05:11:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gear-30t-ghb-1340-6218>: HTTP status code is not handled or not allowed 2026-01-25 05:11:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rear-sway-braces-48-wide-units>: HTTP status code is not handled or not allowed 2026-01-25 05:11:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-dog-house->: HTTP status code is not handled or not allowed 2026-01-25 05:11:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/goop-hand-cleaner-creme-14-oz-can>: HTTP status code is not handled or not allowed 2026-01-25 05:11:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/office_machines/laminators_pouches already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:11:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-cabinets-fs-227-p-storage-cabinet-assembled-27x15x72-putty>: HTTP status code is not handled or not allowed 2026-01-25 05:11:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/power_tools/power_saws_blades/band_saws/band_saw_blades>: HTTP status code is not handled or not allowed 2026-01-25 05:11:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 68030 pages (at 97 pages/min), scraped 34045 items (at 50 items/min) 2026-01-25 05:11:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-hanford-triple-post-only-with-support-brace-with-ornate-base>: HTTP status code is not handled or not allowed 2026-01-25 05:11:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-cabinets-maah-36782rb-g-mobile-storage-cabinet-assembled-36x24x78-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:11:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/cabinets/full_height_storage_cabinets/stationary_storage_cabinets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:11:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-precision-iso-metric-roller-chain-16b-1-1-pitch-connecting-link>: HTTP status code is not handled or not allowed 2026-01-25 05:11:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cando-hand-held-massage-gun>: HTTP status code is not handled or not allowed 2026-01-25 05:11:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-foot-x-30-foot-forest-green-tarp>: HTTP status code is not handled or not allowed 2026-01-25 05:11:15 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/office_machines/laminators_pouches landed on page that is not a product page. 2026-01-25 05:11:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-w-x-36-d-production-workbench-maple-butcher-block-safety-edge-complete-bench-tan>: HTTP status code is not handled or not allowed 2026-01-25 05:11:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6811z-rib-knit-hat-zipper-with-bump-cap-insert-black>: HTTP status code is not handled or not allowed 2026-01-25 05:11:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cross-screw-1230r-5232261>: HTTP status code is not handled or not allowed 2026-01-25 05:11:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9415pc-double-sided-removable-film-tape-34-x-72-yds-2-mil-clear-48pack>: HTTP status code is not handled or not allowed 2026-01-25 05:11:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g1950p-eye-face-wash-shower-plastic-bowl-plastic-bowl>: HTTP status code is not handled or not allowed 2026-01-25 05:11:21 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/cabinets/full_height_storage_cabinets/stationary_storage_cabinets landed on page that is not a product page. 2026-01-25 05:11:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pointer-vbs-14081610-9700>: HTTP status code is not handled or not allowed 2026-01-25 05:11:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-foot-x-36-foot-blue-tarp>: HTTP status code is not handled or not allowed 2026-01-25 05:11:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-x-36-x-30-drawer-shelf-shop-stand-shop-top-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:11:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hl400-lpf-refill-dispenser-refill-canister-t-shape-2-canisters-800-pair>: HTTP status code is not handled or not allowed 2026-01-25 05:11:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/gloves_sleeves/cut_resistant_gloves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:11:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-precision-ansi-nickel-plated-roller-chain-40-1np-1-2-pitch-connecting-link>: HTTP status code is not handled or not allowed 2026-01-25 05:11:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smooth-paper-clips-wire-jumbo-silver-100box>: HTTP status code is not handled or not allowed 2026-01-25 05:11:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-foot-x-12-foot-high-visibility-orange-tarp>: HTTP status code is not handled or not allowed 2026-01-25 05:11:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25-foot-x-40-foot-blue-tarp>: HTTP status code is not handled or not allowed 2026-01-25 05:11:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ez-carbon-steel-truck-dump-insert-for-6-foot-bed>: HTTP status code is not handled or not allowed 2026-01-25 05:11:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-key-4x4x12-jvm836-44>: HTTP status code is not handled or not allowed 2026-01-25 05:11:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nut-jtm-4vs-mk-014>: HTTP status code is not handled or not allowed 2026-01-25 05:11:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/right-flange-jmd15-226>: HTTP status code is not handled or not allowed 2026-01-25 05:11:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-w-3-drawer-binder-lateral-file-black-1>: HTTP status code is not handled or not allowed 2026-01-25 05:11:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-tube-light-rl70le-24-n-led-22-26v-dc-125w-glass-146>: HTTP status code is not handled or not allowed 2026-01-25 05:11:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/dump_truck_inserts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:11:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/gloves_sleeves/cut_resistant_gloves landed on page that is not a product page. 2026-01-25 05:11:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-precision-ansi-double-roller-chain-35-2r-3-8-pitch-100ft-reel>: HTTP status code is not handled or not allowed 2026-01-25 05:11:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cylinder-cap-jet-0t-y31002020>: HTTP status code is not handled or not allowed 2026-01-25 05:11:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scroll-mount-address-post-no-base-with-horsehead-finial>: HTTP status code is not handled or not allowed 2026-01-25 05:11:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40192-hose-adapter-steel>: HTTP status code is not handled or not allowed 2026-01-25 05:11:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-kit-for-electric-pallet-jack-gwk-ew60-lw-xl-fits-baker-model-ew60>: HTTP status code is not handled or not allowed 2026-01-25 05:11:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4150064-air-inlet-adapter-steel>: HTTP status code is not handled or not allowed 2026-01-25 05:11:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crossover-bridge-20-inch-l-x-24-inch-w-perforated>: HTTP status code is not handled or not allowed 2026-01-25 05:11:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/dump_truck_inserts>: HTTP status code is not handled or not allowed 2026-01-25 05:11:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/minigrip174-reclosable-poly-bags-12w-x-15l-4-mil-clear-500pack>: HTTP status code is not handled or not allowed 2026-01-25 05:11:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprng-pin-jtm-1055-jtm1055-v45-1>: HTTP status code is not handled or not allowed 2026-01-25 05:11:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/05800su-fc22t5sp830-22w-fluorescent-t5-circline-bulb-2gx13-warm-white>: HTTP status code is not handled or not allowed 2026-01-25 05:11:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss-tong-16-white>: HTTP status code is not handled or not allowed 2026-01-25 05:11:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inner-liner-can-option-slc-12>: HTTP status code is not handled or not allowed 2026-01-25 05:11:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pretzels-peanut-butter-tub-24oz>: HTTP status code is not handled or not allowed 2026-01-25 05:11:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/45-qt-1-4-size-steam-table-pan-anti-jam-23-ga>: HTTP status code is not handled or not allowed 2026-01-25 05:11:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-cal-recycled-rubber-flooring-1-4-x-4-x-5-rolls-black-rubber-mats>: HTTP status code is not handled or not allowed 2026-01-25 05:11:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11699-window-underlay-12-x-30-polystyrene>: HTTP status code is not handled or not allowed 2026-01-25 05:11:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/trash_recycling/trash_recycling_containers/indoor_trash_cans/metal_indoor_trash_cans already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:11:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-cal-diamond-grip-rubber-runner-mats-2mm-x-4-x-40-rubber-flooring-rolls-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:11:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speedpress-6-12-rhino-cutting-mat-grid>: HTTP status code is not handled or not allowed 2026-01-25 05:12:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sketch-pla-filament-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:12:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/174-1791078k-pm1300tx-bk-dust-collector>: HTTP status code is not handled or not allowed 2026-01-25 05:12:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-liquid-level-display-tube-mountin-261990-641250-641263-641407-floor-scrubbers>: HTTP status code is not handled or not allowed 2026-01-25 05:12:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-upright-vacuum-12-cleaning-path>: HTTP status code is not handled or not allowed 2026-01-25 05:12:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-handle-60-x-30-structural-foam-plastic-platform-trucks>: HTTP status code is not handled or not allowed 2026-01-25 05:12:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/trash_recycling/trash_recycling_containers/indoor_trash_cans/metal_indoor_trash_cans>: HTTP status code is not handled or not allowed 2026-01-25 05:12:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uj306018a-sectionalizing-cabinet-3000x6000x1800-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 05:12:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cal-osha-kit-for-ladders-16in-w-steps-yellow-grcal05ec-y>: HTTP status code is not handled or not allowed 2026-01-25 05:12:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-head-screw-jtm-1050evs-ivs-051>: HTTP status code is not handled or not allowed 2026-01-25 05:12:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 68124 pages (at 94 pages/min), scraped 34085 items (at 40 items/min) 2026-01-25 05:12:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-foam-squares-12-x-12-3125-mil-white-1296roll>: HTTP status code is not handled or not allowed 2026-01-25 05:12:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-blue-scrubbing-pad-5-per-case>: HTTP status code is not handled or not allowed 2026-01-25 05:12:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/600-gph-manual-lift-pump>: HTTP status code is not handled or not allowed 2026-01-25 05:12:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10oz-piston-pump-for-70mm-5-6-7-15-30-and-55>: HTTP status code is not handled or not allowed 2026-01-25 05:12:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guard-belt-3250135>: HTTP status code is not handled or not allowed 2026-01-25 05:12:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-slip-on-shoes-size-12m-white>: HTTP status code is not handled or not allowed 2026-01-25 05:12:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-dress-mary-jane-shoes-size-55m-black>: HTTP status code is not handled or not allowed 2026-01-25 05:12:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a30r308hcr-hinged-cover-medium-type-3r-3000x3000x800-galvanizedpaint>: HTTP status code is not handled or not allowed 2026-01-25 05:12:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/optiloc-3-sections-14>: HTTP status code is not handled or not allowed 2026-01-25 05:12:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-quilt>: HTTP status code is not handled or not allowed 2026-01-25 05:12:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stanco-flame-resistant-13-atpv-jacket-frc9630orcs-l>: HTTP status code is not handled or not allowed 2026-01-25 05:12:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/outdoor-resin-stack-chair-white-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-25 05:12:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/minigrip174-reclosable-poly-bags-6w-x-8l-4-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 05:12:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-36-2-drawer-with-steel-top-workbench>: HTTP status code is not handled or not allowed 2026-01-25 05:12:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dinex-clear-dome-lid-fits-dxfc507-5-oz-cup-1000-cs-clear-dx55000174>: HTTP status code is not handled or not allowed 2026-01-25 05:12:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/outdoor-umbrella-w-tilt-mechanism-olefin-fabric-8-1-2-w-royal-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:12:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washer-jsm-6196195b-sm-61944>: HTTP status code is not handled or not allowed 2026-01-25 05:12:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waterhog-classic-diamond-mat-bordeaux-6-x-6>: HTTP status code is not handled or not allowed 2026-01-25 05:12:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/125-lf-2-12-union-npt-female>: HTTP status code is not handled or not allowed 2026-01-25 05:12:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/janitorial-shadow-board-blue-on-black-industrial-grade-aluminum-sb101al>: HTTP status code is not handled or not allowed 2026-01-25 05:12:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caring-woven-gauze-sponges-4-x-4-sterile-12-ply-25-sponges-box>: HTTP status code is not handled or not allowed 2026-01-25 05:12:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/asg18x18x6nk-pull-box-screw-cover-1800x1800x600-galvanized>: HTTP status code is not handled or not allowed 2026-01-25 05:12:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nl34-stanzoil-neoprene-gloves-12-l-medium-weight-1-pair-size-6-334946>: HTTP status code is not handled or not allowed 2026-01-25 05:12:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dx-power-workstation-hutch-roller-cabinet-41w-x-25dorange-wblack-handles-trim>: HTTP status code is not handled or not allowed 2026-01-25 05:12:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/machinery_parts_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:12:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-ss-316-cast-hex-bushing-12-x-38-npt-male-x-female>: HTTP status code is not handled or not allowed 2026-01-25 05:12:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bar-rb-25-6290339>: HTTP status code is not handled or not allowed 2026-01-25 05:12:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ingersoll-rand-2235timax-2-1-2-impactool-ext-anvil>: HTTP status code is not handled or not allowed 2026-01-25 05:12:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cursor-hf2-112>: HTTP status code is not handled or not allowed 2026-01-25 05:12:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ahe10x10x6-pull-box-hinged-cover-1000x1000x600-steelgray>: HTTP status code is not handled or not allowed 2026-01-25 05:12:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-manifold-1-npt-inlet-x-34-npt-outlet-5-ports>: HTTP status code is not handled or not allowed 2026-01-25 05:12:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e6pba-push-button-encl-sloped-front-6pb-x-305mm>: HTTP status code is not handled or not allowed 2026-01-25 05:12:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/screw-tap-ds-6295641>: HTTP status code is not handled or not allowed 2026-01-25 05:12:50 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/machinery_parts_accessories landed on page that is not a product page. 2026-01-25 05:12:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speedi-boot-90-175-register-vent-boot-w-adj-hangers-for-hvac-duct-work-10-x-10-x-10>: HTTP status code is not handled or not allowed 2026-01-25 05:12:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-plastic-garbage-can-with-lid-dolly-55-gal-green>: HTTP status code is not handled or not allowed 2026-01-25 05:12:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/american-pride-writing-pads-white-5-x-8-50-sheets-pad-12-pads-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:12:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-unit-5-tier-48w-x-24d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 05:12:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a606012lpg-2-door-wfloor-stands-type-12-encl-6006x6006x1206-steelgray>: HTTP status code is not handled or not allowed 2026-01-25 05:12:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speedi-grille-hands-free-ceiling-or-wall-vent-grille-vent-register-cover-w-4-way-reflexctor-14-x-14>: HTTP status code is not handled or not allowed 2026-01-25 05:12:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a14n128-small-type-1-enclosure-1400x1200x800-steelgray>: HTTP status code is not handled or not allowed 2026-01-25 05:12:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/notebooks_pads already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:12:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/warning-label-54a-6296150>: HTTP status code is not handled or not allowed 2026-01-25 05:12:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scotch-r-general-purpose-masking-tape-234-natural-48mm-x-55m-24-per-case>: HTTP status code is not handled or not allowed 2026-01-25 05:13:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bad-boy-chrome>: HTTP status code is not handled or not allowed 2026-01-25 05:13:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-5-tier-60w-x-18d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 05:13:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a806nf-j-box-clamp-cover-type-4-800x600x350-steelgray>: HTTP status code is not handled or not allowed 2026-01-25 05:13:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-12-cross-fitting>: HTTP status code is not handled or not allowed 2026-01-25 05:13:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wshpl-padlocking-handle-kit>: HTTP status code is not handled or not allowed 2026-01-25 05:13:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ce1000-1-75hp-2-stage-30-gallon-vertical-portable-air-compressor>: HTTP status code is not handled or not allowed 2026-01-25 05:13:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/paper_stationery/notebooks_pads>: HTTP status code is not handled or not allowed 2026-01-25 05:13:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a1008chs-continuous-hinge-wclamps-type-12-1150x894-in-steelgray>: HTTP status code is not handled or not allowed 2026-01-25 05:13:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-s2172b-black-epoxy-wire-shelf-72-w-x-21-d-with-clips>: HTTP status code is not handled or not allowed 2026-01-25 05:13:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 68219 pages (at 95 pages/min), scraped 34129 items (at 44 items/min) 2026-01-25 05:13:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s-lp-10-thermaflex-flexible-hvac-duct-5-diameter>: HTTP status code is not handled or not allowed 2026-01-25 05:13:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pivot-shaft-jms12scms-18>: HTTP status code is not handled or not allowed 2026-01-25 05:13:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26x12x72-solar-star-greenhouse-w-solid-polycarbonate-gas-heater>: HTTP status code is not handled or not allowed 2026-01-25 05:13:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/power_tools/power_saws_blades/band_saws/band_saw_machines already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:13:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/laptop-wall-desk-combination-lock-beige>: HTTP status code is not handled or not allowed 2026-01-25 05:13:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-green-wire-shelving-add-on-unit-5-tier-48w-x-24d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 05:13:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-green-wire-shelving-add-on-unit-5-tier-60w-x-24d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 05:13:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60440106627-utility-cloth-sheet-314d-9w-x-11l-p180-grit>: HTTP status code is not handled or not allowed 2026-01-25 05:13:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/small-animal-hut-81w-x-62h-x-16l>: HTTP status code is not handled or not allowed 2026-01-25 05:13:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spectrum-ellipse-slim-double-recycling-station-for-cans-bottles-16-gal-capblueblk>: HTTP status code is not handled or not allowed 2026-01-25 05:13:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classical-b-v-belt-b43>: HTTP status code is not handled or not allowed 2026-01-25 05:13:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26x12x72-solar-star-greenhouse-w-poly-top-and-ends-roll-up-sides-gas-heater>: HTTP status code is not handled or not allowed 2026-01-25 05:13:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/power_tools/power_saws_blades/band_saws/band_saw_machines landed on page that is not a product page. 2026-01-25 05:13:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:13:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shaft-drive-pf3-48-6288948>: HTTP status code is not handled or not allowed 2026-01-25 05:13:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/krylon-kamar-varnish-tint-base>: HTTP status code is not handled or not allowed 2026-01-25 05:13:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spectrum-quad-recycling-station-for-organics-paper-92-gallon-cap-whitegrayblack>: HTTP status code is not handled or not allowed 2026-01-25 05:13:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-15hp-1ph-115230v-31a-81>: HTTP status code is not handled or not allowed 2026-01-25 05:13:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roller-208-planer-6292674>: HTTP status code is not handled or not allowed 2026-01-25 05:13:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/majestic-greenhouse-28w-x-24l-drop-down-w-film>: HTTP status code is not handled or not allowed 2026-01-25 05:13:30 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:13:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/armorultra-job-on-pallet-2-layer-epoxy-system-1000-sq-ft-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:13:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-25 05:13:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bpo200-basic-oil-only-pads15w-x-17llight-weightbale200>: HTTP status code is not handled or not allowed 2026-01-25 05:13:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-dry-hanger-chrome>: HTTP status code is not handled or not allowed 2026-01-25 05:13:35 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/foam/foam_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:13:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/worm-20-planer-mdl-208-6292676>: HTTP status code is not handled or not allowed 2026-01-25 05:13:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fabric-uphostered-restaurant-chair-with-ladder-back-blue-1>: HTTP status code is not handled or not allowed 2026-01-25 05:13:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/peel-seal-glamour-mailers-9-12w-x-12-34l-translucent-silver-250pack>: HTTP status code is not handled or not allowed 2026-01-25 05:13:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-25 05:13:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/86522-danger-hands-off-do-not-operate-tag-2-sided-10pkg-polyester-3w-x-5-34h>: HTTP status code is not handled or not allowed 2026-01-25 05:13:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-1-2-x-4-1-2-x-48-half-slotted-container-outer>: HTTP status code is not handled or not allowed 2026-01-25 05:13:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labels-with-small-parts-enclosed-print-3l-x-2w-fluorescent-green-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-25 05:13:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/foam/foam_sheets>: HTTP status code is not handled or not allowed 2026-01-25 05:13:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cast-acrylic-bar-48-l-x-4-w-x-1-8-thick-red>: HTTP status code is not handled or not allowed 2026-01-25 05:13:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bushing-20-planer-mdl-208-6292809>: HTTP status code is not handled or not allowed 2026-01-25 05:13:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-x-12-x-12-d-w-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-25 05:13:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/45a-3-motor-220v60hz3ph-6288988>: HTTP status code is not handled or not allowed 2026-01-25 05:13:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-silicone-strip-36-l-x-7-w-x-1-8-thick-blue-detectable-60a-high-temp-adhesive-bulk-rs-s60md-83>: HTTP status code is not handled or not allowed 2026-01-25 05:13:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/indicator-pja-c41-15-planer-6284750>: HTTP status code is not handled or not allowed 2026-01-25 05:13:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-x-14-x-7-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-25 05:13:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/comfort-curve-bus-box-20-15-7grey>: HTTP status code is not handled or not allowed 2026-01-25 05:13:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-duralast-wheel-3-14x2-34-roller>: HTTP status code is not handled or not allowed 2026-01-25 05:13:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-x-16-x-24-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-25 05:13:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-z-brite-wire-shelving-starter-unit-1-dunnage-shelf-48w-x-24d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 05:13:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-x-16-x-16-doublewall-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-25 05:13:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700505yel-pegboard-gift-card-display-8w-x-20h-yl-1-pc>: HTTP status code is not handled or not allowed 2026-01-25 05:13:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:13:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-poly-tech-wheel-5x2-34-roller>: HTTP status code is not handled or not allowed 2026-01-25 05:14:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-versa-tech-wheel-5x1-38-12-ball>: HTTP status code is not handled or not allowed 2026-01-25 05:14:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/51345-lockout-padlock-keyed-differently-1-12-plasticsteel-green-6pack>: HTTP status code is not handled or not allowed 2026-01-25 05:14:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brake-fixed-base-jds-12bpds-12cs-612118>: HTTP status code is not handled or not allowed 2026-01-25 05:14:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700356blk-pegboard-powerwing-display-1675w-x-60h-bk-1-pc>: HTTP status code is not handled or not allowed 2026-01-25 05:14:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22-x-17-x-12-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-25 05:14:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-plastex-wheel-5x1-12-1-316-plain>: HTTP status code is not handled or not allowed 2026-01-25 05:14:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10a-2-bushing-29d-x-23d-x-93-6289135>: HTTP status code is not handled or not allowed 2026-01-25 05:14:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-seal-bubble-mailers-5-10-12w-x-16l-kraft-70pack>: HTTP status code is not handled or not allowed 2026-01-25 05:14:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 68312 pages (at 93 pages/min), scraped 34170 items (at 41 items/min) 2026-01-25 05:14:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 05:14:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:14:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-plastex-wheel-3-14-x-2-34-roller>: HTTP status code is not handled or not allowed 2026-01-25 05:14:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simonize-digest-enzyme-producing-bacterial-liquid-gallon>: HTTP status code is not handled or not allowed 2026-01-25 05:14:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:14:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-x-18-x-16-14-12-10-multi-depth-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-25 05:14:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700460org-pinwheel-pegboard-display-wrb-135w-x-60h-rg-1-pc>: HTTP status code is not handled or not allowed 2026-01-25 05:14:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700357blk-pegboard-powerwing-display-16w-x-2025h-bk-1-pc>: HTTP status code is not handled or not allowed 2026-01-25 05:14:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/193732-triple-sided-sign-hlr-wrb-85-x-11-clr-1pack>: HTTP status code is not handled or not allowed 2026-01-25 05:14:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-foam-epdm-sheet-12-l-x-12-w-x-1-2-thick-black>: HTTP status code is not handled or not allowed 2026-01-25 05:14:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-dia-round-number-paper-labels-with-15-print-fluorescent-orange-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-25 05:14:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_tube_stock landed on page that is not a product page. 2026-01-25 05:14:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-48-inch-led-aluminum-shop-light-35-watts-4000k-3850-lumens-48-inch-adjustable-height-6-foot-cord>: HTTP status code is not handled or not allowed 2026-01-25 05:14:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stop-quick-re511-asa-3761160>: HTTP status code is not handled or not allowed 2026-01-25 05:14:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labels-with-fragile-print-5l-x-3w-redwhite-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-25 05:14:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-heavy-duty-quick-disconnect-warning-whip-w-o-light-12-x11-orange-rectangle-flag>: HTTP status code is not handled or not allowed 2026-01-25 05:14:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700024-curve-hook-for-gridwall-05w-x-225h-metal-20pack>: HTTP status code is not handled or not allowed 2026-01-25 05:14:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-wall-mounted-shelf-15-inch-60-inch-shelf>: HTTP status code is not handled or not allowed 2026-01-25 05:14:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/600086-cchannel-6h-clr-5-pc>: HTTP status code is not handled or not allowed 2026-01-25 05:14:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-x-2-1-2-clevis-pin-low-carbon-steel-zinc-clear-clpz-1000-2500>: HTTP status code is not handled or not allowed 2026-01-25 05:14:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:14:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-fire-hydrant-flange-mount-w-bright-zinc-finish>: HTTP status code is not handled or not allowed 2026-01-25 05:14:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labels-with-priority-print-5l-x-3w-fluorescent-red-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-25 05:14:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-magnetic-retractable-belt-barrier-black-yellow-mount-black-yellow-15-ft-l>: HTTP status code is not handled or not allowed 2026-01-25 05:14:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/112736-vert-l-shaped-acrylic-sign-hlr-3w-x-4h-10pack>: HTTP status code is not handled or not allowed 2026-01-25 05:14:32 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-25 05:14:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powerback-rubber-duct-5-ft-heavy-duty-single-1-14-ch>: HTTP status code is not handled or not allowed 2026-01-25 05:14:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plug-blade-retainer-3598046>: HTTP status code is not handled or not allowed 2026-01-25 05:14:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700421yel-pinwheel-pegboard-display-wwheels-16w-x-60h-yl-1-pc>: HTTP status code is not handled or not allowed 2026-01-25 05:14:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-32-x-1-2-cotter-pin-316-stainless-steel-asme-18-8-1-made-in-usa>: HTTP status code is not handled or not allowed 2026-01-25 05:14:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/rubber/rubber_sheets>: HTTP status code is not handled or not allowed 2026-01-25 05:14:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reusable-sphyg-cuff-one-tube-latex-free-burgundy>: HTTP status code is not handled or not allowed 2026-01-25 05:14:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/88in-w-x44in-d-reception-station-with-non-electric-raceway-gray-counter-blue-panel>: HTTP status code is not handled or not allowed 2026-01-25 05:14:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/work-height-platform-truck-rnh-3048-6py-ah-with-hardboard-top-30-x-48-adj-height>: HTTP status code is not handled or not allowed 2026-01-25 05:14:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/124in-w-x-44in-d-reception-station-with-electric-raceway-gray-counter-gray-panel>: HTTP status code is not handled or not allowed 2026-01-25 05:14:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nhl-new-york-rangers-grill-mat-14243>: HTTP status code is not handled or not allowed 2026-01-25 05:14:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kansas-hitch-cover-4-1-2-x-3-3-8-15094>: HTTP status code is not handled or not allowed 2026-01-25 05:14:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fs7-4sw-flow-switch-1-1-4-npt>: HTTP status code is not handled or not allowed 2026-01-25 05:14:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_angle_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:14:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-20-gallon-perforated-steel-receptacle-w-flat-lid-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:14:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-foam-epdm-sheet-36-l-x-36-w-x-1-thick-black-acrylic-adhesive>: HTTP status code is not handled or not allowed 2026-01-25 05:14:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/texas-am-hitch-cover-4-1-2-x-3-3-8-15103>: HTTP status code is not handled or not allowed 2026-01-25 05:14:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/415c-strainer-1-1-4-in-npt>: HTTP status code is not handled or not allowed 2026-01-25 05:14:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/00-5x10-kra-padded-mailer>: HTTP status code is not handled or not allowed 2026-01-25 05:14:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tapcon-1-4x4-concrete-anchor-phillips-25-pk>: HTTP status code is not handled or not allowed 2026-01-25 05:14:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nhl-phoenix-coyotes-grill-mat-14246>: HTTP status code is not handled or not allowed 2026-01-25 05:14:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pneumatic-bit-8-cold-chisel>: HTTP status code is not handled or not allowed 2026-01-25 05:14:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/taylor-percussion-hammer-with-256-cps-tuning-fork>: HTTP status code is not handled or not allowed 2026-01-25 05:14:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nba-miami-heat-grill-mat-14210>: HTTP status code is not handled or not allowed 2026-01-25 05:14:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/floor_mats/grill_mats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:14:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/daddy-long-legs-gable-end-12w-tan>: HTTP status code is not handled or not allowed 2026-01-25 05:14:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_angle_stock>: HTTP status code is not handled or not allowed 2026-01-25 05:14:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-48-l-x-2-1-2-w-x-1-thick-black-antistatic>: HTTP status code is not handled or not allowed 2026-01-25 05:14:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-electric-office-partition-panel-with-partial-window-48-1-4w-x-101-1-2h-gray-gl>: HTTP status code is not handled or not allowed 2026-01-25 05:14:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-9-1-2x14-1-2-self-seal-white-padded-mailers>: HTTP status code is not handled or not allowed 2026-01-25 05:14:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mext584vp-exit-safety-sign-10w-x-7h-plastic>: HTTP status code is not handled or not allowed 2026-01-25 05:14:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-socket-1-2-drive-6-point-1-1-2-short>: HTTP status code is not handled or not allowed 2026-01-25 05:14:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-hand-riveter-w-4-nosepieces>: HTTP status code is not handled or not allowed 2026-01-25 05:15:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-socket-3-8-drive-6-point-9-16-deep>: HTTP status code is not handled or not allowed 2026-01-25 05:15:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/17-x-21-self-seal-white-stayflat-mailers>: HTTP status code is not handled or not allowed 2026-01-25 05:15:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mgsh904va-safety-first-sign-report-all-accidents-14w-x-10h-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 05:15:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/floor_mats/grill_mats>: HTTP status code is not handled or not allowed 2026-01-25 05:15:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/13-x-18-self-seal-white-stayflat-mailers>: HTTP status code is not handled or not allowed 2026-01-25 05:15:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-profile-connectable-spill-berm-5-l-x-4-w-x-4-1-2-h>: HTTP status code is not handled or not allowed 2026-01-25 05:15:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-sheet-48-l-x-48-w-x-1-8-thick-black-antistatic>: HTTP status code is not handled or not allowed 2026-01-25 05:15:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-x-20-open-end-static-shielding-bags>: HTTP status code is not handled or not allowed 2026-01-25 05:15:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-door-ovation-letter-board-burgundy-w-gray-frame-24-1-8w-x-33-3-4h>: HTTP status code is not handled or not allowed 2026-01-25 05:15:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-36-l-x-2-1-2-w-x-3-16-thick-black-antistatic>: HTTP status code is not handled or not allowed 2026-01-25 05:15:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:15:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 68404 pages (at 92 pages/min), scraped 34198 items (at 28 items/min) 2026-01-25 05:15:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-socket-3-8-drive-12-point-11-16-deep>: HTTP status code is not handled or not allowed 2026-01-25 05:15:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/colored-thermal-transfer-labels-4w-x-6l-3-core-orange-4pack>: HTTP status code is not handled or not allowed 2026-01-25 05:15:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ksk115-lockouttagout-pouch-kit-nylon>: HTTP status code is not handled or not allowed 2026-01-25 05:15:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnetic-wall-mount-barrier-15-yellow-black-diagonal-stripe-banner>: HTTP status code is not handled or not allowed 2026-01-25 05:15:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/laminate-center-drawer-22w-mahogany-finish>: HTTP status code is not handled or not allowed 2026-01-25 05:15:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/impact-socket-set-1-2-drive-6-point-deep-12-piece>: HTTP status code is not handled or not allowed 2026-01-25 05:15:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/07157-30834-torx-t8-x-2-bit>: HTTP status code is not handled or not allowed 2026-01-25 05:15:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mchl174vs-danger-sign-acetylene-14w-x-10h-adhesive-vinyl>: HTTP status code is not handled or not allowed 2026-01-25 05:15:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-36-l-x-1-w-x-1-thick-black-antistatic>: HTTP status code is not handled or not allowed 2026-01-25 05:15:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-9-in-10-1-2-in-12-in-jbt-r2000tbk>: HTTP status code is not handled or not allowed 2026-01-25 05:15:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/desks_workstations/office_computer_desks>: HTTP status code is not handled or not allowed 2026-01-25 05:15:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-high-performance-lubricant-w-ptfe-11-oz-aerosol-petroleum-solvent>: HTTP status code is not handled or not allowed 2026-01-25 05:15:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/battery-powered-portable-pulling-and-lifting-tool-24v>: HTTP status code is not handled or not allowed 2026-01-25 05:15:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:15:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mvhr830vs-notice-sign-wheels-must-be-chocked-before-loading-10w-x-7h-adhesive-vinyl>: HTTP status code is not handled or not allowed 2026-01-25 05:15:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rectangular-inventory-paper-labels-4l-x-4w-black-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-25 05:15:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-t-general-clamping-tube-pipe-hydraulic-hose>: HTTP status code is not handled or not allowed 2026-01-25 05:15:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-gallon-jumbo-stacking-flammable-cabinet-self-closing-30w-x-18-1-2d-x-32-1-2h>: HTTP status code is not handled or not allowed 2026-01-25 05:15:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sr-57-shop-desk-pigeonhole-riser-drawer-and-shelf-34-1-2-w-x-29-d-x-53-h-medium-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:15:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f6-woven-split-wrappable-sleeve-1-dia-50-black>: HTTP status code is not handled or not allowed 2026-01-25 05:15:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-16-size-col-series-cushion-clamps>: HTTP status code is not handled or not allowed 2026-01-25 05:15:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/msmk243vp-danger-sign-flammable-material-no-smoking-or-open-flames-14w-x-10h-plastic>: HTTP status code is not handled or not allowed 2026-01-25 05:15:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mase807vs-notice-sign-this-area-is-under-24-hour-video-14w-x-10h-adhesive-vinyl>: HTTP status code is not handled or not allowed 2026-01-25 05:15:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions>: HTTP status code is not handled or not allowed 2026-01-25 05:15:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-tube-od-stainless-steel-anti-vibe-cush-a-clamp>: HTTP status code is not handled or not allowed 2026-01-25 05:15:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mcsp140va-danger-sign-confined-space-authorized-personnel-only-10w-x-7h-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 05:15:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ratchet-type-banding-tool-for-stainless-steel-strapping>: HTTP status code is not handled or not allowed 2026-01-25 05:15:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-wire-security-truck-36-x-18-x-69-1200-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 05:15:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-tube-od-zinc-plated-anti-vibration-cush-a-clamp>: HTTP status code is not handled or not allowed 2026-01-25 05:15:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-rack-guard-back-panel-with-offset-bracket-96l-x-1-14w-x-36h>: HTTP status code is not handled or not allowed 2026-01-25 05:15:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cpvc-bar-24-l-x-4-w-x-1-16-thick-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:15:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-sheet-96-l-x-48-w-x-3-16-thick-clear-antistatic>: HTTP status code is not handled or not allowed 2026-01-25 05:15:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7448-pro-hand-pad-6-x-9-in-20-padsbox-3-boxescase>: HTTP status code is not handled or not allowed 2026-01-25 05:15:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-wr-84v-cnc-router-table-with-t-slots-7-12-hp-1ph-220v>: HTTP status code is not handled or not allowed 2026-01-25 05:15:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mcsp130vp-danger-sign-confined-space-enter-by-permit-only-10w-x-7h-plastic>: HTTP status code is not handled or not allowed 2026-01-25 05:15:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/legacy-racetrack-conference-table-with-charging-port-144l-x-52w-x-29h-noble-oak>: HTTP status code is not handled or not allowed 2026-01-25 05:15:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2flexo-pet-general-purpose-sleeving-dia-500-white>: HTTP status code is not handled or not allowed 2026-01-25 05:15:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-pvc-type-1-sheet-6-l-x-6-w-x-3-16-thick-clear>: HTTP status code is not handled or not allowed 2026-01-25 05:15:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mdt185ctp-danger-tag-pf-cardstock-25pack>: HTTP status code is not handled or not allowed 2026-01-25 05:15:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/abrasives/sanding_discs/psa_sanding_discs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:15:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/madc801vp-notice-sign-authorized-personnel-only-14w-x-10h-plastic>: HTTP status code is not handled or not allowed 2026-01-25 05:15:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12x30-4-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 05:15:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_drawers_keyboard_trays already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:15:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-socket-1-4-drive-10mm-deep>: HTTP status code is not handled or not allowed 2026-01-25 05:15:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-socket-1-4-drive-14mm-deep>: HTTP status code is not handled or not allowed 2026-01-25 05:15:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/male-single-swivel-socket-member-2>: HTTP status code is not handled or not allowed 2026-01-25 05:15:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/impact-socket-3-4-drive-1-7-16-6-point-deep>: HTTP status code is not handled or not allowed 2026-01-25 05:15:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trs211ptp-fire-extinguisher-recharge-record-tag-pf-cardstock-25pack>: HTTP status code is not handled or not allowed 2026-01-25 05:15:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-sheet-72-l-x-48-w-x-3-32-thick-clear>: HTTP status code is not handled or not allowed 2026-01-25 05:15:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durawrap-1050-ballistic-nylon-wrap-w-velcro-closure-4-dia-25-black>: HTTP status code is not handled or not allowed 2026-01-25 05:15:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:15:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-1-2-x-10-7-8-x-7-lewisbins-divider-box-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:15:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cortech-usa-86484p-5-chair-razorback-plum-5-pack-5-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:15:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/abrasives/sanding_discs/psa_sanding_discs landed on page that is not a product page. 2026-01-25 05:15:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-2flexo-pet-general-purpose-sleeving-dia-200-red>: HTTP status code is not handled or not allowed 2026-01-25 05:15:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_drawers_keyboard_trays>: HTTP status code is not handled or not allowed 2026-01-25 05:15:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-rectangular-tube-72-l-x-3-4-w-clear>: HTTP status code is not handled or not allowed 2026-01-25 05:15:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kdd160-lockout-120240-circuit-breaker-lockout-single-pole>: HTTP status code is not handled or not allowed 2026-01-25 05:15:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-x-12-open-end-static-shielding-bags>: HTTP status code is not handled or not allowed 2026-01-25 05:15:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/detect-smoke-alarm-aa-battery-powered>: HTTP status code is not handled or not allowed 2026-01-25 05:15:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-1-2flexo-pet-general-purpose-sleeving-dia-200-green>: HTTP status code is not handled or not allowed 2026-01-25 05:16:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-nylon-6-6-bar-12-l-x-6-w-x-3-8-thick-black>: HTTP status code is not handled or not allowed 2026-01-25 05:16:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-25 05:16:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bronco-waste-container-55-gal-yellow>: HTTP status code is not handled or not allowed 2026-01-25 05:16:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-head-grout-line-brush-nylon-bristle-7-1-2-black>: HTTP status code is not handled or not allowed 2026-01-25 05:16:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machined-type-needle-roller-bearing-metric-taf81516-8mm-bore-15mm-od>: HTTP status code is not handled or not allowed 2026-01-25 05:16:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g22-150-ap-30hp-oil-injected-rotary-screw-132-gal-tank-mnt-150psi-3ph-208-230-460v>: HTTP status code is not handled or not allowed 2026-01-25 05:16:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-led-radar-feedback-trailer-sign-solar-powered-55w-x-106d-x-97h-blackwhite>: HTTP status code is not handled or not allowed 2026-01-25 05:16:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9x18-1-5-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 05:16:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:16:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mchl170vp-danger-sign-oxygen-14w-x-10h-plastic>: HTTP status code is not handled or not allowed 2026-01-25 05:16:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/efp-b-passive-welding-helmet-10-fixed-shade-clear-lens-captain-america-graphic>: HTTP status code is not handled or not allowed 2026-01-25 05:16:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machined-type-needle-roller-bearing-metric-separable-cage-30mm-bore-40mm-od>: HTTP status code is not handled or not allowed 2026-01-25 05:16:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/efp-gen-x-auto-darkening-welding-helmet-10-fixed-shade-doa-graphic>: HTTP status code is not handled or not allowed 2026-01-25 05:16:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:16:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/duraflex-woven-high-abrasion-sleeve-175-dia-50-black>: HTTP status code is not handled or not allowed 2026-01-25 05:16:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 68496 pages (at 92 pages/min), scraped 34221 items (at 23 items/min) 2026-01-25 05:16:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cpvc-bar-12-l-x-2-w-x-1-thick-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:16:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mvhr626va-caution-sign-driver-must-chock-wheels-14w-x-10h-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 05:16:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hurricane-360-hoist-and-trolley-combo-geared-10-ton-capacity-20-lift>: HTTP status code is not handled or not allowed 2026-01-25 05:16:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 05:16:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hurricane-360-hoist-and-trolley-combo-geared-20-ton-capacity-10-lift>: HTTP status code is not handled or not allowed 2026-01-25 05:16:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mstf645vs-caution-sign-watch-your-step-10w-x-7h-adhesive-vinyl>: HTTP status code is not handled or not allowed 2026-01-25 05:16:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/upsable-bubble-wrap-roll-48-x-300-x-316-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:16:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 05:16:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/orange-document-envelopes-612-x-5-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:16:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-sheet-12-l-x-12-w-x-3-16-thick-clear-scratch-and-uv-resistant>: HTTP status code is not handled or not allowed 2026-01-25 05:16:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hurricane-360-hoist-and-trolley-combo-standard-1-ton-capacity-20-lift>: HTTP status code is not handled or not allowed 2026-01-25 05:16:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hepa-cartridge-filter-for-strike153-upright--zephyr153-backpack-vacuum>: HTTP status code is not handled or not allowed 2026-01-25 05:16:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/omnimed-stainless-twin-narcotic-cabinet-double-door-4-adjustable-shelves-22w-x-8d-x-15h>: HTTP status code is not handled or not allowed 2026-01-25 05:16:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5x12-4-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 05:16:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6x10-4-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 05:16:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n407-replacement-needles-14-piece-4mm-x-7-needles>: HTTP status code is not handled or not allowed 2026-01-25 05:16:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4x12-4-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 05:16:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-sheet-48-l-x-24-w-x-1-8-thick-clear-scratch-and-uv-resistant>: HTTP status code is not handled or not allowed 2026-01-25 05:16:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8mm-e-z-crimper-standard-seal>: HTTP status code is not handled or not allowed 2026-01-25 05:16:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cpvc-bar-12-l-x-1-w-x-1-4-thick-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:16:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12x20-3-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 05:16:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/saf-t-grip-6-board-system-cbg1520kc>: HTTP status code is not handled or not allowed 2026-01-25 05:16:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tuff-cut-cutting-boards-tc152012>: HTTP status code is not handled or not allowed 2026-01-25 05:16:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2x3-3-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 05:16:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1838-series-5pc-hss-multi-purpose-carbide-tipped-masonry-drill-set-1-8-5-32-3-16-1-4-5-16>: HTTP status code is not handled or not allowed 2026-01-25 05:16:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-emergency-light-dual-head-self-diagnostic-remote-capacity>: HTTP status code is not handled or not allowed 2026-01-25 05:16:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dax-carbon-fiber-full-brim-black-camo>: HTTP status code is not handled or not allowed 2026-01-25 05:16:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gun-safe-electronic-lock-1-hr-fire-rating-23x20x59-12-gun-cap-black>: HTTP status code is not handled or not allowed 2026-01-25 05:16:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20x20-2-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 05:16:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-handle-w-stud-3-8-16-x-1-57-stud-2-56-l-x-1-77-h-die-cast-handle-mh-305>: HTTP status code is not handled or not allowed 2026-01-25 05:16:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pull-type-foam-cup-dispenser-c4210pfsd>: HTTP status code is not handled or not allowed 2026-01-25 05:16:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:16:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5a-battery-charger-5-bank-on-board-battery-charger-waterproof-at-ip68-genius5>: HTTP status code is not handled or not allowed 2026-01-25 05:16:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guardair-blind-hole-gun-12-angled-tip>: HTTP status code is not handled or not allowed 2026-01-25 05:16:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caw-stationary-electric-pressure-washer-2000-psi-6-hp-39-gpm-3-phase-230v>: HTTP status code is not handled or not allowed 2026-01-25 05:16:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guardair-lazer-pistol-grip-18-alum-ext-alum-nozzle>: HTTP status code is not handled or not allowed 2026-01-25 05:16:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guardair-ultra-venturi-12-alum-ext-alloy-nozzle-long-trigger>: HTTP status code is not handled or not allowed 2026-01-25 05:16:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_film_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:16:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-hdpe-sheet-24-l-x-12-w-x-3-4-thick-white-marine>: HTTP status code is not handled or not allowed 2026-01-25 05:16:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vci-reclosable-poly-bags-4w-x-6l-4-mil-blue-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 05:16:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/entrance-mat-3-8-thick-3-x-10-gray-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:16:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/freestanding-3-panel-corner-room-divider-24-x-72-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:16:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 05:16:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/euro-ez-fit-c2210c18>: HTTP status code is not handled or not allowed 2026-01-25 05:16:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-grid-panel-48-x-18-poly-green>: HTTP status code is not handled or not allowed 2026-01-25 05:16:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vci-reclosable-poly-bags-8w-x-10l-4-mil-blue-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 05:16:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rapi-kool-premium-2-pack-rcu1282>: HTTP status code is not handled or not allowed 2026-01-25 05:16:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_film_stock>: HTTP status code is not handled or not allowed 2026-01-25 05:16:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guardair-ultra-xtra-thrust-48-alum-ext-alloy-nozzle-long-trigger>: HTTP status code is not handled or not allowed 2026-01-25 05:16:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sn-20016-fire-resistant-safety-netting-86-ft-x-150-ft-black-1-roll>: HTTP status code is not handled or not allowed 2026-01-25 05:16:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1200-1250-specific-gravity-astm-129h-hydrometer-for-heavy-liquids>: HTTP status code is not handled or not allowed 2026-01-25 05:16:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:16:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guardair-inforcer-48-alum-ext-venturi-nozzle>: HTTP status code is not handled or not allowed 2026-01-25 05:16:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-organizer-with-12-removable-bins>: HTTP status code is not handled or not allowed 2026-01-25 05:16:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wet-zone-shadow-board-yellow-black-68-x-30-aluminum-sb124al>: HTTP status code is not handled or not allowed 2026-01-25 05:16:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guardair-syphon-spray-gun-12-extension>: HTTP status code is not handled or not allowed 2026-01-25 05:16:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-rod-24-l-x-9-16-dia-white>: HTTP status code is not handled or not allowed 2026-01-25 05:17:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pilaster-to-wall-bracket-kit-steel-partition>: HTTP status code is not handled or not allowed 2026-01-25 05:17:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/046lb-weighted-beaker-flask-holder-with-vikem-vinyl-coating-for-100ml-beakers>: HTTP status code is not handled or not allowed 2026-01-25 05:17:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/electrical_power_testing/multimeters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:17:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dry-zone-shadow-board-green-red-68-x-30-acp-aluminum-composite-panel-sb134acp>: HTTP status code is not handled or not allowed 2026-01-25 05:17:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 05:17:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-sided-bulk-truck-1-1-2-retaining-lip-3600-lbs-mesh-sides-24-x-60-deck>: HTTP status code is not handled or not allowed 2026-01-25 05:17:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluke-tl71-precision-dmm-probe-assembly-cat-iii-300-v-3-a-rating-ul-listed>: HTTP status code is not handled or not allowed 2026-01-25 05:17:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/electrical_power_testing/multimeters already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:17:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-ptfe-tube-72-l-x-5-8-id-x-13-16-od-white>: HTTP status code is not handled or not allowed 2026-01-25 05:17:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-3-4-x-2-3-8-5-repairable-rework>: HTTP status code is not handled or not allowed 2026-01-25 05:17:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conway-diffusion-cell-83mm-od-3pk>: HTTP status code is not handled or not allowed 2026-01-25 05:17:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/koch-filter-dust-collector-cartridge-open-closed-with-bolt-hole-paper-base>: HTTP status code is not handled or not allowed 2026-01-25 05:17:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commercial-sensor-faucet-elkay-lkb721c>: HTTP status code is not handled or not allowed 2026-01-25 05:17:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-hdpe-bar-12-l-x-1-1-2-w-x-1-4-thick-white-marine>: HTTP status code is not handled or not allowed 2026-01-25 05:17:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 68589 pages (at 93 pages/min), scraped 34247 items (at 26 items/min) 2026-01-25 05:17:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-rod-24-l-x-1-1-2-dia-black>: HTTP status code is not handled or not allowed 2026-01-25 05:17:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-gallon-used-fluid-drain-black-polyethylene-3626>: HTTP status code is not handled or not allowed 2026-01-25 05:17:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/testing_inspection_instruments/electrical_power_testing/multimeters>: HTTP status code is not handled or not allowed 2026-01-25 05:17:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/autoclavable-polypropylene-carboy-without-spigot-20-liters-53-gallons>: HTTP status code is not handled or not allowed 2026-01-25 05:17:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-hdpe-bar-12-l-x-1-w-x-1-2-thick-white-marine>: HTTP status code is not handled or not allowed 2026-01-25 05:17:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/autoclavable-polypropylene-carboy-with-spigot-20-liters-53-gallons>: HTTP status code is not handled or not allowed 2026-01-25 05:17:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cloud-foam-roll-12w-x-18l-x-2-thick-white-1-roll>: HTTP status code is not handled or not allowed 2026-01-25 05:17:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/testing_inspection_instruments/electrical_power_testing/multimeters>: HTTP status code is not handled or not allowed 2026-01-25 05:17:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clamp-grease-gun-g160>: HTTP status code is not handled or not allowed 2026-01-25 05:17:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-174-chrome-wire-linen-cart-with-nylon-cover-4-shelves-72-l-x-18-w-x-80-h>: HTTP status code is not handled or not allowed 2026-01-25 05:17:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/calibrated-electronic-stainless-steel-stem-thermometer-50-300c-58-572f>: HTTP status code is not handled or not allowed 2026-01-25 05:17:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-275-containment-sump-360-gallon-capacity-with-no-drain>: HTTP status code is not handled or not allowed 2026-01-25 05:17:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lewisbins-conductive-hanging-stacking-part-bin-8-1316w-x-9-516d-x-5h>: HTTP status code is not handled or not allowed 2026-01-25 05:17:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-rod-36-l-x-2-dia-white>: HTTP status code is not handled or not allowed 2026-01-25 05:17:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-edge-2w-18l-2-ply-eye-eye-poly-web-sling-flat-eyes>: HTTP status code is not handled or not allowed 2026-01-25 05:17:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roundup-1-1-2w-18l-endless-poly-roundsling-red>: HTTP status code is not handled or not allowed 2026-01-25 05:17:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/orbis-heavy-duty-fixed-wall-bulkpak-48l-x-45w-x-19-516h-2000-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-25 05:17:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-1-4-x-2-5-8-6-light-green-shipping-tag>: HTTP status code is not handled or not allowed 2026-01-25 05:17:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-1-4-x-2-5-8-6-gray-shipping-tag>: HTTP status code is not handled or not allowed 2026-01-25 05:17:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/orbis-stakpak-modular-straight-wall-container-12l-x-7-1332w-x-5h-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:17:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yuasa-yix30l-bs-battery-30-amps-12v-b-terminals>: HTTP status code is not handled or not allowed 2026-01-25 05:17:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48h-single-sided-floor-stand-with-96-drawer-interlocking-storage-cabinet>: HTTP status code is not handled or not allowed 2026-01-25 05:17:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-rod-24-l-x-2-1-2-dia-black>: HTTP status code is not handled or not allowed 2026-01-25 05:17:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-rack-24x36x60-with-64-blue-drawers>: HTTP status code is not handled or not allowed 2026-01-25 05:17:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-60-x-36-adj-height-workbench-w-drawer-black-maple-square-edge-top>: HTTP status code is not handled or not allowed 2026-01-25 05:17:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h-b-easy-read-general-purpose-liquid-in-glass-thermometer-35-to-50c-total-immersion>: HTTP status code is not handled or not allowed 2026-01-25 05:17:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h-b-frio-temp-dual-zone-electronic-verification-thermometer-50-70c-58-158f-10-50c-14-122f>: HTTP status code is not handled or not allowed 2026-01-25 05:17:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-4-x-2-7-8-7-red-fluorescent-tag>: HTTP status code is not handled or not allowed 2026-01-25 05:17:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-hdpe-sheet-48-l-x-12-w-x-1-2-thick-white-marine>: HTTP status code is not handled or not allowed 2026-01-25 05:17:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/euro-drawer-8-3-8w-x-17-5-8d-x-4-5-8h-yellow>: HTTP status code is not handled or not allowed 2026-01-25 05:17:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/55-gallon-cover>: HTTP status code is not handled or not allowed 2026-01-25 05:17:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-1-4-x-3-1-8-8-pink-fluorescent-wired-tag>: HTTP status code is not handled or not allowed 2026-01-25 05:17:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h-b-durac-maximum-registering-autoclave-bi-metal-thermometer-20-to-150c-0-to-300f>: HTTP status code is not handled or not allowed 2026-01-25 05:17:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-drawer-10-3-4w-dividers>: HTTP status code is not handled or not allowed 2026-01-25 05:17:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kendall-howard-advanced-classroom-training-table-30-x-72-african-mahogany>: HTTP status code is not handled or not allowed 2026-01-25 05:17:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interlocking-storage-cabinet-11-3-4w-11-3-8d-x-11h-14-gray-drawers>: HTTP status code is not handled or not allowed 2026-01-25 05:17:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h-b-frio-temp-incubator-verification-thermometer-60-to-120f>: HTTP status code is not handled or not allowed 2026-01-25 05:17:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-alloy-latching-j-hook-style-a-2300-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 05:17:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h-b-enviro-safe-general-purpose-liquid-in-glass-thermometer-30-to-120f-total-immersion>: HTTP status code is not handled or not allowed 2026-01-25 05:17:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-174-galvanized-steel-linen-cart-with-nylon-cover-4-shelves-72-l-x-24-w-x-69-h>: HTTP status code is not handled or not allowed 2026-01-25 05:17:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interlocking-storage-cabinet-11-3-4w-11-3-8d-x-11h-16-blue-drawers>: HTTP status code is not handled or not allowed 2026-01-25 05:17:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:17:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ff7lbimed2-built-in-undercounter-auto-defrost-medical-all-refrigerator-5-5-cu-ft-capacity>: HTTP status code is not handled or not allowed 2026-01-25 05:17:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plus-astm-like-liquid-in-glass-thermometer-40c-solvents-distillation-72-to-126c>: HTTP status code is not handled or not allowed 2026-01-25 05:17:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tempco-channel-strip-240v-t3-23-34l-1000w>: HTTP status code is not handled or not allowed 2026-01-25 05:17:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leather-guest-sofa-black-hercules-diplomat-series>: HTTP status code is not handled or not allowed 2026-01-25 05:17:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pe-buchner-table-top-funnel-with-coarse-porosity-removable-plate-24-id-13-height>: HTTP status code is not handled or not allowed 2026-01-25 05:17:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paint-marker-oil-based-fine-green-ink>: HTTP status code is not handled or not allowed 2026-01-25 05:17:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paint-marker-oil-based-medium-silver-ink>: HTTP status code is not handled or not allowed 2026-01-25 05:17:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/individually-calibrated-9-21-degree-api-astm-2h-hydrometer>: HTTP status code is not handled or not allowed 2026-01-25 05:17:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:17:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers>: HTTP status code is not handled or not allowed 2026-01-25 05:17:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spruce-general-use-spray-paint-16-oz-gloss-white-12-cans-case-98-2>: HTTP status code is not handled or not allowed 2026-01-25 05:17:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-ptfe-tube-36-l-x-1-5-8-id-x-1-15-16-od-white>: HTTP status code is not handled or not allowed 2026-01-25 05:17:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simplehuman-174-wall-mount-grocery-bag-holder>: HTTP status code is not handled or not allowed 2026-01-25 05:17:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buyers-stainless-steel-underbody-box-w-t-handle-polished-gray-18x18x30>: HTTP status code is not handled or not allowed 2026-01-25 05:17:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/estate-4w-x-4h-oxford-gy-alum-composite-adjustable-single-fence-gate-kit-in-ground-only-ef-et1204>: HTTP status code is not handled or not allowed 2026-01-25 05:17:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/estate-10w-x-6h-king-cedar-alum-composite-adj-fence-double-gate-kit-in-ground-only-ef-et2216>: HTTP status code is not handled or not allowed 2026-01-25 05:17:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-wall-mount-wire-shelving-72w-x-24d-x-34h-2-shelf-starter-poly-green>: HTTP status code is not handled or not allowed 2026-01-25 05:18:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-tech-enamel-16-oz-safety-blue-6-cans-case-16-129>: HTTP status code is not handled or not allowed 2026-01-25 05:18:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnetic-bead-separation-rack-for-15-to-20ml-microcentrifuge-tubes>: HTTP status code is not handled or not allowed 2026-01-25 05:18:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lattice-top-6w-x-6h-oxford-grey-alum-composite-horizontal-fence-starter-section-in-ground-ef-lg1206>: HTTP status code is not handled or not allowed 2026-01-25 05:18:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 05:18:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:18:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lab-wall-freestanding-cabinet-35w-x-18d-x-84-1-4h-2-steel-doors-5-adj-shelves-model-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:18:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deep-well-horizontal-coolers-dw-frosty-brew-95w-dw94-b-29>: HTTP status code is not handled or not allowed 2026-01-25 05:18:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-swivel-caster-for-dandux-bulk-trucks-986834>: HTTP status code is not handled or not allowed 2026-01-25 05:18:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plus-calibrated-liquid-in-glass-thermometer-20-to-150c-total-immersion>: HTTP status code is not handled or not allowed 2026-01-25 05:18:06 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:18:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n55h15f50n-forest-lite-single-jacket-fire-hose-1-12-x-50-ft-300-psi-white>: HTTP status code is not handled or not allowed 2026-01-25 05:18:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plus-pfa-safety-coated-liquid-in-glass-thermometer-20-to-110c-total-immersion>: HTTP status code is not handled or not allowed 2026-01-25 05:18:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/sockets_bits/masonry_hammer_drill_bits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:18:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 68682 pages (at 93 pages/min), scraped 34274 items (at 27 items/min) 2026-01-25 05:18:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-wire-shelf-stem-caster-truck-black-ep-18-w-x-60-l-x-69-h>: HTTP status code is not handled or not allowed 2026-01-25 05:18:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyethylene-150ml-single-piece-buchner-funnel>: HTTP status code is not handled or not allowed 2026-01-25 05:18:12 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers landed on page that is not a product page. 2026-01-25 05:18:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/itm-1-8-ship-auger-pouched>: HTTP status code is not handled or not allowed 2026-01-25 05:18:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/32d-undercounter-refrigerator-freezer-food-prep-119w-ucr119>: HTTP status code is not handled or not allowed 2026-01-25 05:18:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-48-l-x-2-w-x-1-4-thick-white>: HTTP status code is not handled or not allowed 2026-01-25 05:18:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-209ml-standard-stem-funnels-4pk>: HTTP status code is not handled or not allowed 2026-01-25 05:18:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:18:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers>: HTTP status code is not handled or not allowed 2026-01-25 05:18:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-26ml-micro-funnels-12pk>: HTTP status code is not handled or not allowed 2026-01-25 05:18:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:18:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kendall-howard-48-4post-lan-rack>: HTTP status code is not handled or not allowed 2026-01-25 05:18:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/sockets_bits/masonry_hammer_drill_bits>: HTTP status code is not handled or not allowed 2026-01-25 05:18:19 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:18:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plus-pocket-liquid-in-glass-thermometer-10-to-110c-0-to-220f-window-plastic-case>: HTTP status code is not handled or not allowed 2026-01-25 05:18:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plus-blood-bank-verification-thermometer-5-20c>: HTTP status code is not handled or not allowed 2026-01-25 05:18:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reach-in-refrigerator-horizon-solid-half-solid-doors-52w-hrps2-1s>: HTTP status code is not handled or not allowed 2026-01-25 05:18:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-solid-plastic-shelving-unit-nexelon-posts-18-w-x-36-l-x-63-h>: HTTP status code is not handled or not allowed 2026-01-25 05:18:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/special_purpose_carts_trucks/medical_supply_carts/medical_storage_carts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:18:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions landed on page that is not a product page. 2026-01-25 05:18:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-wall-mount-wire-shelving-54w-x-24d-x-34h-2-shelf-starter-poly-green>: HTTP status code is not handled or not allowed 2026-01-25 05:18:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-wire-shelf-stem-caster-truck-2-with-brakes-nexelon-24-w-x-48-l-x-69-h>: HTTP status code is not handled or not allowed 2026-01-25 05:18:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:18:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock landed on page that is not a product page. 2026-01-25 05:18:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-wall-mount-wire-shelving-36w-x-14d-x-14h-1-shelf-starter-poly-green>: HTTP status code is not handled or not allowed 2026-01-25 05:18:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kendall-howard-96-performance-work-bench-no-upper-shelving>: HTTP status code is not handled or not allowed 2026-01-25 05:18:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:18:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reach-in-refrigerator-horizon-solid-half-solid-doors-35w-hr1w-1s>: HTTP status code is not handled or not allowed 2026-01-25 05:18:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-25 05:18:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-wall-mount-wire-shelving-30w-x-14d-x-63h-4-shelf-starter-poly-green>: HTTP status code is not handled or not allowed 2026-01-25 05:18:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-weather-aed-surface-mount-storage-labeling-kit-alarmed>: HTTP status code is not handled or not allowed 2026-01-25 05:18:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-acetal-bar-12-l-x-4-w-x-1-2-thick-off-white-antistatic>: HTTP status code is not handled or not allowed 2026-01-25 05:18:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-24-maple-square-edge-double-4-drawer>: HTTP status code is not handled or not allowed 2026-01-25 05:18:34 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/special_purpose_carts_trucks/medical_supply_carts/medical_storage_carts landed on page that is not a product page. 2026-01-25 05:18:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cast-aluminum-propel-dolly-non-tilt-6-roller-9000-cap-lbs>: HTTP status code is not handled or not allowed 2026-01-25 05:18:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 05:18:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-ptfe-bar-24-l-x-3-w-x-3-8-thick-black>: HTTP status code is not handled or not allowed 2026-01-25 05:18:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers>: HTTP status code is not handled or not allowed 2026-01-25 05:18:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heartsine-samaritan-aed-package-with-prescription-mounting-cabinet>: HTTP status code is not handled or not allowed 2026-01-25 05:18:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mylar-emergency-response-blanket-25pack>: HTTP status code is not handled or not allowed 2026-01-25 05:18:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-sheet-12-l-x-12-w-x-1-2-thick-black-bulk-ps-acb-1827>: HTTP status code is not handled or not allowed 2026-01-25 05:18:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bumpon-resiliant-rollstock-sj5832-4-1-2-x-72-yds-black-1-32-thick-70006459799>: HTTP status code is not handled or not allowed 2026-01-25 05:18:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commercial-surface-mounted-ceiling-heater-480v-3000w-6826-btu>: HTTP status code is not handled or not allowed 2026-01-25 05:18:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-labeled-2-color-acetone-wide-mouth-wash-bottles-500ml-16oz-pe-w-red-pp-cap-6pk>: HTTP status code is not handled or not allowed 2026-01-25 05:18:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thermostat-heat-only-line-voltage-double-pole-with-off-position>: HTTP status code is not handled or not allowed 2026-01-25 05:18:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-hay-rack-corner-stall-feeder-enclosed-back-w-rounded-edges-21l-x24h>: HTTP status code is not handled or not allowed 2026-01-25 05:18:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:18:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plug-in-thermostat-control-for-cool-120v-analog-40-95-f>: HTTP status code is not handled or not allowed 2026-01-25 05:18:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/17065-10239-btu-forced-air-cabinet-unit-heater-480v>: HTTP status code is not handled or not allowed 2026-01-25 05:18:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/hvac_controls/thermostats_temperature_controls/thermostats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:18:51 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:18:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/middle-buster-implement-for-tractors-under-50hp-category-1>: HTTP status code is not handled or not allowed 2026-01-25 05:18:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/folded-stacked-chair-tote>: HTTP status code is not handled or not allowed 2026-01-25 05:18:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/01-120-entree-only-grab-and-go-kit-120-servings>: HTTP status code is not handled or not allowed 2026-01-25 05:18:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leviton-5252-w-narrow-body-duplex-receptacle-straight-blade-industrial-grade-white>: HTTP status code is not handled or not allowed 2026-01-25 05:18:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mk01-120-powdered-milk-bucket-120-servings>: HTTP status code is not handled or not allowed 2026-01-25 05:18:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-5-tier-30w-x-18d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 05:18:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leviton-1221-2-20-amp-120-277-volt-toggle-single-pole-ac-quiet-switch-extra-heavy-duty-spec-grade-brown>: HTTP status code is not handled or not allowed 2026-01-25 05:18:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-25 05:19:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-36-l-x-5-w-x-3-4-thick-white>: HTTP status code is not handled or not allowed 2026-01-25 05:19:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/hvac_controls/thermostats_temperature_controls/thermostats>: HTTP status code is not handled or not allowed 2026-01-25 05:19:01 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_thermometers_hydrometers landed on page that is not a product page. 2026-01-25 05:19:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-sign-frame-wwedge-base-and-adjustable-stem-4l-x-11w-black>: HTTP status code is not handled or not allowed 2026-01-25 05:19:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4000-watt-commercial-ceiling-heater-14-3-16-l-208v>: HTTP status code is not handled or not allowed 2026-01-25 05:19:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:19:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-36-l-x-3-w-x-1-2-thick-white>: HTTP status code is not handled or not allowed 2026-01-25 05:19:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-chair-carton-of-two>: HTTP status code is not handled or not allowed 2026-01-25 05:19:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ap1-90-half-frame-safety-glasses-io-lens-black-frame>: HTTP status code is not handled or not allowed 2026-01-25 05:19:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-36-l-x-6-w-x-2-1-2-thick-white>: HTTP status code is not handled or not allowed 2026-01-25 05:19:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xt1-11-foam-lined-frame-safety-glasses-clear-af-lens-black-frame>: HTTP status code is not handled or not allowed 2026-01-25 05:19:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-sheet-12-l-x-12-w-x-2-3-4-thick-black-bulk-ps-acb-2205>: HTTP status code is not handled or not allowed 2026-01-25 05:19:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sv2z-economy-class-2-solid-safety-vest-with-zipper-hi-viz-green-5xl>: HTTP status code is not handled or not allowed 2026-01-25 05:19:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 68768 pages (at 86 pages/min), scraped 34298 items (at 24 items/min) 2026-01-25 05:19:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-abs-bar-12-l-x-1-1-2-w-x-3-8-thick-black>: HTTP status code is not handled or not allowed 2026-01-25 05:19:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jr0110id-half-frame-safety-glasses-clear-lens-black-frame>: HTTP status code is not handled or not allowed 2026-01-25 05:19:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/stainless_steel_bathroom_partitions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:19:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-section-locker>: HTTP status code is not handled or not allowed 2026-01-25 05:19:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-72w-x-30d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 05:19:13 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock landed on page that is not a product page. 2026-01-25 05:19:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-general-purpose-polyethylene-tray-without-faucet-12-x-16-x-3>: HTTP status code is not handled or not allowed 2026-01-25 05:19:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-48w-x-24d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 05:19:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/saturn-spinbar-teflon-magnetic-stirring-bar-40mm-white>: HTTP status code is not handled or not allowed 2026-01-25 05:19:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/back-pack-harness-system-for-type-5000-outdoor-cases-11-l-x-3-w-x-17-h-black>: HTTP status code is not handled or not allowed 2026-01-25 05:19:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-12-l-x-1-1-2-w-x-1-1-4-thick-white>: HTTP status code is not handled or not allowed 2026-01-25 05:19:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-48w-x-14d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 05:19:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-36w-x-18d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 05:19:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/janitorial-facility-maintenance/restroom_partitions_stalls/stainless_steel_bathroom_partitions>: HTTP status code is not handled or not allowed 2026-01-25 05:19:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-muff-headset-w-in-line-ptt>: HTTP status code is not handled or not allowed 2026-01-25 05:19:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mounting-kit-with-18-w-pegboard-36-w-louver-60-w-workbench-tan>: HTTP status code is not handled or not allowed 2026-01-25 05:19:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-30w-x-14d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 05:19:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roller-stand-with-27-to-42-height-range-1760-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 05:19:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spinbar-teflon-elliptical-egg-shaped-magnetic-stirring-bar-76-x-191mm-white>: HTTP status code is not handled or not allowed 2026-01-25 05:19:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-48w-x-21d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 05:19:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_dippers_scoops_cups already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:19:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cast-acrylic-bar-48-l-x-2-w-x-3-16-thick-clear-antistatic>: HTTP status code is not handled or not allowed 2026-01-25 05:19:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u-shaped-1-1-4-round-tubing-hangrail-semi-gloss-black>: HTTP status code is not handled or not allowed 2026-01-25 05:19:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spinbar-teflon-polygon-magnetic-stirring-bar-30-x-8mm-white-without-pivot-ring>: HTTP status code is not handled or not allowed 2026-01-25 05:19:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-60w-x-21d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 05:19:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:19:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-42w-x-18d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 05:19:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plano-68-quart-storage-trunk-30-l-x-14-1-4-w-x-12-3-4-h-black>: HTTP status code is not handled or not allowed 2026-01-25 05:19:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:19:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14d-x-23-1-2w-straight-shelf-black>: HTTP status code is not handled or not allowed 2026-01-25 05:19:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-wide-welded-steel-locker-triple-tier-24x24x74-3-door-gray-2>: HTTP status code is not handled or not allowed 2026-01-25 05:19:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-round-amber-flood-light-12-24vdc-54-leds-1492111>: HTTP status code is not handled or not allowed 2026-01-25 05:19:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_dippers_scoops_cups>: HTTP status code is not handled or not allowed 2026-01-25 05:19:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hytrol-stationary-floor-support-36w-max-leg-45-5-8h>: HTTP status code is not handled or not allowed 2026-01-25 05:19:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-sbr-roll-720-l-x-16-w-x-1-4-thick-black-75a>: HTTP status code is not handled or not allowed 2026-01-25 05:19:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_rod_stock>: HTTP status code is not handled or not allowed 2026-01-25 05:19:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-neoprene-roll-720-l-x-12-w-x-3-32-thick-black-60a>: HTTP status code is not handled or not allowed 2026-01-25 05:19:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/laboratory_heating_cooling/laboratory_hotplates_stirrers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:19:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_dippers_scoops_cups already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:19:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-304-stainless-steel-locker-12w-x-18d-x-78h-six-tier-1-wide-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 05:19:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 05:19:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flexible-truck-step-30-inch-w-5231130>: HTTP status code is not handled or not allowed 2026-01-25 05:19:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/butterfly-joining-clip-semi-gloss-black>: HTTP status code is not handled or not allowed 2026-01-25 05:19:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-epoxy-wire-shelving-add-on-24545b-54w-x-24d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 05:19:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-deluxe-hook-semi-gloss-black>: HTTP status code is not handled or not allowed 2026-01-25 05:19:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-underbed-truck-box-24x24x24-gray-1735130>: HTTP status code is not handled or not allowed 2026-01-25 05:19:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-square-clear-flood-light-12-24vdc-5-leds-1492117>: HTTP status code is not handled or not allowed 2026-01-25 05:19:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/laboratory_heating_cooling/laboratory_hotplates_stirrers>: HTTP status code is not handled or not allowed 2026-01-25 05:19:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_dippers_scoops_cups>: HTTP status code is not handled or not allowed 2026-01-25 05:19:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-plastic-ladle-50ml-9-handle>: HTTP status code is not handled or not allowed 2026-01-25 05:19:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-three-tier-locker-starter-1-wide-12w-x-18d-x-24h-unassembled-blue-frost>: HTTP status code is not handled or not allowed 2026-01-25 05:19:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lk60-chrome-continuous-waste-drain-for-3-compartment-sink>: HTTP status code is not handled or not allowed 2026-01-25 05:19:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slim-cd-cases-clear-25-cases-per-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:19:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-optional-upright-for-dmb-1-chrome>: HTTP status code is not handled or not allowed 2026-01-25 05:19:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-epoxy-wire-shelving-18605b-60w-x-18d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 05:19:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/end-cap-for-1-1-4-1-5-16-round-tubing-chrome>: HTTP status code is not handled or not allowed 2026-01-25 05:19:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-epoxy-wire-shelving-18305b-30w-x-18d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 05:19:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_dippers_scoops_cups already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:19:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22-x-28-double-bulletin-sign-holder-w-rectangular-tubing-base-matte-black>: HTTP status code is not handled or not allowed 2026-01-25 05:20:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/450002-kltm-guard-locking-switchmechanical-only-12npt-die-cast>: HTTP status code is not handled or not allowed 2026-01-25 05:20:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/height-adjustable-cafe-table-pneumatic-32-round-white>: HTTP status code is not handled or not allowed 2026-01-25 05:20:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20015hpad-ultra-quiet-oil-free-15-hp-20-gal-175-psi-two-stage-air-compressor>: HTTP status code is not handled or not allowed 2026-01-25 05:20:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/seq-dual-speed-electric-chain-hoist-1-8-ton-20-lift-56-9-3-ft-min-230v>: HTTP status code is not handled or not allowed 2026-01-25 05:20:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-x-30-plastic-safety-edge-mobile-production-bench-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:20:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-double-tier-locker-2-wide-12w-x-18d-x-36h-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:20:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2w-x-7h-slatgrid-panel-semi-gloss-white>: HTTP status code is not handled or not allowed 2026-01-25 05:20:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-30-stainless-steel-square-edge-mobile-production-bench-tan>: HTTP status code is not handled or not allowed 2026-01-25 05:20:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_dippers_scoops_cups>: HTTP status code is not handled or not allowed 2026-01-25 05:20:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 68863 pages (at 95 pages/min), scraped 34332 items (at 34 items/min) 2026-01-25 05:20:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/140014-rope-kit-ss-30m-ss>: HTTP status code is not handled or not allowed 2026-01-25 05:20:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/seq-dual-speed-electric-chain-hoist-1-8-ton-10-lift-56-9-3-ft-min-115v>: HTTP status code is not handled or not allowed 2026-01-25 05:20:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-plastic-forceps-5-6pk>: HTTP status code is not handled or not allowed 2026-01-25 05:20:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eq-dual-speed-electric-chain-hoist-1-2-ton-10-lift-25-4-2-ft-min-230v>: HTTP status code is not handled or not allowed 2026-01-25 05:20:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vacuum-aspirator-bottle-05-gal-plastic>: HTTP status code is not handled or not allowed 2026-01-25 05:20:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/SVT-1872-72-W-x-18-D-18-ga-T-430-Stainless-Steel-Workbench>: HTTP status code is not handled or not allowed 2026-01-25 05:20:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/SVTR-1872-18-W-x-72-D-T-430-18-ga-Stainless-Steel-Workbench-with-a-15-backsplash>: HTTP status code is not handled or not allowed 2026-01-25 05:20:15 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:20:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-double-tier-locker-2-wide-12w-x-18d-x-36h-assembled-almond>: HTTP status code is not handled or not allowed 2026-01-25 05:20:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sterile-disposable-plastic-vaccu-pette-96-pipetting-device>: HTTP status code is not handled or not allowed 2026-01-25 05:20:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_clamps_tube_fittings already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:20:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-head-thumb-screw-m8-x-1-25-30mm-thread-30mm-head-dia-19-5mm-head-h-pkg-of-5>: HTTP status code is not handled or not allowed 2026-01-25 05:20:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-buna-n-strip-120-l-x-1-4-w-x-1-8-thick-black-high-strength-80a>: HTTP status code is not handled or not allowed 2026-01-25 05:20:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4l-round-tubing-w-1-1-16-dia-chrome>: HTTP status code is not handled or not allowed 2026-01-25 05:20:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precision-thumb-screw-w-shoulder-10-32-3-8-thread-1-2-head-dia-stainless-pkg-of-5>: HTTP status code is not handled or not allowed 2026-01-25 05:20:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precision-thumb-screw-w-shoulder-4-40-1-4-thread-3-8-head-dia-steel-pkg-of-5>: HTTP status code is not handled or not allowed 2026-01-25 05:20:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lf-brass-full-port-ball-valve-threaded-1-1-4>: HTTP status code is not handled or not allowed 2026-01-25 05:20:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/statfree-uc2-8482-66322-dissipative-dual-layer-rubber-mat-24-d-x-36-w-sky-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:20:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 05:20:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-side-action-serratable-sealer-hd-for-5-8w-polypropylene>: HTTP status code is not handled or not allowed 2026-01-25 05:20:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/labware_utensils/laboratory_clamps_tube_fittings>: HTTP status code is not handled or not allowed 2026-01-25 05:20:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-head-t-knob-thumb-screw-5-16-18-1-14-thread-1-77-head-dia-94-head-h-pkg-of-5>: HTTP status code is not handled or not allowed 2026-01-25 05:20:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-side-double-notch-action-sealer-for-1w-x-1thickness>: HTTP status code is not handled or not allowed 2026-01-25 05:20:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precision-brass-thumb-screw-w-shoulder-8-32-3-8-thread-1-2-head-dia-pkg-of-10>: HTTP status code is not handled or not allowed 2026-01-25 05:20:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-shower-curtain-rod-stainless-steel-9538-036000>: HTTP status code is not handled or not allowed 2026-01-25 05:20:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/bar_restaurant_equipment/dispensers_organizers/cup_lid_organizers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:20:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-buna-n-sheet-12-l-x-12-w-x-3-8-thick-black-high-strength-90a-bulk-rs-h90-34>: HTTP status code is not handled or not allowed 2026-01-25 05:20:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-94dv2a-n5-division-2-xenon-strobe-amber-120v-ac>: HTTP status code is not handled or not allowed 2026-01-25 05:20:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/patio-bar-height-stools-all-weather-textilene-sling-fabric-seat-back-navy-2-pk>: HTTP status code is not handled or not allowed 2026-01-25 05:20:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cm-1133-74-one-by-one-rotating-cup-and-lid-organizer-10-inch-w-x-10-inch-d-x-15-3-4-inch-h-platinum>: HTTP status code is not handled or not allowed 2026-01-25 05:20:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mount-anywhere-basket-12-6-6-w>: HTTP status code is not handled or not allowed 2026-01-25 05:20:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beverage-center-for-built-in-or-free-standing-use-residentialcommercial-ada-compliant>: HTTP status code is not handled or not allowed 2026-01-25 05:20:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/die-cast-adjustable-lever-m5-x-0-8-30mm-handle-50mm-stud-black-5n50a19k-gn300>: HTTP status code is not handled or not allowed 2026-01-25 05:20:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-chm344a-5airover-hp-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:20:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/bar_restaurant_equipment/dispensers_organizers/cup_lid_organizers landed on page that is not a product page. 2026-01-25 05:20:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ecp4308t-40-hp-1190-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:20:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/support-ring-m24-o-d-x-17mm-i-d-x-1-50mm-thick-spring-steel-din-988>: HTTP status code is not handled or not allowed 2026-01-25 05:20:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signal-5531m-120n5-multi-tone-electron-signal-4-inputs-outputs-120v-ac-input-120vac-power>: HTTP status code is not handled or not allowed 2026-01-25 05:20:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ecp4316t-75-hp-1780-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:20:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/174-150741-inspection-timer-30-day-10-pack-led-lights-abs-plastic-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:20:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-125xbrmb24db-125xbr-xtra-brite-led-multi-mode-blue-24-vdc>: HTTP status code is not handled or not allowed 2026-01-25 05:20:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/draft-inducer-blower-3250-rpm-115v-oao-1-6-fl-amps>: HTTP status code is not handled or not allowed 2026-01-25 05:20:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-6536-g5-hornstrobe-24v-ac>: HTTP status code is not handled or not allowed 2026-01-25 05:20:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-hinged-top-basket-truck-12-bu-green-vinyl-wood-base-all-swivel>: HTTP status code is not handled or not allowed 2026-01-25 05:20:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cjm3155-2-hp-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:20:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/174-j20-260-2595-153-vinyl-label-3-35-h-x-2-17-w-340-roll-white>: HTTP status code is not handled or not allowed 2026-01-25 05:20:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-hinged-top-basket-truck-6-bu-gray-vinyl-wood-base-all-swivel>: HTTP status code is not handled or not allowed 2026-01-25 05:20:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cwdl3514-1-5-hp-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:20:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mighty-products-mn-ms90-b1016-90-shade-cloth-shade-tarp-10-x-16>: HTTP status code is not handled or not allowed 2026-01-25 05:20:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/banner-safety-protects-people-quality-protects-jobs-3-x-10>: HTTP status code is not handled or not allowed 2026-01-25 05:20:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pool-pump-motor-3-4-hp-3450-rpm-230-115v-odp-60-hertz>: HTTP status code is not handled or not allowed 2026-01-25 05:20:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/filament-tape-898-36mm-x-55m-6-6-mil-clear>: HTTP status code is not handled or not allowed 2026-01-25 05:21:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/direct-drive-motor-1-4-hp-1050-rpm-277v-oao>: HTTP status code is not handled or not allowed 2026-01-25 05:21:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-long-aluminum-construction-pipe-safety-railing>: HTTP status code is not handled or not allowed 2026-01-25 05:21:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-hinged-top-basket-truck-16-bu-tan-vinyl-wood-base-all-swivel>: HTTP status code is not handled or not allowed 2026-01-25 05:21:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-52c-n5-40wh-rotating-beacon-clear-120v-ac>: HTTP status code is not handled or not allowed 2026-01-25 05:21:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-boltless-shelving-96-inch-w-x-48-inch-d-1-level-w-ez-decking-black>: HTTP status code is not handled or not allowed 2026-01-25 05:21:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-270ledsy24ad-smd-steady-led-module-and-light-source-yellow-24v-acdc>: HTTP status code is not handled or not allowed 2026-01-25 05:21:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/removable-repositionable-tape-9425-case-1-2-x-72-yds-10-mil-clear>: HTTP status code is not handled or not allowed 2026-01-25 05:21:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/specialty-molded-dock-bumper-8-wx18-lx2-h-1>: HTTP status code is not handled or not allowed 2026-01-25 05:21:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 68957 pages (at 94 pages/min), scraped 34369 items (at 37 items/min) 2026-01-25 05:21:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ctm3770t-7-5-hp-1765-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:21:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/room-air-conditioner-motor-1-6-hp-1075-rpm-115v-oao-42y-frame>: HTTP status code is not handled or not allowed 2026-01-25 05:21:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-boltless-shelving-96-inch-w-x-18-inch-d-x-84-inch-h-3-levels-add-on-w-wood-decking-black>: HTTP status code is not handled or not allowed 2026-01-25 05:21:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/generators_inverters_power_distribution_units/standby_generators already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:21:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spring-counterbalanced-cart-5>: HTTP status code is not handled or not allowed 2026-01-25 05:21:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cem3586t-5-2-hp-3490-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:21:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-bm3569-33-hp-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:21:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clearview-industrial-cabinet-with-single-door-30-w-x-24-d-x-75-h-red>: HTTP status code is not handled or not allowed 2026-01-25 05:21:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-storage-cabinet-36-w-x-24-d-x-75-h-brown>: HTTP status code is not handled or not allowed 2026-01-25 05:21:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/freeman-professional-fine-wire-stapler>: HTTP status code is not handled or not allowed 2026-01-25 05:21:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-storage-cabinet-36-w-x-24-d-x-75-h-beige>: HTTP status code is not handled or not allowed 2026-01-25 05:21:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-cabinet-with-single-door-30-w-x-24-d-x-75-h-brown>: HTTP status code is not handled or not allowed 2026-01-25 05:21:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/generators_inverters_power_distribution_units/standby_generators>: HTTP status code is not handled or not allowed 2026-01-25 05:21:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hydraulic-drum-dumper-stationary-4>: HTTP status code is not handled or not allowed 2026-01-25 05:21:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jlh-series-compact-lever-chain-hoist-1-2-ton-10-ft-lift>: HTTP status code is not handled or not allowed 2026-01-25 05:21:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paper-astrobrights-paper-yellow-8-1-2in-x-11in-24-lb-500-sheets-ream>: HTTP status code is not handled or not allowed 2026-01-25 05:21:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/condenser-fans-motor-1-8-hp-1075-rpm-208-230v-teao-48-frame>: HTTP status code is not handled or not allowed 2026-01-25 05:21:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbl-rivet-boltless-shelving-60wx18dx84h-4-levels-starter-no-decking-1200-lbs-shelf-cap-tan>: HTTP status code is not handled or not allowed 2026-01-25 05:21:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbl-rivet-boltless-shelving-48wx48dx84h-4-levels-starter-no-decking-750-lbs-shelf-cap-tan>: HTTP status code is not handled or not allowed 2026-01-25 05:21:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-shift-industrial-cabinet-48-w-x-24-d-x-75-h-black>: HTTP status code is not handled or not allowed 2026-01-25 05:21:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-shift-industrial-cabinet-48-w-x-24-d-x-75-h-red>: HTTP status code is not handled or not allowed 2026-01-25 05:21:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-uniform-cabinet-with-full-width-rod-48-w-x-24-d-x-75-h-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:21:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-boltless-shelving-w-center-support-96-inch-w-x-36-inch-d-1-level-w-o-decking-black>: HTTP status code is not handled or not allowed 2026-01-25 05:21:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-uniform-cabinet-with-4-shelves-36-w-x-24-d-x-75-h-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:21:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c-oem-replacement-motor-1-5-hp-825-rpm-208-230v-teao>: HTTP status code is not handled or not allowed 2026-01-25 05:21:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-cabinet-with-7-drawers-36-w-x-24-d-x-75-h-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:21:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbl-rivet-boltless-shelving-48wx36dx84h-4-levels-add-on-no-decking-1220-lbs-shelf-cap-tan>: HTTP status code is not handled or not allowed 2026-01-25 05:21:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-straight-female-4mm-tube-x-1-8-bspp-944-l>: HTTP status code is not handled or not allowed 2026-01-25 05:21:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pre-configured-6x8-partitioned-office-72-h-add-on-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:21:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-polymer-shelf-with-grid-mat-30-w-x-21-d>: HTTP status code is not handled or not allowed 2026-01-25 05:21:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-size-stacking-bin-11x23-7-8x10-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:21:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/toolbox-t700-white-jumbo-roll-870-sheets-roll-1-roll-case-78300>: HTTP status code is not handled or not allowed 2026-01-25 05:21:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pressure-washer-motor-6-hp-3450-rpm-230v-odp>: HTTP status code is not handled or not allowed 2026-01-25 05:21:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oem-replacement-motor-1-4-hp-1100-rpm-208-230v-teao>: HTTP status code is not handled or not allowed 2026-01-25 05:21:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-l1409t-5-hp-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:21:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shop-ticket-holders-stitched-both-sides-clear-6-x-9-25-bx>: HTTP status code is not handled or not allowed 2026-01-25 05:21:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-boltless-shelving-72-inch-w-x-18-inch-d-x-84-inch-h-3-levels-add-on-w-wood-decking-black>: HTTP status code is not handled or not allowed 2026-01-25 05:21:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-swivel-elbow-5-32-tube-x-1-4-nptf-2-3-l>: HTTP status code is not handled or not allowed 2026-01-25 05:21:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-l1322tm-2-hp-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:21:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/113079000-00580669-mach-led-plus-30-led-machine-luminaire-20-watts>: HTTP status code is not handled or not allowed 2026-01-25 05:21:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/17-3-4-x-8-1-8-x-9-wire-mesh-stack-and-hang-bin-chrome>: HTTP status code is not handled or not allowed 2026-01-25 05:21:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-canvas-functional-cargo-short-charcoal-38x12-wp90ch3812>: HTTP status code is not handled or not allowed 2026-01-25 05:21:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-boltless-shelving-60-inch-w-x-18-inch-d-x-84-inch-h-3-levels-starter-w-wire-decking-black>: HTTP status code is not handled or not allowed 2026-01-25 05:21:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/filter-multi-pleat-xl10-18-w-x-24-h-x-1-d-12-ctn>: HTTP status code is not handled or not allowed 2026-01-25 05:21:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-rack-4-shelves-with-12-span-track-flow-units-48-w-x-48-d-x-72-h-textured-black>: HTTP status code is not handled or not allowed 2026-01-25 05:21:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-boltless-shelving-48-inch-w-x-18-inch-d-x-84-inch-h-3-levels-add-on-w-wood-decking-black>: HTTP status code is not handled or not allowed 2026-01-25 05:21:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-3-8-l-x-11-w-x-3-h-ultra-series-stack-and-hang-bin-ivory-polyethylene-3-slots>: HTTP status code is not handled or not allowed 2026-01-25 05:21:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v-grip-wire-shelving-spill-containment-system-4-shelves-trays-36-w-x-18-d-x-84-h-starter-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:21:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-30-mobile-security-cabinet-bench-maple-safety-edge-1>: HTTP status code is not handled or not allowed 2026-01-25 05:21:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/combi-oven-38-9-16-w-x-34-d-x-35-h-208-240v-63-63a-6-shelves-digital-controls>: HTTP status code is not handled or not allowed 2026-01-25 05:21:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rivetwell-particle-board-decking-69-inch-w-x-15-inch-d-x-5-8-inch>: HTTP status code is not handled or not allowed 2026-01-25 05:21:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/maxiscrub-9-sponge-mop-refill-12-case-94205>: HTTP status code is not handled or not allowed 2026-01-25 05:22:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/windex-ready-to-use-glass-cleaner-gallon-refill>: HTTP status code is not handled or not allowed 2026-01-25 05:22:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-x-36-reflective-roll-up-vinyl-sign-transition-symbol-right>: HTTP status code is not handled or not allowed 2026-01-25 05:22:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-m1219-75-33-hp-1725-1140-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:22:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-deck-for-rivetwell-60-inch-w-x-36-inch-d-x-1-4-inch-h-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:22:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/warming-cabinet-30-w-x-35-1-2-h-x-26-5-8-d-undercounter-1-solid-door-2-adj-shelves>: HTTP status code is not handled or not allowed 2026-01-25 05:22:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-swivel-run-tee-4mm-tube-x-m5-unf-2-3-l>: HTTP status code is not handled or not allowed 2026-01-25 05:22:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-flow-control-valve-flow-screw-adjustment-3-8-tube-x-3-8-universal-thread-1-1-6-l>: HTTP status code is not handled or not allowed 2026-01-25 05:22:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-ball-valve-male-x-female-1-8-nptf-1-1-2-l>: HTTP status code is not handled or not allowed 2026-01-25 05:22:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/warming-cabinet-30-w-x-60-h-x-20-5-8-d-recessed-1-solid-door-3-adj-shelves>: HTTP status code is not handled or not allowed 2026-01-25 05:22:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 69054 pages (at 97 pages/min), scraped 34406 items (at 37 items/min) 2026-01-25 05:22:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-mesh-fabric-tack-board-72-x-48>: HTTP status code is not handled or not allowed 2026-01-25 05:22:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-straight-female-1-4-tube-x-1-8-nptf-1-l>: HTTP status code is not handled or not allowed 2026-01-25 05:22:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-l3355-17-hp-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:22:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/17mm-external-snap-ring-standard-duty-stamped-carbon-spring-steel-phosphate-din-471>: HTTP status code is not handled or not allowed 2026-01-25 05:22:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-floor-pads-black-21>: HTTP status code is not handled or not allowed 2026-01-25 05:22:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xtra-st600h-ultrasonic-cleaner-240v-dual-frequency-25-45-khz-high-power-12-gallons>: HTTP status code is not handled or not allowed 2026-01-25 05:22:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xactasaw-deluxe-5hp-1ph-50-rip>: HTTP status code is not handled or not allowed 2026-01-25 05:22:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:22:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-tear-drop-pallet-rack-add-on-108-w-x-36-d-x-96-h-796567n>: HTTP status code is not handled or not allowed 2026-01-25 05:22:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mri-safe-howard-instrument-table-20-l-x-36-w-x-34-h-on-casters-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 05:22:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-needle-valve-knob-adjustment-8mm-tube-x-1-4-swift-fit-universal-thread-1-l>: HTTP status code is not handled or not allowed 2026-01-25 05:22:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-tear-drop-pallet-rack-starter-96-w-x-48-d-x-120-h-796604n>: HTTP status code is not handled or not allowed 2026-01-25 05:22:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/plumbing/water_filtration/filter_housings already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:22:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-m1524t-5-2-2-hp-1725-1140-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:22:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/freestanding-clear-desk-divider-acrylic-30W-x-24H>: HTTP status code is not handled or not allowed 2026-01-25 05:22:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-needle-valve-screw-adjustment-12mm-tube-x-3-8-swift-fit-universal-thread-1-1-4-l>: HTTP status code is not handled or not allowed 2026-01-25 05:22:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/knee-boot-size-11-15-h-composite-toe-chevron-plus-outsole-black>: HTTP status code is not handled or not allowed 2026-01-25 05:22:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wheel-bed-workstation-96-w-x-48-d-x-72-h-add-on-section>: HTTP status code is not handled or not allowed 2026-01-25 05:22:28 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/pallet_racks/pallet_rack_kits landed on page that is not a product page. 2026-01-25 05:22:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-needle-valve-screw-adjustment-1-4-fm-nptf-x-1-4-swift-fit-universal-thread-1-l>: HTTP status code is not handled or not allowed 2026-01-25 05:22:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/atrix-omega-crevice-tool-black>: HTTP status code is not handled or not allowed 2026-01-25 05:22:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b30-series-printer-ribbon-4-21-64-w-x-200-l-black-b30-r4300>: HTTP status code is not handled or not allowed 2026-01-25 05:22:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/plumbing/water_filtration/filter_housings>: HTTP status code is not handled or not allowed 2026-01-25 05:22:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interlake-mecalux-pallet-rack-tear-drop-add-on-96-w-x-42-d-x-120-h>: HTTP status code is not handled or not allowed 2026-01-25 05:22:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-fdl3712tm-10-hp-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:22:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dunnage-rack-gray-48-w-x-22-d-x-12-h>: HTTP status code is not handled or not allowed 2026-01-25 05:22:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_lockout_identification_tags already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:22:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-em3313t-10-hp-1770-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:22:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-em3313t-5-10-hp-1770-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:22:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waterhog-eco-elite-3-wide-4-up-to-60-southern-pine>: HTTP status code is not handled or not allowed 2026-01-25 05:22:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interlake-mecalux-pallet-rack-tear-drop-starter-96-w-x-42-d-x-120-h-new>: HTTP status code is not handled or not allowed 2026-01-25 05:22:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interlake-mecalux-pallet-rack-tear-drop-starter-96-w-x-36-d-x-120-h>: HTTP status code is not handled or not allowed 2026-01-25 05:22:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-lightweight-knee-boot-men-s-size-7-15-h-plain-toe-cleated-outsole-black>: HTTP status code is not handled or not allowed 2026-01-25 05:22:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/locks_lockout_devices/lockout_tagout_devices/lockout_devices already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:22:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-x-6-x-4-white-corrugated-boxes-200lb-test-ect-32>: HTTP status code is not handled or not allowed 2026-01-25 05:22:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polly-products-large-message-center-2-sided-2-posts-green>: HTTP status code is not handled or not allowed 2026-01-25 05:22:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kraft-corrugated-mailers-6-x-4-x-2-200lb-ect-32-b-50-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:22:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/palmjet-12-straight-extension>: HTTP status code is not handled or not allowed 2026-01-25 05:22:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-gas-heater-infrared-positive-pressure-straight-tube-50000-btu-30l>: HTTP status code is not handled or not allowed 2026-01-25 05:22:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g2-pvc-overshoe-size-extra-small-17-h-cleated-outsole-black-with-black-sole>: HTTP status code is not handled or not allowed 2026-01-25 05:22:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fireproof-file-cabinet-and-safe-legal-letter-size-17-3-4-w-x-22-1-8-d-x-27-3-4-h>: HTTP status code is not handled or not allowed 2026-01-25 05:22:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ectm2332t-10airover-hp-1180-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:22:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ejpm2534t-30-hp-3510-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:22:48 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/safety_lockout_identification_tags landed on page that is not a product page. 2026-01-25 05:22:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sealed-motor-low-profile-air-mover-w-gci-power-outlets-1-3-hp-1150-cfm-xl-730a>: HTTP status code is not handled or not allowed 2026-01-25 05:22:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyurethane-cleanroom-stool-with-casters-high-bench-height-black>: HTTP status code is not handled or not allowed 2026-01-25 05:22:50 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/locks_lockout_devices/lockout_tagout_devices/lockout_devices landed on page that is not a product page. 2026-01-25 05:22:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vertical-baby-changing-table-wall-mounted-stainless-steel-kb311-sswm>: HTTP status code is not handled or not allowed 2026-01-25 05:22:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-gas-heater-infrared-positive-pressure-u-tube-tough-guy-75000-btu-20l>: HTTP status code is not handled or not allowed 2026-01-25 05:22:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/vehicle_flood_utility_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:22:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coverallzipper-fronthoodelastic-wrist-and-anklestormflapserged-seamyellow3x>: HTTP status code is not handled or not allowed 2026-01-25 05:22:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-canister-of-sugar-20-oz-3-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:22:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coverall-hood-and-socks-boots-serged-seam-white-3x>: HTTP status code is not handled or not allowed 2026-01-25 05:22:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coverall-hood-and-socks-boots-serged-seam-white-4x>: HTTP status code is not handled or not allowed 2026-01-25 05:22:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/desiccant-pak-claytyveksilicon-dioxide-300-count-pail>: HTTP status code is not handled or not allowed 2026-01-25 05:22:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jacket-size-tall-men-s-2t-type-r-class-3-fluorescent-yellow-green-black>: HTTP status code is not handled or not allowed 2026-01-25 05:22:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zero-calorie-sweetener-1g-packet-115-box>: HTTP status code is not handled or not allowed 2026-01-25 05:22:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-look-out-sensor-w-floor-burst-ceiling-hung-1-box-4-sensors-4-lights>: HTTP status code is not handled or not allowed 2026-01-25 05:23:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/amber-oval-mini-light-bar-with-40-led-9-875-x-6-75-x-1-625-inch-8891080>: HTTP status code is not handled or not allowed 2026-01-25 05:23:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/75in-round-marker-clearance-lights-1-led-red-with-male-bullets-5627511>: HTTP status code is not handled or not allowed 2026-01-25 05:23:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/vehicle_flood_utility_lights>: HTTP status code is not handled or not allowed 2026-01-25 05:23:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/35mm-economy-rainsuits-yellow-35mm-pvc-on-polyester-5x>: HTTP status code is not handled or not allowed 2026-01-25 05:23:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oil-safe-mini-spout-lid-yellow>: HTTP status code is not handled or not allowed 2026-01-25 05:23:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ectm4110t-40airover-hp-1775-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:23:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/humidity-indicator-card-5-10-15-range-125-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:23:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dot-light-plug-3-wire-amp-style-plug-with-3-pin-right-angle-pl-3-male-plug-5620354>: HTTP status code is not handled or not allowed 2026-01-25 05:23:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pressure-sensitive-pipe-marker-gasoline-1>: HTTP status code is not handled or not allowed 2026-01-25 05:23:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-ac-valve-core-remover-and-installer-kit-r-12-r-134a-msc58490>: HTTP status code is not handled or not allowed 2026-01-25 05:23:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/executive-chair-with-flip-up-armrests-black-leather-upholstered-277492>: HTTP status code is not handled or not allowed 2026-01-25 05:23:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/hearing_protection/earmuffs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:23:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 69144 pages (at 90 pages/min), scraped 34432 items (at 26 items/min) 2026-01-25 05:23:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precoiled-strap-on-pipe-marker-air-3>: HTTP status code is not handled or not allowed 2026-01-25 05:23:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-efm3211t-8-3-hp-1760-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:23:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2000-coverall-front-zipper-serged-seams-yellow-lg>: HTTP status code is not handled or not allowed 2026-01-25 05:23:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precoiled-strap-on-pipe-marker-storm-sewer-8>: HTTP status code is not handled or not allowed 2026-01-25 05:23:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-mounts-for-flood-and-spot-lights-1492106>: HTTP status code is not handled or not allowed 2026-01-25 05:23:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-5-gallon-200-psi-portable-electric-trim-air-compressor>: HTTP status code is not handled or not allowed 2026-01-25 05:23:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/small-brake-caliper-press-kas277>: HTTP status code is not handled or not allowed 2026-01-25 05:23:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precoiled-strap-on-pipe-marker-water-11>: HTTP status code is not handled or not allowed 2026-01-25 05:23:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-16-x-1-2-square-drive-bit-holder-w-c-ring-x-1-3-4-gunmetal-grey-4140-steel>: HTTP status code is not handled or not allowed 2026-01-25 05:23:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/hearing_protection/earmuffs>: HTTP status code is not handled or not allowed 2026-01-25 05:23:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pressure-sensitive-pipe-marker-direction-arrow-green-1>: HTTP status code is not handled or not allowed 2026-01-25 05:23:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jw-winco-aluminum-cabinet-u-handle-512-102-black>: HTTP status code is not handled or not allowed 2026-01-25 05:23:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pressure-sensitive-pipe-marker-direction-arrow-blue-1>: HTTP status code is not handled or not allowed 2026-01-25 05:23:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-to-3-1-2-33-degree-paper-tape-framing-nailer>: HTTP status code is not handled or not allowed 2026-01-25 05:23:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-mag-bit-holder-1pc-stainless-steel-w-c-ring-x-2-3-8>: HTTP status code is not handled or not allowed 2026-01-25 05:23:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coil-siding-nail-316-stainless-steel-1-1-2-x-090-ring-1800-qty>: HTTP status code is not handled or not allowed 2026-01-25 05:23:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15300-visitor-arrivaldeparture-chime-battery-operated-2-34w-x-2d-x-4-14h-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:23:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2007-14c-top-hanger-executive-series-17l-wood-cy-pkg-qty-50>: HTTP status code is not handled or not allowed 2026-01-25 05:23:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/412-dress-hanger-infant-super-heavy-weight-12l-plastic-cl-pkg-qty-100>: HTTP status code is not handled or not allowed 2026-01-25 05:23:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corner-reflector-straight-u-shaped-infrared-heaters>: HTTP status code is not handled or not allowed 2026-01-25 05:23:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1414rc-suit-hanger-super-heavy-weight-14l-plastic-wh-pkg-qty-100>: HTTP status code is not handled or not allowed 2026-01-25 05:23:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-8482-magnetic-whiteboard-48-w-x-48-h-steel-surface-aluminum-frame>: HTTP status code is not handled or not allowed 2026-01-25 05:23:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/privacy-screen-beamatic-screen-5-sections>: HTTP status code is not handled or not allowed 2026-01-25 05:23:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-vem3770t-71-2-hp-1770-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:23:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foaming-glass-cleaner-20oz-can>: HTTP status code is not handled or not allowed 2026-01-25 05:23:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-pick-shelving-5-level-single-straight-78-h-x-50-w-x-24-d-add-on>: HTTP status code is not handled or not allowed 2026-01-25 05:23:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torq-set-8-insert-bit-x-1-s2-modified-steel-gunmetal-grey>: HTTP status code is not handled or not allowed 2026-01-25 05:23:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-5-gauge-1-2-crown-2-floorng-staples-7728-qty>: HTTP status code is not handled or not allowed 2026-01-25 05:23:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-weather-faux-rattan-chair-chocolate-brown-with-beige-cushions>: HTTP status code is not handled or not allowed 2026-01-25 05:23:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-venm3581t-1-hp-1745-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:23:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ambidex-disposable-nitrile-industrial-grade-4-mil-textured-m>: HTTP status code is not handled or not allowed 2026-01-25 05:23:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-vem3584t-5-1-5-hp-1760-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:23:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-vinyl-sign-danger-keep-out-2>: HTTP status code is not handled or not allowed 2026-01-25 05:23:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ambidex-disposable-nitrile-industrial-grade-4-mil-textured-s>: HTTP status code is not handled or not allowed 2026-01-25 05:23:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-vinyl-sign-danger-no-smoking-4>: HTTP status code is not handled or not allowed 2026-01-25 05:23:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manual-chain-hoist-with-overload-protection-1-ton-capacity-10-lift>: HTTP status code is not handled or not allowed 2026-01-25 05:23:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/top-grain-goatskin-leather-protector-for-novax-gloves-slip-on-size-8>: HTTP status code is not handled or not allowed 2026-01-25 05:23:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-aluminum-sign-danger-this-equipment-starts-automatically>: HTTP status code is not handled or not allowed 2026-01-25 05:23:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beam-trolley-with-clamp-3-ton-capacity>: HTTP status code is not handled or not allowed 2026-01-25 05:23:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mounted-pipe-guard-46-inch-tall-safety-yellow-protects-6-inch-pipe>: HTTP status code is not handled or not allowed 2026-01-25 05:23:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esp-kit-1-pair-esp-glove-1-pair-cow-protector-class-2-size-11>: HTTP status code is not handled or not allowed 2026-01-25 05:23:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-m3537-5-5-hp-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:23:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/k-flex-6rxlo100448-6-insul-lock-ds-pipe-insulation-4-1-2-id-x-1-thick-w-overla>: HTTP status code is not handled or not allowed 2026-01-25 05:23:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-turbo-rim-diamond-blade-for-smooth-cuts-0087segment-thick>: HTTP status code is not handled or not allowed 2026-01-25 05:23:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sds-plusbulldog8482xtreme-rotary-hammer-bit-38dia-x-4lx6l>: HTTP status code is not handled or not allowed 2026-01-25 05:23:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torq-set-3-insert-bit-x-1-s2-modified-steel-gunmetal-grey>: HTTP status code is not handled or not allowed 2026-01-25 05:23:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sb72-se-speed-bump-w-cat-eye-refelctors-recycled-rubber-w-reflective-stripes-6-l>: HTTP status code is not handled or not allowed 2026-01-25 05:23:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gtek-maxiflex-plus-iii-nitrile-dotted-palm-fully-black-microfoam-s>: HTTP status code is not handled or not allowed 2026-01-25 05:24:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zurn-7-x-7-square-floor-drain-wscrews-nickel>: HTTP status code is not handled or not allowed 2026-01-25 05:24:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3308-eg-lr-drill-kit-type-iii-barricade-w-plastic-uprights-angle-iron-feet-engineer-striped>: HTTP status code is not handled or not allowed 2026-01-25 05:24:02 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/appliances/refrigerators_freezers/refrigerators/undercounter_refrigerators already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:24:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-40-flange-nut-1-4-hex-3-8-flange-dia-7-32-height-stainless-steel-cn-094ss>: HTTP status code is not handled or not allowed 2026-01-25 05:24:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yoke-roller-sealed-crowned-od-125-roller-dia>: HTTP status code is not handled or not allowed 2026-01-25 05:24:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jat-413-1-4-straight-aluminum-die-grinder>: HTTP status code is not handled or not allowed 2026-01-25 05:24:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lynx-pow-x-7-her-in-module-16-w-x-7-h-blue-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 05:24:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/watts-filters-20x2-12-softening-cartridge>: HTTP status code is not handled or not allowed 2026-01-25 05:24:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cam-follower-sealed-hex-head-crowned-od-1-1-2-l-stud-0875-w-roller-1625-roller-dia>: HTTP status code is not handled or not allowed 2026-01-25 05:24:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 69241 pages (at 97 pages/min), scraped 34473 items (at 41 items/min) 2026-01-25 05:24:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standing-height-desk-60-x-30-charcoal-top-black-base>: HTTP status code is not handled or not allowed 2026-01-25 05:24:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/open-return-desk-48-x-24-charcoal-top-black-base>: HTTP status code is not handled or not allowed 2026-01-25 05:24:11 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/appliances/refrigerators_freezers/refrigerators/undercounter_refrigerators landed on page that is not a product page. 2026-01-25 05:24:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/watts-sediment-melt-blown-cartridges-2-12odx9-78l-50-micron>: HTTP status code is not handled or not allowed 2026-01-25 05:24:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-4-tier-esd-14w-x-48l-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 05:24:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/watts-wound-filter-ff-cartridges-9-34lx4-12od-1-micron>: HTTP status code is not handled or not allowed 2026-01-25 05:24:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spanish-aluminum-sign-peligro-blank>: HTTP status code is not handled or not allowed 2026-01-25 05:24:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelf-truck-esd-24w-x-42l-x-80h-polyurethane-4-swivel-2-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 05:24:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-174-reach-refrigerator-2-door-54-wx32-2-dx82-5-h-47-cu>: HTTP status code is not handled or not allowed 2026-01-25 05:24:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-battery-cable-with-5-16-ring-terminals-11-0-awg-12-ft-black>: HTTP status code is not handled or not allowed 2026-01-25 05:24:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hang-rail-for-storability-bins-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:24:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leviton-80436-t-6-gang-decora-gfci-device-decora-standard-light-almond>: HTTP status code is not handled or not allowed 2026-01-25 05:24:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-battery-cable-with-3-8-ring-terminals-2-0-awg-5-ft-red>: HTTP status code is not handled or not allowed 2026-01-25 05:24:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/surgical-protective-wear-organizer-10-1-8-w-x-3-15-16-d-x-20-3-8-h-beige>: HTTP status code is not handled or not allowed 2026-01-25 05:24:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mesh-stacking-chair-fabric-black>: HTTP status code is not handled or not allowed 2026-01-25 05:24:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-4-tier-esd-14w-x-24l-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 05:24:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/danger-sign-14x10-aluminum-flammable-material-keep-fire-away>: HTTP status code is not handled or not allowed 2026-01-25 05:24:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-4-tier-esd-36w-x-48l-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 05:24:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/protective-wear-isolation-bundle-15-15-16-w-x-2-15-16-d-x-10-1-8-h-transparent>: HTTP status code is not handled or not allowed 2026-01-25 05:24:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-inch-x-4-inch-x-4-inch-corrugated-boxes>: HTTP status code is not handled or not allowed 2026-01-25 05:24:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-battery-cable-with-3-8-ring-terminals-12-0-awg-12-ft-red>: HTTP status code is not handled or not allowed 2026-01-25 05:24:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smartsolar-charge-controller-mppt-75-15-retail-packaging-blue-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 05:24:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/danger-sign-10x7-rigid-plastic-electrical-hazard>: HTTP status code is not handled or not allowed 2026-01-25 05:24:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-esd-18w-x-30l-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 05:24:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/46633-satin-disc-wheel-1-8-mandrel-1-1-2-aluminum-oxide-coarse>: HTTP status code is not handled or not allowed 2026-01-25 05:24:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ve-bus-smart-dongle-black-abs-plastic>: HTTP status code is not handled or not allowed 2026-01-25 05:24:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vital-cluster-box-unit-w-vogue-classic-accessories-8-unit-2-parcel-lockers-sandstone>: HTTP status code is not handled or not allowed 2026-01-25 05:24:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/39194-bore-polisher-spindle-3-1-8-to-4-aluminum-oxide-fine>: HTTP status code is not handled or not allowed 2026-01-25 05:24:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11890-cartridge-roll-1-2-x-1-5-x-1-8-aluminum-oxide-medium>: HTTP status code is not handled or not allowed 2026-01-25 05:24:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-shelf-cart-esd-3-shelf-24w-x-42l-polyurethane-4-swivel-2-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 05:24:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/notice-sign-10x7-rigid-plastic-keep-door-closed>: HTTP status code is not handled or not allowed 2026-01-25 05:24:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-shelf-truck-esd-3-shelf-21w-x-72l-x-40h-polyurethane-4-swivel-2-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 05:24:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quad-divided-glove-box-dispenser-5-5-16-w-x-5-5-16-d-x-9-h>: HTTP status code is not handled or not allowed 2026-01-25 05:24:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honeywell-pilot-burner-natgas-bcr-18-orifice-front-tip-a-mounting-non-primary>: HTTP status code is not handled or not allowed 2026-01-25 05:24:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/warning-sign-14x10-rigid-plastic-machine-starts-automatically>: HTTP status code is not handled or not allowed 2026-01-25 05:24:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11508-coated-psa-disc-stick-on-12-aluminum-oxide-medium>: HTTP status code is not handled or not allowed 2026-01-25 05:24:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/19828-cartridge-roll-5-8-x-1-5-x-3-16-aluminum-oxide-medium>: HTTP status code is not handled or not allowed 2026-01-25 05:24:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-gallon-sharps-container-w-split-rotor-lid-color-red>: HTTP status code is not handled or not allowed 2026-01-25 05:24:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ls-hollow-core-extension-pole-6-feet>: HTTP status code is not handled or not allowed 2026-01-25 05:24:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/utility-cart-esd-2-shelf-18w-x-24l-polyurethane-4-swivel-2-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 05:24:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bug-blocking-manual-side-seal-door-8w-x-8h-black>: HTTP status code is not handled or not allowed 2026-01-25 05:24:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/82456-psa-disc-5-dia-100-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-25 05:24:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cloth-belt-79196-3-1-2-15-1-2-60-grit-ceramic>: HTTP status code is not handled or not allowed 2026-01-25 05:24:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/training-panel-leg-folding-table-18in-x-96in-walnut-top>: HTTP status code is not handled or not allowed 2026-01-25 05:24:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/material-handling/pallets_pallet_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:24:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-plastic-garbage-can-with-lid-44-gal-white>: HTTP status code is not handled or not allowed 2026-01-25 05:24:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-4-tier-esd-24w-x-48l-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 05:24:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-shelf-cart-esd-2-shelf-21w-x-42l-polyurethane-4-swivel-2-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 05:24:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/individual-character-stencil-12-letter-l>: HTTP status code is not handled or not allowed 2026-01-25 05:24:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mw-50506-sf-display-base-pallet-spot-merchandiser-50w-x-50d-x-6h-black>: HTTP status code is not handled or not allowed 2026-01-25 05:24:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/training-panel-leg-folding-table-24in-x-60in-walnut-top>: HTTP status code is not handled or not allowed 2026-01-25 05:25:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fda-silicone-tubing-116id-x-18od-x-100-ft>: HTTP status code is not handled or not allowed 2026-01-25 05:25:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rechargeable-battery-for-x-talker-series-white-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-25 05:25:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fda-epdm-sanitary-gasket-for-6-tube>: HTTP status code is not handled or not allowed 2026-01-25 05:25:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fore-mp-white-multipurpose-paper-102848-11-x-17-white-500-sheetsream>: HTTP status code is not handled or not allowed 2026-01-25 05:25:04 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/material-handling/pallets_pallet_accessories landed on page that is not a product page. 2026-01-25 05:25:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rust-oleum-v7400-series-340-voc-dtm-alkyd-enamel-silver-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:25:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clean-room-fda-buna-n-sanitary-gasket-for-15-tube>: HTTP status code is not handled or not allowed 2026-01-25 05:25:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelf-truck-esd-24w-x-30l-x-69h-polyurethane-4-swivel-2-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 05:25:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8x1-1-4-bugle-head-drywall-screws-0820CQG>: HTTP status code is not handled or not allowed 2026-01-25 05:25:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-3-tier-esd-21w-x-72l-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 05:25:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 69338 pages (at 97 pages/min), scraped 34510 items (at 37 items/min) 2026-01-25 05:25:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rust-ole-com-cv740-sys-100-voc-dtm-alkyd-ena-rust-prev-mnt-pnt-sfty-rd>: HTTP status code is not handled or not allowed 2026-01-25 05:25:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/paper_stationery/notebooks_pads already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:25:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-office-partition-panel-wpartial-window-pass-thru-cable-48-14wx65-12h-tan>: HTTP status code is not handled or not allowed 2026-01-25 05:25:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-office-partition-panel-with-partial-window-60-14-w-x-73-12-h-tan>: HTTP status code is not handled or not allowed 2026-01-25 05:25:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-office-partition-panel-with-pass-thru-cable-36-14-w-x-65-12-h-tan>: HTTP status code is not handled or not allowed 2026-01-25 05:25:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metal-detectable-silicone-o-ring-dash-228-pack-of-1>: HTTP status code is not handled or not allowed 2026-01-25 05:25:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conductive-silicone-o-ring-dash-228-pack-of-5>: HTTP status code is not handled or not allowed 2026-01-25 05:25:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/overhead_clearance_bars_door_guards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:25:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clearance-bar-kit-7-d-x-80-l-white-bar-red-tapes>: HTTP status code is not handled or not allowed 2026-01-25 05:25:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rust-oleum-5600-sys-100-voc-acryl-urethane-flr-paint-silver-gry-5-gal>: HTTP status code is not handled or not allowed 2026-01-25 05:25:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/office-partition-panel-with-pass-thru-cable-24-14-w-x-46-h--tan>: HTTP status code is not handled or not allowed 2026-01-25 05:25:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buna-n-o-ring-1mm-wide-2-5mm-id-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 05:25:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rust-oleum-9100-system-340voc-dtm-epoxy-mastic-safety-green>: HTTP status code is not handled or not allowed 2026-01-25 05:25:20 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/paper_stationery/notebooks_pads landed on page that is not a product page. 2026-01-25 05:25:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/push-spike-aerator-17-inch-l-x-8-inch-w-x-38-inch-h-black>: HTTP status code is not handled or not allowed 2026-01-25 05:25:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rk-05-small-holder-rack-with-tilt-adjustment>: HTTP status code is not handled or not allowed 2026-01-25 05:25:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-safety-sign-fire-hose-vinyl-4>: HTTP status code is not handled or not allowed 2026-01-25 05:25:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rust-oleum-v7400-series-450-voc-dtm-alkyd-enamel-primer-damp-proof-red>: HTTP status code is not handled or not allowed 2026-01-25 05:25:25 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_railings_machine_guards/overhead_clearance_bars_door_guards landed on page that is not a product page. 2026-01-25 05:25:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rust-oleum-9100-system-340voc-dtm-epoxy-mastic-navy-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:25:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/easel-pads-self-stick-easel-pads-559vad6pk-25-x-30-white-30-sheets-6pack>: HTTP status code is not handled or not allowed 2026-01-25 05:25:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brown-viton-o-ring-dash-115-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-25 05:25:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lockout-tags-do-not-close>: HTTP status code is not handled or not allowed 2026-01-25 05:25:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jtm-1050vs2-3-axis-millwr-w-air-draw-bar>: HTTP status code is not handled or not allowed 2026-01-25 05:25:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8x2-modified-truss-head-self-drilling-screws-0832KPM>: HTTP status code is not handled or not allowed 2026-01-25 05:25:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-16-serrated-flange-lock-nuts-37NR>: HTTP status code is not handled or not allowed 2026-01-25 05:25:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brown-viton-o-ring-1-5mm-wide-8-5mm-id-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-25 05:25:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:25:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lucky-dog-european-style-modular-dog-kennel-72-h-x-120-w-x-120-l-black>: HTTP status code is not handled or not allowed 2026-01-25 05:25:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6x3-8-flat-head-self-drilling-screws-0606KPU410>: HTTP status code is not handled or not allowed 2026-01-25 05:25:36 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/shoe_covers_guards already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:25:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-rubber-overboots-mens-black-size-810-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 05:25:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-rubber-overboots-mens-black-size-810-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 05:25:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lockout-tags-this-tag-lock-to-be-removed-only-by-the-person-shown-on-back>: HTTP status code is not handled or not allowed 2026-01-25 05:25:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-paper-filter-bag-for-cat-c06v-wet-dry-vacuum-641758>: HTTP status code is not handled or not allowed 2026-01-25 05:25:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/envirco-hospi-gard-174-isoclean-174-hepa-filtration-system-freestanding-115v>: HTTP status code is not handled or not allowed 2026-01-25 05:25:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metal-platform-bed-frame-with-steel-slat-support-14-h-queen-size>: HTTP status code is not handled or not allowed 2026-01-25 05:25:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/beds_mattresses/beds_headboards>: HTTP status code is not handled or not allowed 2026-01-25 05:25:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-ton-short-bottle-jack-4412s>: HTTP status code is not handled or not allowed 2026-01-25 05:25:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-composite-air-impact-wrench-6-ext-anvil-1-drive-2200-max-torque>: HTTP status code is not handled or not allowed 2026-01-25 05:25:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/shoe_covers_guards>: HTTP status code is not handled or not allowed 2026-01-25 05:25:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-16-18x3-8-socket-set-screws-cone-point-3106SSNI>: HTTP status code is not handled or not allowed 2026-01-25 05:25:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/soft-silicone-o-ring-dash-116-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-25 05:25:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buna-n-o-ring-dash-016-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 05:25:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10x3-phillips-bugle-head-exterior-wood-deck-screws-18-8-stainless-steel-1048DPG188>: HTTP status code is not handled or not allowed 2026-01-25 05:25:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ingersoll-rand-dust-filter-21-cfm>: HTTP status code is not handled or not allowed 2026-01-25 05:25:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/micropore-stamp-pad-6-1-4-x-3-1-4-black>: HTTP status code is not handled or not allowed 2026-01-25 05:25:53 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:25:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-25-11110-200-psikpa-14-lm-plastic>: HTTP status code is not handled or not allowed 2026-01-25 05:25:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premier-stainless-steel-solid-door-laboratory-refrigerator-72-cu-ft-abt-hc-ssp-72>: HTTP status code is not handled or not allowed 2026-01-25 05:25:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/traffic_parking_lot_safety/temporary_pavement_markers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:25:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-inking-heavy-duty-stamps>: HTTP status code is not handled or not allowed 2026-01-25 05:25:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lux-lspp2b-o3x-ls-wick-b-eye-polo-org-3x>: HTTP status code is not handled or not allowed 2026-01-25 05:25:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-25-11112-160-psikpa-14-cbm-plastic>: HTTP status code is not handled or not allowed 2026-01-25 05:26:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bar-storage-rack-starter-single-sided>: HTTP status code is not handled or not allowed 2026-01-25 05:26:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/800-lumen-rechargeable-led-work-light>: HTTP status code is not handled or not allowed 2026-01-25 05:26:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pencils-hb-2-black-lead-yellow-barrel-72-box>: HTTP status code is not handled or not allowed 2026-01-25 05:26:03 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges landed on page that is not a product page. 2026-01-25 05:26:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-20x3-4x1-wing-nuts-1412NWA>: HTTP status code is not handled or not allowed 2026-01-25 05:26:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-2-11112-15-psi-14-cbm-plastic>: HTTP status code is not handled or not allowed 2026-01-25 05:26:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/refill-for-pilot-frixion-pens-fine-point-purple-pink-turquoise-3-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:26:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/traffic_parking_lot_safety/temporary_pavement_markers landed on page that is not a product page. 2026-01-25 05:26:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-16x3-4-hex-tap-bolts-grade-5-3712BHT5>: HTTP status code is not handled or not allowed 2026-01-25 05:26:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-25-11110-30inhg60psi-14-lm-plastic>: HTTP status code is not handled or not allowed 2026-01-25 05:26:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/flashlights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:26:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 69427 pages (at 89 pages/min), scraped 34541 items (at 31 items/min) 2026-01-25 05:26:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/recycled-open-plan-landscape-wall-calendar-12-x-12-2015-aag88200>: HTTP status code is not handled or not allowed 2026-01-25 05:26:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:26:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glow-nyc-door-mount-sign-le>: HTTP status code is not handled or not allowed 2026-01-25 05:26:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ez-breeze-20-white-115v-1-10hp-1ph-1sp-w-bracket-cord>: HTTP status code is not handled or not allowed 2026-01-25 05:26:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glow-sign-vinyl-do-not-use-elevators>: HTTP status code is not handled or not allowed 2026-01-25 05:26:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pilot-light-operator-22mm-red>: HTTP status code is not handled or not allowed 2026-01-25 05:26:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/easy-peel-laser-mailing-labels-2-x-4-clear-500-box-5663>: HTTP status code is not handled or not allowed 2026-01-25 05:26:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-stick-ballpoint-counter-pen-medium-1mm-black-ink-barrel-6-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:26:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/pencils already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:26:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-square-trash-can-waste-lid-36-gal-black>: HTTP status code is not handled or not allowed 2026-01-25 05:26:18 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/flashlights landed on page that is not a product page. 2026-01-25 05:26:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6003nr-open-wsnap-ring-17mm-bore-35mm-od>: HTTP status code is not handled or not allowed 2026-01-25 05:26:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/folding-bed-wedge-10>: HTTP status code is not handled or not allowed 2026-01-25 05:26:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-20-thin-weld-nuts-with-tab-base-14NWS2>: HTTP status code is not handled or not allowed 2026-01-25 05:26:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-hi-polymer-lead-refills-0-5-mm-h-black-12-tube>: HTTP status code is not handled or not allowed 2026-01-25 05:26:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oil-only-heavy-weight-sorbent-pads-White>: HTTP status code is not handled or not allowed 2026-01-25 05:26:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glow-tape-1-x100-1>: HTTP status code is not handled or not allowed 2026-01-25 05:26:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-truck-tire-inflator-gauge-10-to-160-psi-7ft-hose-550>: HTTP status code is not handled or not allowed 2026-01-25 05:26:24 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/tools/testing_inspection_instruments/pressure_vacuum_gauges/dial_pressure_vacuum_gauges landed on page that is not a product page. 2026-01-25 05:26:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4x1-1-4-split-drive-anchors-1420ADSPR>: HTTP status code is not handled or not allowed 2026-01-25 05:26:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/print-to-the-edge-easy-peel-labels-with-trueblock-1-1-2-x-1-1-2-white-600-pk-22805>: HTTP status code is not handled or not allowed 2026-01-25 05:26:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/print-or-write-removable-multi-use-labels-3-4-x-1-1-2-white-504-pack-05430>: HTTP status code is not handled or not allowed 2026-01-25 05:26:27 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/pencils landed on page that is not a product page. 2026-01-25 05:26:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-16-expansion-anchors-31AED>: HTTP status code is not handled or not allowed 2026-01-25 05:26:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-125-internal-retaining-ring-212RIBP>: HTTP status code is not handled or not allowed 2026-01-25 05:26:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-40x1-4-hex-jackscrews-100404JM>: HTTP status code is not handled or not allowed 2026-01-25 05:26:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n3c-r-edge-series-light-diffuser-focuser>: HTTP status code is not handled or not allowed 2026-01-25 05:26:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/fans_blowers/cooling_fans/home_office_fans already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:26:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/repositionable-shipping-labels-for-laser-printers-2-x-4-white-250-box-58163>: HTTP status code is not handled or not allowed 2026-01-25 05:26:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6307zz-double-shielded-35mm-bore-80mm-od>: HTTP status code is not handled or not allowed 2026-01-25 05:26:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14x1-round-spacers-501614RSN>: HTTP status code is not handled or not allowed 2026-01-25 05:26:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/automatic-label-applicator-for-square-rectangular-items-up-to-8-w-boxmate-610>: HTTP status code is not handled or not allowed 2026-01-25 05:26:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-clearview-stock-truck-48x30-4-adjustable-2-fixed-shelves>: HTTP status code is not handled or not allowed 2026-01-25 05:26:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-pocket-verticle-literature-rack-black>: HTTP status code is not handled or not allowed 2026-01-25 05:26:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keystock-rod-rack>: HTTP status code is not handled or not allowed 2026-01-25 05:26:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-16x7-16-tee-nuts-3707NT3>: HTTP status code is not handled or not allowed 2026-01-25 05:26:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-16-dia-x-12-working-length-j-hook-style-a>: HTTP status code is not handled or not allowed 2026-01-25 05:26:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/hvac/fans_blowers/cooling_fans/home_office_fans>: HTTP status code is not handled or not allowed 2026-01-25 05:26:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-daily-whiteboard-cleaner-32-oz-trigger-spray-6-pack-pl9869-6>: HTTP status code is not handled or not allowed 2026-01-25 05:26:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tall-isolation-cart-3-drawers-standard-key-lock-37-1-2-wx22-lx42-63-100-h-purple>: HTTP status code is not handled or not allowed 2026-01-25 05:26:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/upright-freezer-4-2-cu-ft-white-duf480we>: HTTP status code is not handled or not allowed 2026-01-25 05:26:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-visibility-safety-beanie-hat-with-reflective-woven-stripe-lime-one-size>: HTTP status code is not handled or not allowed 2026-01-25 05:26:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/catheter-cart-with-basket-chrome-2-casters-w-brakes-24l-x-48w-x-68h>: HTTP status code is not handled or not allowed 2026-01-25 05:26:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/44-pocket-rotary-literature-rack-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:26:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-grid-panel-poly-green-24-x-72>: HTTP status code is not handled or not allowed 2026-01-25 05:26:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6908-2rs-double-sealed-40mm-bore-62mm-od>: HTTP status code is not handled or not allowed 2026-01-25 05:26:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dra-ing-stool-fabric-black>: HTTP status code is not handled or not allowed 2026-01-25 05:26:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rmt-545-diamond-wheel-for-dremel>: HTTP status code is not handled or not allowed 2026-01-25 05:26:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-bases-9-deep-parts-bins>: HTTP status code is not handled or not allowed 2026-01-25 05:26:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_microscopes/digital_microscopes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:26:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vp300-main-sack-filter-1471432500>: HTTP status code is not handled or not allowed 2026-01-25 05:26:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-tall-emergency-crash-cart-6-drawers-36-3-4-w-x-26-l-x-43-h-forest-green>: HTTP status code is not handled or not allowed 2026-01-25 05:26:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/bins_totes_containers/organizers_caddies/small_parts_organizers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:26:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-size-stacking-bin-23-7-8-x-8-1-4-x-7-red>: HTTP status code is not handled or not allowed 2026-01-25 05:26:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-series-narrow-medical-cart-6-drawers-e-lock-24-3-4-wx22-lx43-3-4-h-purple>: HTTP status code is not handled or not allowed 2026-01-25 05:26:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-120-economy-h-beam-adjustable-length-yellow-126-l-x-40-w-x-36-h-915lbs-8000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 05:26:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6215zznr-double-shielded-wsnap-ring-75mm-bore-130mm-od>: HTTP status code is not handled or not allowed 2026-01-25 05:26:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-fixed-spreader-beam-yellow-100000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 05:26:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s163-ansi-class-3-long-raincoat-hi-vis-orange-lg-62036>: HTTP status code is not handled or not allowed 2026-01-25 05:27:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-dia-x-12-working-length-j-hook-style-a>: HTTP status code is not handled or not allowed 2026-01-25 05:27:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dot-placard-four-digit-3257-1>: HTTP status code is not handled or not allowed 2026-01-25 05:27:02 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/medical-lab/lab_equipment_instruments/laboratory_microscopes/digital_microscopes landed on page that is not a product page. 2026-01-25 05:27:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/61335-s762p-non-ansi-vest-tricot-hook-loop-closure-hi-vis-pink-3xl>: HTTP status code is not handled or not allowed 2026-01-25 05:27:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-duty-nylon-glove-polyurethane-coated-black-33-b125-xl>: HTTP status code is not handled or not allowed 2026-01-25 05:27:05 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/storage/bins_totes_containers/organizers_caddies/small_parts_organizers landed on page that is not a product page. 2026-01-25 05:27:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triple-column-storage-cabinet-electronic-keypad-lock-60-w-x-28-d-x-81-h-hammertone-red>: HTTP status code is not handled or not allowed 2026-01-25 05:27:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/i-fit-flex-safety-glasses-apple-green-frame-clear-lens-16266>: HTTP status code is not handled or not allowed 2026-01-25 05:27:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stretch-wrap-film-18-1500-80-hand-film>: HTTP status code is not handled or not allowed 2026-01-25 05:27:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6808-open-40mm-bore-52mm-od>: HTTP status code is not handled or not allowed 2026-01-25 05:27:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 69519 pages (at 92 pages/min), scraped 34569 items (at 28 items/min) 2026-01-25 05:27:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-series-aluminium-medical-cart-tall-height-36-3-4-w-x-22-d-x-43-3-4-h-taupe>: HTTP status code is not handled or not allowed 2026-01-25 05:27:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/refrigerated-merchandiser-single-section-upright-full-glass-door>: HTTP status code is not handled or not allowed 2026-01-25 05:27:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/19390-americana-full-brim-hard-hat-w-accy-slots-standard-4-pt-suspension-hi-viz-lime>: HTTP status code is not handled or not allowed 2026-01-25 05:27:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n-ferno-6466-thermal-jacket-black-xl-41115>: HTTP status code is not handled or not allowed 2026-01-25 05:27:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-8-x-6-working-length-j-hook-style-9>: HTTP status code is not handled or not allowed 2026-01-25 05:27:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/61814-s383p-class-2-mesh-vest-hi-vis-lime-contrasting-trim-sm>: HTTP status code is not handled or not allowed 2026-01-25 05:27:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-16012-open-60mm-bore-95mm-od>: HTTP status code is not handled or not allowed 2026-01-25 05:27:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/61816-s383p-class-2-mesh-vest-hi-vis-lime-contrasting-trim-lg>: HTTP status code is not handled or not allowed 2026-01-25 05:27:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-72-spread-adjustable-lift-yellow-15000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 05:27:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/19292-americana-full-brim-hard-hat-w-accessory-slots-standard-4-pt-suspension-yellow>: HTTP status code is not handled or not allowed 2026-01-25 05:27:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/19291-americana-full-brim-hard-hat-w-accessory-slots-standard-4-pt-suspension-white>: HTTP status code is not handled or not allowed 2026-01-25 05:27:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6024-2rs-double-sealed-120mm-bore-180mm-od>: HTTP status code is not handled or not allowed 2026-01-25 05:27:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/squids-3109fx-double-locking-single-carabiner-w-swivel-25-lbs-capacity-19809>: HTTP status code is not handled or not allowed 2026-01-25 05:27:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dot-placard-oxidizer-5-1-4>: HTTP status code is not handled or not allowed 2026-01-25 05:27:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-dia-x-9-working-length-j-hook-style-a>: HTTP status code is not handled or not allowed 2026-01-25 05:27:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6815zz-double-shielded-75mm-bore-95mm-od>: HTTP status code is not handled or not allowed 2026-01-25 05:27:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tall-savary-dilator-drying-cart-hepa-filter-23-89-100-wx22-lx51-89-100-h-cream>: HTTP status code is not handled or not allowed 2026-01-25 05:27:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6310zznr-double-shielded-wsnap-ring-50mm-bore-110mm-od>: HTTP status code is not handled or not allowed 2026-01-25 05:27:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-72-economy-h-beam-adjustable-length-yellow-78-l-x-40-w-x-36-h-285lbs-4000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 05:27:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-tall-emergency-crash-cart-6-drawers-36-3-4-w-x-22-l-x-43-h-navy>: HTTP status code is not handled or not allowed 2026-01-25 05:27:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6221nr-open-wsnap-ring-105mm-bore-190mm-od>: HTTP status code is not handled or not allowed 2026-01-25 05:27:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-min-twist-timer-w-metal-wallplate>: HTTP status code is not handled or not allowed 2026-01-25 05:27:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/120v-spdt-20a-24-hr-duty-cycle>: HTTP status code is not handled or not allowed 2026-01-25 05:27:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-tall-emergency-crash-cart-6-drawers-36-3-4-w-x-22-l-x-43-h-taupe>: HTTP status code is not handled or not allowed 2026-01-25 05:27:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6309-2rsnr-double-sealed-wsnap-ring-45mm-bore-100mm-od>: HTTP status code is not handled or not allowed 2026-01-25 05:27:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-2-shelf-container-rack-bk183052n-30w-x-18d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 05:27:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6834m-open-170mm-bore-215mm-od>: HTTP status code is not handled or not allowed 2026-01-25 05:27:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/relief-valve-for-jet-and-wilton-band-saws>: HTTP status code is not handled or not allowed 2026-01-25 05:27:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6312-2rs-double-sealed-60mm-bore-130mm-od>: HTTP status code is not handled or not allowed 2026-01-25 05:27:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-series-aluminium-isolation-cart-6-drawers-30-w-x-22-l-x-43-3-4-h-hammertone-red>: HTTP status code is not handled or not allowed 2026-01-25 05:27:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hand-crank-operated-mechanical-post-lift-table-30-x-48-cast-iron-wheels>: HTTP status code is not handled or not allowed 2026-01-25 05:27:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/semi-trash-water-pump-with-g-force-6919>: HTTP status code is not handled or not allowed 2026-01-25 05:27:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/strobe-120vac-amber-vals-120a>: HTTP status code is not handled or not allowed 2026-01-25 05:27:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drylin-r-open-linear-polymer-bearing-with-shell-1-2-dia-shaft>: HTTP status code is not handled or not allowed 2026-01-25 05:27:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triple-column-storage-cabinet-electronic-keypad-lock-60-w-x-28-d-x-81-h-beige>: HTTP status code is not handled or not allowed 2026-01-25 05:27:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/strobe-120vac-red-lp1-120r>: HTTP status code is not handled or not allowed 2026-01-25 05:27:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/boss-reception-return-shell-42w-x-24d-mahogany>: HTTP status code is not handled or not allowed 2026-01-25 05:27:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-container-foam-lined-855-20-fl-22l-x-20w-x-20h-black>: HTTP status code is not handled or not allowed 2026-01-25 05:27:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cast-acrylic-plastic-sheet-116-thick-x-24w-x-24l>: HTTP status code is not handled or not allowed 2026-01-25 05:27:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ptfe-plastic-rod-1-dia-x-3l>: HTTP status code is not handled or not allowed 2026-01-25 05:27:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_cabinets_pedestals already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:27:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/boss-36-x-24-reversible-return-mahogany>: HTTP status code is not handled or not allowed 2026-01-25 05:27:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/warm-20-68f-3-window-time-temperature-indicators-100-box>: HTTP status code is not handled or not allowed 2026-01-25 05:27:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tall-isolation-cart-3-drawers-standard-key-lock-37-1-2-wx22-lx42-63-100-h-hammertone-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:27:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cell-lifter-double-end-flat-blade-narrow-blade-sterile-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 05:27:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/engraved-occupancy-sign-use-vacant-brown>: HTTP status code is not handled or not allowed 2026-01-25 05:27:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-wall-mount-shelf-with-1-5-lip-18-gauge-430-stainless-steel-48-x-12>: HTTP status code is not handled or not allowed 2026-01-25 05:27:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-rack-tear-drop-backstop-144-l-0-offset>: HTTP status code is not handled or not allowed 2026-01-25 05:27:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lopro-series-96-wide-door-unheated-208-230160-obsidian-black>: HTTP status code is not handled or not allowed 2026-01-25 05:27:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/desks_workstations/desk_components_accessories/desk_cabinets_pedestals landed on page that is not a product page. 2026-01-25 05:27:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/250ml-bottle-top-filter-pes-filter-material-0-45-181-m-75mm-sterile-24-pk>: HTTP status code is not handled or not allowed 2026-01-25 05:27:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-24-nsert-for-hard-wood-brass-400-624>: HTTP status code is not handled or not allowed 2026-01-25 05:27:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/400ml-graduated-beaker-polypropylene-non-sterile-50-pk>: HTTP status code is not handled or not allowed 2026-01-25 05:27:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15ml-centrifuge-tube-black-bag-sterile-polypropylene-500-pk>: HTTP status code is not handled or not allowed 2026-01-25 05:27:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/515-tank-heavy-duty-shipping-case-tube-without-wheels-10l-x-10w-x-38h-blk>: HTTP status code is not handled or not allowed 2026-01-25 05:27:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/engraved-occupancy-sign-open-closed-red>: HTTP status code is not handled or not allowed 2026-01-25 05:28:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ec-6w115-unit-bearing-1-125-hp-1550-rpm-115v>: HTTP status code is not handled or not allowed 2026-01-25 05:28:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/telescoping-case-276-carrying-case-with-wheels-46l-x-22w-x-14h-black>: HTTP status code is not handled or not allowed 2026-01-25 05:28:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/selftapping-concrete-screw-hex-head-304-stainless-steel-14-x-114>: HTTP status code is not handled or not allowed 2026-01-25 05:28:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/architectural-sign-use-other-door>: HTTP status code is not handled or not allowed 2026-01-25 05:28:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/d0746-oem-motor-for-penn-vent-1-12-hp-1550-1300-rpm-115v-open>: HTTP status code is not handled or not allowed 2026-01-25 05:28:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1161209-comfort-construction-style-climbing-harness-pass-through-s>: HTTP status code is not handled or not allowed 2026-01-25 05:28:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/train-horn-three-black-abs-trumpet-and-12-volt-solenoid-with-on-board-air-system-having-a-25-gallon-tank-847-858>: HTTP status code is not handled or not allowed 2026-01-25 05:28:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/500ml-bottle-top-filter-nylon-filter-material-0-45-181-m-90mm-sterile-24-pk>: HTTP status code is not handled or not allowed 2026-01-25 05:28:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 69619 pages (at 100 pages/min), scraped 34606 items (at 37 items/min) 2026-01-25 05:28:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/275-omni-telescoping-case-trade-show-case-46l-x-42w-x-8h-black>: HTTP status code is not handled or not allowed 2026-01-25 05:28:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/half-height-double-column-med-storage-cabinet-keylock-40-1-10-wx29-1-4-dx40-3-4-h-mauve>: HTTP status code is not handled or not allowed 2026-01-25 05:28:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/emergency_vehicle_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:28:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-32-tapered-ultrasonic-insert-double-vane-dv-008-th>: HTTP status code is not handled or not allowed 2026-01-25 05:28:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/suredry-high-volume-16-scope-cabinet-dri-scope-aid-key-lock-44-wx24-dx93-h-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:28:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_repair_maintenance/horns_horn_accessories already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:28:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/suredry-high-volume-16-scope-cabinet-dri-scope-aid-key-lock-44-wx24-dx93-h-mauve>: HTTP status code is not handled or not allowed 2026-01-25 05:28:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/13460-dynisher-finishing-tool-versatility-kit-1hp-right-angle-2800-rpm-rear-exhaust>: HTTP status code is not handled or not allowed 2026-01-25 05:28:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-20-flanged-press-insert-brass-260-4-br>: HTTP status code is not handled or not allowed 2026-01-25 05:28:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classic-soft-precut-gauntlet-thumb-post-splint-1-16-micro-perforated-small>: HTTP status code is not handled or not allowed 2026-01-25 05:28:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vertical-paper-dispenser-w-cutter-36-w-x-9-dia-roll-size>: HTTP status code is not handled or not allowed 2026-01-25 05:28:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polly-products-monarque-6-ft-backed-bench-with-arms-brown-bench-brown-frame>: HTTP status code is not handled or not allowed 2026-01-25 05:28:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zinc-die-cast-adjustable-lever-steel-10-24-63-stud-177l>: HTTP status code is not handled or not allowed 2026-01-25 05:28:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/emergency_vehicle_lights>: HTTP status code is not handled or not allowed 2026-01-25 05:28:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a48h3612sslp-continuous-hinge-wclamps-type-4x-4800x3600x1200in-ss-type-304>: HTTP status code is not handled or not allowed 2026-01-25 05:28:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_repair_maintenance/horns_horn_accessories>: HTTP status code is not handled or not allowed 2026-01-25 05:28:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tele-pro-davit-crane-1200-lb-capcity-w-manual-winch-and-brake>: HTTP status code is not handled or not allowed 2026-01-25 05:28:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylon-plastic-adjustable-lever-steel-38-16-tapped-307l>: HTTP status code is not handled or not allowed 2026-01-25 05:28:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylon-plastic-adjustable-lever-steel-516-18-78-stud-248l>: HTTP status code is not handled or not allowed 2026-01-25 05:28:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbi-sala-lad-saf-20-ft-swaged-stainless-steel-cable-3-8-dia-1-7-strand-6105020>: HTTP status code is not handled or not allowed 2026-01-25 05:28:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbi-sala-lad-saf-80-ft-vertical-safety-system-galvanized-steel-cable-6118080>: HTTP status code is not handled or not allowed 2026-01-25 05:28:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machine-labels-notice-cylinders-must-be-chained-at-all-times>: HTTP status code is not handled or not allowed 2026-01-25 05:28:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/surface-mounted-end-wall-90-degrees-corners-2-ft-ft-wings-4-ft-h-w-caps-6-inch-w-chablis-recycled-vinyl>: HTTP status code is not handled or not allowed 2026-01-25 05:28:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/siesta-sunshine-resin-dining-arm-chair-white>: HTTP status code is not handled or not allowed 2026-01-25 05:28:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/la-vertical-lifting-180-degree-turning-plate-clamp-steel-2000-lbs-cap-3-4-opening>: HTTP status code is not handled or not allowed 2026-01-25 05:28:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/think-osha-10x14-vinyl-safety-first>: HTTP status code is not handled or not allowed 2026-01-25 05:28:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-glass-door-gasket-for-nexel-model-243004>: HTTP status code is not handled or not allowed 2026-01-25 05:28:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/csd303012ss-conceptwall-mt-encl-type-4x-3000x3000x1200in-ss-type-304>: HTTP status code is not handled or not allowed 2026-01-25 05:28:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pur-fill-1g-600-cold-weather-gun-foam-600ml-cw01>: HTTP status code is not handled or not allowed 2026-01-25 05:28:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pictorial-osha-sign-plastic-first-aid>: HTTP status code is not handled or not allowed 2026-01-25 05:28:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-3-pc-ss-npt-ball-valve-w-sr-pneumatic>: HTTP status code is not handled or not allowed 2026-01-25 05:28:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/open-plan-office-desk-60-w-x-30-d-x-29-h-gray-top-with-black-legs>: HTTP status code is not handled or not allowed 2026-01-25 05:28:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/restricted-area-aluminum-bilingual-authorized-employees-only>: HTTP status code is not handled or not allowed 2026-01-25 05:28:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-3-pc-ss-npt-ball-valve-w-nema-4-115vac>: HTTP status code is not handled or not allowed 2026-01-25 05:28:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-3-way-l-port-brass-npt-ball-valve-w-sr-pneumatic>: HTTP status code is not handled or not allowed 2026-01-25 05:28:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unisex-esdanti-static-tech-coat-white-polyesternylon-xl>: HTTP status code is not handled or not allowed 2026-01-25 05:28:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/narrow-coil-c-hook-7000-lbs-capacity-yellow-12-max-coil-width>: HTTP status code is not handled or not allowed 2026-01-25 05:28:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-sheet-18-thick-x-16-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 05:28:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uhmw-polyethylene-plastic-bar-18-thick-x-5-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 05:28:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/semi-clear-fda-silicone-rubber-sheet-no-adhesive-40a-116-thick-x-36-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 05:28:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-4in-basic-fixed-flat-ball-mount-black-1803100cr>: HTTP status code is not handled or not allowed 2026-01-25 05:28:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/neoprene-foam-sheet-no-adhesive-316-thick-x-36-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 05:28:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:28:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jumbo-roll-bath-tissue-white-1000-roll-12-rolls-case>: HTTP status code is not handled or not allowed 2026-01-25 05:28:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-boot-for-7-way-connectors-tc1007b>: HTTP status code is not handled or not allowed 2026-01-25 05:28:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:28:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-sheet-18-thick-x-12-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 05:28:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-light-13-semi-flush-alabaster-glass-brushed-nickel>: HTTP status code is not handled or not allowed 2026-01-25 05:28:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-lab-coat-light-blue-polycombed-cotton-2xl>: HTTP status code is not handled or not allowed 2026-01-25 05:28:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brush-set-for-w-40-xl-w-40-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 05:28:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/door-frame-guard-3-wing-4l-white>: HTTP status code is not handled or not allowed 2026-01-25 05:28:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-long-sleeve-work-shirt-fluorescent-yellowgreen-tall-2xl>: HTTP status code is not handled or not allowed 2026-01-25 05:29:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/battery-connector-for-8300-8400-8500-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 05:29:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20in-burnisher-pad-low-freq-medium-to-hard-finish-5-per-case>: HTTP status code is not handled or not allowed 2026-01-25 05:29:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits>: HTTP status code is not handled or not allowed 2026-01-25 05:29:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x36x74-chrome-wire-shelving-with-64-6-h-shelf-bins-red>: HTTP status code is not handled or not allowed 2026-01-25 05:29:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pin-for-mp-mpb-040-ac-pallet-trucks-yl-641786101>: HTTP status code is not handled or not allowed 2026-01-25 05:29:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/connector-handle-for-pe-4000-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 05:29:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits>: HTTP status code is not handled or not allowed 2026-01-25 05:29:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14x36x74-chrome-wire-shelving-with-28-giant-stacking-bins-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:29:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/american-lock-high-security-solid-aluminum-padlock-5-pin-cylinders-red>: HTTP status code is not handled or not allowed 2026-01-25 05:29:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x36x74-chrome-wire-shelving-with-21-giant-stacking-bins-yellow>: HTTP status code is not handled or not allowed 2026-01-25 05:29:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-built-in-combination-deadbolt-lock-purple-dial-right-hinged>: HTTP status code is not handled or not allowed 2026-01-25 05:29:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bhg06xxl-box-handler-gloves-1-pair-black-2x-large>: HTTP status code is not handled or not allowed 2026-01-25 05:29:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x36x74-chrome-wire-shelving-with-15-giant-stacking-bins-green>: HTTP status code is not handled or not allowed 2026-01-25 05:29:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-general-security-combo-padlock-lh-shackle-green-dial>: HTTP status code is not handled or not allowed 2026-01-25 05:29:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x36x74-chrome-wire-shelving-with-20-giant-stacking-bins-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:29:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 69716 pages (at 97 pages/min), scraped 34639 items (at 33 items/min) 2026-01-25 05:29:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washable-school-glue-stick>: HTTP status code is not handled or not allowed 2026-01-25 05:29:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-bar-3-thick-x-6-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 05:29:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hanging-locker-shelf-black-6-case>: HTTP status code is not handled or not allowed 2026-01-25 05:29:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/office-konnect-153-stackable-desk-organizer-4-piece-white-6-case-white>: HTTP status code is not handled or not allowed 2026-01-25 05:29:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-assembly-for-wpt60-pallet-trucks-bj-1280-130011-00-xl-a>: HTTP status code is not handled or not allowed 2026-01-25 05:29:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-built-in-combination-lock-with-metal-dial-left-hinged>: HTTP status code is not handled or not allowed 2026-01-25 05:29:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/boost-x-12v-2500a-ultrasafe-lithium-jump-starter>: HTTP status code is not handled or not allowed 2026-01-25 05:29:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bowl-sqr-base-16-7oz-slate-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:29:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:29:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-ner-electric-chain-hoist-motor-trolley-1-10-14>: HTTP status code is not handled or not allowed 2026-01-25 05:29:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-116-thick-x-1-12-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 05:29:21 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:29:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-ner-dual-speed-electric-chain-hoist-18-20-559>: HTTP status code is not handled or not allowed 2026-01-25 05:29:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/security-business-checks-with-marble-top-8-1-2-inch-x-11-inch-green-500-sheets-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:29:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/koch-filter-merv-11-high-capacity-xl11-pleated-panel-extended-surface-12-inch-w-x-24-inch-h-x-1-inch-d>: HTTP status code is not handled or not allowed 2026-01-25 05:29:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rp-351-battery-kit-for-rp-50-rp51-press-tool-18v-li-ion>: HTTP status code is not handled or not allowed 2026-01-25 05:29:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-ner-electric-chain-hoist-hook-suspension-2-12-20-22>: HTTP status code is not handled or not allowed 2026-01-25 05:29:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camcover-versa-6-1-2-antique-parchment>: HTTP status code is not handled or not allowed 2026-01-25 05:29:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits>: HTTP status code is not handled or not allowed 2026-01-25 05:29:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ptfe-plastic-rod-38-diameter-x-4-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 05:29:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kids-sensory-table-clear-p1108-48l-x-16w-x-6h>: HTTP status code is not handled or not allowed 2026-01-25 05:29:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 05:29:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-132-thick-x-1-12-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 05:29:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-bank-top-chest-w-7-drawers-27-w-x-22-3-8-d-x-27-2-3-h-tan>: HTTP status code is not handled or not allowed 2026-01-25 05:29:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plate-dw-3comp-9-white>: HTTP status code is not handled or not allowed 2026-01-25 05:29:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kl-lever-hoists-2200-lb-capacity-15-ft-lift>: HTTP status code is not handled or not allowed 2026-01-25 05:29:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buna-n-rubber-strip-no-adhesive-60a-132-thick-x-38-wide-x-10-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 05:29:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slb-lever-hoists-2200-lb-capacity-5-ft-lift-14-38-head-room>: HTTP status code is not handled or not allowed 2026-01-25 05:29:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storage-tote-yellow-t142-19l-x-12-1-2w-x-4-3-8h>: HTTP status code is not handled or not allowed 2026-01-25 05:29:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/composites/composite_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:29:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storage-tote-red-t142-19l-x-12-1-2w-x-4-3-8h>: HTTP status code is not handled or not allowed 2026-01-25 05:29:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pivot-shaft-for-pe-3000-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 05:29:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/550s-top-chest-w-12-drawers-50-w-x-25-1-4-d-x-27-h-red-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:29:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-double-arm-fixed-shelf-bracket-chrome>: HTTP status code is not handled or not allowed 2026-01-25 05:29:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shaft-for-gpw-series-pallet-trucks-cr-042214>: HTTP status code is not handled or not allowed 2026-01-25 05:29:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/top-chest-w-5-drawers-34-w-x-25-1-4-d-x-27-h-red>: HTTP status code is not handled or not allowed 2026-01-25 05:29:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-sner-electric-chain-hoist-hook-suspension-14-10-14>: HTTP status code is not handled or not allowed 2026-01-25 05:29:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-ner-electric-chain-hoist-motor-trolley-3-10-17>: HTTP status code is not handled or not allowed 2026-01-25 05:29:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-sheet-34-thick-x-12-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 05:29:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12u-linier-fixed-wall-mount-cabinet-vented-door>: HTTP status code is not handled or not allowed 2026-01-25 05:29:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-uhmw-polyethylene-plastic-bar-3-thick-x-4-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 05:29:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/composites/composite_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 05:29:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-flat-polyurethane-coated-gloves-white-large-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 05:29:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sos-outdoor-storage-container-42-x-29-x-30-11-cu-ft-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:29:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72x36-maple-square-edge-mobile-power-apron-production-bench-tan>: HTTP status code is not handled or not allowed 2026-01-25 05:29:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-salvage-drum-20-gallon>: HTTP status code is not handled or not allowed 2026-01-25 05:29:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nps-8ft-mobile-cafeteria-fixed-bench-unit-with-mdf-core-top-protect-edge-walnut>: HTTP status code is not handled or not allowed 2026-01-25 05:29:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-for-7400-reach-pallet-trucks-ra-632-052-008>: HTTP status code is not handled or not allowed 2026-01-25 05:29:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gripper-front-butcher-coat-wexterior-top-pocket-spun-polyester-white-l>: HTTP status code is not handled or not allowed 2026-01-25 05:29:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cast-acrylic-plastic-sheet-18-thick-x-48-wide-x-72-long>: HTTP status code is not handled or not allowed 2026-01-25 05:29:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gripper-front-butcher-coat-wexterior-top-pocket-spun-polyester-white-xs>: HTTP status code is not handled or not allowed 2026-01-25 05:29:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/solar-power-ai-smart-1600-lumens-cree-led-parking-light>: HTTP status code is not handled or not allowed 2026-01-25 05:29:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-rectangular-picnic-table-w-seat-cushions-expanded-metal-black>: HTTP status code is not handled or not allowed 2026-01-25 05:29:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bushing-for-mpb-040-e-b827-mpw-045-e-b802-pallet-trucks-yl-524149643>: HTTP status code is not handled or not allowed 2026-01-25 05:30:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/surface-bumper-guard-type-c-yellow-black>: HTTP status code is not handled or not allowed 2026-01-25 05:30:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tandem-load-wheel-assembly-for-pe-4500-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 05:30:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3800-series-single-truck-bus-v-bar-tire-chains-pair-0382955>: HTTP status code is not handled or not allowed 2026-01-25 05:30:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-w-lse-acrylic-adhesive-12-thick-x-4-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 05:30:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-5203-double-row-angular-contact-bearing-open-17mm-bore-x-40mm-od-x-17-5mm-w>: HTTP status code is not handled or not allowed 2026-01-25 05:30:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-bar-3-thick-x-4-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 05:30:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12x36x74-chrome-wire-shelving-with-77-4-h-shelf-bins-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:30:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/maxtrac-snow-blower-garden-tractor-tire-chains-2-link-spacing-pair-1093956>: HTTP status code is not handled or not allowed 2026-01-25 05:30:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-assembly-for-pe-3000-pallet-trucks-cr-154840-301-88>: HTTP status code is not handled or not allowed 2026-01-25 05:30:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/biohazard-specimen-transport-bag-6-x-9-zipper-score-line-document-pouch-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:30:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 69811 pages (at 95 pages/min), scraped 34673 items (at 34 items/min) 2026-01-25 05:30:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x36x74-chrome-wire-shelving-with-55-4-h-shelf-bins-ivory>: HTTP status code is not handled or not allowed 2026-01-25 05:30:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/56-quart-53-liter14988004-wheeled-clear-storage-tote-white-lid-blue-latches-33-7-8-x-18-3-4-x-7>: HTTP status code is not handled or not allowed 2026-01-25 05:30:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ballymore-8-step-perforated-cantilever-ladder-42-overhang>: HTTP status code is not handled or not allowed 2026-01-25 05:30:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/optional-type-xwl-grooved-toe-plate-for-liftkar-sal-stair-climbing-trucks>: HTTP status code is not handled or not allowed 2026-01-25 05:30:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-assembly-for-mpw-060len-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 05:30:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ballymore-12-step-serrated-cantilever-ladder-28-overhang>: HTTP status code is not handled or not allowed 2026-01-25 05:30:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/neckerchief-red-polyestercotton-40-x-20>: HTTP status code is not handled or not allowed 2026-01-25 05:30:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rcl-7000is-2150ir>: HTTP status code is not handled or not allowed 2026-01-25 05:30:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:30:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/head-harness-for-102xm-walkie-pallet-truck>: HTTP status code is not handled or not allowed 2026-01-25 05:30:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-x-14-red-13-oz-flame-retardant-vinyl>: HTTP status code is not handled or not allowed 2026-01-25 05:30:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/collarless-butcher-wrap-wexterior-pockets-white-polyestercombed-cotton-4xl>: HTTP status code is not handled or not allowed 2026-01-25 05:30:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-round-counter-height-breakroom-table-black>: HTTP status code is not handled or not allowed 2026-01-25 05:30:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stinger-berm-spill-containment-puncture-resistant-6-l-x-6-w-x-12-h>: HTTP status code is not handled or not allowed 2026-01-25 05:30:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-sheet-1-12-thick-x-48-wide-x-72-long>: HTTP status code is not handled or not allowed 2026-01-25 05:30:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:30:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eggcrate-return-grille-surface-mount-12-x-12-white-pk2>: HTTP status code is not handled or not allowed 2026-01-25 05:30:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rolling-ladder-4-step-aluminum-perforated-21-deep-top-step-with-handrails-24-w-step>: HTTP status code is not handled or not allowed 2026-01-25 05:30:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-magazine-12-brochure-oak-acrylic-wall-display-medium-oak>: HTTP status code is not handled or not allowed 2026-01-25 05:30:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bushing-for-w-40z-b218-45z-c215-pallet-trucks-hy-2791987>: HTTP status code is not handled or not allowed 2026-01-25 05:30:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-men-s-boot-16-goliath-black-plain-toe-w-power-lug-outsole-pvc-size-12>: HTTP status code is not handled or not allowed 2026-01-25 05:30:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spacer-for-mpe-060-mpe-080-pallet-trucks-yl-503480200>: HTTP status code is not handled or not allowed 2026-01-25 05:30:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-flaker-ice-machine-2030-ppd-water-cooled>: HTTP status code is not handled or not allowed 2026-01-25 05:30:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fuse-for-m-series-stacker-pallet-trucks-cr-071902-002>: HTTP status code is not handled or not allowed 2026-01-25 05:30:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-38-thick-x-1-14-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 05:30:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-piece-interlocking-multi-purpose-storage-compartment-organizer-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:30:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_sheet_stock>: HTTP status code is not handled or not allowed 2026-01-25 05:30:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drive-tire-for-st-3000-pallet-trucks-cr-127248-001>: HTTP status code is not handled or not allowed 2026-01-25 05:30:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/complete-wheel-kit-for-manual-pallet-jack-gwk-vj2-ck-valu-jack-model-5500-2-hydraulic-unit>: HTTP status code is not handled or not allowed 2026-01-25 05:30:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/complete-wheel-kit-for-manual-pallet-jack-gwk-hpt25-ck-fits-toyota-model-hpt25>: HTTP status code is not handled or not allowed 2026-01-25 05:30:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-dc-control-bc142-6-dc-scr-control-115-230v-1ph-chassis>: HTTP status code is not handled or not allowed 2026-01-25 05:30:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-assembly-for-mpb-040-e-b827-pallet-trucks-yl-524148750-a1>: HTTP status code is not handled or not allowed 2026-01-25 05:30:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-tier-paper-file-tray-with-5-sliding-trays-black>: HTTP status code is not handled or not allowed 2026-01-25 05:30:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-332-thick-x-4-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 05:30:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x36x74-chrome-wire-truck-with-30-4-h-shelf-bins-red>: HTTP status code is not handled or not allowed 2026-01-25 05:30:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-sitex-48-yellow-coat-w-detachable-hood-pvc-size-small>: HTTP status code is not handled or not allowed 2026-01-25 05:30:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-sitex-yellow-3-piece-suit-pvc-size-4x>: HTTP status code is not handled or not allowed 2026-01-25 05:30:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-pedestal-post-cd-premium-fan-292650>: HTTP status code is not handled or not allowed 2026-01-25 05:30:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uhmw-polyethylene-plastic-bar-38-thick-x-5-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 05:30:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x36x74-chrome-wire-shelving-with-36-6-h-nest-stack-shipping-totes-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:30:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-1-14-thick-x-2-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 05:30:44 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/carts-trucks/pallet_truck_parts_accessories/pallet_truck_wheel_kits landed on page that is not a product page. 2026-01-25 05:30:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-sanitex-green-bib-overall-plain-front-pvc-on-polyester-size-small>: HTTP status code is not handled or not allowed 2026-01-25 05:30:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/semi-clear-fda-silicone-rubber-sheet-whigh-temp-adhesive-60a-132-thick-x-36w-x-36l>: HTTP status code is not handled or not allowed 2026-01-25 05:30:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leadhead-auto-darkening-welding-helmet-manual-98x87mm-flame>: HTTP status code is not handled or not allowed 2026-01-25 05:30:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/male-34-body-form-black-wblack-wood-tripod-base>: HTTP status code is not handled or not allowed 2026-01-25 05:30:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ptfe-plastic-sheet-332-thick-x-16-wide-x-16-long>: HTTP status code is not handled or not allowed 2026-01-25 05:30:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/base-for-body-forms-12-x-12-x-30h>: HTTP status code is not handled or not allowed 2026-01-25 05:30:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/epdm-rubber-sheet-no-adhesive-60a-18-thick-x-18-wide-x-18-long>: HTTP status code is not handled or not allowed 2026-01-25 05:30:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/female-mannequin-headless-hands-by-side-legs-apart-matte-finish>: HTTP status code is not handled or not allowed 2026-01-25 05:30:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milwaukee-14-abrasive-chop-saw>: HTTP status code is not handled or not allowed 2026-01-25 05:30:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-bar-38-thick-x-1-14-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 05:30:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extruded-aluminum-hose-cable-bridge-silver-36-x-278-x-716>: HTTP status code is not handled or not allowed 2026-01-25 05:30:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rigid-dock-shelter-10-w-x-10-h-with-36-projection>: HTTP status code is not handled or not allowed 2026-01-25 05:30:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ridgeline-vented-full-brim-hard-hat-matte-black-graphite-pattern-4-point-ratchet-suspension>: HTTP status code is not handled or not allowed 2026-01-25 05:30:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milwaukee-6-5-tpi-sawzall-blades-5-48-00-5035>: HTTP status code is not handled or not allowed 2026-01-25 05:30:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/abs-plastic-bar-18-thick-x-6-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 05:30:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-men-s-boot-16-sureflex-brown-cream-steel-toe-pvc-size-5>: HTTP status code is not handled or not allowed 2026-01-25 05:31:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-head-cap-screw-m6-x-1-0-x-20mm-steel-zinc-clear-class-8-8-din-933-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 05:31:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-men-s-boot-14-white-plain-toe-w-safety-lock-pvc-size-7>: HTTP status code is not handled or not allowed 2026-01-25 05:31:01 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:31:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slide-box-for-100-slides-cork-lined-stainless-steel-lock-red>: HTTP status code is not handled or not allowed 2026-01-25 05:31:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/louvered-wall-panel-without-bins-18x19-blue-1>: HTTP status code is not handled or not allowed 2026-01-25 05:31:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-barometric-relief-damper-weighted-arm>: HTTP status code is not handled or not allowed 2026-01-25 05:31:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-3-thick-x-6-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 05:31:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-deluxe-office-partition-panel-24-1-4w-x-61-1-2h-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:31:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-centrifugal-fan-plastic-housing-432-cfm>: HTTP status code is not handled or not allowed 2026-01-25 05:31:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slide-box-for-100-slides-cork-lined-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:31:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fine-rib-corrugated-rubber-floor-mat-1-8-thick-4-x-4-black>: HTTP status code is not handled or not allowed 2026-01-25 05:31:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 69906 pages (at 95 pages/min), scraped 34703 items (at 30 items/min) 2026-01-25 05:31:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-bin-shelf-cart-w-phenolic-casters-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:31:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-materials/plastics/plastic_bar_stock>: HTTP status code is not handled or not allowed 2026-01-25 05:31:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cap-flange-plug-13mm-orange-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:31:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/first-responder-bag-orange>: HTTP status code is not handled or not allowed 2026-01-25 05:31:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-battery-6v-200ah-agm-c49r-641748-sweeper>: HTTP status code is not handled or not allowed 2026-01-25 05:31:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/omega-ii-cap-with-accessory-slots-and-6-point-mega-ratchet-suspension-brown>: HTTP status code is not handled or not allowed 2026-01-25 05:31:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-deluxe-office-partition-panel-with-window-60-1-4w-x-61-1-2h-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:31:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/built-in-combination-lock-manual-dead-bolt-control-key-access-left-hinged>: HTTP status code is not handled or not allowed 2026-01-25 05:31:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-bar-18-thick-x-14-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 05:31:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waterproof-quilted-thermal-bib-rain-pants-ansi-isea-class-e-lime-black-l-lqbbip-ce-l>: HTTP status code is not handled or not allowed 2026-01-25 05:31:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-skatewheel-conveyor-18-oaw-5-oal-16-wpf>: HTTP status code is not handled or not allowed 2026-01-25 05:31:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-w-lse-acrylic-adhesive-116-thick-x-2-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 05:31:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-drawer-full-height-36-w-cabinet-everest-blue-2>: HTTP status code is not handled or not allowed 2026-01-25 05:31:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gun-safe-dehumidifier-md36-43-1-2-h-500-cubic-ft-coverage-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 05:31:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-piece-raincoat-35-mil-pvcpolyester-snap-closure-yellow-3xl>: HTTP status code is not handled or not allowed 2026-01-25 05:31:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-132-thick-x-1-14-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 05:31:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/rainwear already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:31:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/i675-americana-vent-slide-lock-suspension>: HTTP status code is not handled or not allowed 2026-01-25 05:31:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-1-2-inch-dial-1-4-inch-npt-bottom-0-30-psi-furnished-in-drawn-steel-case>: HTTP status code is not handled or not allowed 2026-01-25 05:31:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-acetal-plastic-bar-18-thick-x-34-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 05:31:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-drawer-counter-high-36-w-cabinet-light-gray-1>: HTTP status code is not handled or not allowed 2026-01-25 05:31:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-drawer-bench-high-48-w-cabinet-everest-blue-1>: HTTP status code is not handled or not allowed 2026-01-25 05:31:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storage-bin-cabinet-114-yellow-hook-on-bins-48w-24d-72h>: HTTP status code is not handled or not allowed 2026-01-25 05:31:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-drawer-counter-high-36-w-cabinet-avalanche-blue-3>: HTTP status code is not handled or not allowed 2026-01-25 05:31:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gh-1880zx-with-newall-dp700-with-taper-attachment-and-collet-closer>: HTTP status code is not handled or not allowed 2026-01-25 05:31:37 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/rainwear landed on page that is not a product page. 2026-01-25 05:31:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-drawer-counter-high-30-w-cabinet-red-3>: HTTP status code is not handled or not allowed 2026-01-25 05:31:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cap-universal-fits-most-12mm-13mm-and-16mm-tubes-black-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:31:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-guard-w-rounded-top-bottom-edges-rec-plastic-clip-retainer-system-7-3-4-h-x-12-l-white>: HTTP status code is not handled or not allowed 2026-01-25 05:31:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4in-single-round-powder-coated-steel-light-box-w-knockouts>: HTTP status code is not handled or not allowed 2026-01-25 05:31:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/vehicle_flood_utility_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:31:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12in-heavy-duty-series-cree-combo-bar-60w-4-200lm>: HTTP status code is not handled or not allowed 2026-01-25 05:31:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90-120-day-undated-horizontal-erasable-wall-planner-36x24-white-blue-pm23928>: HTTP status code is not handled or not allowed 2026-01-25 05:31:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-equipment-dolly-mover-8000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 05:31:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jtm-1254vs-with-x-y-z-powerfeeds>: HTTP status code is not handled or not allowed 2026-01-25 05:31:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/siesta-sky-24-square-resin-folding-bar-table-black>: HTTP status code is not handled or not allowed 2026-01-25 05:31:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-drawer-full-height-36-w-cabinet-red-1>: HTTP status code is not handled or not allowed 2026-01-25 05:31:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/superevac-lcd-vacuum-gauge-69075>: HTTP status code is not handled or not allowed 2026-01-25 05:31:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/culture-tube-borosilicate-glass-13-x-100mm-10ml1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:31:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-drawer-counter-high-30-w-cabinet-everest-blue-3>: HTTP status code is not handled or not allowed 2026-01-25 05:31:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jtm-1254vs-with-newall-dp700-3x-k-dro-x-y-powerfeeds-air-power-drawbar>: HTTP status code is not handled or not allowed 2026-01-25 05:31:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/vehicle_lighting/vehicle_flood_utility_lights>: HTTP status code is not handled or not allowed 2026-01-25 05:31:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transfer-pipet-58ml-fine-tip-147mm-5000-pk>: HTTP status code is not handled or not allowed 2026-01-25 05:31:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/female-double-swivel-ring-m12>: HTTP status code is not handled or not allowed 2026-01-25 05:31:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-swivel-ring-dsr-u-100>: HTTP status code is not handled or not allowed 2026-01-25 05:31:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/refrigerated-countertop-display-4-2-cu-ft>: HTTP status code is not handled or not allowed 2026-01-25 05:31:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ante-brush-belt-for-push-sweeper-ref7-rp9019>: HTTP status code is not handled or not allowed 2026-01-25 05:31:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/female-double-swivel-ring-u050>: HTTP status code is not handled or not allowed 2026-01-25 05:31:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/freezer-merch-2-section-54-18w-29-78d-78-58h-gdm-49f>: HTTP status code is not handled or not allowed 2026-01-25 05:31:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11-camwear-perforated-spoon-red>: HTTP status code is not handled or not allowed 2026-01-25 05:31:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/draft-beer-cooler-69-18w-27-18d-37h-tdd-3>: HTTP status code is not handled or not allowed 2026-01-25 05:32:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jps-10-10-proshop-1-75hp-1ph-115v-52-fence-system-steel-wing-w-riving-knife>: HTTP status code is not handled or not allowed 2026-01-25 05:32:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jtm-1254rvs-with-newall-dp700-3x-q-dro-x-powerfeed-air-power-drawbar>: HTTP status code is not handled or not allowed 2026-01-25 05:32:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jtm-1254vs-with-acu-rite-203-dro-x-y-z-powerfeeds-air-power-drawbar>: HTTP status code is not handled or not allowed 2026-01-25 05:32:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wireless-tube-rack-16mm-full-size-72-place-nylon-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:32:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kraft-corrugated-sheets-20-x-16>: HTTP status code is not handled or not allowed 2026-01-25 05:32:03 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=jtm-1254vs-with-acu-rite-203-3x-k-dro-x-y-powerfeeds already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:32:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-welded-gear-locker-with-door-legs-24x18x72-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:32:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c10-bump-cap-for-minor-bumps-with-shield-attachment-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:32:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wireless-tube-rack-25mm-half-size-16-place-nylon-green>: HTTP status code is not handled or not allowed 2026-01-25 05:32:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-welded-gear-locker-with-door-foot-locker-24x24x72-putty>: HTTP status code is not handled or not allowed 2026-01-25 05:32:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insulated-shipping-containers-16-3-4-x-16-3-4-x-15>: HTTP status code is not handled or not allowed 2026-01-25 05:32:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sc-6-safety-hard-hat-4-pt-ratchet-suspension-cap-style-orange->: HTTP status code is not handled or not allowed 2026-01-25 05:32:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wesco-vertical-cut-side-mounted-drum-deheader>: HTTP status code is not handled or not allowed 2026-01-25 05:32:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 69998 pages (at 92 pages/min), scraped 34734 items (at 31 items/min) 2026-01-25 05:32:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-corrugated-bookfolds-7-1-2-x-5-1-2-x-2->: HTTP status code is not handled or not allowed 2026-01-25 05:32:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-barbed-equal-tee-glass-filled-black-nylon>: HTTP status code is not handled or not allowed 2026-01-25 05:32:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leviton-ods10-idt-decora-passive-infrared-wall-switch-occupancy-sensor-light-almond>: HTTP status code is not handled or not allowed 2026-01-25 05:32:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/etm-949-mill-with-2-axis-newall-dp700-dro-and-x-axis-jet-powerfeed>: HTTP status code is not handled or not allowed 2026-01-25 05:32:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-adapter-face-shield-with-shade-8-ir-flip-visor-anti-fog-quad500-series>: HTTP status code is not handled or not allowed 2026-01-25 05:32:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stud-mid-sole-footwear-traction-aid-tungsten-carbide-spikes-universal-blackgreen>: HTTP status code is not handled or not allowed 2026-01-25 05:32:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evs-949-mill-3-axis-acu-rite-203-knee-dro-x-y-axis-jet-powerfeeds-usa-pwrd-drw-br>: HTTP status code is not handled or not allowed 2026-01-25 05:32:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-ash-trash-receptacle-alum-lid-13-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:32:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crinkle-latex-coated-gloves-polyester-knit-black-blue-large-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 05:32:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/egh-1740-with-acu-rite-203-dro>: HTTP status code is not handled or not allowed 2026-01-25 05:32:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kwikool-portable-water-cooled-air-conditioner-kwib12043-10-ton-120000-btu>: HTTP status code is not handled or not allowed 2026-01-25 05:32:17 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=jtm-1254vs-with-acu-rite-203-3x-k-dro-x-y-powerfeeds landed on page that is not a product page. 2026-01-25 05:32:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tube-stand-reinforced-polypropylene-32mm-12-place-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:32:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/etm-949-mill-with-x-y-z-axis-jet-powerfeeds>: HTTP status code is not handled or not allowed 2026-01-25 05:32:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stackbin-1012-series-esd-w-t-molding-72-w-x-36-d-black>: HTTP status code is not handled or not allowed 2026-01-25 05:32:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wireless-tube-rack-16mm-half-size-36-place-nylon-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:32:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stackbin-4000-series-plastic-laminate-w-t-molding-72-w-x-30-d-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:32:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-corrugated-mailer-10-x-6-x-2->: HTTP status code is not handled or not allowed 2026-01-25 05:32:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-coated-string-knit-gloves-natural-blue-large-1-dozen>: HTTP status code is not handled or not allowed 2026-01-25 05:32:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/melamine-dry-erase-whiteboard-4-x-8-double-sided-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-25 05:32:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/market-tray-13x18-yellow>: HTTP status code is not handled or not allowed 2026-01-25 05:32:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/market-tray-13x18-white>: HTTP status code is not handled or not allowed 2026-01-25 05:32:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conveyor-magnet-36in-l>: HTTP status code is not handled or not allowed 2026-01-25 05:32:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1847-sc-11-1-4-x-1-8-double-cut-cylindrical-ball-nose-bur>: HTTP status code is not handled or not allowed 2026-01-25 05:32:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1850-sm-51-1-4-x-1-8-standard-cut-pointed-cone-bur>: HTTP status code is not handled or not allowed 2026-01-25 05:32:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evs-949-mill-with-3-axis-newall-dp700-knee-dro-and-x-y-axis-jet-powerfeeds>: HTTP status code is not handled or not allowed 2026-01-25 05:32:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-ton-shop-press-with-plc-and-portable-power-unit-1-phase-230-volt>: HTTP status code is not handled or not allowed 2026-01-25 05:32:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-step-stock-picking-ladder-perforated-tread-spl-6-14p>: HTTP status code is not handled or not allowed 2026-01-25 05:32:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camrack-glass-rack-25-compartments-10-1-8-inch-max-height-red-nsf>: HTTP status code is not handled or not allowed 2026-01-25 05:32:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-step-stock-picking-ladder-expanded-tread-spl-9-14x>: HTTP status code is not handled or not allowed 2026-01-25 05:32:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-cantilever-arm-48-inch-2600-no-cap>: HTTP status code is not handled or not allowed 2026-01-25 05:32:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-cantilever-192-inch-upright-48-inch-brace-kit-starter>: HTTP status code is not handled or not allowed 2026-01-25 05:32:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/justrite-smokers-cease-fire-cigarette-butt-receptacle-1-gallon-beige>: HTTP status code is not handled or not allowed 2026-01-25 05:32:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securall-60g-self-close-hazardous-waste-drum-storage-cabinet-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:32:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securall-16-cylinder-vertical-lp-oxygen-cabinet-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 05:32:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securall-120g-self-close-vertical-flammable-drum-cabinet-yellow>: HTTP status code is not handled or not allowed 2026-01-25 05:32:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/howard-feed-n-wax-wood-polish-conditioner-8-oz-bottle-12-case>: HTTP status code is not handled or not allowed 2026-01-25 05:32:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1847-sc-51-6-35mm-x-12-7mm-cylindrical-ball-nose-bur>: HTTP status code is not handled or not allowed 2026-01-25 05:32:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-security-truck-poly-z-brite-with-brakes-2-shelves-24-w-x-48-l-x-69-h>: HTTP status code is not handled or not allowed 2026-01-25 05:32:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wireless-tube-rack-13mm-half-size-36-place-nylon-magenta>: HTTP status code is not handled or not allowed 2026-01-25 05:32:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lt241g11-supreme-1-hole-sg-lavatory-colonial-white>: HTTP status code is not handled or not allowed 2026-01-25 05:32:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securall-45g-self-close-flammable-cabinet-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 05:32:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/open-front-smoking-shelter-freestanding-7-ft-w-x-3-ft-6-inch-d-x-7-ft-11-inch-h-clear-roof>: HTTP status code is not handled or not allowed 2026-01-25 05:32:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-cash-handling-thermal-printer-kp-1>: HTTP status code is not handled or not allowed 2026-01-25 05:32:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securall-60g-manual-close-haz-waste-drum-storage-cabinet-red>: HTTP status code is not handled or not allowed 2026-01-25 05:32:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securall-60g-manual-close-paint-ink-cabinet-red>: HTTP status code is not handled or not allowed 2026-01-25 05:32:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/matching-flange-30-duct-fans>: HTTP status code is not handled or not allowed 2026-01-25 05:32:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/44xl100-standard-timing-belt-xl-1-x-4-3-8-t22-trapezoidal>: HTTP status code is not handled or not allowed 2026-01-25 05:32:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1847-sc-42-1-8-x-1-8-standard-cut-cylindrical-ball-nose-bur>: HTTP status code is not handled or not allowed 2026-01-25 05:32:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1846-sb-43-3-00mm-x-14-3mm-standard-cut-cylindrical-bur-w-end-cut>: HTTP status code is not handled or not allowed 2026-01-25 05:32:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forklift-container-ramp-72in-w-x-84in-l-20-000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 05:32:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-x-48-mat-ultimat-light-blue-um-3048lb>: HTTP status code is not handled or not allowed 2026-01-25 05:32:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-x-36-mat-ultimat-clean-room-white-cm-2436>: HTTP status code is not handled or not allowed 2026-01-25 05:32:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/security-laptop-transport-stand>: HTTP status code is not handled or not allowed 2026-01-25 05:32:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dura-spike-narrow-w-bond-bird-spikes-102-ft-bp-sr200>: HTTP status code is not handled or not allowed 2026-01-25 05:32:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1852-sl-6-5-8-x-1-4-standard-cut-included-angle-bur>: HTTP status code is not handled or not allowed 2026-01-25 05:32:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pop-up-ball-transfer-platform-19-w-x-32-l>: HTTP status code is not handled or not allowed 2026-01-25 05:32:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unex-gravity-flow-roller-rack-with-knuckled-span-track-add-on-96w-x-96d-x-84h-with-4-levels>: HTTP status code is not handled or not allowed 2026-01-25 05:32:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unex-gravity-flow-roller-rack-with-knuckled-span-track-starter-96w-x-72d-x-84h-with-4-levels>: HTTP status code is not handled or not allowed 2026-01-25 05:32:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trustat-common-point-ground-cord-w-10-mm-stud-15ft-black-cord>: HTTP status code is not handled or not allowed 2026-01-25 05:32:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forkli-truck-fork-caddy>: HTTP status code is not handled or not allowed 2026-01-25 05:33:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cc263-5-3piece-aluminum-broom-thread-handle>: HTTP status code is not handled or not allowed 2026-01-25 05:33:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cap-plug-16mm-blue-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:33:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-1-4-x-77-1-2-h-deluxe-office-partition-panel-with-electric-blue-277544bl>: HTTP status code is not handled or not allowed 2026-01-25 05:33:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/teller-vault-note-bus-cabinet-585-a-6-compartment-18w-x-19d-x-38-1-2h-champagne>: HTTP status code is not handled or not allowed 2026-01-25 05:33:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/27-NRR-reusable-uncorded-soft-earplugs-200-box>: HTTP status code is not handled or not allowed 2026-01-25 05:33:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gp250-usb-digital-bench-scale-with-usb-port-250-x-0-5-lb-12-1-2-x-11-platform>: HTTP status code is not handled or not allowed 2026-01-25 05:33:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/supco-freeze-protection-control>: HTTP status code is not handled or not allowed 2026-01-25 05:33:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 70101 pages (at 103 pages/min), scraped 34769 items (at 35 items/min) 2026-01-25 05:33:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ice-tray-with-lid-9-liter-red>: HTTP status code is not handled or not allowed 2026-01-25 05:33:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/funnel-analytical-polypropylene-25mm-20-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:33:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gate-valve-stem-shield-cable-lock-18-inch-stem-9109s>: HTTP status code is not handled or not allowed 2026-01-25 05:33:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/420h300-standard-timing-belt-h-3-x-42-t84-trapezoidal>: HTTP status code is not handled or not allowed 2026-01-25 05:33:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/430h100-standard-timing-belt-h-1-x-43-t86-trapezoidal>: HTTP status code is not handled or not allowed 2026-01-25 05:33:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/schedule-40-pvc-3way-l-fitting-12dia-white-31-u>: HTTP status code is not handled or not allowed 2026-01-25 05:33:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-inch-round-bistro-table-4-barstool-set-designer-white-table-top-with-espresso-barstools>: HTTP status code is not handled or not allowed 2026-01-25 05:33:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-inch-square-bistro-table-4-barstool-set-designer-white-table-top-with-espresso-barstools>: HTTP status code is not handled or not allowed 2026-01-25 05:33:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clean-mop-store-board-ultra-aluma-lite-blue-on-black>: HTTP status code is not handled or not allowed 2026-01-25 05:33:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wet-zone-store-board-max-duty-aluminum-blue-on-black>: HTTP status code is not handled or not allowed 2026-01-25 05:33:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-height-bath-stool-20-x-12-seat-14-5-21-5-h-white-dc-hy3410l-wh-gg>: HTTP status code is not handled or not allowed 2026-01-25 05:33:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-or-countertop-magazine-rack-11-w-mahogany>: HTTP status code is not handled or not allowed 2026-01-25 05:33:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ha018240-home-advantage-ii-electric-tankless-water-heater-18-kw-75a>: HTTP status code is not handled or not allowed 2026-01-25 05:33:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/folding-security-gate-6-6-hx12-w-use>: HTTP status code is not handled or not allowed 2026-01-25 05:33:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-angle-beam-60-w-x-36-d-x-84-h-4-level-add-on-no-deck-tan>: HTTP status code is not handled or not allowed 2026-01-25 05:33:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/extra_heavy_boltless_shelves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:33:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beaker-diamond-essentials-griffin-style-low-form-printed-graduations-pmp-250ml-6-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:33:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-boltless-shelving-96-w-x-24-d-x-84-h-3-level-add-on-no-deck-tan>: HTTP status code is not handled or not allowed 2026-01-25 05:33:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-tect-runner-174-protection-36in-w-x-150ft-l-roll>: HTTP status code is not handled or not allowed 2026-01-25 05:33:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-oneline-guardrail-steel-hdpe-plastic-yellow-96-x-1434>: HTTP status code is not handled or not allowed 2026-01-25 05:33:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/protective-window-film-12in-w-x-500ft-l-4-mil>: HTTP status code is not handled or not allowed 2026-01-25 05:33:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securall-312x96x100-agri-chemical-building>: HTTP status code is not handled or not allowed 2026-01-25 05:33:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6004-ammonia-methylamine-cartridge-2-bag>: HTTP status code is not handled or not allowed 2026-01-25 05:33:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carbon-monoxide-gas-cylinder-1016-50-ppm-bal-air-17l-p>: HTTP status code is not handled or not allowed 2026-01-25 05:33:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slide-file-folder-with-clear-hinged-lids-20-place-hi-polystyrene-san-yellow-12-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:33:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/shelving/boltless_shelving/extra_heavy_boltless_shelves>: HTTP status code is not handled or not allowed 2026-01-25 05:33:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/active-air-mgmlp1-3-way-meter-test-moisture-light-and-ph-levels>: HTTP status code is not handled or not allowed 2026-01-25 05:33:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ice-caddies-beige-175-lbs-cap-tall-2-fixed-2-swivel-1-with-brake>: HTTP status code is not handled or not allowed 2026-01-25 05:33:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/tools/outdoor_tools_power_equipment/spreaders_sprayers/salt_spreaders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:33:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thermal-leno-weave-blankets-twin-size-white-12-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:33:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-ratchet-1-2-inch-drive-65-ft-lb-torque>: HTTP status code is not handled or not allowed 2026-01-25 05:33:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chesapeake-86776-14-inch-high-pvc-boot-safety-toe-size-14-black>: HTTP status code is not handled or not allowed 2026-01-25 05:33:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelf-mobile-computer-lanstation-workstation-keyboard-tray-40-hx24-wx48-l-black-3-shelf>: HTTP status code is not handled or not allowed 2026-01-25 05:33:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3775l-squids-174-large-can-bottle-holder-trap-black>: HTTP status code is not handled or not allowed 2026-01-25 05:33:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cassette-biopsy-embedding-with-attached-lid-35-writing-area-orange-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:33:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8366-glowear-174-type-r-class-3-lightweight-hi-vis-rain-jacket-orange-4xl>: HTTP status code is not handled or not allowed 2026-01-25 05:33:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/earplug-dispenser-small-nrr-28db-250-pairs-dispenser>: HTTP status code is not handled or not allowed 2026-01-25 05:33:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transport-tube-5ml-polypropylene-ss-attached-white-screwcap-mg-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:33:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/structural-plastic-vented-shelf-36-x-36>: HTTP status code is not handled or not allowed 2026-01-25 05:33:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/tools/outdoor_tools_power_equipment/spreaders_sprayers/salt_spreaders>: HTTP status code is not handled or not allowed 2026-01-25 05:33:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-o-smith-unit-heater-motor-115-volts-1075-rpm-1-4hp-2>: HTTP status code is not handled or not allowed 2026-01-25 05:33:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/structural-plastic-solid-shelf-36-x-24>: HTTP status code is not handled or not allowed 2026-01-25 05:33:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/groz-44133-vertical-lift-safety-pump-5-gallon-pail>: HTTP status code is not handled or not allowed 2026-01-25 05:33:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microtube-05ml-self-standing-polypropylene-no-cap-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:33:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-12bs25hx38-metric-34-pitch-38mm-finished-bore-25-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:33:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-pallet-covers-41-x-31-x-72-2-mil-50-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:33:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beaker-polypropylene-molded-graduations-2000ml-4-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:33:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-5-5-8-permanent-split-capacitor-motor-208-230-volts-1075-rpm-13>: HTTP status code is not handled or not allowed 2026-01-25 05:33:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-care-16-x16-x1-electrostatic-air-filter-merv-8>: HTTP status code is not handled or not allowed 2026-01-25 05:33:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/armor-poly-vci-bags-4-mil-13-x-11-x-25-blue-250-roll>: HTTP status code is not handled or not allowed 2026-01-25 05:33:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-3-3-shaded-pole-open-motor-115-volts-1500-rpm-1>: HTTP status code is not handled or not allowed 2026-01-25 05:34:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-position-rotary-switch>: HTTP status code is not handled or not allowed 2026-01-25 05:34:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/graduated-cylinder-polypropylene-molded-graduations-25ml-30-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:34:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-pallet-covers-68-x-65-x-87-3-mil-50-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:34:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inoculation-loop-flexible-10ul-calibration-certificate-sterile-polystyrene-yellow-pp-250-pk>: HTTP status code is not handled or not allowed 2026-01-25 05:34:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-o-smith-5-split-capacitor-split-capacitor-fan-coil-motor-1625-rpm-115v>: HTTP status code is not handled or not allowed 2026-01-25 05:34:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sore-muscle-rub-heating-therapy-gel-2-7-oz-14-1279>: HTTP status code is not handled or not allowed 2026-01-25 05:34:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/partial-carton-2-6-red-white>: HTTP status code is not handled or not allowed 2026-01-25 05:34:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/digital-battery-load-tester-130amp>: HTTP status code is not handled or not allowed 2026-01-25 05:34:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bright-yellow-4-dia-discs>: HTTP status code is not handled or not allowed 2026-01-25 05:34:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/econo-plastic-seatcovers-200-roll-1-0mil>: HTTP status code is not handled or not allowed 2026-01-25 05:34:08 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/definite_purpose_electric_motors/oem_replacement_ac_motors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:34:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/graduated-cylinder-pmp-tpx-printed-graduations-100ml-30-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:34:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 70193 pages (at 92 pages/min), scraped 34801 items (at 32 items/min) 2026-01-25 05:34:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-3-3-shaded-pole-dra-inducer-motor-208-230-volts-3000-rpm-2>: HTTP status code is not handled or not allowed 2026-01-25 05:34:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/charger-6-12v-70-62-2a-agm-265-amp-cranking>: HTTP status code is not handled or not allowed 2026-01-25 05:34:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-bags-4-x-6-2-mil-blue-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:34:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-bags-4-x-6-2-mil-white-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:34:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/screw-cap-for-microtube-with-o-ring-white-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:34:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/number-1-adjustable-height-complete-packing-workbench-laminate-safety-edge-with-t-mold-72x36-black>: HTTP status code is not handled or not allowed 2026-01-25 05:34:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/long-poly-mailers-13-x-45-50-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:34:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/definite_purpose_electric_motors/oem_replacement_ac_motors>: HTTP status code is not handled or not allowed 2026-01-25 05:34:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fragile-4-6-white-red-black>: HTTP status code is not handled or not allowed 2026-01-25 05:34:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-inch-pastry-detail-brush-white>: HTTP status code is not handled or not allowed 2026-01-25 05:34:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-4-4-shaded-pole-motor-115-volts-1500-rpm-3>: HTTP status code is not handled or not allowed 2026-01-25 05:34:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-flex-hood-only-green>: HTTP status code is not handled or not allowed 2026-01-25 05:34:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/right-to-know-hazard-2-2-white-red-black-blue-yellow>: HTTP status code is not handled or not allowed 2026-01-25 05:34:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flite-safety-toe-work-shoe-black-size-7>: HTTP status code is not handled or not allowed 2026-01-25 05:34:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/six-tier-18-door-premium-steel-locker-12-w-x-18-d-x-12-h-black-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 05:34:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/to-go-rack-silver-ep-21w-x-30l-x-34h-3-wire-shelves-dividers-ledges>: HTTP status code is not handled or not allowed 2026-01-25 05:34:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-trash-can-20-gallon-brown>: HTTP status code is not handled or not allowed 2026-01-25 05:34:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-12a28h-metric-a-plate-34-pitch-19mm-bore-28-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:34:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/contractor-grade-high-adhesion-masking-tape-natural-48mm-x-55m-pack-of-24>: HTTP status code is not handled or not allowed 2026-01-25 05:34:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mixed-carton-bilngual-3-5-fluorescent-green-black>: HTTP status code is not handled or not allowed 2026-01-25 05:34:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-day-shurrelease-blue-painters-tape-multi-surface-blue-36mm-x-55m>: HTTP status code is not handled or not allowed 2026-01-25 05:34:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-caution-label-stanchion-and-chain-kit-40-height-and-50-of-2-chain>: HTTP status code is not handled or not allowed 2026-01-25 05:34:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-two-tone-hooded-rain-coat-black-l-xl>: HTTP status code is not handled or not allowed 2026-01-25 05:34:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-3-new-designed-lightweight-shirt-rip-stop-bottom-down-shirt-w-spf-50-orange-xl>: HTTP status code is not handled or not allowed 2026-01-25 05:34:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-multi-color-vest-blue-2xl-3xl>: HTTP status code is not handled or not allowed 2026-01-25 05:34:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-3-3-shaded-pole-dra-inducer-blower-208-240-volts-3480-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:34:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-3-3-shaded-pole-dra-inducer-blower-208-230-volts-3200-rpm-1>: HTTP status code is not handled or not allowed 2026-01-25 05:34:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-shaded-pole-dra-inducer-blower-115-volts-3000-rpm-3>: HTTP status code is not handled or not allowed 2026-01-25 05:34:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/curbside-truck-chrome-18w-x-36l-x-72h-4-wire-shelves-pneumatic-2-rigid-casters>: HTTP status code is not handled or not allowed 2026-01-25 05:34:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-class-2-five-point-breakaway-vest-orange-5xl>: HTTP status code is not handled or not allowed 2026-01-25 05:34:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stationary-tip-out-bin-unit-single-sided-24-h-white>: HTTP status code is not handled or not allowed 2026-01-25 05:34:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60btl36-taper-bushed-34-pitch-36-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:34:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dont-break-stretch-wrap-3-10-fluorescent-red-black>: HTTP status code is not handled or not allowed 2026-01-25 05:34:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-sided-floor-stand-tip-out-bins-48-h>: HTTP status code is not handled or not allowed 2026-01-25 05:34:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-inch-slantwall-exhaust-fan-belt-drive-3-4-hp-13207-cfm-115-230v>: HTTP status code is not handled or not allowed 2026-01-25 05:34:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arrows-up-red-3-4-1-2-white-red>: HTTP status code is not handled or not allowed 2026-01-25 05:34:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-dia-disc-with-5-bright-yellow-black>: HTTP status code is not handled or not allowed 2026-01-25 05:34:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polymer-ada-inward-swing-partition-door-35-3-5in-w-x-58in-h-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:34:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exq-pro-extreme-power-workstation-hutch-72-w-x-30-d-x-h-26-3-8-blue-w-chrome-handle>: HTTP status code is not handled or not allowed 2026-01-25 05:34:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-wd13aa2pz7-1-3-hp-1-phase-1700-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:34:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/46000k-2-serialized-framed-impact-labels-10g-range-teal-50-box>: HTTP status code is not handled or not allowed 2026-01-25 05:34:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kisses-milk-chocolates-red-66-7-oz-gr>: HTTP status code is not handled or not allowed 2026-01-25 05:34:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/antimicrobial-round-reception-ottoman-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:34:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-paper-recycling-lid-green>: HTTP status code is not handled or not allowed 2026-01-25 05:34:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-solid-recycling-lid-w-handle-green>: HTTP status code is not handled or not allowed 2026-01-25 05:34:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/big-healthy-snack-box-gr>: HTTP status code is not handled or not allowed 2026-01-25 05:35:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-step-16-w-stainless-steel-ladder-with-rails-perforated-tread>: HTTP status code is not handled or not allowed 2026-01-25 05:35:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-z-brite-4-tier-microwave-station-cart-kit-24-l-x-18-w-x-63-h>: HTTP status code is not handled or not allowed 2026-01-25 05:35:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vertical-25-2-drawer-letter-fire-and-water-resistant-file-cabinet-lt-gray-ltr2w25lg>: HTTP status code is not handled or not allowed 2026-01-25 05:35:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lateral-31-2-drawer-fire-and-water-resistant-file-cabinet-black-lat2w31b>: HTTP status code is not handled or not allowed 2026-01-25 05:35:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-inch-sds-max-rotary-hammer-d25773k>: HTTP status code is not handled or not allowed 2026-01-25 05:35:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-1-4-inch-compact-jobsite-table-saw-dwe7485>: HTTP status code is not handled or not allowed 2026-01-25 05:35:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/janitorial-cabinet-18-d-x-36-w-x-72-h-putty>: HTTP status code is not handled or not allowed 2026-01-25 05:35:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-t3c1p18-3-hp-1-phase-3500-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:35:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-wide-4-drawer-lateral-file-cabinet-arctic-silver>: HTTP status code is not handled or not allowed 2026-01-25 05:35:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/non-adjustable-tank-top-li>: HTTP status code is not handled or not allowed 2026-01-25 05:35:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 70290 pages (at 97 pages/min), scraped 34842 items (at 41 items/min) 2026-01-25 05:35:09 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-t1cm2jcr-1-hp-1-phase-1725-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:35:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tda080p279s-3pk-3-pack-of-serrated-blades-for-tda080-dispensers-10-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:35:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-wd5e2d-5-hp-3-phase-1755-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:35:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-72-vinyl-shower-curtain-white-taffeta-embossed-6-gauge-12-per-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:35:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compact-pro-industrial-led-high-bay-15-000-lumens-120v-277v-5000k-white>: HTTP status code is not handled or not allowed 2026-01-25 05:35:13 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-wds32e2ac-1-5-hp-3-phase-1755-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:35:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-d13ba2n4a-1-3-hp-1-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:35:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-u5p2dk-5-hp-3-phase-1760-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:35:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=us-motor-t3c1p18-3-hp-1-phase-3500-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:35:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wwe-khn77-50-1-h-184tc-cast-iron-helical-bevel-speed-reducer-184tc-input-flange-50-1-foot-mt>: HTTP status code is not handled or not allowed 2026-01-25 05:35:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-actuated-hydraulic-bottle-jack-30-ton>: HTTP status code is not handled or not allowed 2026-01-25 05:35:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=us-motor-t1cm2jcr-1-hp-1-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:35:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-d10c2k21-10-hp-1-phase-1740-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:35:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/workbench-plastic-laminate-adjustable-riser-shelf-with-back-stop-blue-2>: HTTP status code is not handled or not allowed 2026-01-25 05:35:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flask-erlenmeyer-globe-glass-screw-cap-dual-graduations-astm-e1404-500ml-6-box>: HTTP status code is not handled or not allowed 2026-01-25 05:35:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wwe-khn37-10-1-h-143-5tc-cast-iron-helical-bevel-speed-reducer-143-5tc-input-flange-10-1-foot-mt>: HTTP status code is not handled or not allowed 2026-01-25 05:35:23 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=us-motor-wds32e2ac-1-5-hp-3-phase-1755-rpm landed on page that is not a product page. 2026-01-25 05:35:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cylinder-graduated-globe-glass-class-a-to-deliver-td-dual-grads-astm-e1272-250ml>: HTTP status code is not handled or not allowed 2026-01-25 05:35:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-ee612-2-hp-3-phase-3510-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:35:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-knock-down-corrosion-resistant-locker-double-tier-1-wide-12x18x78>: HTTP status code is not handled or not allowed 2026-01-25 05:35:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-assembled-electronic-access-locker-six-tier-3-wide-12x18x78>: HTTP status code is not handled or not allowed 2026-01-25 05:35:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-u15p2d-15-hp-3-phase-1775-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:35:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-rivet-steel-shelving-with-wood-deck-96-w-x-18-d-x-84-h-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:35:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-galvanized-steel-blower-wheel-3-13-16-diameter-5-16-bore-4>: HTTP status code is not handled or not allowed 2026-01-25 05:35:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-un1t3bc-1-hp-3-phase-1150-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:35:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-galvanized-steel-blower-wheel-9-1-8-diameter-1-2-bore-1>: HTTP status code is not handled or not allowed 2026-01-25 05:35:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-master-link-2-inch-link-safety-orange-10-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:35:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-yc10p1b-10-hp-3-phase-3500-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:35:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-ho400v2slh-400-hp-3-phase-1775-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:35:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-un5t2bc-5-hp-3-phase-1770-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:35:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-d3p2h-3-hp-3-phase-1770-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:35:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-c400p2c-400-hp-3-phase-1785-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:35:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/connecting-s-hooks-1-5-inch-yellow-10-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:35:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-master-link-2-inch-heavy-duty-link-safety-green-10-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:35:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-c32p2b-1-5-hp-3-phase-1750-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:35:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-c2p1b-2-hp-3-phase-3495-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:35:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-d1p2d-1-hp-3-phase-1755-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:35:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cotter-pin-assortment-carbon-steel-zinc-clear-100-piece-handi-chek-version>: HTTP status code is not handled or not allowed 2026-01-25 05:35:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-3275-3-4-hp-1-phase-1075-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:35:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-un12t2gc-0-5-hp-3-phase-1750-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:35:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camshelving-stationary-post-kit-24x72>: HTTP status code is not handled or not allowed 2026-01-25 05:35:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/searchResult?q=us-motor-7041-1-2-hp-1-phase-825-rpm already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:35:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camshelving-stationary-starter-5-vented-shelves-21x36x64>: HTTP status code is not handled or not allowed 2026-01-25 05:35:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-prong-tool-wrench-holder>: HTTP status code is not handled or not allowed 2026-01-25 05:35:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-rod-1-30-degree-bend-2>: HTTP status code is not handled or not allowed 2026-01-25 05:35:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wwe-kan37-10-1-h-143-5tc-cast-iron-helical-bevel-speed-reducer-143-5tc-input-flange-10-1>: HTTP status code is not handled or not allowed 2026-01-25 05:35:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wwe-kan77-50-1-h-184tc-cast-iron-helical-bevel-speed-reducer-184tc-input-flange-50-1>: HTTP status code is not handled or not allowed 2026-01-25 05:35:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-8052-1-3-hp-1-phase-825-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:35:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-3134-1-4-hp-1-phase-1075-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:35:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wwe-kan87-15-1-h-213-5tc-cast-iron-helical-bevel-speed-reducer-213-5tc-input-flange-15-1>: HTTP status code is not handled or not allowed 2026-01-25 05:35:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camwear-measuring-cup-1-quart-clear-polycarbonate-molded-handle-dishwasher-safe-nsf>: HTTP status code is not handled or not allowed 2026-01-25 05:35:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camcarrier-heated-ultra-pancarrier-front-loading-110v-black>: HTTP status code is not handled or not allowed 2026-01-25 05:35:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-desk-cherry-2>: HTTP status code is not handled or not allowed 2026-01-25 05:35:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/searchResult?q=us-motor-7041-1-2-hp-1-phase-825-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:35:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interion-deluxe-freestanding-office-partition-panel-60-1-4-w-x-96-h-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:35:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/versa-food-bars-serving-buffet-cold-food-60-x-36-hot-red->: HTTP status code is not handled or not allowed 2026-01-25 05:35:58 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/searchResult?q=us-motor-c2p1b-2-hp-3-phase-3495-rpm landed on page that is not a product page. 2026-01-25 05:35:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-bar-675-sedona>: HTTP status code is not handled or not allowed 2026-01-25 05:35:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/afs49ml-pharma-lab-performance-series-upright-pharmacy-freezer-solid-doors-49-cu-ft>: HTTP status code is not handled or not allowed 2026-01-25 05:36:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-core-cable-with-drum-assembly-1-2-x-50-for-670439-670440>: HTTP status code is not handled or not allowed 2026-01-25 05:36:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mesh-desk-organizer-3-horizontal-5-upright>: HTTP status code is not handled or not allowed 2026-01-25 05:36:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camwear-food-pan-plastic-full-size-2-1-2-deep-polycarbonate-white-nsf>: HTTP status code is not handled or not allowed 2026-01-25 05:36:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camwear-food-pan-1-2-size-2-1-2-deep-polycarbonate-clear-nsf>: HTTP status code is not handled or not allowed 2026-01-25 05:36:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manual-12-diameter-turntable-4-h-500-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 05:36:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mid-back-ergonomic-task-chair-black>: HTTP status code is not handled or not allowed 2026-01-25 05:36:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-cabinet-2-3-4-backsplash-hinged-doors-galv-30-x-72>: HTTP status code is not handled or not allowed 2026-01-25 05:36:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-camcart-food-pan-carrier-front-loading-approximately-cap-90-qt-brick-red>: HTTP status code is not handled or not allowed 2026-01-25 05:36:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 70385 pages (at 95 pages/min), scraped 34875 items (at 33 items/min) 2026-01-25 05:36:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-angle-frame-mirror-24-wx72-h>: HTTP status code is not handled or not allowed 2026-01-25 05:36:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/riser-fits-1000lcd-19-1-2-d-x-15-3-8-w-x-4-1-2-h-coffee-beige-nsf>: HTTP status code is not handled or not allowed 2026-01-25 05:36:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/meal-delivery-cart-low-profile-2-doors-48-1-2x32-1-2x44-granite-sand>: HTTP status code is not handled or not allowed 2026-01-25 05:36:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-camcart-food-pan-carrier-front-loading-approximately-cap-45-qt-dark-brown>: HTTP status code is not handled or not allowed 2026-01-25 05:36:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-workbench-2-3-4-backsplash-galv-steel-undershelf-72-w-x-24-d>: HTTP status code is not handled or not allowed 2026-01-25 05:36:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-flat-top-workbench-72-w-x-30-d-2>: HTTP status code is not handled or not allowed 2026-01-25 05:36:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cast-iron-jaw-coupling-hub-style-l090-3-4-inch-bore-diameter-3-16-x-3-32-keyway>: HTTP status code is not handled or not allowed 2026-01-25 05:36:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-male-metal-release-collet-internal-hex-6mm-tube-x-1-4-swift-fit>: HTTP status code is not handled or not allowed 2026-01-25 05:36:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hytrel-jaw-coupling-insert-style-l099-100>: HTTP status code is not handled or not allowed 2026-01-25 05:36:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cast-iron-jaw-coupling-hub-style-l035-5-16-inch-bore-diameter-no-keyway>: HTTP status code is not handled or not allowed 2026-01-25 05:36:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6mm-balldriver-l-wrench-10968>: HTTP status code is not handled or not allowed 2026-01-25 05:36:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-npt-breather-vent-brass-body-100-micron-14-to-176-f>: HTTP status code is not handled or not allowed 2026-01-25 05:36:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forced-air-circulation-incubator-53-liters-115-volt>: HTTP status code is not handled or not allowed 2026-01-25 05:36:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chamfer-gage-0-5mm-graduation>: HTTP status code is not handled or not allowed 2026-01-25 05:36:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-electronic-waterproof-depth-gage-0-8-inch-0-200mm-range>: HTTP status code is not handled or not allowed 2026-01-25 05:36:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/protractor-6-5-8-inch-l-x-3-11-16-inch-w-0-180->: HTTP status code is not handled or not allowed 2026-01-25 05:36:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shield-octagonal-36-inch-reach-w-o-interlock-handle-3-arms-24-inch-l-x-10-inch-w-25-lb>: HTTP status code is not handled or not allowed 2026-01-25 05:36:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-sidedshield-24-inch-reach-w-interlock-24-inch-l-x-9-inch-w-x-6-1-2-inch-h-25-lb>: HTTP status code is not handled or not allowed 2026-01-25 05:36:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-ply-cotton-blend-looped-end-mop-w-1-green-band-medium-blue-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-25 05:36:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-ply-looped-end-mop-w-5-red-band-large-blue-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-25 05:36:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ecp-vertical-paper-poly-dispenser-60-inch-l>: HTTP status code is not handled or not allowed 2026-01-25 05:36:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/birdslide-small-12-seam-connectors-gray-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-25 05:36:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ez-catch-trap-xl-36l-x-36w>: HTTP status code is not handled or not allowed 2026-01-25 05:36:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stealthnet-1-18-standard-bird-netting-50l-x-75w-black>: HTTP status code is not handled or not allowed 2026-01-25 05:36:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/net-ring-tool-for-applying-net-rings-to-bird-netting>: HTTP status code is not handled or not allowed 2026-01-25 05:36:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/synthetic-winch-line-extension-w-integrated-shackle-58-dia-x-25l>: HTTP status code is not handled or not allowed 2026-01-25 05:36:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbw-vinyl-bumper-truck-2-rigid-2-swivel-casters-18-bushel-capacity-beige>: HTTP status code is not handled or not allowed 2026-01-25 05:36:37 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_tables/outdoor_dining_bar_tables already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:36:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cape-coral-night-lines-slab-door-wood-36w-x-80h-whiskey-oak>: HTTP status code is not handled or not allowed 2026-01-25 05:36:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ocala-night-lines-slab-door-wood-36w-x-80h-white>: HTTP status code is not handled or not allowed 2026-01-25 05:36:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:36:40 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:36:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ocala-silver-lines-slab-door-wood-30w-x-80h-white>: HTTP status code is not handled or not allowed 2026-01-25 05:36:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/naples-night-lines-slab-door-wood-32w-x-80h-white>: HTTP status code is not handled or not allowed 2026-01-25 05:36:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_plates_braces already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:36:43 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:36:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flush-bolt-nickel>: HTTP status code is not handled or not allowed 2026-01-25 05:36:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pensacola-glazed-light-slab-door-wood--glass-32w-x-80h-whiskey-oak>: HTTP status code is not handled or not allowed 2026-01-25 05:36:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:36:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_plates_braces already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:36:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flush-pull-handle-gold>: HTTP status code is not handled or not allowed 2026-01-25 05:36:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hog-heavenanti-fatigue-mat-58-thick-30l-x-5w-black>: HTTP status code is not handled or not allowed 2026-01-25 05:36:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hog-heavenanti-fatigue-mat-78-thick-60l-x-2w-black>: HTTP status code is not handled or not allowed 2026-01-25 05:36:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hog-heavenanti-fatigue-mat-78-thick-40l-x-3w-black>: HTTP status code is not handled or not allowed 2026-01-25 05:36:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:36:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors>: HTTP status code is not handled or not allowed 2026-01-25 05:36:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cushion-max-anti-fatigue-mat-58-thick-5l-x-2w-charcoal>: HTTP status code is not handled or not allowed 2026-01-25 05:36:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_plates_braces>: HTTP status code is not handled or not allowed 2026-01-25 05:36:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/doors_windows/doors/interior-doors>: HTTP status code is not handled or not allowed 2026-01-25 05:36:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_tables/outdoor_dining_bar_tables landed on page that is not a product page. 2026-01-25 05:36:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polywoven-dunnage-air-bags-4-ply-96l-x-46-12w-pack-of-190>: HTTP status code is not handled or not allowed 2026-01-25 05:36:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polywoven-dunnage-air-bags-8-ply-60l-x-46-12w-pack-of-190>: HTTP status code is not handled or not allowed 2026-01-25 05:36:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/building-supplies/doors_windows/doors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:36:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/fasteners/door_window_hardware/door_plates_braces>: HTTP status code is not handled or not allowed 2026-01-25 05:36:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/infinity-2-tier-2-door-locker-18-in-w-x-21-in-d-x-36-in-h-gray-assembled>: HTTP status code is not handled or not allowed 2026-01-25 05:36:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/first-aid-drop-leg-kit-103-pieces>: HTTP status code is not handled or not allowed 2026-01-25 05:36:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-hivis-3-in-1-rainwear-jacket-xl-black-lime>: HTTP status code is not handled or not allowed 2026-01-25 05:36:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/building-supplies/doors_windows/doors landed on page that is not a product page. 2026-01-25 05:36:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-mediumweight-knife-white-pack-of-1000>: HTTP status code is not handled or not allowed 2026-01-25 05:36:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smartstock-refill-knives-mediumweight-bio-blend-7-l-beige-pack-of-960>: HTTP status code is not handled or not allowed 2026-01-25 05:37:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conex-complements-portion-medicine-cups-3-25-oz-clear-pack-of-2500>: HTTP status code is not handled or not allowed 2026-01-25 05:37:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-tall-pet-drink-cups-9-oz-clear-pack-of-1000>: HTTP status code is not handled or not allowed 2026-01-25 05:37:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/building-supplies/doors_windows/doors>: HTTP status code is not handled or not allowed 2026-01-25 05:37:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flexstyle-double-poly-paper-container-3-5-8-dia-x-2-13-16-h-white-pack-of-500>: HTTP status code is not handled or not allowed 2026-01-25 05:37:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gen-freezer-paper-1-000-l-x-18-w-white>: HTTP status code is not handled or not allowed 2026-01-25 05:37:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/comet-plastic-squat-tumblers-5-oz-clear-pack-of-1000>: HTTP status code is not handled or not allowed 2026-01-25 05:37:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tray-w-56-fork-56-knives-56-spoons-combo-pack-white-pack-of-1008>: HTTP status code is not handled or not allowed 2026-01-25 05:37:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-cold-drink-cups-polypropylene-16-oz-translucent-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-25 05:37:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 70474 pages (at 89 pages/min), scraped 34904 items (at 29 items/min) 2026-01-25 05:37:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smartlock-large-container-w-3-comp-9-1-4-l-x-9-w-x-3-1-4-h-pack-of-150>: HTTP status code is not handled or not allowed 2026-01-25 05:37:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/container-9-l-x-5-3-8-w-x-3-1-2-h-clear-pack-of-250>: HTTP status code is not handled or not allowed 2026-01-25 05:37:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insulated-foam-bowls-6-oz-white-pack-of-1000>: HTTP status code is not handled or not allowed 2026-01-25 05:37:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paper-hot-drink-cups-12-oz-white-pack-of-1k>: HTTP status code is not handled or not allowed 2026-01-25 05:37:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/max-lithium-ion-battery-24v-80ah>: HTTP status code is not handled or not allowed 2026-01-25 05:37:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oscillating-5-piece-tiling-project-kit>: HTTP status code is not handled or not allowed 2026-01-25 05:37:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/colormaxx-paint-primer-12-oz-gloss-black-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 05:37:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/colormaxx-paint-primer-12-oz-satin-black-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 05:37:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/colormaxx-paint-primer-12-oz-gloss-peekaboo-blue-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 05:37:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ductless-mini-split-air-conditioner-w-heat-pump-wifi-enabled-12000-btu-1-ton-25ft-lineset>: HTTP status code is not handled or not allowed 2026-01-25 05:37:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scotch-weld-structural-plastic-adhesive-duo-pak-45-ml-capacity-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:37:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/outdoor-dining-set-36-in-round-x-36-in-h-table-4-chairs-black>: HTTP status code is not handled or not allowed 2026-01-25 05:37:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arkyn-rl-replacement-lens-for-otg-safety-goggles-anti-scratch-anti-fog-smoke>: HTTP status code is not handled or not allowed 2026-01-25 05:37:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vinyl-post-op-shoe-w-hook-and-loop-closure-mens-large>: HTTP status code is not handled or not allowed 2026-01-25 05:37:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sterile-half-surgical-drape-40-l-x-58-w-20-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:37:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gel-foam-wheelchair-cushion-for-pressure-redistribution-24l-x-18w-x-2h-white>: HTTP status code is not handled or not allowed 2026-01-25 05:37:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/terry-adult-bib-w-hook-loop-strap-10-oz-33l-x-21w-white-12-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:37:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guardian-k3-wheelchair-w-full-length-arms-elevating-leg-rests-16w-seat>: HTTP status code is not handled or not allowed 2026-01-25 05:37:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guardian-k3-wheelchair-w-height-adj-full-length-arms-swing-away-footrests-18w-seat>: HTTP status code is not handled or not allowed 2026-01-25 05:37:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/curad-instant-cold-pack-24-per-box-4-boxes-per-case>: HTTP status code is not handled or not allowed 2026-01-25 05:37:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/activicetopical-pain-reliever-gel-4-oz-tube-pack-of-24>: HTTP status code is not handled or not allowed 2026-01-25 05:37:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/comprecares-reusable-compression-garment-kit-small-tall-black>: HTTP status code is not handled or not allowed 2026-01-25 05:37:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tool-trolley-w-round-ring-control-and-removable-parts-tray-vinyl-black>: HTTP status code is not handled or not allowed 2026-01-25 05:37:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torch-regulator-welding-gloves-x-large-black>: HTTP status code is not handled or not allowed 2026-01-25 05:37:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rolling-service-cart-steel-1200-lb-capacity-66-1-4l-x-30-1-4w-x-55-5-8h>: HTTP status code is not handled or not allowed 2026-01-25 05:37:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-turn-angle-stop-chrome-plated-12-pex-x-38-od-comp-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 05:37:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bu800-20bt-battery-burnisher-200ah-25-hp>: HTTP status code is not handled or not allowed 2026-01-25 05:37:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-stream-recycling-trash-can-86-gallon-black>: HTTP status code is not handled or not allowed 2026-01-25 05:37:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/countertop-microwave-900-watts-09-cuft-capacity-black>: HTTP status code is not handled or not allowed 2026-01-25 05:37:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hoist-bracket-for-t100108-8-ft-tripod>: HTTP status code is not handled or not allowed 2026-01-25 05:37:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sliding-door-cabinet-assembled-36w-x-21d-x-72h-wine-red>: HTTP status code is not handled or not allowed 2026-01-25 05:37:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bookcase-assembled-36w-x-18d-x-45h-putty>: HTTP status code is not handled or not allowed 2026-01-25 05:37:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bookcase-assembled-36w-x-13d-x-60h-pastel-green>: HTTP status code is not handled or not allowed 2026-01-25 05:37:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/teardrop-pallet-rack-starter-48-in-w-x-24-in-d-x-120-in-h>: HTTP status code is not handled or not allowed 2026-01-25 05:37:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-hype-lite-heavy-duty-vest-4xl5xl-black>: HTTP status code is not handled or not allowed 2026-01-25 05:37:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onyx-standard-safety-vest-w-lime-contrasting-trim-sm-black>: HTTP status code is not handled or not allowed 2026-01-25 05:37:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/friedrich--smart-window-air-conditioner-e-star-992-watts-115-v-12000-btu>: HTTP status code is not handled or not allowed 2026-01-25 05:37:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-wagon-truck-w-flush-deck-3000-lb-capacity-36l-x-24w-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:37:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/welded-5-shelf-service-cart-1200-lb-capacity-53-12l-x-24w-x-61-12h-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:37:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-class-2-hi-vis-safety-vest-2-pockets-mesh-lime-bk-l-xl>: HTTP status code is not handled or not allowed 2026-01-25 05:37:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-class-3-hi-vis-safety-vest-3-refl-strips-4-pockets-mesh-2xl-3xl>: HTTP status code is not handled or not allowed 2026-01-25 05:37:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-class-3-hi-vis-safety-vest-3-refl-strips-mesh-or-l-xl>: HTTP status code is not handled or not allowed 2026-01-25 05:37:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-hi-2-reflective-strips-4-pockets-lime-size-l>: HTTP status code is not handled or not allowed 2026-01-25 05:37:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-817wp-thermal-waterproof-winter-work-gloves-w-reinforced-palms-2xl-black>: HTTP status code is not handled or not allowed 2026-01-25 05:37:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8381-hi-vis-4-in-1-winter-bomber-jacket-5xl-orange>: HTTP status code is not handled or not allowed 2026-01-25 05:37:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/hvac/air_conditioners_chillers/ductless_air_conditioners/ductless_air_handlers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:37:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8287-hi-vis-winter-jacket--vest-w-detachable-sleeves-medium-lime>: HTTP status code is not handled or not allowed 2026-01-25 05:37:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8384-hi-vis-quilted-parka-winter-jacket-medium-orange>: HTTP status code is not handled or not allowed 2026-01-25 05:37:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/squids-3111fx-lanyard-w-dual-ss-carabiner-15-lb-capacity-orangegray>: HTTP status code is not handled or not allowed 2026-01-25 05:37:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chill-its-6689-cooling-long-sleeve-sun-shirt-w-uv-protection-2xl-lime>: HTTP status code is not handled or not allowed 2026-01-25 05:37:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8376k-lightweight-hi-vis-rain-suit-class-3-polyester-l-lime>: HTTP status code is not handled or not allowed 2026-01-25 05:37:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8376k-lightweight-hi-vis-rain-suit-kit-polyester-large-orange>: HTTP status code is not handled or not allowed 2026-01-25 05:38:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8372-hi-vis-hooded-sweatshirt-black-bottom-full-zip-class-3-5xl-lime>: HTTP status code is not handled or not allowed 2026-01-25 05:38:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-7031-cut-resistant-gloves-nitrile-coated-ansi-a3-xl-gray-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 05:38:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8379-hi-vis-3-in-1-winter-bomber-jacket-3xl-orange>: HTTP status code is not handled or not allowed 2026-01-25 05:38:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n-ferno-6468-duck-canvas-work-jacket-large-black>: HTTP status code is not handled or not allowed 2026-01-25 05:38:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-air-impact-wrench-atexhazardous-duty-2750-ft-lbs-torque-d-handle-inside-trigger-titanium-hammercase>: HTTP status code is not handled or not allowed 2026-01-25 05:38:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extended-air-die-grinder-14-and-6mm-collets-burr-18000-rpm-1-hp-rear-exhaust>: HTTP status code is not handled or not allowed 2026-01-25 05:38:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/right-angle-air-grinder-45-type-27-wheel-58-in-11-thread-12000-rpm-1-hp-rear-exhaust>: HTTP status code is not handled or not allowed 2026-01-25 05:38:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-duty-air-chisel-scaler-3000-bpm-14-stroke-length-093-bore>: HTTP status code is not handled or not allowed 2026-01-25 05:38:08 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/hvac/air_conditioners_chillers/ductless_air_conditioners/ductless_air_handlers landed on page that is not a product page. 2026-01-25 05:38:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-tier-18-door-digital-locker-36in-w-x-12in-d-x-78in-h-gray-assembled>: HTTP status code is not handled or not allowed 2026-01-25 05:38:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 70571 pages (at 97 pages/min), scraped 34941 items (at 37 items/min) 2026-01-25 05:38:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-tier-6-door-digital-locker-12in-w-x-18in-d-x-78in-h-tan-assembled>: HTTP status code is not handled or not allowed 2026-01-25 05:38:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/medical_laboratory_refrigeration/medical_freezers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:38:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/msa-safety--full-brim-hard-hat-vented-fas-trac-iii-hi-viz-yellow>: HTTP status code is not handled or not allowed 2026-01-25 05:38:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/momcube153-breast-milk-refrigerator-12-cuft-solid-door>: HTTP status code is not handled or not allowed 2026-01-25 05:38:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pc-board-for-w-series--tisa-auto-sealers>: HTTP status code is not handled or not allowed 2026-01-25 05:38:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heater-terminal-for-fkr-a-double-impulse-portable-sealers>: HTTP status code is not handled or not allowed 2026-01-25 05:38:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-par30-reflector-light-bulb-120v-9w-medium-screw-base-4000k-cool-white>: HTTP status code is not handled or not allowed 2026-01-25 05:38:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/top-handle-jig-saw-65-amp-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:38:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/radius-tree-shaped-bur-double-cut-1-12l-x-18-shank-dia-14-len-of-cut>: HTTP status code is not handled or not allowed 2026-01-25 05:38:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pointed-cone-bur-single-cut-2-18l-x-18-shank-dia>: HTTP status code is not handled or not allowed 2026-01-25 05:38:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cylindrical-no-endcut-bur-2-12l-x-14-shank-dia-516-cut-dia>: HTTP status code is not handled or not allowed 2026-01-25 05:38:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pointed-tree-shaped-bur-double-cut-7l-x-14-shank-dia>: HTTP status code is not handled or not allowed 2026-01-25 05:38:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cylindrical-no-endcut-bur-double-cut-2-34l-x-14-shank-dia-1-cut-dia>: HTTP status code is not handled or not allowed 2026-01-25 05:38:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lite-trapped-blister-cordless-rotary-kit-36v>: HTTP status code is not handled or not allowed 2026-01-25 05:38:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-round-mini-panel-motion-sensor-fixture-18w-1400-lumens-90-cri-12-size-white>: HTTP status code is not handled or not allowed 2026-01-25 05:38:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/medical_laboratory_cold_storage already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:38:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t5-led-tube-light-bulb-miniature-bi-pin-base-16w-1800-lumens-3000k-warm-white>: HTTP status code is not handled or not allowed 2026-01-25 05:38:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10mm-tungsten-carbide-grinding-balls>: HTTP status code is not handled or not allowed 2026-01-25 05:38:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/myfuge-mini-centrifuge-w-2-rotors-6000-rpm-240v-clear-lid>: HTTP status code is not handled or not allowed 2026-01-25 05:38:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precision-balance-1200g-capacity-001g-readability-115v>: HTTP status code is not handled or not allowed 2026-01-25 05:38:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tuberoller-w-5-rollers-230v-5060-hz-2-80-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:38:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/100-ultrasound-gel-5-liters-refillable-dispenser>: HTTP status code is not handled or not allowed 2026-01-25 05:38:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sentrysafe-file-organizer-accessory-for-16-20-cubic-feet-safes>: HTTP status code is not handled or not allowed 2026-01-25 05:38:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-octagon-sign-base-w-wheels-60-lb-black>: HTTP status code is not handled or not allowed 2026-01-25 05:38:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbi-sala174-sealed-blok-self-retracting-lifeline-stainless-steel-cable-180l>: HTTP status code is not handled or not allowed 2026-01-25 05:38:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tread-plate-deck-platform-truck-aluminum-6-polyurethane-caster-63l-x-36w-2600-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 05:38:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/medical_laboratory_cold_storage>: HTTP status code is not handled or not allowed 2026-01-25 05:38:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/excelon-plus-oil-vapour-removal-filter-w-closed-metal-bowl-12-iso-g-0003-m>: HTTP status code is not handled or not allowed 2026-01-25 05:38:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hookit-paper-disc-236u-p400-grit-3-dia-x-nh-pack-of-500000>: HTTP status code is not handled or not allowed 2026-01-25 05:38:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/optional-bottom-shelf-for-30w-tmt-cart-28w-x-28-1116d-x-1-516h-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:38:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-starter-36l-x-32w-x-48h-antique-white>: HTTP status code is not handled or not allowed 2026-01-25 05:38:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-add-on-48l-x-24w-x-72h-black>: HTTP status code is not handled or not allowed 2026-01-25 05:38:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-add-on-48l-x-36w-x-96h-antique-white>: HTTP status code is not handled or not allowed 2026-01-25 05:38:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-starter-36l-x-36w-x-54h-black>: HTTP status code is not handled or not allowed 2026-01-25 05:38:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-starter-48l-x-24w-x-84h-antique-white>: HTTP status code is not handled or not allowed 2026-01-25 05:38:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/optimus-power-scissor-lift-table-foot-operated-5000-cap-1ph-230v-56l-x-32w>: HTTP status code is not handled or not allowed 2026-01-25 05:38:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/marque-plexi-triple-unit-reception-desk-workstation-143-12w-x-71d-x-45-12h-cherry>: HTTP status code is not handled or not allowed 2026-01-25 05:38:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/marque-plexi-single-unit-reception-desk-workstation-72w-x-32d-x-45-12h-driftwood-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:38:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/squids-5542-phone-style-barcode-scanner-holster-w-belt-loop-small-black>: HTTP status code is not handled or not allowed 2026-01-25 05:38:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cylinder-graduated-globe-glass-class-b-to-deliver-td-dual-grads-astm-e1272-250ml-2-box>: HTTP status code is not handled or not allowed 2026-01-25 05:38:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/skullerz-aegir-afas-half-frame-safety-glasses-clear-lens-matte-black-frame>: HTTP status code is not handled or not allowed 2026-01-25 05:38:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-drum-strap-36986>: HTTP status code is not handled or not allowed 2026-01-25 05:38:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/skullerz-loki-afas-safety-spoggles-w-strap-clear-lens-matte-black-frame>: HTTP status code is not handled or not allowed 2026-01-25 05:38:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-flex-ii-vinyl-bulk-roll-low-temp-scratch-guard-8-x-0198-x-75-clear>: HTTP status code is not handled or not allowed 2026-01-25 05:38:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spout-cap-black-small-34404>: HTTP status code is not handled or not allowed 2026-01-25 05:38:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/skullerz-8947-lightweight-baseball-hat-bump-cap-insert-ml-lime>: HTTP status code is not handled or not allowed 2026-01-25 05:38:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/array153-signal-iaq-sensor-monitor-120v-ac-white>: HTTP status code is not handled or not allowed 2026-01-25 05:38:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-recycled-paper-towel-rolls-by-gp-pro-brown-6-rolls-case>: HTTP status code is not handled or not allowed 2026-01-25 05:38:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear--mild-foam-hand-wash-unscented-1250-ml-capacity-pack-of-3>: HTTP status code is not handled or not allowed 2026-01-25 05:38:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/digital-healthcare-scale-400-lb-cap-12l-x-12w-platform>: HTTP status code is not handled or not allowed 2026-01-25 05:38:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h1-2-bar-aluminum-ladder-rack-for-ram-promaster-cargo-van-2013-on-white>: HTTP status code is not handled or not allowed 2026-01-25 05:38:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h1-2-bar-aluminum-ladder-rack-system-for-dodge-ram-van-1981-on-white>: HTTP status code is not handled or not allowed 2026-01-25 05:38:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-polypropylene-rope-per-foot-7200l-x-12w-yellow>: HTTP status code is not handled or not allowed 2026-01-25 05:38:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-ratchet-tie-down-for-h2-rack-system>: HTTP status code is not handled or not allowed 2026-01-25 05:38:56 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/medical-lab/medical_devices_instruments/vital_signs_monitors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:38:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precision-series-trash-container-w-cigarette-receptacle-40-gal-cap-silver>: HTTP status code is not handled or not allowed 2026-01-25 05:38:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hip-pack-w-guard-logo-first-aid-supply-royal-blue-49-pieces>: HTTP status code is not handled or not allowed 2026-01-25 05:38:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cprb-led-round-high-bay-120001500018000-lumens-40005000k-80-cri-white>: HTTP status code is not handled or not allowed 2026-01-25 05:38:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-ultimate-ems-backpack-13l-x-18w-x-20h-navy-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:38:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mdg100df4-mobile-diesel-generator-80-85kw-j-d-engine-w-single-axle-trailer>: HTTP status code is not handled or not allowed 2026-01-25 05:39:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/military-grade-cargo-lifting-net-4500-lb-capacity-10-opening-168l-x-168w-black>: HTTP status code is not handled or not allowed 2026-01-25 05:39:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-wide-body-diesel-vertical-mast-led-light-tower-8kw-mit-engine-electric-winch>: HTTP status code is not handled or not allowed 2026-01-25 05:39:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/military-grade-cargo-lifting-net-4500-lb-capacity-10-opening-72l-x-72w-black>: HTTP status code is not handled or not allowed 2026-01-25 05:39:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-ceramic-utility-heater-w-pivot-power-1500w-black>: HTTP status code is not handled or not allowed 2026-01-25 05:39:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60bs33x1716-34-pitch-1-716-finished-bore-33-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:39:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/medical-lab/medical_devices_instruments/vital_signs_monitors>: HTTP status code is not handled or not allowed 2026-01-25 05:39:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-w-locker-bench-laminate-top-steel-tube-pedestals>: HTTP status code is not handled or not allowed 2026-01-25 05:39:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-20bs12hx32-metric-1-14-pitch-32mm-finished-bore-12-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:39:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-120e22h-qd-bushed-1-12-pitch-22-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:39:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 70665 pages (at 94 pages/min), scraped 34969 items (at 28 items/min) 2026-01-25 05:39:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ada-height-upright-controlled-room-temperature-cabinet-6-cuft-capacity-glass-door>: HTTP status code is not handled or not allowed 2026-01-25 05:39:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/upright-controlled-room-temperature-cabinet-6-cuft-capacity-115-lb-weight-solid-door>: HTTP status code is not handled or not allowed 2026-01-25 05:39:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tuff-job-windshield-towels-2-ply-9-1-2-inch-x-10-1-4-inch-blue-168-pack-12-packs-carton>: HTTP status code is not handled or not allowed 2026-01-25 05:39:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/intensive-care-essential-healing-body-lotion-3-4-oz-squeeze-tube-12-carton>: HTTP status code is not handled or not allowed 2026-01-25 05:39:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/healthcare-advanced-gentle-foam-hand-sanitizer-1200-ml-refill-for-es8-dispensers-2pk>: HTTP status code is not handled or not allowed 2026-01-25 05:39:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strong-scented-tall-white-kitchen-bags-13-gal-0-9-mil-white-80-box>: HTTP status code is not handled or not allowed 2026-01-25 05:39:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trailgrip-work-athletic-hiker-boots-alloy-toe-size-11-5w-4-inch-h-coyote-black>: HTTP status code is not handled or not allowed 2026-01-25 05:39:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35bs35hx1316-38-pitch-1-316-finished-bore-35-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:39:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-41bs20hx34-12-pitch-34-finished-bore-20-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:39:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35bs26hx118-38-pitch-1-18-finished-bore-26-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:39:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-safety-crafted-waterproof-logger-work-boots-steel-toe-size-6w-8-inch-h-dark-brown>: HTTP status code is not handled or not allowed 2026-01-25 05:39:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80sf34-1-pitch-qd-bushed-34-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:39:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-safety-crafted-waterproof-logger-work-boots-steel-toe-size-8-5w-8-inch-h-black>: HTTP status code is not handled or not allowed 2026-01-25 05:39:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-24bs16hx48-metric-1-12-pitch-48mm-finished-bore-16-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:39:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-24bs15hx45-metric-1-12-pitch-45mm-finished-bore-15-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:39:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/indoor-flashing-led-sign-24-w-x-24-h-white-red>: HTTP status code is not handled or not allowed 2026-01-25 05:39:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-41a36x34-a-plate-12-pitch-34-bore-36-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:39:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trailgrip-work-athletic-hiker-boots-alloy-toe-size-8w-4-inch-h-coyote-black>: HTTP status code is not handled or not allowed 2026-01-25 05:39:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/defib-pads-2346n-6-inch-x-4-5-inch-10-pack-carton-10-carton-case>: HTTP status code is not handled or not allowed 2026-01-25 05:39:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/groundbreaker-work-boots-w-internal-met-guard-steel-toe-size-8-5w-6-inch-h-brown>: HTTP status code is not handled or not allowed 2026-01-25 05:39:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steri-drape-pouch-with-ioban-2-incise-film-6697-122-inch-x-77-inch-5-each-case>: HTTP status code is not handled or not allowed 2026-01-25 05:39:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:39:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zig-elusion-heritage-work-high-top-sneaker-composite-toe-size-13m-black-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:39:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zig-elusion-heritage-work-men-s-low-cut-sneaker-composite-toe-size-10w-gray-black>: HTTP status code is not handled or not allowed 2026-01-25 05:39:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/truflex-fly-skylar-work-zip-bootie-composite-toe-size-6m-4-inch-h-black>: HTTP status code is not handled or not allowed 2026-01-25 05:39:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zig-elusion-heritage-work-men-s-low-cut-sneaker-composite-toe-size-7-5m-black-red>: HTTP status code is not handled or not allowed 2026-01-25 05:39:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/groundbreaker-work-boots-w-internal-met-guard-steel-toe-size-12m-6-inch-h-brown>: HTTP status code is not handled or not allowed 2026-01-25 05:39:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zig-elusion-heritage-work-high-top-sneaker-composite-toe-size-10-5w-black-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:39:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-boltless-shelving-72-in-w-x-36-in-d-x-84-in-h-5-shelves-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:39:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-100bs19hx114-1-14-pitch-1-14-finished-bore-19-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:39:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-microcentrifuge-tubes-with-cap-sterile-1-5-ml-clear-500-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:39:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-10bs16hx32-metric-58-pitch-32mm-finished-bore-16-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:39:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-25 05:39:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/truflex-fly-skylar-sneaker-composite-toe-size-10m-black>: HTTP status code is not handled or not allowed 2026-01-25 05:39:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-120a36x114-a-plate-1-12-pitch-1-14-bore-36-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:39:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-140btl21-1-34-pitch-taper-bushed-21-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:39:39 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:39:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-halo-single-channel-pipettor-lightweight-2-20ul>: HTTP status code is not handled or not allowed 2026-01-25 05:39:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-workstation-rack-box-for-0-2-ml-tubes-strips-plates-96-place-orange-5-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:39:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:39:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:39:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-microcentrifuge-tubes-sterile-1-5-ml-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:39:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-propette-le-8-channel-pipette-1-10ul>: HTTP status code is not handled or not allowed 2026-01-25 05:39:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:39:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/true-skate-inspired-work-shoes-composite-toe-size-8-5m-black>: HTTP status code is not handled or not allowed 2026-01-25 05:39:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/true-skate-inspired-work-shoes-composite-toe-size-14w-black>: HTTP status code is not handled or not allowed 2026-01-25 05:39:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-100btl35-taper-bushed-1-14-pitch-35-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:39:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polaris-600-snowmobile-replacement-battery-1985-1987-12v-i>: HTTP status code is not handled or not allowed 2026-01-25 05:39:49 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-25 05:39:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/true-skate-inspired-work-shoes-composite-toe-size-10-5w-navy>: HTTP status code is not handled or not allowed 2026-01-25 05:39:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/truflex-fly-mudguard-slip-on-sneaker-composite-toe-size-12m-black>: HTTP status code is not handled or not allowed 2026-01-25 05:39:52 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-25 05:39:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35b28hx12mpb-38-pitch-12-plain-bore-28-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:39:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60bs13hx1-34-pitch-1-finished-bore-13-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:39:53 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-25 05:39:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kawasaki-mule-2010-540cc-utv-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:39:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-40bs18hx1-12-pitch-1-finished-bore-18-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:39:55 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-25 05:39:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35bs28hx114-38-pitch-1-14-finished-bore-28-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:39:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ge-amx-iii-medical-replacement-battery-18ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:39:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50sds25h-qd-bushed-58-pitch-25-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:39:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stone-skate-inspired-work-shoes-composite-toe-size-11-5w-gray-black>: HTTP status code is not handled or not allowed 2026-01-25 05:40:00 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:40:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nanoflex-tr-athletic-work-shoes-composite-toe-size-12m-navy-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:40:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35sh48-38-pitch-qd-bushed-48-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:40:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50a27hx34-a-plate-58-pitch-34-bore-27-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:40:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premier-work-casual-slip-on-sneaker-steel-toe-size-13d-cognac>: HTTP status code is not handled or not allowed 2026-01-25 05:40:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premier-work-casual-slip-on-sneaker-steel-toe-size-12eee-cognac>: HTTP status code is not handled or not allowed 2026-01-25 05:40:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premier-work-casual-oxford-shoes-steel-toe-size-13eee-navy>: HTTP status code is not handled or not allowed 2026-01-25 05:40:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-12bs15hx25-metric-34-pitch-25mm-finished-bore-15-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:40:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paper-twist-ties-4-x-5-32-green-2000-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:40:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paper-twist-ties-12-x-5-32-green-2000-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:40:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adi-vista-10se-alarm-replacement-battery-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:40:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bb-hr8-12-sealed-lead-acid-replacement-battery-8ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:40:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 70754 pages (at 89 pages/min), scraped 34990 items (at 21 items/min) 2026-01-25 05:40:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-12bs16hx35-metric-34-pitch-35mm-finished-bore-16-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:40:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-25 05:40:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-bags-12-x-15-2-mil-blue-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:40:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-10bs22hx24-metric-58-pitch-24mm-finished-bore-22-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:40:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stone-op-art-skate-inspired-work-shoes-composite-toe-size-11m-dark-gray-charcoal>: HTTP status code is not handled or not allowed 2026-01-25 05:40:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nanoflex-tr-athletic-work-shoes-composite-toe-size-6-5w-black-seafoam-green-white>: HTTP status code is not handled or not allowed 2026-01-25 05:40:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-twist-ties-12-x-5-32-yellow-2000-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:40:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nanoflex-tr-athletic-work-shoes-composite-toe-size-9-5w-navy-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:40:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stone-op-art-skate-inspired-work-shoes-composite-toe-size-7-5w-dark-gray-charcoal>: HTTP status code is not handled or not allowed 2026-01-25 05:40:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stone-op-art-skate-inspired-work-shoes-composite-toe-size-12w-dark-brown-khaki>: HTTP status code is not handled or not allowed 2026-01-25 05:40:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-10bs21hx32-metric-58-pitch-32mm-finished-bore-21-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:40:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/severance-chatter-free-stop-countersink-cutter-90-3-8-30-pilot-hole>: HTTP status code is not handled or not allowed 2026-01-25 05:40:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/severance-chatter-free-stop-countersink-cutter-82-3-4-3-8-pilot-hole>: HTTP status code is not handled or not allowed 2026-01-25 05:40:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/r-d-5391-sealed-lead-acid-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:40:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60bs16hx114-34-pitch-1-14-finished-bore-16-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:40:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ledm001nkl-1912-led-ceiling-fixtures-35w-4000-cct-2900-lumens-82-cri-white>: HTTP status code is not handled or not allowed 2026-01-25 05:40:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leduc42brz-42-led-undercabinet-16w-3000-cct-1080-lumens-82-cri-bronze>: HTTP status code is not handled or not allowed 2026-01-25 05:40:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-40bs28hx1-12-pitch-1-finished-bore-28-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:40:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simplex-112-113-alarm-replacement-battery-100ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:40:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ledsm4dnkl-4-dimmable-led-can-disk-light-15w-3000-cct-1095-lumen-82-cri-bronze>: HTTP status code is not handled or not allowed 2026-01-25 05:40:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/solex-x-ptx-b4-12-alarm-replacement-battery-4-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:40:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securitron-bps241-alarm-replacement-battery-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:40:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-ingredient-bin-148bin-wh-14-5-8-l-x-9-1-4-w-x-23-1-4-h-white>: HTTP status code is not handled or not allowed 2026-01-25 05:40:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-40bs26hx118-12-pitch-1-18-finished-bore-26-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:40:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/altronix-lps5c12x-alarm-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:40:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nano-x1-adventure-athletic-work-shoes-composite-toe-size-11m-slate-blue-salmon>: HTTP status code is not handled or not allowed 2026-01-25 05:40:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-08bs24hx22-metric-12-pitch-22mm-finished-bore-24-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:40:28 [py.warnings] (PID: 96) WARNING: /var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/extensions/bq_feedstorage.py:33: ScrapyDeprecationWarning: scrapy.extensions.feedexport.build_storage() is deprecated, call the builder directly. 2026-01-25 05:40:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:40:29 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (1000 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:35 2026-01-25 05:40:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6001-super-tugger-cable-puller-power-unit>: HTTP status code is not handled or not allowed 2026-01-25 05:40:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96-w-x-30-d-pedestal-workbench-w-7-drawers-plastic-laminate-square-edge-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:40:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60btl11h-taper-bushed-34-pitch-11-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:40:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securitron-m62g-alarm-replacement-battery-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:40:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-40bs15hx78-12-pitch-78-finished-bore-15-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:40:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lavante-trail-2-running-work-shoes-composite-toe-size-8m-gray-red-black>: HTTP status code is not handled or not allowed 2026-01-25 05:40:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nano-x1-adventure-athletic-work-mid-cut-shoes-composite-toe-size-5m-black>: HTTP status code is not handled or not allowed 2026-01-25 05:40:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/altronix-smp7pmp8cb-alarm-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:40:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80bs21x2716-1-pitch-2-716-finished-bore-21-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:40:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-25bs18hx12-14-pitch-12-finished-bore-18-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:40:40 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-25 05:40:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lavante-trail-2-running-work-shoes-composite-toe-size-13m-gray-black>: HTTP status code is not handled or not allowed 2026-01-25 05:40:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-25a40x38-a-plate-14-pitch-38-bore-40-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:40:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/damaco-d90-wheelchair-replacement-battery-18ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:40:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-d60bs23hx112-double-34-pitch-1-12-finished-bore-23-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:40:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nano-x1-adventure-athletic-work-shoes-composite-toe-size-11-5m-gray-black>: HTTP status code is not handled or not allowed 2026-01-25 05:40:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merits-p314-wheelchair-replacement-battery-55ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:40:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nano-x1-adventure-athletic-work-mid-cut-shoes-composite-toe-size-8w-black>: HTTP status code is not handled or not allowed 2026-01-25 05:40:46 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:40:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merits-p31362-wheelchair-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:40:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-06a17h-metric-a-plate-38-pitch-10mm-bore-17-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:40:48 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:40:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-40bs12hx12-12-pitch-12-finished-bore-12-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:40:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evolve-skate-inspired-high-top-work-shoes-composite-toe-size-9m-triple-black>: HTTP status code is not handled or not allowed 2026-01-25 05:40:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/food_storage/food_storage_containers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:40:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evolve-skate-inspired-high-top-work-shoes-composite-toe-size-8-5w-triple-black>: HTTP status code is not handled or not allowed 2026-01-25 05:40:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hybrid-skate-inspired-work-shoes-composite-toe-size-10w-navy-celestial-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:40:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/food_storage/food_storage_containers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:40:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60bs30hx134-34-pitch-1-34-finished-bore-30-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:40:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/union-mx-12170-wheelchair-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:40:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/foodservice/food_storage/food_storage_containers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:40:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/floatride-energy-3-adventure-men-s-athletic-work-shoes-composite-toe-size-6m-black>: HTTP status code is not handled or not allowed 2026-01-25 05:40:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-25 05:40:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lawn-boy-lawn-and-garden-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 05:40:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-25 05:40:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hybrid-skate-inspired-work-shoes-composite-toe-size-8w-navy-celestial-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:40:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelter-deluxe-log-rack-cover-small-slrcd-s>: HTTP status code is not handled or not allowed 2026-01-25 05:41:00 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/food_storage/food_storage_containers landed on page that is not a product page. 2026-01-25 05:41:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evolve-skate-inspired-work-shoes-composite-toe-size-14m-olive-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:41:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-tapered-roller-bearing-e32207j-35mm-bore-72mm-od>: HTTP status code is not handled or not allowed 2026-01-25 05:41:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/foodservice/food_storage/food_storage_containers>: HTTP status code is not handled or not allowed 2026-01-25 05:41:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bb4500-work-high-top-work-sneaker-composite-toe-size-9m-black-holographic-spectrum>: HTTP status code is not handled or not allowed 2026-01-25 05:41:04 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/foodservice/food_storage/food_storage_containers landed on page that is not a product page. 2026-01-25 05:41:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bb4500-work-high-top-work-sneaker-composite-toe-size-9-5m-black-holographic-spectrum>: HTTP status code is not handled or not allowed 2026-01-25 05:41:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-ds70-70cc-atv-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:41:05 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:41:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-u-handle-547l-102-dia-black-m8-516-socket-cap-screw>: HTTP status code is not handled or not allowed 2026-01-25 05:41:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reportpro-10-pocket-project-organizer-letter-black>: HTTP status code is not handled or not allowed 2026-01-25 05:41:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/side-opening-pressguard-report-cover-prong-fastener-letter-yellow>: HTTP status code is not handled or not allowed 2026-01-25 05:41:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 70850 pages (at 96 pages/min), scraped 35016 items (at 26 items/min) 2026-01-25 05:41:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honeywell-pressuretrol-limit-controller-l4079b1033-manual-reset-215-psi-141-kgcm>: HTTP status code is not handled or not allowed 2026-01-25 05:41:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-channel-4-cord-dropover-cable-guard-red>: HTTP status code is not handled or not allowed 2026-01-25 05:41:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/liftmaster-475-garage-door-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:41:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/46-square-expanded-metal-picnic-table-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:41:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/46-ada-square-expanded-metal-picnic-table-black>: HTTP status code is not handled or not allowed 2026-01-25 05:41:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-slide-base-w445t>: HTTP status code is not handled or not allowed 2026-01-25 05:41:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bmw-k75s-750cc-atv-replacement-battery-1985-1987-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:41:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes>: HTTP status code is not handled or not allowed 2026-01-25 05:41:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42u-rack-enclosure-server-cabinet-doors-and-sides-2400lb-capacity-sr2400>: HTTP status code is not handled or not allowed 2026-01-25 05:41:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/daystart-slip-on-work-shoes-steel-toe-size-12m-navy>: HTTP status code is not handled or not allowed 2026-01-25 05:41:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bb4500-work-high-top-work-sneaker-composite-toe-size-8-5w-black-holographic-spectrum>: HTTP status code is not handled or not allowed 2026-01-25 05:41:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commercial-bottleless-cold-cooler-w-filtration-black-w-blue-trim>: HTTP status code is not handled or not allowed 2026-01-25 05:41:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evo-120-electric-scooter-replacement-battery-5ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 05:41:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bb4500-work-high-top-work-sneaker-composite-toe-size-11-5m-black-holographic-spectrum>: HTTP status code is not handled or not allowed 2026-01-25 05:41:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:41:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pm-dc-motor-wpmdc2-18-180v-145tcb-tenv-tefc-145tc-180v-2-hp-1800-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:41:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vapor-x12-electric-scooter-replacement-battery-7ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 05:41:22 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:41:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bb4500-work-high-top-work-sneaker-composite-toe-size-10-5w-black-holographic-spectrum>: HTTP status code is not handled or not allowed 2026-01-25 05:41:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/z-grip-retractable-ballpoint-pen-black-ink-medium-24-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:41:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/general_purpose_electric_motors/three_phase_motors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:41:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cc-pump-motor-wwe3-36-182jm-tefc-rigid-c-3-ph-182jm-3-hp-3600-rpm>: HTTP status code is not handled or not allowed 2026-01-25 05:41:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-gtx-500ss-597cc-snowmobile-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:41:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/definite_purpose_electric_motors/severe_duty_motors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:41:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/can-am-defender-xt-1000cc-utv-replacement-battery-2016-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:41:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-frame-motor-wwht10-12-256tc-gp-tefc-rigid-c-3-ph-256tc-134-fla>: HTTP status code is not handled or not allowed 2026-01-25 05:41:30 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-25 05:41:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-gsx-380f-377cc-snowmobile-replacement-battery-2005-2006-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:41:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-plate-leveling-manual-caster-110-lbs-63mm-dia-nylon-wheel-2-5-32-x-2-5-32-plate>: HTTP status code is not handled or not allowed 2026-01-25 05:41:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbi-sala-nano-lok-xl-self-retracting-lifeline-w-quick-connect-steel-rebar-lock-hook-9l>: HTTP status code is not handled or not allowed 2026-01-25 05:41:33 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_shoes landed on page that is not a product page. 2026-01-25 05:41:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/apc-smart-ups-sc-620va-sc620-ups-replacement-battery-10ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 05:41:35 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/general_purpose_electric_motors/three_phase_motors landed on page that is not a product page. 2026-01-25 05:41:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-extinguisher-cabinet-solid-door-semi-recessed-6d-steel-3-rolled>: HTTP status code is not handled or not allowed 2026-01-25 05:41:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dunkin-hazelnut-coffee-regular-medium-ground-k-cup-pods-037-oz-capacity-pack-of-22>: HTTP status code is not handled or not allowed 2026-01-25 05:41:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carton-sealing-tape-rejected-2-x-55-yds-blackred-pack-of-36>: HTTP status code is not handled or not allowed 2026-01-25 05:41:38 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/electric_motors/definite_purpose_electric_motors/severe_duty_motors landed on page that is not a product page. 2026-01-25 05:41:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-rated-insulated-metal-access-panel-14l-x-14w>: HTTP status code is not handled or not allowed 2026-01-25 05:41:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/parks-medical-2-doppler-medical-replacement-battery-1-3ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:41:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ft-one-fit-non-belted-full-body-harness-standard-3-d-ring-quick-connect-legs-small>: HTTP status code is not handled or not allowed 2026-01-25 05:41:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/greenline-636-hand-truck-w-dual-handle-solid-rubber-wheels-500-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 05:41:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kontron-1334336-01334336-medical-replacement-battery-10ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:41:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ft-one-fit-non-belted-full-body-harness-standard-3-d-ring-tongue-buckle-legs-small>: HTTP status code is not handled or not allowed 2026-01-25 05:41:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usb-30-charging-hub-2x-usb-30-4x-usb-20-1x-charging-ipad2>: HTTP status code is not handled or not allowed 2026-01-25 05:41:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hill-rom-versacare-bed-medical-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:41:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mark-bracket-for-wall-mounting-of-fire-extinguisher-for-models-cosmic-6>: HTTP status code is not handled or not allowed 2026-01-25 05:41:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rotary-screw-air-compressor-w-dryer-230v-115psi-185cfm-5hp-3ph-120-gal-tank>: HTTP status code is not handled or not allowed 2026-01-25 05:41:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rotary-screw-air-compressor-w-dryer-200v-115psi-28cfm-75hp-3ph-120-gal-tank>: HTTP status code is not handled or not allowed 2026-01-25 05:41:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/digitech-safety-view-plus-4-tier-4-doors-wardrobe-locker-12w-x-18d-x-78h-assembled>: HTTP status code is not handled or not allowed 2026-01-25 05:41:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jump-n-carry-jnc100-jump-starter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:41:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ready-built-ii153-1-tier-1-door-locker-12w-x-15d-x-83h-black-assembled>: HTTP status code is not handled or not allowed 2026-01-25 05:41:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2500-nonslip-coating-durable-antislip-paint-5-gal-safety-yellow-2500-ye-pl>: HTTP status code is not handled or not allowed 2026-01-25 05:41:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-blade-for-bigboy-360mm-medium-teeth>: HTTP status code is not handled or not allowed 2026-01-25 05:41:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tq3104-air-compressor-5hp-240v-80-gal>: HTTP status code is not handled or not allowed 2026-01-25 05:41:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/amstar-ams2000-jump-starter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:41:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/100-amp-2500-watt-single-load-manual-transfer-switch>: HTTP status code is not handled or not allowed 2026-01-25 05:41:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-tech-closed-stand-alone-shelving-6-shelves-375-lb-cap-48w-x-18d-x-87h-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:41:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vent-cap-slip-mount-3-1>: HTTP status code is not handled or not allowed 2026-01-25 05:41:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-level-drum-alarm-polypropylene-threaded-collar-knob-55-gallon-1>: HTTP status code is not handled or not allowed 2026-01-25 05:42:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powered-drum-truck-f89484l-locking-cover-chime-hook-12-wheels>: HTTP status code is not handled or not allowed 2026-01-25 05:42:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/health-o-meter-professional-height-rod-for-499kl-and-599kl-scales>: HTTP status code is not handled or not allowed 2026-01-25 05:42:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/closed-side-base-for-locker-18w-x-18d-x-6h-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:42:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/calm50-ta-torque-arm-for-calm-series-50mm-aluminum-worm-gear-reducer>: HTTP status code is not handled or not allowed 2026-01-25 05:42:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caterpillar-e110-industrial-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:42:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/calm40-201-56c-aluminum-worm-gear-reducer-40mm-201-56c-nema-frame>: HTTP status code is not handled or not allowed 2026-01-25 05:42:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bb-bp35-12f-sealed-lead-acid-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:42:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sensory-table-mat-large-54-x-72-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:42:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/book-browser-flushback>: HTTP status code is not handled or not allowed 2026-01-25 05:42:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 70944 pages (at 94 pages/min), scraped 35046 items (at 30 items/min) 2026-01-25 05:42:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/koyo-np20-12-sealed-lead-acid-replacement-battery-18ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:42:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vision-cp12120-sealed-lead-acid-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 05:42:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leoch-lp12-7-2-sealed-lead-acid-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:42:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-in-stainless-steel-wall-exhaust-shutter>: HTTP status code is not handled or not allowed 2026-01-25 05:42:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mighty-products-mn-dn-0810-debris-safety-netting-8-x-10>: HTTP status code is not handled or not allowed 2026-01-25 05:42:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-tech-open-style-metal-shelving-8-shelves-800-lb-cap-36w-x-18d-x-87h-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:42:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shockstrap-tie-downs-ratchet-strap-3000-lb-capacity-15l-x-1-12w-black>: HTTP status code is not handled or not allowed 2026-01-25 05:42:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/champion-7000-sealed-lead-acid-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:42:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cove-base-vinyl-4x080x48-fawn>: HTTP status code is not handled or not allowed 2026-01-25 05:42:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e-track-strap-link-coupling-cinch-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-25 05:42:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exell-eb1250f1-sealed-lead-acid-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:42:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-dolly-pdsh4248-6ph-with-pipe-handle-solid-deck-42-x-48>: HTTP status code is not handled or not allowed 2026-01-25 05:42:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spark-resistant-manual-chain-hoist-5-ton-capacity-10-lift>: HTTP status code is not handled or not allowed 2026-01-25 05:42:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/instrument-cart-with-hand-guard-30-x-48>: HTTP status code is not handled or not allowed 2026-01-25 05:42:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exair-nano-super-air-nozzle-mnpt-14-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 05:42:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exair-soft-grip-safety-air-gun-wadjustable-super-air-nozzle-72-extension-and-chip-shield-zinc>: HTTP status code is not handled or not allowed 2026-01-25 05:42:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cryo-coder-for-star-round-base-vials-blue-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 05:42:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adt-safewatch-pro-3000-alarm-replacement-battery-4-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:42:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n242424-2-shelving-unit-2-shelf-24w-x-24h-x-24l-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 05:42:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spark-resistant-push-beam-trolley-12-ton-capacity>: HTTP status code is not handled or not allowed 2026-01-25 05:42:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n204836-4-chl2-mobile-shelving-unit-4-shelf-20w-x-54h-x-36l-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 05:42:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/altronix-al1002ul2adaj-alarm-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 05:42:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dehumidifier-w-wifi-115v-50-pint-4500-sq-ft-coverage>: HTTP status code is not handled or not allowed 2026-01-25 05:42:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eaton-ex-1500-rt2u-pulsl1500r-xl2u-86704-12v-7ah-ups-battery>: HTTP status code is not handled or not allowed 2026-01-25 05:42:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-abs-plastic-bowl-for-open-bowl-eye-face-wash-station>: HTTP status code is not handled or not allowed 2026-01-25 05:42:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/johnson-and-johnson-adhesive-bandages-flexible-1l-100bx-fabric-beige>: HTTP status code is not handled or not allowed 2026-01-25 05:42:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oneac-desk-power-650-ups-battery-kit-specifications-12v-7ah-battery>: HTTP status code is not handled or not allowed 2026-01-25 05:42:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/petri-dishes-w-two-compartments-3-916dia-x-916h-pack-of-500>: HTTP status code is not handled or not allowed 2026-01-25 05:42:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merits-p12011-tbmu-wheelchair-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:42:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bike-dock-2-bike-flange-mount-436755>: HTTP status code is not handled or not allowed 2026-01-25 05:42:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-stirring-rods-pp-8l-x-14-dia-white-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-25 05:42:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/type-ii-accuflow-steel-safety-can-5-gal-5-8-metal-hose-blue-7250320>: HTTP status code is not handled or not allowed 2026-01-25 05:42:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sears-16375-16376-wheelchair-replacement-battery-55ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:42:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6216zz-p53-deep-groove-ball-bearing-double-shielded-abec-5-80mm-bore-140mm-od>: HTTP status code is not handled or not allowed 2026-01-25 05:42:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leisure-lift-u1-wheelchair-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:42:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-straight-leg-slotted-shim-130-tool-crib-kit-box-b>: HTTP status code is not handled or not allowed 2026-01-25 05:42:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/62102rsnr-deep-groove-ball-bearing-double-sealed-snap-ring-50mm-bore-90mm-od>: HTTP status code is not handled or not allowed 2026-01-25 05:42:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cryo-rack-for-2ml-capacity-vials-81-places-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-25 05:42:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quickie-p222-gp24-agm-wheelchair-replacement-battery-75ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:42:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/test-tube-rack-plastic-round-shaped-24-places-white>: HTTP status code is not handled or not allowed 2026-01-25 05:42:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/63022rs-p53-deep-groove-ball-bearing-double-sealed-abec-5-15mm-bore-42mm-od>: HTTP status code is not handled or not allowed 2026-01-25 05:42:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quantum-rehab-q1420-wheelchair-replacement-battery-75ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:42:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6303zz-p53-deep-groove-ball-bearing-double-shielded-abec-5-17mm-bore-47mm-od>: HTTP status code is not handled or not allowed 2026-01-25 05:42:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evaporator-fan-motor-assembly-for-global-industrial-1-2-ton-portable-outdoor-ac>: HTTP status code is not handled or not allowed 2026-01-25 05:43:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-tier-1-door-wood-locker-12-w-x-15-d-x-72-h-gray-assembled>: HTTP status code is not handled or not allowed 2026-01-25 05:43:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crown-cltxpa1222nb-wheelchair-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:43:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/type-1-fiberglass-step-ladder-4-ft-dxl311004>: HTTP status code is not handled or not allowed 2026-01-25 05:43:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-cart-black-epoxy-42l-x-24w-x-40h-polyurethane-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 05:43:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:43:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6209zz-deep-groove-ball-bearing-double-shielded-45mm-bore-85mm-od>: HTTP status code is not handled or not allowed 2026-01-25 05:43:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelf-truck-54l-x-24w-x-69h-polyurethane-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 05:43:07 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:43:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-w-x-30-d-standing-height-workbench-maple-butcher-block-square-edge-black>: HTTP status code is not handled or not allowed 2026-01-25 05:43:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/marksalot-permanent-marker-large-chisel-tip-brown-ink>: HTTP status code is not handled or not allowed 2026-01-25 05:43:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 71036 pages (at 92 pages/min), scraped 35086 items (at 40 items/min) 2026-01-25 05:43:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sears-520-2581-lawn-and-garden-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:43:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/energize-deluxe-automatic-pencil-latex-rubber-grip-05mm-silverblack-barrel>: HTTP status code is not handled or not allowed 2026-01-25 05:43:11 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:43:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60440211799-cloth-belt-340d-37w-x-75l-50-grit>: HTTP status code is not handled or not allowed 2026-01-25 05:43:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/desk-highlighter-chisel-tip-assorted-inks-6set>: HTTP status code is not handled or not allowed 2026-01-25 05:43:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers>: HTTP status code is not handled or not allowed 2026-01-25 05:43:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/61500088663-film-backed-belt-6w-x-264l-crs-grit>: HTTP status code is not handled or not allowed 2026-01-25 05:43:16 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/pencils already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:43:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers>: HTTP status code is not handled or not allowed 2026-01-25 05:43:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/61500123155-light-cleansing-hand-pad-7445b-6w-x-9l-sfn>: HTTP status code is not handled or not allowed 2026-01-25 05:43:18 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:43:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sunworks-groundwood-construction-paper-18x12-bright-white-50-sheets>: HTTP status code is not handled or not allowed 2026-01-25 05:43:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/frixion-ball-erasable-gel-pen-fine-07mm-red-barrelink>: HTTP status code is not handled or not allowed 2026-01-25 05:43:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers>: HTTP status code is not handled or not allowed 2026-01-25 05:43:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/permanent-marker-fine-point-purple-ink-1-each>: HTTP status code is not handled or not allowed 2026-01-25 05:43:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/utility-uniform-pant-navy-42x32-pt62>: HTTP status code is not handled or not allowed 2026-01-25 05:43:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/murray-50563x8-lawn-and-garden-replacement-battery-3-2ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:43:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uni-paint-marker-oilbased-medium-metallic-silver-ink>: HTTP status code is not handled or not allowed 2026-01-25 05:43:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/basic_supplies/pens_pencils_markers/pencils>: HTTP status code is not handled or not allowed 2026-01-25 05:43:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precise-grip-rolling-ball-pen-extra-fine-05mm-black-barrelink>: HTTP status code is not handled or not allowed 2026-01-25 05:43:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8020-25-series-1-x-1-1516-six-t-slotted-extrusion-profile-238-316l-stock-bar>: HTTP status code is not handled or not allowed 2026-01-25 05:43:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60070005180-261x-8-1-2w-x-11l-m12-grit>: HTTP status code is not handled or not allowed 2026-01-25 05:43:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-containment-berm-economy-model-30-mil-rpe-thickness-10l-x-10w-x-1h>: HTTP status code is not handled or not allowed 2026-01-25 05:43:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/york-wide-light-ar-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:43:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/energel-retractable-gel-pen-07mm-metal-tip-violet-ink-dozen>: HTTP status code is not handled or not allowed 2026-01-25 05:43:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/energel-liquid-gel-refill-05mm-needle-tip-blue-ink>: HTTP status code is not handled or not allowed 2026-01-25 05:43:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bisley-x-back-airflow153-womens-long-sleeve-shirt-class-3-m-hi-vis-yellow>: HTTP status code is not handled or not allowed 2026-01-25 05:43:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/machining/machinery/grinders_buffers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:43:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bisley-womens-short-sleeve-t-shirt-s-hi-vis-orange>: HTTP status code is not handled or not allowed 2026-01-25 05:43:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8020-10-series-1-x-1-mono-t-slotted-extrusion-profile-120l-stock-bar>: HTTP status code is not handled or not allowed 2026-01-25 05:43:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15452-20mm-loop-thandle-hex-key-6-handle>: HTTP status code is not handled or not allowed 2026-01-25 05:43:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8020-10-series-1-x-2-six-t-slotted-extrusion-profile-120l-stock-bar>: HTTP status code is not handled or not allowed 2026-01-25 05:43:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xag03z-18v-lxt-lithium-ion-brushless-4-12-cut-offangle-grinder-tool-only>: HTTP status code is not handled or not allowed 2026-01-25 05:43:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f402-ballpoint-retractable-pen-07mm-black-ink-stainless-steel-barrel-2pack>: HTTP status code is not handled or not allowed 2026-01-25 05:43:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rack-mount-inverter-1000-watt-416-amp>: HTTP status code is not handled or not allowed 2026-01-25 05:43:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elsar-16256-emergency-light-replacement-battery-26ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:43:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnum-permanent-marker-extra-large-chisel-black-ink-1-each>: HTTP status code is not handled or not allowed 2026-01-25 05:43:41 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/electrical/batteries_chargers/power_tool_batteries_chargers landed on page that is not a product page. 2026-01-25 05:43:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-cv3-emergency-light-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:43:42 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/machining/machinery/grinders_buffers landed on page that is not a product page. 2026-01-25 05:43:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g2-gel-retractable-rollerball-pen-extra-fine-05mm-black-ink-dozen>: HTTP status code is not handled or not allowed 2026-01-25 05:43:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mpsr2-d45w-ge-double-action-dpdt-cat-45-key-reset-weatherproof>: HTTP status code is not handled or not allowed 2026-01-25 05:43:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/holophane-ms-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:43:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hyperlite-extension-ladder-w-cable-hooks-v-rung-16-type-iaa-375-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 05:43:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lithonia-ind616-emergency-light-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:43:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wg4wts-surface-skirt-for-genesis-wg4-appliance-family-white>: HTTP status code is not handled or not allowed 2026-01-25 05:43:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electrostatic-air-filter-merv-8-18w-x-1d-x-25h>: HTTP status code is not handled or not allowed 2026-01-25 05:43:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/petzl-asap-ascender-sorber-accessory-kit-for-hyperlite-limited-arrest-system>: HTTP status code is not handled or not allowed 2026-01-25 05:43:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sd-rj5-rj45-wiring-harness-5>: HTTP status code is not handled or not allowed 2026-01-25 05:43:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alexander-ps1270-sealed-lead-acid-agm-vrla-battery>: HTTP status code is not handled or not allowed 2026-01-25 05:43:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ds-xl-dual-sided-step-ladder-w-side-tip-wheels-8-type-iaa-7-step-375-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 05:43:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sure-lites-aa-2-emergency-light-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:43:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-as80i-emergency-light-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:43:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rhino-sla712-12v-7ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-25 05:43:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sigmas-sp612-sealed-lead-acid-agm-vrla-battery>: HTTP status code is not handled or not allowed 2026-01-25 05:44:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-ub1272-12v-7ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-25 05:44:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interlake-bulk-storage-rack-beam-ms-60-inch-l-for-wire-decking>: HTTP status code is not handled or not allowed 2026-01-25 05:44:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qp-two-stage-pressure-lubricated-air-compressor-175-psi-14-hp-30-gallon-cap-horizontal>: HTTP status code is not handled or not allowed 2026-01-25 05:44:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qp-max-two-stage-air-compressor-vertical-10-hp-120-gallon-capacity-3-phase-460v>: HTTP status code is not handled or not allowed 2026-01-25 05:44:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-cvt3gb-emergency-light-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:44:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yuntong-yt645-sealed-lead-acid-agm-vrla-battery>: HTTP status code is not handled or not allowed 2026-01-25 05:44:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergi-lite-csm3-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:44:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-cvec30-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:44:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 71129 pages (at 93 pages/min), scraped 35120 items (at 34 items/min) 2026-01-25 05:44:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1530lite72-112-x-3-lite-tslotted-profile-72-stock-bar>: HTTP status code is not handled or not allowed 2026-01-25 05:44:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/framed-view-binder-with-one-touch-ezd-rings-3inch-capacity-black>: HTTP status code is not handled or not allowed 2026-01-25 05:44:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1530242-112-x-3-tslotted-profile-242-stock-bar>: HTTP status code is not handled or not allowed 2026-01-25 05:44:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2337-triangular-top-plate-caster-swivel-wbrake>: HTTP status code is not handled or not allowed 2026-01-25 05:44:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/johnlite-2922eb-emergency-light-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:44:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inground-kit-for-uplay-today-deer-creek-commercial-playground-playset>: HTTP status code is not handled or not allowed 2026-01-25 05:44:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2565-45-degree-support-6l>: HTTP status code is not handled or not allowed 2026-01-25 05:44:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6860-single-unibearing-bearing>: HTTP status code is not handled or not allowed 2026-01-25 05:44:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portalac-pe7-2f1-emergency-light-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:44:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6426-double-flange-linear-bearing-brake-kit-ready>: HTTP status code is not handled or not allowed 2026-01-25 05:44:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-center-puller-with-3-jaws-10-ton-capacity-23-58-opening-13-34-reach>: HTTP status code is not handled or not allowed 2026-01-25 05:44:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hubbell-hp12100-emergency-light-replacement-battery-26ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:44:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergi-lite-dse18-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:44:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vestil-46-diameter-hd-precision-turret-bearing-ptb-46-6000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 05:44:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/solid-rod-inside-micrometer-set-plain-thimble-2-12-range-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 05:44:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tissue-box-holder-medium-black>: HTTP status code is not handled or not allowed 2026-01-25 05:44:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lightalarms-2ds3-emergency-light-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:44:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/puller-with-2-reversible-double-end-jaws-11-ton-capacity-11-78-opening-10-14-reach>: HTTP status code is not handled or not allowed 2026-01-25 05:44:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3703-slidein-tnut>: HTTP status code is not handled or not allowed 2026-01-25 05:44:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/puller-with-2-swiveling-jaws-11-ton-capacity-15-34-opening-15-34-adjustable-reach>: HTTP status code is not handled or not allowed 2026-01-25 05:44:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-performance-puller-with-3-jaws-22-ton-capacity-20-12-opening-19-34-reach>: HTTP status code is not handled or not allowed 2026-01-25 05:44:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6523-double-flange-linear-bearing-brake-kit-ready>: HTTP status code is not handled or not allowed 2026-01-25 05:44:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3660-anchor-fastener>: HTTP status code is not handled or not allowed 2026-01-25 05:44:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portalac-gs-pwl7-emergency-light-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:44:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-w-x-30-d-production-workbench-with-drawer-upright-shelf-plastic-laminate-safety-edge-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:44:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96-w-x-36-d-production-workbench-w-drawer-upright-shelf-birch-butcher-block-square-edge-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:44:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/back-plunger-dial-indicator-200-range-0-100-dial-face-001-graduations>: HTTP status code is not handled or not allowed 2026-01-25 05:44:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/no-smoking-in-this-vehicle-sign-adhesive-vinyl-3-12-x-5-blackredwhite-pack-of-5>: HTTP status code is not handled or not allowed 2026-01-25 05:44:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-im-12180-12v-18ah-lawn-and-garden-battery>: HTTP status code is not handled or not allowed 2026-01-25 05:44:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/if-you-cant-see-my-mirrors-i-cant-see-you-sign-reflective-sheet-7-x-10-yellow>: HTTP status code is not handled or not allowed 2026-01-25 05:44:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-ski-doo-ds-450-450cc-atv-replacement-battery-2008-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:44:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/safety_signs_tags/parking_traffic_signs/parking_traffic_signs_1 already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:44:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6121-10-series-replacement-bushing>: HTTP status code is not handled or not allowed 2026-01-25 05:44:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/not-for-hire-truck-safety-sign-adhesive-reflective-sheet-4-x-12-whitered>: HTTP status code is not handled or not allowed 2026-01-25 05:44:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portalac-pe4-5f1-emergency-light-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:44:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6131-access-hole-drills>: HTTP status code is not handled or not allowed 2026-01-25 05:44:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portalac-lac-pe7f1-emergency-light-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:44:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nerp005s-20-ner-electric-hoist-push-trolley-12-ton-20-lift-29-ftmin-230v>: HTTP status code is not handled or not allowed 2026-01-25 05:44:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergi-lite-tsm36-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:44:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nerp020ld-15-ner-electric-hoist-push-trolley-2-ton-15-lift-1425-ftmin-208v>: HTTP status code is not handled or not allowed 2026-01-25 05:44:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6108-t-handle-hex-wrench-5mm>: HTTP status code is not handled or not allowed 2026-01-25 05:44:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mbk-xc-flame-125cc-motorcycle-replacement-battery-1984-2008-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:44:49 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/safety_signs_tags/parking_traffic_signs/parking_traffic_signs_1 landed on page that is not a product page. 2026-01-25 05:44:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ner001hd-20-ner-dual-speed-electric-chain-hoist-18-ton-20-lift-559-ftmin-208v>: HTTP status code is not handled or not allowed 2026-01-25 05:44:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergency-medical-care-traffic-safety-sign-hip-aluminum-18-x-24-whiteblue>: HTTP status code is not handled or not allowed 2026-01-25 05:44:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3675-slidein-economy-tnut>: HTTP status code is not handled or not allowed 2026-01-25 05:44:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ner015sd-15-ner-dual-speed-electric-chain-hoist-1-12-ton-15-lift-183-ftmin-208v>: HTTP status code is not handled or not allowed 2026-01-25 05:44:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motron-sting-50cc-motorcycle-replacement-battery-1998-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:44:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/star-of-life-traffic-safety-sign-hip-aluminum-24-x-24-whiteblue>: HTTP status code is not handled or not allowed 2026-01-25 05:44:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/parklet-with-8-panels-powdercoat-wire-mesh-panel-insert-212l-x-104w-x-44h>: HTTP status code is not handled or not allowed 2026-01-25 05:44:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ner005s-20-ner-electric-hoist-hook-suspension-12-ton-20-lift-29-ftmin-208v>: HTTP status code is not handled or not allowed 2026-01-25 05:44:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honda-xr250r-250cc-motorcycle-replacement-battery-1991-1996-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:44:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/huot-toolscoot-tree-for-hsk63a-cnc-holders>: HTTP status code is not handled or not allowed 2026-01-25 05:44:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/parklet-extension-with-2-panels-solid-wood-panel-insert-78l-x-78w-x-42h>: HTTP status code is not handled or not allowed 2026-01-25 05:44:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mbk-kilibre-300cc-motorcycle-replacement-battery-2004-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:44:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ner001h-20-ner-electric-hoist-hook-suspension-18-ton-20-lift-55-ftmin-230v>: HTTP status code is not handled or not allowed 2026-01-25 05:44:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/titan-coyote-112cc-motorcycle-replacement-battery-1995-2008-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:45:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nerp015s-15-ner-electric-hoist-push-trolley-1-12-ton-15-lift-18-ftmin-208v>: HTTP status code is not handled or not allowed 2026-01-25 05:45:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/contoured-reversible-puffer-vest-class-2-xl-hi-vis-yellow>: HTTP status code is not handled or not allowed 2026-01-25 05:45:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7312bmuc3-angular-contact-ball-bearing-flush-ground-60mm-bore-x-130mm-od-x-31mm-w>: HTTP status code is not handled or not allowed 2026-01-25 05:45:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7316bmuc3-angular-contact-ball-bearing-flush-ground-80mm-bore-x-170mm-od-x-39mm-w>: HTTP status code is not handled or not allowed 2026-01-25 05:45:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ner010ld-15-ner-dual-speed-electric-chain-hoist-1-ton-15-lift-1425-ftmin-230v>: HTTP status code is not handled or not allowed 2026-01-25 05:45:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/retaining-magnet-assembly-thru-hole-126-dia-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 05:45:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/robot-s7-semi-automatic-portable-stretch-wrapper-86h>: HTTP status code is not handled or not allowed 2026-01-25 05:45:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-mobile-utility-table-20-x-16-undershelf-guardrail--4-drawers>: HTTP status code is not handled or not allowed 2026-01-25 05:45:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nerm020c-l-15-ner-electric-hoist-motor-trolley-2-ton-15-lift-7-ftmin-208v>: HTTP status code is not handled or not allowed 2026-01-25 05:45:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/al100-hand-chain-hoist-aluminum-3-ton-capacity-30-ft-lift>: HTTP status code is not handled or not allowed 2026-01-25 05:45:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 71225 pages (at 96 pages/min), scraped 35150 items (at 30 items/min) 2026-01-25 05:45:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pgo-galaxy-25cc-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:45:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-drum-roller-2-belts-20-rpm-1-phase-115-motor>: HTTP status code is not handled or not allowed 2026-01-25 05:45:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/help-conserve-energy-turn-off-lights-when-leaving-label-vinyl-3-12-x-5-pk-of-5>: HTTP status code is not handled or not allowed 2026-01-25 05:45:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nerp010ld-fg-10-ner-food-grade-hoist-with-push-trolley-1-ton-capacity-208v>: HTTP status code is not handled or not allowed 2026-01-25 05:45:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-20-oz-looped-end-wet-mop-w-quick-change-handle-4-pack-979>: HTTP status code is not handled or not allowed 2026-01-25 05:45:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/peugeot-tkr-100cc-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:45:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rascal-rover-mobility-scooter-replacement-battery-55ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:45:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kymco-movie-125cc-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:45:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-channel-variable-pipettor-71211105-2-20ul-volume-range>: HTTP status code is not handled or not allowed 2026-01-25 05:45:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foam-tube-insert-18900022-test-tubes-12mm-requires-universal-adapter>: HTTP status code is not handled or not allowed 2026-01-25 05:45:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ceramic-hang-it-magnet-rb80eb-with-attached-eyebolt-95-lbs-pull-chrome-plating>: HTTP status code is not handled or not allowed 2026-01-25 05:45:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/karma-ks-848-mobility-scooter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:45:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anti-scratchanti-fog-frameless-safety-glasses-gray-lens-gray-temple-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-25 05:45:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/asfogless-3sixty-frameless-safety-glasses-clear-lens-clear-temple-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-25 05:45:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thompson-h600-electric-scooter-replacement-battery-18ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:45:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/astrobrights-colored-paper-8-1-2-inch-x-11-inch-24-lb-terra-green-500-sheets-ream>: HTTP status code is not handled or not allowed 2026-01-25 05:45:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96-x-48-slatwall-panel-brushed-aluminum-hpl-without-inserts-pkg-qty-4>: HTTP status code is not handled or not allowed 2026-01-25 05:45:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96-x-48-slatwall-panel-almond-laminate-without-inserts-pkg-qty-4>: HTTP status code is not handled or not allowed 2026-01-25 05:45:28 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:45:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/loop-tape-individual-strips-with-adhesive-75l-x-34w-black>: HTTP status code is not handled or not allowed 2026-01-25 05:45:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/linen-cover-stock-paper-8-1-2-inch-x-11-inch-65-lb-linen-white-100-sheets-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:45:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arctic-cat-panther-570-esr-international-565cc-snowmobile-replacement-battery-12v>: HTTP status code is not handled or not allowed 2026-01-25 05:45:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arctic-cat-cougar-mountain-cat-500cc-snowmobile-replacement-battery-1989-1990-12v>: HTTP status code is not handled or not allowed 2026-01-25 05:45:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/snugboot-craftsman-mid-calf-boots-cleated-outsole-plain-toe-size-13-11-12h-black>: HTTP status code is not handled or not allowed 2026-01-25 05:45:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acifort-metmax-full-safety-boots-cleated-outsole-steel-toe-size-10-15h-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:45:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/business-card-2-inch-x-3-1-2-inch-white-160-cards-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:45:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fore-mp-colors-paper-8-1-2-inch-x-11-inch-20-lb-salmon-500-sheets-ream>: HTTP status code is not handled or not allowed 2026-01-25 05:45:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-side-printable-inkjet-business-card-2-inch-x-3-1-2-inch-matte-white-400-cards-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:45:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/foot_protection/safety_toe_work_boots>: HTTP status code is not handled or not allowed 2026-01-25 05:45:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-graphite-nebula-round-table-w-4-chair-set-in-natural-finish>: HTTP status code is not handled or not allowed 2026-01-25 05:45:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/envirotec-pad-8-1-2-inch-x-11-3-4-inch-jr-legal-ruled-white-50-sheet-pad-12-pads-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:45:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/Classic-Chair-Sensor-Pad-7-x-15-45-Day-10110>: HTTP status code is not handled or not allowed 2026-01-25 05:45:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hook-tape-individual-strips-with-adhesive-75l-x-2w-black>: HTTP status code is not handled or not allowed 2026-01-25 05:45:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/import-pulley-hand-tap-1-4-20-6-inch-oal>: HTTP status code is not handled or not allowed 2026-01-25 05:45:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hook-tape-individual-dots-with-adhesive-58l-x-58w-white-pack-of-1200>: HTTP status code is not handled or not allowed 2026-01-25 05:45:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/schumacher-electric-pp-2200-jump-starter-replacement-battery-75ah-12v-it>: HTTP status code is not handled or not allowed 2026-01-25 05:45:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s-hook-3-l-metal-chrome>: HTTP status code is not handled or not allowed 2026-01-25 05:45:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sign-holder-gridwall-11-w-x-7-h-332-thickness-acrylic-clear>: HTTP status code is not handled or not allowed 2026-01-25 05:45:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glass-316-dia-round-for-30-round-racks-temper-clear>: HTTP status code is not handled or not allowed 2026-01-25 05:45:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/schumacher-electric-psj-4424-jump-starter-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:45:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nilfisk-advance-micromatic-5800-industrial-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:45:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dsr-psj4424-pro-series-12-24v-jump-starter-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:45:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bright-way-group-bw-1280-sealed-lead-acid-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:45:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/schumacher-electric-scupsj3612-jump-starter-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:45:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-day-essential-emergency-survival-kit-4-person-backpack-60-pieces>: HTTP status code is not handled or not allowed 2026-01-25 05:45:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelf-unit-36w-x-12d-x-74h-with-28-ssb423-stackable-shelf-bins-black>: HTTP status code is not handled or not allowed 2026-01-25 05:45:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kraft-paper-roll-50-lbs-36-x-1000-ft-natural>: HTTP status code is not handled or not allowed 2026-01-25 05:45:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/schumacher-electric-ip-2200i-950a-jump-starter-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:45:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelf-unit-36w-x-21d-x-74h-with-21-ssb485-stackable-shelf-bins-black>: HTTP status code is not handled or not allowed 2026-01-25 05:45:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storage-bin-station-w-4-fixed-compartments-1-shelf-24w-x-12d-x-10-1-8h>: HTTP status code is not handled or not allowed 2026-01-25 05:45:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-power-group-ub12120-sealed-lead-acid-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 05:45:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bubble-sof-tred-12-thick-safety-anti-fatigue-floor-mat-3x5-black>: HTTP status code is not handled or not allowed 2026-01-25 05:45:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bubble-sof-tred-12-thick-safety-anti-fatigue-floor-mat-2x3-blackyellow>: HTTP status code is not handled or not allowed 2026-01-25 05:45:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yov-445-4-12-apprentice-series-utility-bench-vise>: HTTP status code is not handled or not allowed 2026-01-25 05:46:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conduit-hanger-w-hex-nut-and-bolt-stainless-steel-1-14-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 05:46:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/935-2005-935-case-w-padded-divider-220l-x-140w-x-90h-silver>: HTTP status code is not handled or not allowed 2026-01-25 05:46:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-nestable-shelf-bin-4-1-8w-x-17-7-8d-x-6h>: HTTP status code is not handled or not allowed 2026-01-25 05:46:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/960-0007-960-case-254l-x-200w-x-145h-graphite>: HTTP status code is not handled or not allowed 2026-01-25 05:46:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-super-size-11w-x-23-7-8d-x-10h>: HTTP status code is not handled or not allowed 2026-01-25 05:46:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machine-screw-hex-nut-silicon-bronze-10-24-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 05:46:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c-style-threaded-conduit-body-w-cover-and-gasket-2-size>: HTTP status code is not handled or not allowed 2026-01-25 05:46:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-person-stage-24-inch-w-x-20-ft-l-2620>: HTTP status code is not handled or not allowed 2026-01-25 05:46:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmx-iiia-flame-resistant-premium-coverall-cnb2-navy-45-oz-size-48-regular>: HTTP status code is not handled or not allowed 2026-01-25 05:46:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-insulated-liquid-tight-connector-alloy-steel-1-12-size-5-pk>: HTTP status code is not handled or not allowed 2026-01-25 05:46:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 71323 pages (at 98 pages/min), scraped 35185 items (at 35 items/min) 2026-01-25 05:46:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triple-grip-plastic-anchor-6-size-beige-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 05:46:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ef-flame-resistant-classic-coverall-cec2-navy-size-46-regular>: HTTP status code is not handled or not allowed 2026-01-25 05:46:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ef-ct-fr-dress-uniform-shirt-slu2-navy-size-3xl-regular>: HTTP status code is not handled or not allowed 2026-01-25 05:46:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/orthofab-lifestyles-770-wheelchair-gp24-wheelchair-replacement-battery-75ah-12v>: HTTP status code is not handled or not allowed 2026-01-25 05:46:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ef-flame-resistant-relaxed-fit-denim-jeans-pej2-dark-denim-125-oz-size-42-x-37u>: HTTP status code is not handled or not allowed 2026-01-25 05:46:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merits-junior-p320-p3201-p32011-p32012-wheelchair-replacement-battery-22ah-12v>: HTTP status code is not handled or not allowed 2026-01-25 05:46:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/armored-ground-clamp-bronze-12-1-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 05:46:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insulated-no-thread-connector-34-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 05:46:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bare-ground-clamp-bronze-4-12-6-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 05:46:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dwa-dwe305-12-amp-keyless-variable-speed-corded-reciprocating-saw>: HTTP status code is not handled or not allowed 2026-01-25 05:46:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dwa-dcs331m1-20v-max-jigsaw-kit-40-ah>: HTTP status code is not handled or not allowed 2026-01-25 05:46:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s20240b-infrared-heater-20-kw-208240v-black>: HTTP status code is not handled or not allowed 2026-01-25 05:46:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standup-adjustable-height-workstation-desk-47-1-4w-x-29-1-2d-x-29-42-3-4-h>: HTTP status code is not handled or not allowed 2026-01-25 05:46:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-decker-905080-11-lawn-and-garden-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:46:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5501-moisture-wicking-short-sleeve-safety-tshirt-with-chest-pocket-lime-medium>: HTTP status code is not handled or not allowed 2026-01-25 05:46:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/murray-16hp-42-16-hp-35ah-lawn-and-garden-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:46:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1501-multipurpose-class-2-two-tone-mesh-zipper-6-pockets-vest-lime-2xl>: HTTP status code is not handled or not allowed 2026-01-25 05:46:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/murray-16hp-46-16-hp-35ah-lawn-and-garden-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:46:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triple-grip-anchor-kit-w-square-pan-head-screw-10-size-tuff-pack-201-pieces>: HTTP status code is not handled or not allowed 2026-01-25 05:46:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pacesaver-eclipse-premier-titan-4-wheel-wheelchair-replacement-battery-35ah-12v>: HTTP status code is not handled or not allowed 2026-01-25 05:46:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1005-standard-class-2-two-tone-mesh-zipper-safety-vest-lime-large>: HTTP status code is not handled or not allowed 2026-01-25 05:46:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electric-mobility-600c-indoor-scooter-wheelchair-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:46:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-60-yds-black-3-pack-ac36-cloth-duct-tape-10-mil>: HTTP status code is not handled or not allowed 2026-01-25 05:46:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibration-mount-2-tapped-holes-150-inch-dia-100-inch-h-5-16-18-thread>: HTTP status code is not handled or not allowed 2026-01-25 05:46:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1505-multipurpose-class-2-mesh-zipper-6-pockets-safety-vest-lime-3xl>: HTTP status code is not handled or not allowed 2026-01-25 05:46:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/line-package-nylon-12-x-150-size>: HTTP status code is not handled or not allowed 2026-01-25 05:46:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-12-255-12255-0120255-w-leads-emergency-light-replacement-battery-5ah-6v>: HTTP status code is not handled or not allowed 2026-01-25 05:46:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sonnenschein-nga51201d1hsosa-emergency-light-replacement-battery-1-3ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:46:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/worksurface-4-brushed-16-gage-wrapped-and-polished-corners-pb-core-80-inch-w-x-30-inch-d-x-1-1-2-inch-h>: HTTP status code is not handled or not allowed 2026-01-25 05:46:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sheet-metal-screw-square-pan-head-carbon-steel-6-x-1-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 05:46:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grip-24-w-12-step-stl-roll-lad-14-d-top-stp-hndrl-lck-orng-fs123214go>: HTTP status code is not handled or not allowed 2026-01-25 05:46:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/box-box-file-mobile-pedestal-file-15-inchw-x-19-inchd-x-28-inchh-black>: HTTP status code is not handled or not allowed 2026-01-25 05:46:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grip-24-w-15-step-stl-roll-lad-21-d-top-stp-hndrl-lck-orng-fs154021go>: HTTP status code is not handled or not allowed 2026-01-25 05:46:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-drawer-heavy-duty-vertical-file-cabinet-15-inchw-x-25-inchd-x-28-3-8-inchh-putty>: HTTP status code is not handled or not allowed 2026-01-25 05:46:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/wall_mount_belts already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:46:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/can-am-outlander-500-efi-dps-500cc-atv-replacement-battery-2013-2015-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:46:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lucky-duck-rapid-flyer-mallard-hen-combo-decoy-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:46:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5bf09a-aluminum-solid-disk-handwheel-wo-handle-787-dia-x-625-bore>: HTTP status code is not handled or not allowed 2026-01-25 05:46:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-gallon-water-bottle-storage-rack-4-bottle-capacity-797083>: HTTP status code is not handled or not allowed 2026-01-25 05:46:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96w-40h-mobile-reversible-magnetic-whiteboard>: HTTP status code is not handled or not allowed 2026-01-25 05:46:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-ski-doo-outlander-400-max-400cc-atv-replacement-battery-2004-2015-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:46:50 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/pneumatic_valves/directional_control_valves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:46:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sheet-metal-screw-square-pan-head-stainless-steel-8-x-1-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 05:46:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/peg-perego-racing-truck-hp0214-ride-on-toys-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:46:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/peg-perego-polaris-trail-boss-igod0052-ride-on-toys-replacement-battery-75ah-12v>: HTTP status code is not handled or not allowed 2026-01-25 05:46:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/short-wall-drive-anchor-carbon-steel-18-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 05:46:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drywall-screw-phillips-bugle-head-fine-thread-8-x-3-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 05:46:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honda-trx420fa2-fourtrax-rancher-4x4-auto-dct-with-eps-420cc-atv-battery-2014-2019-12v>: HTTP status code is not handled or not allowed 2026-01-25 05:46:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrf175601de56c-cast-iron-right-angle-worm-gear-reducer-601-ratio-56c-frame>: HTTP status code is not handled or not allowed 2026-01-25 05:46:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrs20651l-cast-iron-right-angle-worm-gear-reducer-51-ratio>: HTTP status code is not handled or not allowed 2026-01-25 05:46:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mp40-3-hole-punch-30-sheet-capacity-black>: HTTP status code is not handled or not allowed 2026-01-25 05:46:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-ticket-roll-red-2000-roll>: HTTP status code is not handled or not allowed 2026-01-25 05:46:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/saunders-recycled-aluminum-forms-holder-side-opening-8-1-2-x-12-silver>: HTTP status code is not handled or not allowed 2026-01-25 05:46:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrs206101l-cast-iron-right-angle-worm-gear-reducer-101-ratio>: HTTP status code is not handled or not allowed 2026-01-25 05:46:59 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/wall_mount_belts landed on page that is not a product page. 2026-01-25 05:47:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/pneumatic_valves/directional_control_valves>: HTTP status code is not handled or not allowed 2026-01-25 05:47:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrf17551l56c-cast-iron-right-angle-worm-gear-reducer-51-ratio-56c-frame>: HTTP status code is not handled or not allowed 2026-01-25 05:47:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrf133401l56c-cast-iron-right-angle-worm-gear-reducer-401-ratio-56c-frame>: HTTP status code is not handled or not allowed 2026-01-25 05:47:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-head-wood-screw-phillips-carbon-steel-12-x-2-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 05:47:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kawasaki-vulcan-s-abs-650cc-motorcycle-replacement-battery-2015-2017-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:47:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/indian-scout-spirit-1442cc-motorcycle-replacement-battery-2002-2003-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:47:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triumph-rocket-iii-classic-tourer-2300cc-motorcycle-battery-2007-2011-12v>: HTTP status code is not handled or not allowed 2026-01-25 05:47:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plate-strap-carbon-steel-12-gauge-1-12-x-24-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 05:47:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/k-lath-metal-piercing-screw-phillips-drive-wafer-head-8-x-1-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 05:47:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honda-cmx500-rebel-500-500cc-motorcycle-replacement-battery-2017-2019-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:47:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-heavy-duty-staples-25-sheet-capacity-1-4-leg-100-per-strip-1000-box>: HTTP status code is not handled or not allowed 2026-01-25 05:47:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 71420 pages (at 97 pages/min), scraped 35217 items (at 32 items/min) 2026-01-25 05:47:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harley-davidson-softail-deuce-fxstdi-1450cc-motorcycle-battery-2001-2006-12v>: HTTP status code is not handled or not allowed 2026-01-25 05:47:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/american-ironhorse-ranger-t-1820cc-motorcycle-replacement-battery-2003-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:47:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/big-dog-vintage-sport-1750cc-motorcycle-replacement-battery-1999-2001-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:47:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdrf262151h145tc-cast-iron-right-angle-worm-gear-reducer-151-ratio-145t-frame>: HTTP status code is not handled or not allowed 2026-01-25 05:47:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grip-stapler-25-sheet-capacity-black>: HTTP status code is not handled or not allowed 2026-01-25 05:47:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-head-tap-bolt-low-carbon-steel-14-20-x-4-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 05:47:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yamaha-xv1600at-road-star-silverado-1600cc-motorcycle-battery-1999-2003-12v>: HTTP status code is not handled or not allowed 2026-01-25 05:47:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/indian-chief-dark-horse-1720cc-motorcycle-replacement-battery-2010-2013-12v-i>: HTTP status code is not handled or not allowed 2026-01-25 05:47:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lock-washer-silicon-bronze-34-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 05:47:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/indian-springfield-dark-horse-1811cc-motorcycle-replacement-battery-2018-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:47:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waterproof-utility-cases-small-case-8-1-2l-x-6-1-2w-x-3h-black>: HTTP status code is not handled or not allowed 2026-01-25 05:47:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-e-fr-overall-snap-fly-front-fluorescent-orangered-2xl>: HTTP status code is not handled or not allowed 2026-01-25 05:47:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guardian-pool-tile-cleaner-gallon-bottle-4-case-p1128cs>: HTTP status code is not handled or not allowed 2026-01-25 05:47:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-offset-strut-strap-carbon-steel-2-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 05:47:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/closed-add-on-cab85-1842-5x-42-inch-w-x-18-inch-d-x-85-inch-h-5-shelves-18-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:47:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yamaha-xvz1300at-royal-star-tour-classic-1300cc-motorcycle-battery-1996-2000-12v>: HTTP status code is not handled or not allowed 2026-01-25 05:47:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bx1542l-42-inch-w-x-15-inch-d-22-gauge-450-lb-capacity-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:47:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/prta011mpi-two-speed-arrow-white-button-insert-use-wmike-victor-pendants>: HTTP status code is not handled or not allowed 2026-01-25 05:47:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-head-lag-screw-carbon-steel-12-x-4-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 05:47:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/19-dia-x-16-ga-galvanized-roller-42072-16-gp-for-16-oaw-omni-conveyors-abec-bearings>: HTTP status code is not handled or not allowed 2026-01-25 05:47:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-head-lag-screw-carbon-steel-14-x-2-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 05:47:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harley-davidson-cvo-ultra-classic-electra-glide-flhtcuse7-1803cc-motorcycle-battery-12v>: HTTP status code is not handled or not allowed 2026-01-25 05:47:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lock-washer-spring-steel-58-size-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 05:47:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eco-funnel-ef-4-38-006n-4-with-polypropylene-quick-disconnect-adapter-red-lid>: HTTP status code is not handled or not allowed 2026-01-25 05:47:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lime-a-way-lime-calcium-rust-remover-28oz-bottle-6-case-rac87000ct>: HTTP status code is not handled or not allowed 2026-01-25 05:47:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/combat-child-resist-ant-kill-sys-kill-queen-6-traps-bx-12cs-dia45901ct>: HTTP status code is not handled or not allowed 2026-01-25 05:47:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wal-10-3-16-4-flute-fingers>: HTTP status code is not handled or not allowed 2026-01-25 05:47:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kawasaki-jh900-zxi-900cc-personal-watercraft-replacement-battery-1995-1997-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:47:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dawn-heavy-duty-degreaser-gallon-bottle-3-case-pgc04852>: HTTP status code is not handled or not allowed 2026-01-25 05:47:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/import-7-inch-straight-tap-wrench>: HTTP status code is not handled or not allowed 2026-01-25 05:47:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kawasaki-jet-ski-jh750-xir-750cc-personal-watercraft-battery-1992-1999-12v>: HTTP status code is not handled or not allowed 2026-01-25 05:47:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-hcflu206-19-flange-mount-bearing-2-bolt-flange-eccentric-locking-collar-bore-30-1625-mm>: HTTP status code is not handled or not allowed 2026-01-25 05:47:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kawasaki-jet-ski-jt1500-stx-15f-1500cc-personal-watercraft-battery-2004-2016-12v>: HTTP status code is not handled or not allowed 2026-01-25 05:47:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-row-spherical-roller-bearing-22212exw33c3-60mm-bore-110mm-od>: HTTP status code is not handled or not allowed 2026-01-25 05:47:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-row-spherical-roller-bearing-22322exw33kc3-110mm-bore-240mm-od>: HTTP status code is not handled or not allowed 2026-01-25 05:47:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-row-spherical-roller-bearing-22328ew33c3-140mm-bore-300mm-od>: HTTP status code is not handled or not allowed 2026-01-25 05:47:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vector-all-other-models-mobility-scooter-replacement-battery-55ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:47:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-hcpk206-20-pillow-block-bearing-low-base-ecc-lock-collar-bore-31-75-mm>: HTTP status code is not handled or not allowed 2026-01-25 05:47:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-ucp217-55-pillow-block-bearing-set-screw-locking-bore-87-3125-mm>: HTTP status code is not handled or not allowed 2026-01-25 05:47:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sure-grip-ex-safety-cabinet-self-close-12-gallon-cap-23-14w-x-18d-x-35h-royal-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:47:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-ucf211-34-flange-mount-bearing-4-bolt-flange-set-screw-locking-bore-53-975-mm>: HTTP status code is not handled or not allowed 2026-01-25 05:47:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quantum-partition-wall-add-on-unit-7-1017hbc-baskets-18w-x-26d-x-70h-whitechrome>: HTTP status code is not handled or not allowed 2026-01-25 05:47:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8950-bump-cap-lime-short-brim-one-size>: HTTP status code is not handled or not allowed 2026-01-25 05:47:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/odin-safety-glasses-silver-mirror-lens-matte-black-frame>: HTTP status code is not handled or not allowed 2026-01-25 05:47:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-row-spherical-roller-bearing-22248ew33kc3-240mm-bore-440mm-od>: HTTP status code is not handled or not allowed 2026-01-25 05:47:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/piaggio-vespa-nexus-500ie-500cc-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:47:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dagr-safety-glasses-silver-mirror-lens-black-frame>: HTTP status code is not handled or not allowed 2026-01-25 05:47:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-row-cylindrical-roller-bearing-nj205-25mm-bore-52mm-od>: HTTP status code is not handled or not allowed 2026-01-25 05:47:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6705ct-evaporative-cooling-triangle-hat-with-cooling-towel-flames>: HTTP status code is not handled or not allowed 2026-01-25 05:47:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3713-elastic-loop-tool-tails153-swivel-10-lbs-lime-standard-11>: HTTP status code is not handled or not allowed 2026-01-25 05:47:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flammable-cabinet-manual-close-double-door-60-gal-cap-34w-x-34d-x-65h-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:47:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ada-braille-sign-all-gender-with-2-symbols--wheelchair-access-6w-x-9h-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:47:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-row-cylindrical-roller-bearing-nu311eg-55mm-bore-120mm-od-high-capacity>: HTTP status code is not handled or not allowed 2026-01-25 05:48:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8091-noncertified-long-sleeve-tshirt-lime-3xl>: HTTP status code is not handled or not allowed 2026-01-25 05:48:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/klip-it-closed-add-on-cak85-2436-5-36-inch-w-x-24-inch-d-x-85-inch-h-5-shelves-20-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:48:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/klip-it-open-add-on-oak73-1248-8-48-inch-w-x-12-inch-d-x-73-inch-h-8-shelves-20-ga-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:48:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jenoptik-250-superhexagon-x9-250cc-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:48:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/golden-technologies-companion-ii-mobility-scooter-replacement-battery-35ah-12v>: HTTP status code is not handled or not allowed 2026-01-25 05:48:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scooterteq-commuter-500zx-rsv-mobility-scooter-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:48:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-chrome-esd-wire-shelving-unit-starter-36w-x-30d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 05:48:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-stainless-steel-wire-shelving-unit-starter-72w-x-24d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 05:48:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-2-in-lead-free-brass-cap-fnpt-125-psi-import>: HTTP status code is not handled or not allowed 2026-01-25 05:48:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electric-mobility-candy-apple-mobility-scooter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:48:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-4-in-304-stainless-steel-coupling-fnpt-125-psi-import>: HTTP status code is not handled or not allowed 2026-01-25 05:48:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pride-mobility-elite-traveller-mobility-scooter-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:48:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shoprider-sunrunner-3777-3-4-mobility-scooter-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 05:48:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 71520 pages (at 100 pages/min), scraped 35252 items (at 35 items/min) 2026-01-25 05:48:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pride-mobility-jazzy-elite-14-mobility-scooter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:48:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bombardier-gtx-600-ho-sdi-593cc-snowmobile-replacement-battery-2005-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:48:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-in-lead-free-brass-plug-mnpt-125-psi-import>: HTTP status code is not handled or not allowed 2026-01-25 05:48:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-chrome-wire-shelving-unit-starter-48w-x-12d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 05:48:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-stainless-steel-wire-shelving-unit-starter-72w-x-21d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 05:48:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-chrome-wire-shelving-unit-starter-72w-x-12d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 05:48:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss68052rs-deep-groove-ball-bearing-stainless-steel-double-sealed-od-37mm-bore-25mmmetric>: HTTP status code is not handled or not allowed 2026-01-25 05:48:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/side-apply-correction-tape-tear-resistant-1-5-in-x-394-in-white>: HTTP status code is not handled or not allowed 2026-01-25 05:48:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-stainless-steel-wire-shelving-unit-starter-24w-x-14d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 05:48:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-grand-touring-se-700cc-snowmobile-replacement-battery-1997-1998-12v-i>: HTTP status code is not handled or not allowed 2026-01-25 05:48:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/permanent-write-on-index-tabs-7-16-x-1-white-416-tabs-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:48:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polaris-600-iq-racer-euro-600cc-snowmobile-replacement-battery-2009-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:48:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pre-inked-message-stamp-approved-1-5-8-x-1-2-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:48:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-chrome-wire-shelving-unit-add-on-48w-x-21d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 05:48:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n499e-snow-safety-sign-notice-ice-and-snow-take-it-slow-24-x-18-white-blue-black>: HTTP status code is not handled or not allowed 2026-01-25 05:48:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/felt-stamp-pad-2-3-4-x-4-1-4-black>: HTTP status code is not handled or not allowed 2026-01-25 05:48:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hxt-60-6-gallon-expansion-tank-hydronic-1-2-in-mnpt>: HTTP status code is not handled or not allowed 2026-01-25 05:48:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-chrome-wire-shelving-unit-add-on-54w-x-14d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 05:48:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/permanent-write-on-index-tabs-7-16-x-1-white-104-tabs-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:48:23 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/carts-trucks/forklift_hoppers_loaders already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:48:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/correction-tape-1-6-in-x-473-in-white-10-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:48:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bombardier-gsx-800-ho-793cc-snowmobile-replacement-battery-2004-2005-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:48:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-corrugated-boxes-20-x-12-x-3-200lb-test-ect-32-25-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:48:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/purchase-order-book-2-part-carbonless-5-9-16-x-8-7-16-50-sets-book>: HTTP status code is not handled or not allowed 2026-01-25 05:48:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-mailers-4-x-4-x-4-200lb-test-ect-32-kraft-50-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:48:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-chrome-wire-shelving-unit-add-on-60w-x-36d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 05:48:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-chrome-wire-shelving-unit-starter-60w-x-21d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 05:48:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-16-x-20-8-mil-clear-250-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:48:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-stainless-steel-wire-shelving-unit-starter-42w-x-21d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 05:48:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-1-2-gas-appliance-regulator-210g-2-1-2-up-to-26000000-btu>: HTTP status code is not handled or not allowed 2026-01-25 05:48:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/carts-trucks/forklift_hoppers_loaders>: HTTP status code is not handled or not allowed 2026-01-25 05:48:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-12-violet-spring-vio-r5310-for-rv53-regulators>: HTTP status code is not handled or not allowed 2026-01-25 05:48:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bombardier-summit-800-x-800cc-snowmobile-replacement-battery-2001-2007-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:48:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/weather-resistant-corrugated-boxes-10-x-10-x-10-250lb-test-25-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:48:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-double-wall-corrugated-boxes-16-x-16-x-16-500lb-test-ect-71-15-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:48:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bombardier-summit-500-f-496cc-snowmobile-replacement-battery-2001-2002-12v-i>: HTTP status code is not handled or not allowed 2026-01-25 05:48:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gusseted-reclosable-poly-bags-10-x-3-x-12-2-mil-clear-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:48:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/66-in-ada-compliant-concrete-oval-picnic-table-brown>: HTTP status code is not handled or not allowed 2026-01-25 05:48:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/66-in-concrete-round-picnic-table-sand>: HTTP status code is not handled or not allowed 2026-01-25 05:48:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honeywell-wiring-subbase-q7800a1005-panel-mount>: HTTP status code is not handled or not allowed 2026-01-25 05:48:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-ski-doo-renegade-backcountry-600-ho-600cc-snowmobile-battery-2014-2016-12v>: HTTP status code is not handled or not allowed 2026-01-25 05:48:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lf177fbq-9mhp-1in-horizontal-keyway-engine>: HTTP status code is not handled or not allowed 2026-01-25 05:48:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-poly-green-wire-shelving-unit-add-on-30w-x-14d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 05:48:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-wire-shelf-basket-divider-18-797101>: HTTP status code is not handled or not allowed 2026-01-25 05:48:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bombardier-mx-z-renegade-600-600cc-snowmobile-replacement-battery-2004-2013-12v>: HTTP status code is not handled or not allowed 2026-01-25 05:48:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-8485-containment-berm-foam-wall-15-x-50-22-oz-pvc>: HTTP status code is not handled or not allowed 2026-01-25 05:48:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arctic-cat-mountain-cat-800-efi-le-785cc-snowmobile-replacement-battery-12v-i>: HTTP status code is not handled or not allowed 2026-01-25 05:48:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-8474-containment-berm-foam-wall-10-x-50-22-oz-pvc>: HTTP status code is not handled or not allowed 2026-01-25 05:48:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/miller-revolution-arc-rated-safety-harness-w-back-d-ring-quick-connect-3xl-black>: HTTP status code is not handled or not allowed 2026-01-25 05:48:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-8484-containment-berm-foam-wall-15-x-40-22-oz-pvc>: HTTP status code is not handled or not allowed 2026-01-25 05:48:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chemical-resistant-gloves-rough-grip-butyl-13-mil-thick-size-11-black>: HTTP status code is not handled or not allowed 2026-01-25 05:48:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36w-square-pedestal-table-with-graphite-nebula-top-round-silver-base>: HTTP status code is not handled or not allowed 2026-01-25 05:48:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hollow-steel-door-36w-x-80h-with-24w-x-24h-louvers-mortise-prep-steelcraft-hinge>: HTTP status code is not handled or not allowed 2026-01-25 05:48:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-mx-z-x-rs-iron-dog-e-tec-600-ho-600cc-snowmobile-battery-2017-2018-12v>: HTTP status code is not handled or not allowed 2026-01-25 05:48:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safetypro-335-no-entry-belt-barrier-40-red-post-35-redwhite-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-25 05:48:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-plastic-locker-11-921215720-double-tier-12-x-15-x-36-1-wide-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:48:52 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:48:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-renegade-backcountry-e-tec-800r-800cc-snowmobile-battery-2011-2017-12v>: HTTP status code is not handled or not allowed 2026-01-25 05:48:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bluff-mobilepro43-yard-ramp-with-hinged-lip--adj-legs-30000-lb-cap-36l-x-84w-x-48h>: HTTP status code is not handled or not allowed 2026-01-25 05:48:55 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/packaging/labels_barcode_equipment/label_dispensers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:48:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-renegade-600-adrenaline-600cc-snowmobile-replacement-battery-2010-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:48:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electrically-conductive-floor-mat-916-thick-2-x-3-black>: HTTP status code is not handled or not allowed 2026-01-25 05:48:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5741wh-device-box-white-4-58l>: HTTP status code is not handled or not allowed 2026-01-25 05:48:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-skandic-550f-tundra-550cc-snowmobile-replacement-battery-2010-2011-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:49:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-ski-doo-renegade-sport-600-ace-600cc-snowmobile-replacement-battery-2011-12v>: HTTP status code is not handled or not allowed 2026-01-25 05:49:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vapor-tension-panel-thermometer-6812-01-3>: HTTP status code is not handled or not allowed 2026-01-25 05:49:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-717-3-piece-full-port-ball-valve-stainless-steel-38-threaded>: HTTP status code is not handled or not allowed 2026-01-25 05:49:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arctic-cat-bearcat-7000-xt-gs-1049cc-snowmobile-replacement-battery-2016-2017-12v>: HTTP status code is not handled or not allowed 2026-01-25 05:49:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/angle-broom-high-performance-bristles-9-w-yellow-gjo58562>: HTTP status code is not handled or not allowed 2026-01-25 05:49:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/packaging/labels_barcode_equipment/label_dispensers>: HTTP status code is not handled or not allowed 2026-01-25 05:49:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5th-generation-smart-padlock-long-shackle-satin-silver-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-25 05:49:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/floor-standing-rinse-tank-11-gallon-red-27110>: HTTP status code is not handled or not allowed 2026-01-25 05:49:07 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/crowd_control/retractable_belt_barriers/belt_stanchions landed on page that is not a product page. 2026-01-25 05:49:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sealer-sales-kf-150cst-portable-direct-heat-sealer-with-15mm-seal-w-pedal-set>: HTTP status code is not handled or not allowed 2026-01-25 05:49:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-717-3-piece-full-port-ball-valve-stainless-steel-1-threaded>: HTTP status code is not handled or not allowed 2026-01-25 05:49:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 71613 pages (at 93 pages/min), scraped 35273 items (at 21 items/min) 2026-01-25 05:49:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/severe-duty-motor-w569-326thfs9076-30hp-460v-1200rpm-3ph-326t-fr-tefc>: HTTP status code is not handled or not allowed 2026-01-25 05:49:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/x030-1-4hp-115-230v-1625rpm-1ph-teao-48yz-fr>: HTTP status code is not handled or not allowed 2026-01-25 05:49:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/centrifugal-pump-motor-kg215-1-3hp-208-230-460v-3600rpm-3ph-56c-fr-dp>: HTTP status code is not handled or not allowed 2026-01-25 05:49:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/argoscabinet-gun-storage-cabinet-awc72h24r-holds-24-rifles-42wx15dx72h-desert-sand>: HTTP status code is not handled or not allowed 2026-01-25 05:49:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/argoscabinet-gun-storage-cabinet-awc50h12r-1-holds-12-rifles-with-2-half-shelves-42w-x-15d-x-50h-stealth>: HTTP status code is not handled or not allowed 2026-01-25 05:49:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storage-argos-72-slotted-back-panel-ws-hap72-ws56-desert-sand>: HTTP status code is not handled or not allowed 2026-01-25 05:49:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/siemens-mobilett-xp-hybrid-x-ray-medical-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:49:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/parks-medical-1bts-doppler-medical-replacement-battery-1-3ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:49:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/philips-e101a-communicator-medical-replacement-battery-1-3ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:49:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/parks-medical-611-doppler-medical-replacement-battery-1-3ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:49:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-365di-butterfly-valve-with-epdm-seals--10-position-handle-3-lug>: HTTP status code is not handled or not allowed 2026-01-25 05:49:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lzsdwssk-filtered-bottle-filling-station-w-single-ada-cooler-stainless>: HTTP status code is not handled or not allowed 2026-01-25 05:49:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vrctlrddwsk-vandal-resistant-bottle-filling-station-w-bi-level-reverse-cooler-stainless>: HTTP status code is not handled or not allowed 2026-01-25 05:49:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lzstlg8wslk-filt-bottle-fill-station-w-versatile-bi-level-green-ada-cooler-refrig-l-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:49:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/parks-medical-806ca-doppler-medical-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 05:49:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/severe-duty-motor-e449-445tstfn6505-200hp-460v-3600rpm-3ph-445ts-fr-tefc>: HTTP status code is not handled or not allowed 2026-01-25 05:49:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fullriver-hgl4-6-sealed-lead-acid-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:49:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/devilbiss-vac-u-aide-7304-d-medical-replacement-battery-1-3ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:49:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ppi-pp-146014-sharps-container-14-qt-red-big-mouth-case-10>: HTTP status code is not handled or not allowed 2026-01-25 05:49:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sebra-1071-mobile-tube-sealer-medical-replacement-battery-26ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:49:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nps-chemical-resistant-adjustable-height-science-table-plain-front-24-x-48-x-29-41-w-oak-legs>: HTTP status code is not handled or not allowed 2026-01-25 05:49:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c1157-5kc39rn383x-1hp-3450rpm-115-208-230v-1ph-56-fr-dp>: HTTP status code is not handled or not allowed 2026-01-25 05:49:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/briggs-stratton-b193043gs-generator-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:49:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss-component-strip-heater-120v-125w-5-12l>: HTTP status code is not handled or not allowed 2026-01-25 05:49:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sigmastek-sp12-22-sealed-lead-acid-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:49:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sunnyway-sw12145w-sealed-lead-acid-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:49:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss-component-strip-heater-240v-1500w-42-14l>: HTTP status code is not handled or not allowed 2026-01-25 05:49:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50558-triple-top-loading-glove-box-holder-with-magnet-mount-white-clear-12-w-x-15-h-x-5-d>: HTTP status code is not handled or not allowed 2026-01-25 05:49:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/k1322-5k42fn4035-1-3hp-1800rpm-575v-3ph-56c-fr-tefc>: HTTP status code is not handled or not allowed 2026-01-25 05:49:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/900-098-bk-cable-tie-hook-tape-8l-x-12w-black-25pk>: HTTP status code is not handled or not allowed 2026-01-25 05:49:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/900-098-rd-cable-tie-hook-tape-8l-x-12w-red-25pk>: HTTP status code is not handled or not allowed 2026-01-25 05:49:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/400-029-receptacle-tester-standard-outlets>: HTTP status code is not handled or not allowed 2026-01-25 05:49:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3460-15000-ntep-wireless-led-crane-scale-15000lb-x-5lb>: HTTP status code is not handled or not allowed 2026-01-25 05:49:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fx6-series-explosion-proof-electric-unit-heater-10000w-480v-3-phase>: HTTP status code is not handled or not allowed 2026-01-25 05:49:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/refrigeration-duty-motor-x531-1-1-2hp-1hp-900rpm-208-230-460v-3ph-56hz-open>: HTTP status code is not handled or not allowed 2026-01-25 05:49:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kung-long-wp18-12-sealed-lead-acid-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:49:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xb-series-explosion-proof-convection-triple-unit-heater-cw-tstat-3000w-480v-3-phase>: HTTP status code is not handled or not allowed 2026-01-25 05:49:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sunnyway-swe12280-sealed-lead-acid-replacement-battery-26ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:49:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ge-series-forced-air-unit-heater-cw-tstat-15000w-600v-13-phase>: HTTP status code is not handled or not allowed 2026-01-25 05:49:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hyperpure-pe-rt-tubing-1id-x-1-18od-x-500-coil-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:49:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-3-three-way-bottom-bomber-jacket-with-removable-liner-yellowblack-5xl>: HTTP status code is not handled or not allowed 2026-01-25 05:49:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hyperpure-pe-rt-tubing-1-14id-x-1-38od-x-300-coil-white>: HTTP status code is not handled or not allowed 2026-01-25 05:49:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oven-fryer-grill-cleaner-characteristic-scent-1-gallon-capacity-bottle-4carton>: HTTP status code is not handled or not allowed 2026-01-25 05:49:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cortina-plastx-type-i-fold-flat-barricade-hip-grade-sheeting-24l-x-8w-panel-orangewhite>: HTTP status code is not handled or not allowed 2026-01-25 05:49:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yuasa-np7-12-250fr-sealed-lead-acid-replacement-battery-7ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 05:49:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oil-burner-motor-o601-48s17t290-1-6hp-1725rpm-115v-1ph-48n-tenv>: HTTP status code is not handled or not allowed 2026-01-25 05:49:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-gal-sat-blk-waste-recept-sat-alum-lid-w-funnel-top-f1533-bk-sa>: HTTP status code is not handled or not allowed 2026-01-25 05:49:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sigmastek-sp12-35-sealed-lead-acid-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:49:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sunnyway-sw1235-sealed-lead-acid-replacement-battery-4-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:49:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-pipe-fitting-single-socket-tee-1-1-4-dia>: HTTP status code is not handled or not allowed 2026-01-25 05:49:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/firstpower-fp640b-sealed-lead-acid-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:49:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/firstpower-fp645b-sealed-lead-acid-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:49:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caterpillar-cs431-cs443-industrial-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:49:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60600110724-stikitfilm-disc-roll-360l-6dia-x-nh-p800-grit>: HTTP status code is not handled or not allowed 2026-01-25 05:49:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coopower-cp12-1-2-sealed-lead-acid-replacement-battery-1-3ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:49:49 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/electrical/renewable_energy_devices/solar_lights already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:49:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/expertpower-exp6120-sealed-lead-acid-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:49:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-nexelon-blue-wire-shelving-unit-add-on-60w-x-24d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 05:49:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/duracell-slaa6-12f2-sealed-lead-acid-replacement-battery-7ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 05:49:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mighty-max-ml26-12-sealed-lead-acid-replacement-battery-26ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:49:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/firstpower-lfp1260d-sealed-lead-acid-replacement-battery-75ah-12v-it>: HTTP status code is not handled or not allowed 2026-01-25 05:49:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-nexelon-blue-wire-shelving-unit-starter-30w-x-14d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 05:49:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smooth-pro-pvc-ceiling-tile-74007-waterproof-washable-2l-x-2w-satin-black>: HTTP status code is not handled or not allowed 2026-01-25 05:49:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/designer-pvc-ceiling-tile-75100-waterproof-washable-drift-2l-x-2w-white>: HTTP status code is not handled or not allowed 2026-01-25 05:49:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/electrical/renewable_energy_devices/solar_lights>: HTTP status code is not handled or not allowed 2026-01-25 05:49:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-nexelon-blue-wire-shelving-unit-starter-60w-x-14d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 05:50:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-nexelon-blue-wire-shelving-unit-starter-60w-x-21d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 05:50:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-nexelon-blue-wire-shelving-unit-starter-72w-x-14d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 05:50:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-cal-neoprene-commercial-grade-60a-rubber-sheet-1-2-thick-x-36-wx-10-l>: HTTP status code is not handled or not allowed 2026-01-25 05:50:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-nexelon-blue-wire-shelving-unit-starter-24w-x-21d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 05:50:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-black-epoxy-wire-shelving-unit-add-on-60w-x-21d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 05:50:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gogreen-power-gg-36725-25-portable-hand-lamp-orange>: HTTP status code is not handled or not allowed 2026-01-25 05:50:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/snack-mix-97-oz-24box>: HTTP status code is not handled or not allowed 2026-01-25 05:50:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/duracell-slaa12-0-8jst-sealed-lead-acid-replacement-battery-7ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 05:50:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thermal-binding-system-covers-3-8-cap-11-x-8-1-2-clear-black-10-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:50:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heat-free-laminator-8-1-2-wide-1-10-maximum-document-thickness>: HTTP status code is not handled or not allowed 2026-01-25 05:50:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 71710 pages (at 97 pages/min), scraped 35295 items (at 22 items/min) 2026-01-25 05:50:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/laminating-pouches-5-mil-2-5-8-x-3-7-8-id-size-25-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:50:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-nexelate-silver-epoxy-wire-shelving-unit-add-on-48w-x-14d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 05:50:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/61500151594-radial-bristle-disc-9-16-dia-x-1-16-arbor-120-grit>: HTTP status code is not handled or not allowed 2026-01-25 05:50:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-55-gal-drum-self-close-flammable-cabinet-incl-drum-support-34w-x-34d-x-65h-red>: HTTP status code is not handled or not allowed 2026-01-25 05:50:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-utility-cart-handle-nexelate-price-each-package-of-2>: HTTP status code is not handled or not allowed 2026-01-25 05:50:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-two-wheel-trash-cart-64-gallon-graystone-ana64-gys>: HTTP status code is not handled or not allowed 2026-01-25 05:50:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-control-instruments-1002010a-alarm-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:50:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fallte-8447-carabiner-large-twist-lock-2-1-4-gate-opening>: HTTP status code is not handled or not allowed 2026-01-25 05:50:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drive-medical-cobalt-x23-wheelchair-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:50:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pride-mobility-blast-hd-wheelchair-replacement-battery-75ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:50:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/retractable-locking-panel-26w-x-3-4d-x-11-1-4h-beige>: HTTP status code is not handled or not allowed 2026-01-25 05:50:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fortress-winner-3-wheel-wheelchair-replacement-battery-55ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:50:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/werner-8-type-1a-fiberglass-podium-ladder-w-casters-300-lb-cap-pd6208-4c>: HTTP status code is not handled or not allowed 2026-01-25 05:50:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/orthofab-lifestyles-755fs-wheelchair-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:50:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/three-wing-free-air-fan-blade-1-2-inch-interchangeable-hub-alum-blade-cw-12-inch-dia-12-pitch>: HTTP status code is not handled or not allowed 2026-01-25 05:50:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-shelf-poly-z-brite-wire-shelving-unit-add-on-60w-x-18d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 05:50:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/five-wing-condenser-fan-blade-interchangeable-hub-aluminum-cw-22-inch-dia-33-pitch>: HTTP status code is not handled or not allowed 2026-01-25 05:50:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/small-plastic-push-on-fan-blade-4-inch-dia-cw-3-16-inch-bore-1-inch-blade-depth-5-blade>: HTTP status code is not handled or not allowed 2026-01-25 05:50:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-inlet-blower-wheel-15-1-2-inch-dia-cw-or-ccw-15-inch-w-1-7-16-inch-bore>: HTTP status code is not handled or not allowed 2026-01-25 05:50:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-poly-z-brite-wire-shelving-unit-starter-42w-x-24d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 05:50:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simplicity-990780-lawn-and-garden-replacement-battery-1-3ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:50:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-inlet-blower-wheel-11-1-8-inch-dia-cw-1650-rpm-3-4-inch-bore-6-inch-w-galvanized>: HTTP status code is not handled or not allowed 2026-01-25 05:50:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-1-8-professional-guillotine-lever-paper-cutter>: HTTP status code is not handled or not allowed 2026-01-25 05:50:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-1-8-premium-guillotine-lever-paper-cutter>: HTTP status code is not handled or not allowed 2026-01-25 05:50:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/husqvarna-zth6123lxpql-lawn-and-garden-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:50:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-decker-cmm1000-lawn-and-garden-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:50:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acl-f-18x48-combination-louver-damper-with-flange-18wx48h>: HTTP status code is not handled or not allowed 2026-01-25 05:50:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simplicity-16cfc-lawn-and-garden-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:50:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dfl-f-30x36-drainable-blade-fixed-louver-with-flange-30wx36h>: HTTP status code is not handled or not allowed 2026-01-25 05:50:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-wall-mounted-receiving-desk>: HTTP status code is not handled or not allowed 2026-01-25 05:50:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-base-cabinet-bc-1600-2-hinged-doors-1-shelf-1-drawer-30x21x36>: HTTP status code is not handled or not allowed 2026-01-25 05:50:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-poly-z-brite-wire-shelving-unit-add-on-72w-x-14d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 05:50:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-poly-z-brite-wire-shelving-unit-starter-72w-x-14d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 05:50:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eagle-6-poly-base-6-post-sleeves-1730-base>: HTTP status code is not handled or not allowed 2026-01-25 05:50:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vinyl-capital-letters-6h-38-letters-black>: HTTP status code is not handled or not allowed 2026-01-25 05:50:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-stainless-steel-solid-shelving-unit-add-on-36w-x-24d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 05:50:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-inlet-blower-wheel-6-1-4-inch-dia-ccw-3450-rpm-5-8-inch-bore-3-3-16-inch-w-steel>: HTTP status code is not handled or not allowed 2026-01-25 05:50:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sonnenschein-df25-emergency-light-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:50:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/200-088-micro-cutter>: HTTP status code is not handled or not allowed 2026-01-25 05:50:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/200-002-5-precision-cutter-flush-cut>: HTTP status code is not handled or not allowed 2026-01-25 05:50:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergi-lite-12csm54-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:50:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergi-lite-002019-emergency-light-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:50:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/900-206-helping-hands-w-solder-iron-holder-25x>: HTTP status code is not handled or not allowed 2026-01-25 05:50:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/900-015-helping-hands-soldering-aid>: HTTP status code is not handled or not allowed 2026-01-25 05:50:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-straight-edge-blades-10-pack-for-tda080-tape-dispenser>: HTTP status code is not handled or not allowed 2026-01-25 05:50:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-wide-boltless-shelving-add-on-72-in-w-x-48-in-d-x-60-in-h>: HTTP status code is not handled or not allowed 2026-01-25 05:50:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-wide-boltless-shelving-add-on-72-in-w-x-24-in-d-x-96-in-h>: HTTP status code is not handled or not allowed 2026-01-25 05:50:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-cell-pc12180-emergency-light-replacement-battery-18ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:50:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-7-8-wide-electronic-heavy-duty-tape-dispenser>: HTTP status code is not handled or not allowed 2026-01-25 05:50:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sonnenschein-100001164-emergency-light-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:50:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-add-on-36-in-w-x-18-in-d-x-96-in-h-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 05:50:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-high-cap-boltless-shelving-add-on-96-in-w-x-24-in-d-x-96-in-h-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 05:50:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/p-touch-m-series-tape-cartridge-1-2w-black-on-white>: HTTP status code is not handled or not allowed 2026-01-25 05:50:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sonnenschein-a50642s-emergency-light-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:50:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-wide-boltless-shelving-add-on-72-in-w-x-48-in-d-x-96-in-h-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 05:50:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/filter-lcd-privacy-215-bk>: HTTP status code is not handled or not allowed 2026-01-25 05:50:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forbes-6266-flip-up-handle-for-forbes-room-service-table-factory-fit-only-black>: HTTP status code is not handled or not allowed 2026-01-25 05:50:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-high-cap-boltless-shelving-steel-deck-add-on-60-in-w-x-24-in-d-x-96-in-h>: HTTP status code is not handled or not allowed 2026-01-25 05:51:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-high-cap-boltless-shelving-steel-deck-add-on-72-in-w-x-18-in-d-x-96-in-h>: HTTP status code is not handled or not allowed 2026-01-25 05:51:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/schedule-planner-kit-24-x-36>: HTTP status code is not handled or not allowed 2026-01-25 05:51:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-mate-pm12260-emergency-light-replacement-battery-26ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:51:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bwr281dz-wine-refrigerator-builtin-dual-temperature-zone-28-bottle-capacity>: HTTP status code is not handled or not allowed 2026-01-25 05:51:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-wide-boltless-shelving-add-on-96-in-w-x-36-in-d-x-60-in-h-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 05:51:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rfb119-b-floor-box-1-gang-blank-device-plate>: HTTP status code is not handled or not allowed 2026-01-25 05:51:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sonnenschein-a51217-0g5-emergency-light-replacement-battery-18ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:51:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/862ck-12-floor-box-w896ck-12-carpet-brass-cover>: HTTP status code is not handled or not allowed 2026-01-25 05:51:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hl10000-series-corner-l-steel-desk-left-hand-return-66-x-72-blk-walnut>: HTTP status code is not handled or not allowed 2026-01-25 05:51:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ibc-spill-containment-pallet-with-drain-365-gallon>: HTTP status code is not handled or not allowed 2026-01-25 05:51:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 71807 pages (at 97 pages/min), scraped 35324 items (at 29 items/min) 2026-01-25 05:51:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fbs-laundry-cart-1112-hd-cloth-37l-x-22w-x-38h-12-bushel-black>: HTTP status code is not handled or not allowed 2026-01-25 05:51:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1722-7bs-1-drop-in-stainless-steel-3-holes-bar-prep-sink>: HTTP status code is not handled or not allowed 2026-01-25 05:51:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elite-tall-storage-cabinet-all-welded-46w-x-24d-x-72h-putty>: HTTP status code is not handled or not allowed 2026-01-25 05:51:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cs-1105-1-club-series-undermount-stainless-steel-compact-bar-prep-sink>: HTTP status code is not handled or not allowed 2026-01-25 05:51:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simplex-20013072-12volt-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 05:51:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bumper-guard-type-bb-164-1-12-1-12-black-60-6820-3>: HTTP status code is not handled or not allowed 2026-01-25 05:51:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/value-line-storage-cabinet-all-welded-30w-x-18d-x-66h-dove-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:51:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/value-line-storage-cabinet-all-welded-30w-x-18d-x-66h-putty>: HTTP status code is not handled or not allowed 2026-01-25 05:51:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elite-tall-storage-cabinet-all-welded-36w-x-24d-x-72h-putty>: HTTP status code is not handled or not allowed 2026-01-25 05:51:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-high-performance-floor-fan-3-speed-3000-cfm-38-hp-1-phase-115v>: HTTP status code is not handled or not allowed 2026-01-25 05:51:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elite-bookcase-assembled-36w-x-12d-x-72h-dove-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:51:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/teledyne-big-beam-s1210-emergency-light-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 05:51:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/five-shelf-extra-hd-boltless-shelving-add-on-36inw-x-12ind-x-96inh-wood-deck-usa>: HTTP status code is not handled or not allowed 2026-01-25 05:51:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/national-power-gt480s9-emergency-light-replacement-battery-75ah-12v-it>: HTTP status code is not handled or not allowed 2026-01-25 05:51:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/five-shelf-heavy-duty-boltless-shelving-add-on-48inw-x-12ind-x-84inh-wood-deck-usa>: HTTP status code is not handled or not allowed 2026-01-25 05:51:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/awsw-partition-80x7-f-slate>: HTTP status code is not handled or not allowed 2026-01-25 05:51:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/three-shelf-wide-boltless-shelving-add-on-48inw-x-24ind-x-60inh-laminate-deck-usa>: HTTP status code is not handled or not allowed 2026-01-25 05:51:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fine-bubble-disc-diffuser-ab-70007-15-inch-dia-3-4-mpt>: HTTP status code is not handled or not allowed 2026-01-25 05:51:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/peg-perego-bombardier-traxter-atv-igor0014-ride-on-toys-battery-35ah-12v>: HTTP status code is not handled or not allowed 2026-01-25 05:51:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honda-trx125-fourtrax-125cc-atv-replacement-battery-1987-1988-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:51:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mohs-hardness-pick-replacement-tip-number-9-007-0031-00-1-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:51:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/three-shelf-boltless-shelving-add-on-solid-deck-3300-lb-cap-96in-w-x-48in-d-x-84in-h-usa>: HTTP status code is not handled or not allowed 2026-01-25 05:51:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/awrd-partition-80x84-f-royal-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:51:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/can-am-outlander-400-xt-400cc-atv-replacement-battery-2004-2015-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:51:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smoking-shelter-63wsvrca-4sided-l-r-open-fr-15l-x-76w-vented-standing-seam-roof-clear>: HTTP status code is not handled or not allowed 2026-01-25 05:51:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smoking-shelter-s52vrdkb-4sided-left-open-front-12l-x-5w-vented-standing-seam-roof-dk-brz>: HTTP status code is not handled or not allowed 2026-01-25 05:51:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-degree-robetjet-mc101r-motorcaddy-and-golf-caddy-battery-26ah-12v>: HTTP status code is not handled or not allowed 2026-01-25 05:51:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/micromicr-toner-cartridge-thn-05a-black>: HTTP status code is not handled or not allowed 2026-01-25 05:51:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/three-shelf-wide-boltless-shelving-add-on-3300-lb-cap-96-in-wx48-in-dx96-in-h-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 05:51:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/schwinn-fly-fsrideon-electric-bicycle-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 05:51:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/three-shelf-wide-boltless-shelving-add-on-48-in-w-x-24-in-d-x-96-in-h-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 05:51:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/loading-dock-containment-netting-4-feet-x-22-feet-OHDB422>: HTTP status code is not handled or not allowed 2026-01-25 05:51:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gothic-5-piece-fireplace-toolset-666>: HTTP status code is not handled or not allowed 2026-01-25 05:51:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aprilia-rst-futura-1000cc-motorcycle-replacement-battery-2001-2004-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:51:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/moto-morini-s-500cc-motorcycle-replacement-battery-1945-1987-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:51:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/915-case-15-3-8l-x-12-1-8w-x-6-13-16h-black>: HTTP status code is not handled or not allowed 2026-01-25 05:51:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/915-case-15-3-8l-x-12-1-8w-x-6-13-16h-olive>: HTTP status code is not handled or not allowed 2026-01-25 05:51:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machine-table-with-lower-shelf-adjustable-height-36w-x-24d-x-26-40h-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:51:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/big-dog-woft-1750cc-motorcycle-replacement-battery-2000-2002-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:51:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m6-a-metric-eyebolt-shoulder-pattern-m6-x-100-short-shank>: HTTP status code is not handled or not allowed 2026-01-25 05:51:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m10-a-metric-eyebolt-shoulder-pattern-m10-x-150-short-shank>: HTTP status code is not handled or not allowed 2026-01-25 05:51:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/925-case-w-foam-18-11-16l-x-14-13-16w-x-7h-black>: HTTP status code is not handled or not allowed 2026-01-25 05:51:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-vis-cooling-safety-t-shirt-short-sleeve-ansi-class-3-4xl-yellow>: HTTP status code is not handled or not allowed 2026-01-25 05:51:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/megasole-anti-fatigue-insole-mens-67womens-89>: HTTP status code is not handled or not allowed 2026-01-25 05:51:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pam-puncture-resistant-anti-fatigue-insole-mens-1213>: HTTP status code is not handled or not allowed 2026-01-25 05:51:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-machine-table-with-lower-shelf-30w-x-16d-x-24h-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:51:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heat-shield-blanket-96x72-9672-1>: HTTP status code is not handled or not allowed 2026-01-25 05:51:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/moto-morini-9-5-1200cc-motorcycle-replacement-battery-2006-2010-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:51:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electric-forced-air-ceiling-heater-whfc1215-1500w-120v-white>: HTTP status code is not handled or not allowed 2026-01-25 05:51:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sc-ba95110li-led-for-n5-series-110v-clear>: HTTP status code is not handled or not allowed 2026-01-25 05:51:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mz-zschopau-tourer-500cc-motorcycle-replacement-battery-1966-2008-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:51:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sea-doo-spark-trixx-900cc-personal-watercraft-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:51:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harley-davidson-superglide-fxe-1340cc-motorcycle-battery-1979-1984-12v>: HTTP status code is not handled or not allowed 2026-01-25 05:51:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/we-wvhs250-18-460-445tp-db-20-vhs-motor-250hp-1800rpm-445tp-460v-wpi>: HTTP status code is not handled or not allowed 2026-01-25 05:51:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/critical-essential-bleeding-control-kit-1-24-persons-8-pieces-plastic-case>: HTTP status code is not handled or not allowed 2026-01-25 05:51:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bsm-inc-elite-ada-ss-sink-1-station-w-deck-electronic-faucet-wall-mount-36l-x-21w-x-5d-sink>: HTTP status code is not handled or not allowed 2026-01-25 05:51:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd-welded-workbench-48-x-30-lower-shelf--louvered-panel-steel-square-edge>: HTTP status code is not handled or not allowed 2026-01-25 05:51:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/289940000-graduated-pitcher-polypropylene-5000ml>: HTTP status code is not handled or not allowed 2026-01-25 05:51:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yamaha-xt250-serow-250cc-motorcycle-replacement-battery-2005-2013-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:51:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ranger-safari-ltd-mobility-scooter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:51:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd-welded-workbench-48-x-30-adj-height-louvered-panel-butcher-block-square-edge>: HTTP status code is not handled or not allowed 2026-01-25 05:51:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gssafe-5112-class-2-microfiber-birdseye-short-sleeve-t-shrit-w-black-bottom-orange-5xl>: HTTP status code is not handled or not allowed 2026-01-25 05:51:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gssafe-1702-class-2-heavy-duty-safety-vest-orange-3xl>: HTTP status code is not handled or not allowed 2026-01-25 05:51:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gssafe-1701-class-2-heavy-duty-safety-vest-lime-2xl>: HTTP status code is not handled or not allowed 2026-01-25 05:51:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bsm-inc-ada-ss-sink-1-station-with-electronic-faucet-5h-backsplash-40l-x-20w-x-6d-sink>: HTTP status code is not handled or not allowed 2026-01-25 05:51:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gssafe-5007-class-3-hi-viz-moisture-wicking-birdseye-short-sleeve-t-shirt-lime-xl-tall>: HTTP status code is not handled or not allowed 2026-01-25 05:51:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/voyager-gt-overboots-with-heels-cleated-outsole-3xl-12h-black>: HTTP status code is not handled or not allowed 2026-01-25 05:51:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pacesaver-espree-mobility-scooter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:51:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gssafe-2702-class-3-heavy-duty-safety-vest-orange-m>: HTTP status code is not handled or not allowed 2026-01-25 05:51:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/z-scooter-z-turbo-electric-scooter-replacement-battery-7ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 05:51:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gssafe-5113-class-3-microfiber-birdseye-short-sleeve-t-shrit-w-black-bottom-lime-xl-tall>: HTTP status code is not handled or not allowed 2026-01-25 05:51:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/summit-ff1532bcss-all-refrigerator-for-built-in-or-freestanding-use-3-cu-ft-15-wide>: HTTP status code is not handled or not allowed 2026-01-25 05:51:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/piaggio-vespa-gts300ie-300cc-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:51:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/albrecht-1-32-1-2-inch-high-precsion-keyless-drill-chuck-2jt>: HTTP status code is not handled or not allowed 2026-01-25 05:52:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-c-emergency-collet-nylon>: HTTP status code is not handled or not allowed 2026-01-25 05:52:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motofino-mf50qt-7-50cc-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:52:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/freedom-644-36-volt-electric-scooter-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 05:52:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-stp-stl-safet-stair-slp-roll-lad-wt-actu-16-perf-stp-wa-sw52414p>: HTTP status code is not handled or not allowed 2026-01-25 05:52:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merits-pioneersp43-mobility-scooter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:52:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-heavy-duty-workbench-adjustable-height-60w-x-36d-x-36h-3600-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 05:52:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bollard-cover-cl1385s-4-1-2dia-x-52h-green-w-white-tape>: HTTP status code is not handled or not allowed 2026-01-25 05:52:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelate-42w-x-18d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 05:52:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bollard-cover-cl1386pnt72-7dia-x-72h-brown-w-no-tape>: HTTP status code is not handled or not allowed 2026-01-25 05:52:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bollard-cover-cl1385hdnt-4-1-2dia-x-52h-orange-w-no-tape>: HTTP status code is not handled or not allowed 2026-01-25 05:52:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelate-24w-x-21d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 05:52:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 71907 pages (at 100 pages/min), scraped 35339 items (at 15 items/min) 2026-01-25 05:52:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelate-54w-x-14d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 05:52:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-nexelate-36w-x-14d-x-60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 05:52:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bollard-cover-cl1386mm72-7dia-x-72h-green-w-no-tape>: HTTP status code is not handled or not allowed 2026-01-25 05:52:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rear-rplcmnt-blade-red-gum-2x14x49-316-17-holes-20-10058>: HTTP status code is not handled or not allowed 2026-01-25 05:52:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leisure-lift-espree-mobility-scooter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:52:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/p132m-kwik-fit-pressure-vacuum-pump-t-handle-for-easy-operation>: HTTP status code is not handled or not allowed 2026-01-25 05:52:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/190m-horizontal-coupling-for-taco-circulators-1-2-x-3-8dia-2l>: HTTP status code is not handled or not allowed 2026-01-25 05:52:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/z1395-34x4-non-freeze-yard-hydrant-exposed-34-x-4>: HTTP status code is not handled or not allowed 2026-01-25 05:52:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelate-30w-x-21d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 05:52:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/linen-storage-unit-locker-llewlsu-gosv-silvervein-31-x-21-1-4-x-84-1-2>: HTTP status code is not handled or not allowed 2026-01-25 05:52:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kn-4sorp-sorting-panel-30-x-48-red>: HTTP status code is not handled or not allowed 2026-01-25 05:52:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelate-36w-x-18d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 05:52:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dalton-4-wheel-sc-s245-electric-scooter-replacement-battery-18ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:52:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eco-funnel-ef-8-sw-8-with-70mm-cap-5-gal-samson-stacker-red-lid>: HTTP status code is not handled or not allowed 2026-01-25 05:52:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelate-36w-x-21d-x-60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 05:52:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/z-scooters-z-cruiserx-electric-scooter-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 05:52:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e-boarder-powersonic-electric-scooter-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:52:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/walmart-style-strip-wms-12>: HTTP status code is not handled or not allowed 2026-01-25 05:52:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pacesaver-fusion-500-mobility-scooter-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 05:52:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sticky-strip-ss-12-12-position>: HTTP status code is not handled or not allowed 2026-01-25 05:52:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-style-acrylic-signholder-top-loading-8-12w-x-11h>: HTTP status code is not handled or not allowed 2026-01-25 05:52:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelate-72w-x-21d-x-60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 05:52:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classic-aluminum-banner-hanger-22l>: HTTP status code is not handled or not allowed 2026-01-25 05:52:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u1249-ns-cantilever-rack-single-sided-upright-49w-x-12h>: HTTP status code is not handled or not allowed 2026-01-25 05:52:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/peel-n-stick-strip-header-3-78w-x-3-716h>: HTTP status code is not handled or not allowed 2026-01-25 05:52:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-drum-spill-containment-platform-1611st-27-w-x-25-11-16-d-x-6-h-steel>: HTTP status code is not handled or not allowed 2026-01-25 05:52:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/peel-stick-literature-pockets-8-58w-x-7h-x-1-14d>: HTTP status code is not handled or not allowed 2026-01-25 05:52:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/econo-line-shelf-dividers-1h-13-916-depth>: HTTP status code is not handled or not allowed 2026-01-25 05:52:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelate-60w-x-14d-x-69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 05:52:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bombardier-tundra-xtreme-e-tec-600-ho-600cc-snowmobile-battery-2018-12v>: HTTP status code is not handled or not allowed 2026-01-25 05:52:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/econo-strip-extra-long>: HTTP status code is not handled or not allowed 2026-01-25 05:52:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grip-tite-suction-cup-vertical-sign-holder-1-pvc-clear>: HTTP status code is not handled or not allowed 2026-01-25 05:52:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/17-wood-hanger-for-mens-suit-standard-hook-walnut-w-brass-hardware-100case>: HTTP status code is not handled or not allowed 2026-01-25 05:52:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelate-60w-x-14d-x-80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 05:52:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roto-clip-extenda-print-protectorsignholder-5-12w-x-3-12h>: HTTP status code is not handled or not allowed 2026-01-25 05:52:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-nexelate-42w-x-24d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 05:52:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grip-tite-flexi-sign-holders-6>: HTTP status code is not handled or not allowed 2026-01-25 05:52:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-gtx-550f-550cc-snowmobile-replacement-battery-2005-2008-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:52:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-max-peg-hook-strip>: HTTP status code is not handled or not allowed 2026-01-25 05:52:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-tundra-300f-300cc-snowmobile-replacement-battery-2006-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:52:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelate-54w-x-21d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 05:52:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-mx-z-500-f-500cc-snowmobile-replacement-battery-2001-2002-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:52:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelate-72w-x-21d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 05:52:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelate-36w-x-18d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 05:52:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelate-72w-x-24d-x-80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 05:52:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ski-doo-legend-800-sdi-793cc-snowmobile-replacement-battery-2004-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:52:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/SVT-7230-72-W-x-30-D-18-ga-T-430-Stainless-Steel-Workbench>: HTTP status code is not handled or not allowed 2026-01-25 05:52:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelate-48w-x-18d-x-60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 05:52:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/schaefer-stacking-transport-container-14-6-2-pl-18-l-x-12-w-x-8-h-green>: HTTP status code is not handled or not allowed 2026-01-25 05:52:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e-series-lock-combo-esl-c>: HTTP status code is not handled or not allowed 2026-01-25 05:52:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arctic-cat-mountain-cat-500-497cc-snowmobile-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 05:52:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-black-epoxy-42w-x-21d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 05:52:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dm24-7-bb-retractable-awning-manual-24-w-x-10-d-x-10-h-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:52:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ba-fans-powerfoil-8-ceiling-fan-20-ft-208v-3ph>: HTTP status code is not handled or not allowed 2026-01-25 05:52:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ba-fans-basic-6-ceiling-fan-16-ft-208v-1ph>: HTTP status code is not handled or not allowed 2026-01-25 05:52:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelite-60w-x-21d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 05:52:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ba-fans-basic-6-ceiling-fan-20-ft-460v-3ph>: HTTP status code is not handled or not allowed 2026-01-25 05:52:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ba-fans-aireye-directional-fan-with-swivel-mount-20-inch>: HTTP status code is not handled or not allowed 2026-01-25 05:52:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ba-fans-aireye-directional-fan-with-wall-mount-aeos-24-inch>: HTTP status code is not handled or not allowed 2026-01-25 05:52:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-black-epoxy-54w-x-18d-x-60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 05:52:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/handheld-scraper-3-inch-stainless-steel-white>: HTTP status code is not handled or not allowed 2026-01-25 05:52:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-a70-4l720-v-belt-1-2-x-72>: HTTP status code is not handled or not allowed 2026-01-25 05:52:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-a57-4l590-v-belt-1-2-x-59>: HTTP status code is not handled or not allowed 2026-01-25 05:52:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wood-18-compartment-mail-sorter>: HTTP status code is not handled or not allowed 2026-01-25 05:53:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vestil-aluminium-powered-stair-climber-350-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 05:53:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-a36-4l380-v-belt-1-2-x-38>: HTTP status code is not handled or not allowed 2026-01-25 05:53:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-b38-5l410-v-belt-5-8-x-41>: HTTP status code is not handled or not allowed 2026-01-25 05:53:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-a88-4l900-v-belt-1-2-x-90>: HTTP status code is not handled or not allowed 2026-01-25 05:53:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elkay-98530c-regulator-kit-w-spring-for-push-button-models>: HTTP status code is not handled or not allowed 2026-01-25 05:53:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exit-device-lever-trim-passage-grade-1-pb-handle>: HTTP status code is not handled or not allowed 2026-01-25 05:53:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelite-42w-x-24d-x-80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 05:53:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8249z-s-hi-vis-surveyors-vest-class-2-economy-single-size-l-lime>: HTTP status code is not handled or not allowed 2026-01-25 05:53:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8249z-s-hi-vis-surveyors-vest-class-2-economy-single-size-4xl-orange>: HTTP status code is not handled or not allowed 2026-01-25 05:53:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8246z-s-two-tone-mesh-hi-vis-safety-vest-class-2-single-size-xl-lime>: HTTP status code is not handled or not allowed 2026-01-25 05:53:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 72007 pages (at 100 pages/min), scraped 35364 items (at 25 items/min) 2026-01-25 05:53:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-black-epoxy-24w-x-24d-x-69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 05:53:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelite-42w-x-24d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 05:53:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelite-36w-x-21d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 05:53:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-black-epoxy-72w-x-24d-x-80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 05:53:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/px-b195-v-belt-5-8-x-198>: HTTP status code is not handled or not allowed 2026-01-25 05:53:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16495p-replacement-gauge-vial-for-scully-and-oil-tank-gauges>: HTTP status code is not handled or not allowed 2026-01-25 05:53:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/supply-144-bulk-lamp-recycling-kit>: HTTP status code is not handled or not allowed 2026-01-25 05:53:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-lockout-padlock-keyed-alike-1-1-2-plastic-nylon-green-6-pk>: HTTP status code is not handled or not allowed 2026-01-25 05:53:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rackguard-5-w-x-2-5-16-d-x-15-3-4-h-yellow>: HTTP status code is not handled or not allowed 2026-01-25 05:53:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iflex-standard-bollard-8-1-4-h>: HTTP status code is not handled or not allowed 2026-01-25 05:53:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/as-40x-1000x-portable-monocular-student-microscope-w-premium-insect-exploration-kit>: HTTP status code is not handled or not allowed 2026-01-25 05:53:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-black-epoxy-48w-x-14d-x-60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 05:53:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hvac-motor-ehm3313t-8-3-ph-10-hp-200-v-1770-rpm-opsb-215t-frame>: HTTP status code is not handled or not allowed 2026-01-25 05:53:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-black-epoxy-54w-x-14d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 05:53:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7520-wine-glass-17-oz-glassware-vina-12-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:53:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/severe-duty-motor-ecp84106t-5-3-ph-20-hp-575-v-3510-rpm-tefc-256t-frame>: HTTP status code is not handled or not allowed 2026-01-25 05:53:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-black-epoxy-48w-x-18d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 05:53:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rb-scale-package-for-65-diameter-turntable>: HTTP status code is not handled or not allowed 2026-01-25 05:53:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-phase-motor-cwdm3542-5-75-hp-1750-rpm-56c-frame-c-face-mount-tenv-575-volts>: HTTP status code is not handled or not allowed 2026-01-25 05:53:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-best-value-wire-shelving-unit-36-w-x-18-d-x-74-h-400-lb-shelf-cap-black-epoxy>: HTTP status code is not handled or not allowed 2026-01-25 05:53:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dicata-intrinsically-safe-low-profile-dual-light-headlamp-red>: HTTP status code is not handled or not allowed 2026-01-25 05:53:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelite-48w-x-24d-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 05:53:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1000va-700w-ups-smart-lcd-smc1000-2u-rm-pure-sine-wave-avr-usb>: HTTP status code is not handled or not allowed 2026-01-25 05:53:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tb108-turnbuckle-body-carbon-3-4-10-dia-x-8-7-16-l>: HTTP status code is not handled or not allowed 2026-01-25 05:53:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tb108-9-hdg-turnbuckle-body-galvanized-3-4-10-dia-x-11-7-16-l>: HTTP status code is not handled or not allowed 2026-01-25 05:53:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ken-forging-tb103-j-e-turnbuckle-assembly-jaw-eye-3-8-16-x-11-7-8-carbon>: HTTP status code is not handled or not allowed 2026-01-25 05:53:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ken-forging-tb105-s-hdg-turnbuckle-assembly-stub-stub-1-2-13-x-14-galvanized>: HTTP status code is not handled or not allowed 2026-01-25 05:53:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evolution-deluxe-6151-short-brim-hard-hat-hdpe-shell-6-pt-suspension-ratchet-adjustment-yellow>: HTTP status code is not handled or not allowed 2026-01-25 05:53:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-black-epoxy-72w-x-14d-x-80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 05:53:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-poly-green-72wx18dx69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 05:53:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washdown-motor-cewdm3610t-3-ph-3-hp-3450-rpm-208-230-460-v-tefc-182tc-fr>: HTTP status code is not handled or not allowed 2026-01-25 05:53:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vesuvio-cap-style-dome-hard-hat-nylon-fiber-resin-shell-4-pt-suspension-rachet-adjustment-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:53:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-poly-green-30wx21dx69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 05:53:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/k352-stud-tire-on-2-piece-wheel-354m-st-34bb-410-350-4-2-1-8-hub-x-3-4-bore>: HTTP status code is not handled or not allowed 2026-01-25 05:53:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dynamic-dom-cap-style-dome-hard-hat-hdpe-shell-6-pt-suspension-rachet-adjustment-orange>: HTTP status code is not handled or not allowed 2026-01-25 05:53:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dynamic-dom-cap-style-dome-hard-hat-hdpe-shell-4-pt-suspension-rachet-adjustment-dark-green>: HTTP status code is not handled or not allowed 2026-01-25 05:53:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1009730-upeghk-u-hook-accessory-for-wire-baskets-bright-silver>: HTTP status code is not handled or not allowed 2026-01-25 05:53:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t103s-rotating-literature-display-w-12-clear-pockets-heavy-duty-base-black>: HTTP status code is not handled or not allowed 2026-01-25 05:53:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hvac-motor-em4117t-g-3-ph-30-hp-230-460-v-1200-rpm-tefc-326t-frame>: HTTP status code is not handled or not allowed 2026-01-25 05:53:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/severe-duty-motor-ecp84313t-5-3-ph-75-hp-575-v-3555-rpm-tefc-365ts-frame>: HTTP status code is not handled or not allowed 2026-01-25 05:53:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1009659-1448bsz-wire-basket-48w-x-14d-x-45-to-85h-bright-silver>: HTTP status code is not handled or not allowed 2026-01-25 05:53:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lc-8-3500-centrifuge-with-8-x-15ml-rotor-max-3500-rpm-120v>: HTTP status code is not handled or not allowed 2026-01-25 05:53:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelon-24w-x-14d-x-69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 05:53:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-poly-green-24w-x-24d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 05:53:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-poly-green-36w-x-14d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 05:53:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-nexelon-48w-x-14d-x-80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 05:53:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inkcart-cli-251-bk>: HTTP status code is not handled or not allowed 2026-01-25 05:53:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dc-explosion-proof-motor-cdpx3436-075-hp-1750-rpm-xpfc-56c>: HTTP status code is not handled or not allowed 2026-01-25 05:53:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hvac-motor-ehm3714t-5-3-ph-10-hp-575-v-1770-rpm-tefc-215t-frame>: HTTP status code is not handled or not allowed 2026-01-25 05:53:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v426-ball-valve-lockout-large-metal-00300>: HTTP status code is not handled or not allowed 2026-01-25 05:53:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-poly-green-42wx14dx69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 05:53:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s59-4016d-point-of-use-thermostatic-mixing-valve-15-gpm>: HTTP status code is not handled or not allowed 2026-01-25 05:53:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6a-rod-end-blank-c1030-1035-5-8-x-3-1-2>: HTTP status code is not handled or not allowed 2026-01-25 05:53:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-5-inch-circular-nylon-brush-for-gvc-models-390-1100-1250-1502-3-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:53:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-round-black-cafeteria-table-with-8-stools-gray-mdf-core-top-blue-stools>: HTTP status code is not handled or not allowed 2026-01-25 05:53:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machined-chrome-steel-needle-roller-bearing-w-inner-ring-95-x-125-x-26mm>: HTTP status code is not handled or not allowed 2026-01-25 05:53:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machined-chrome-steel-needle-roller-bearing-w-inner-ring-130-x-180-x-50mm>: HTTP status code is not handled or not allowed 2026-01-25 05:53:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galvanized-stock-tank-125-gallon-wt225-60-18-l-x-24-58-w-x-23-34-h-price-each>: HTTP status code is not handled or not allowed 2026-01-25 05:53:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-phase-motor-cl3712t-10-hp-230-volts-1740-rpm-tefc-213tc-frame>: HTTP status code is not handled or not allowed 2026-01-25 05:53:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-nexelon-72w-x-14d-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 05:53:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/material-handling/mezzanines_modular_structures/catwalks_mezzanines/mezzanines already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:53:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-motor-230-460-v-05-hp-1740-rpm-3-ph-56-tenv>: HTTP status code is not handled or not allowed 2026-01-25 05:54:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-poly-green-24wx18dx60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 05:54:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11-step-steel-roll-ladd-rear-exit-gate-24-30-plat-serr-rwsr111242-xr>: HTTP status code is not handled or not allowed 2026-01-25 05:54:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inverter-vector-motor-idfrpm28754-3ph-75hp-1785-3565rpm-460v-tefc-fl2898>: HTTP status code is not handled or not allowed 2026-01-25 05:54:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machined-chrome-steel-needle-roller-bearing-w-inner-ring-85-x-120-x-63mm>: HTTP status code is not handled or not allowed 2026-01-25 05:54:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machined-chrome-steel-needle-roller-bearing-w-inner-ring-160-x-200-x-40mm>: HTTP status code is not handled or not allowed 2026-01-25 05:54:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-threshold-ramp-ath1232-12l-x-32w-800-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 05:54:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelon-24w-x-24d-x-69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 05:54:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evap-cooling-bandana-w-cooling-towel-tie-red-western-12563>: HTTP status code is not handled or not allowed 2026-01-25 05:54:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forklift-caddy-clip-shrink-wrap-holder-w-divider-grey>: HTTP status code is not handled or not allowed 2026-01-25 05:54:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/material-handling/mezzanines_modular_structures/catwalks_mezzanines/mezzanines>: HTTP status code is not handled or not allowed 2026-01-25 05:54:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evap-cooling-triangle-hat-w-cooling-towel-red-western-12583>: HTTP status code is not handled or not allowed 2026-01-25 05:54:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 72103 pages (at 96 pages/min), scraped 35389 items (at 25 items/min) 2026-01-25 05:54:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-poly-green-24w-x-24d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 05:54:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-poly-green-60wx21dx60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 05:54:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-poly-green-42w-x-18d-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 05:54:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/125-orange-woven-polyester-strap-x-600-ftcoil3835-lbs-system-strength-423110>: HTTP status code is not handled or not allowed 2026-01-25 05:54:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-shelf-truck-nexelon-48w-x-18d-x-69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 05:54:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ninja-bnf-gloves-18-gauge-nylon-shell-nitrile-coated-palm-fingertips-red-black-2xl>: HTTP status code is not handled or not allowed 2026-01-25 05:54:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/encoder-feedback-kit-417708-120-rl67-dpfv-l440-5-15>: HTTP status code is not handled or not allowed 2026-01-25 05:54:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/feedback-cable-w-assembly-ms-connector-cbl455zd-2-150-ft-extension-length>: HTTP status code is not handled or not allowed 2026-01-25 05:54:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-body-cylinder-10-ton-0-43-stroke>: HTTP status code is not handled or not allowed 2026-01-25 05:54:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/encoder-feedback-kit-no-encoder-417708-202-dpfv-tebc-fl-rl280-l400>: HTTP status code is not handled or not allowed 2026-01-25 05:54:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/idm-idnm-idwnm-motor-encoder-feedback-kit-enc00nv-d1-hs35m-tenv-56-215t-5-24vdc>: HTTP status code is not handled or not allowed 2026-01-25 05:54:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-nexelon-42w-x-21d-x-80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 05:54:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/awn-h22-6k-standing-seam-awning-6-8-w-x-2d-x-2h-black>: HTTP status code is not handled or not allowed 2026-01-25 05:54:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coupler-zinc-regular-flow-female-side-1-4-18-nptf>: HTTP status code is not handled or not allowed 2026-01-25 05:54:20 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/gloves_sleeves/coated_gloves already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:54:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-nexelon-60w-x-24d-x-60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 05:54:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conduit-box-kit-standard-size-33cb5000a01sp-42-nema-frame>: HTTP status code is not handled or not allowed 2026-01-25 05:54:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/d-flange-kit-35ep1506a01sp-tefc-143-145t-nema-frame>: HTTP status code is not handled or not allowed 2026-01-25 05:54:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h-x-1-1-x-66-h-series-quick-detach-steel-bushing-1-bore>: HTTP status code is not handled or not allowed 2026-01-25 05:54:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/constant-velocity-blower-cooling-conversion-kit-blwl09-l-1-ph-115v-254tc-256tc-nema-frame>: HTTP status code is not handled or not allowed 2026-01-25 05:54:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/environmental-particle-counter-up-to-0-3m-built-in-camera>: HTTP status code is not handled or not allowed 2026-01-25 05:54:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-motor-base-b365t-365t-5-8-x-2-1-2-bolt-size-w-2-adjustable-screw>: HTTP status code is not handled or not allowed 2026-01-25 05:54:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o-ring-for-cabinet-42000>: HTTP status code is not handled or not allowed 2026-01-25 05:54:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waring-cac88-container-with-lid-stainless-steel-32-ounce-blade-collar>: HTTP status code is not handled or not allowed 2026-01-25 05:54:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cold-chain-technologies-w10-air-cargo-cover-euro-pallet-uk-47l-x-3112w-x-24h-tyvek-white>: HTTP status code is not handled or not allowed 2026-01-25 05:54:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-4-shelf-truck-nexelon-24w-x-18d-x-69h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 05:54:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/gloves_sleeves/coated_gloves>: HTTP status code is not handled or not allowed 2026-01-25 05:54:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/celltreat-scientific-products-250l-low-retention-pipette-tips-ovation-vistarak-sterile-960-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:54:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/freestanding-air-dry-laundry-rack-locker-dpf-chrome>: HTTP status code is not handled or not allowed 2026-01-25 05:54:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelon-60w-x-21d-x-80h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 05:54:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-shelf-truck-nexelon-72w-x-21d-x-60h-polyurethane-swivel-casters-2-brakes>: HTTP status code is not handled or not allowed 2026-01-25 05:54:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/celltreat-scientific-products-5l-pipette-mla-precision-silver>: HTTP status code is not handled or not allowed 2026-01-25 05:54:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/celltreat-scientific-products-25ml-reagent-reservoir-individually-wrapped-sterile-100-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:54:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/basement-floor-jack-54-to-100-11200-lb-extended-cap-987707>: HTTP status code is not handled or not allowed 2026-01-25 05:54:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/celltreat-scientific-products-2001000l-pipette-mla-digital-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:54:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/we-mc320-125-25-vhs-alternate-coupling-bore-size-125-frame-324tp-or-326tp>: HTTP status code is not handled or not allowed 2026-01-25 05:54:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/we-mc320-15-vhs-alternate-coupling-bore-size-15-frame-324tp-or-326tp>: HTTP status code is not handled or not allowed 2026-01-25 05:54:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forks-heavy-weight-plastic-1000-carton-black>: HTTP status code is not handled or not allowed 2026-01-25 05:54:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/celltreat-scientific-products-50ml-reagent-reservoir-individually-wrapped-sterile-100-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:54:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/celltreat-scientific-products-50ml-wobblenot-serological-pipet-paperplastic-bag-sterile-100-pa>: HTTP status code is not handled or not allowed 2026-01-25 05:54:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/omnicart-cab-40-cap-w-locking-panel-woodgrain-34-1-2w-x-17d-x-70-1-8h>: HTTP status code is not handled or not allowed 2026-01-25 05:54:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n-ferno-6475-cold-storage-thermal-insulated-coverall-m-navy>: HTTP status code is not handled or not allowed 2026-01-25 05:54:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-party-cold-cups-polystyrene-16-oz-1000-carton-red>: HTTP status code is not handled or not allowed 2026-01-25 05:54:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/celltreat-scientific-products-15ml-pipette-mla-macro-adjustable>: HTTP status code is not handled or not allowed 2026-01-25 05:54:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securall-vertical-divider-mg-divider-9-for-securall-medical-gas-cabinets-mg121-and-mg321>: HTTP status code is not handled or not allowed 2026-01-25 05:54:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wasp-ams-kit-2-for-5-channel-wsa-125-kit2-hs>: HTTP status code is not handled or not allowed 2026-01-25 05:54:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commercial-gas-hose-kit-csa-and-ansi-approved-bkg-ghc-5048-sck2>: HTTP status code is not handled or not allowed 2026-01-25 05:54:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rcl-306-arm-complete-head-assembly>: HTTP status code is not handled or not allowed 2026-01-25 05:54:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manway-gasket-spiral-wound-12-x-16-x-1-1-4-0-25-thick-elliptical-2pk>: HTTP status code is not handled or not allowed 2026-01-25 05:54:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lighting-led-ws18-led-wall-sconce-6w-3000-cct-1100-lumens-82-cri-nickel>: HTTP status code is not handled or not allowed 2026-01-25 05:54:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-area-dehumidifier-cd425-220v-with-high-capacity-pump-16-amps-1750-cfm-285-pints>: HTTP status code is not handled or not allowed 2026-01-25 05:54:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/web-winch-bottom-mount-standard>: HTTP status code is not handled or not allowed 2026-01-25 05:54:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/topoge-series-2000-handhole-gasket-312-x-412-x-12-graphite-elliptical-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:54:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/controls-merz-w105-3-aa-reversing-switch-maintained-3-pole-16a-4-hole-front-mount>: HTTP status code is not handled or not allowed 2026-01-25 05:54:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-e-a-ratchet-strap-with-spring-actuated-fitting-12>: HTTP status code is not handled or not allowed 2026-01-25 05:54:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lab-oven-with-forced-air-20af-1050w>: HTTP status code is not handled or not allowed 2026-01-25 05:54:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jurni-multi-purpose-table-with-post-legs--glides-60l-x-24w-x-29h-fusion-maple>: HTTP status code is not handled or not allowed 2026-01-25 05:54:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14000-btu-dual-hose-portable-air-conditioner-with-3m-antimicrobial-filter-arc-143mx>: HTTP status code is not handled or not allowed 2026-01-25 05:54:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dxs2153-standard-anvil-attachment-38-drive>: HTTP status code is not handled or not allowed 2026-01-25 05:54:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/83132-1035-f1-gas-engine-550-series-tiller-log-splitter-horizontal-shaft>: HTTP status code is not handled or not allowed 2026-01-25 05:54:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/topoge-series-180-manway-gasket-18-x-24-x-112-black-rubber-elliptical-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:54:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cl-comfort-pak-6-qt-with-tool-kit-backpack-vacuum-9060610010>: HTTP status code is not handled or not allowed 2026-01-25 05:54:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/twin-series-clamp-for-pipe-tscp038-3-8-inch-id>: HTTP status code is not handled or not allowed 2026-01-25 05:55:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metero-casters-for-wire-shelving-polyurethane-rigid-with-bumper>: HTTP status code is not handled or not allowed 2026-01-25 05:55:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hard-scrub-brush-set-for-vortex-13-multi-surface-scrubber-13w-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-25 05:55:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cep-9125-5500-lumen-led-stubby-light-magnet>: HTTP status code is not handled or not allowed 2026-01-25 05:55:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cep-95132-100-123-stw-string-light-plastic-guards>: HTTP status code is not handled or not allowed 2026-01-25 05:55:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-thread-o-ring-connector-h5315x10x12-5-8-inch-tube-od-3-4-inch-port-size>: HTTP status code is not handled or not allowed 2026-01-25 05:55:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simpson-replacement-spray-nozzles-rated-up-to-4500-psi-orifice-size-40>: HTTP status code is not handled or not allowed 2026-01-25 05:55:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durham-base-for-modular-cabinets-34-1-8x17-1-4x5-3-4>: HTTP status code is not handled or not allowed 2026-01-25 05:55:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tx4000-esd-cleanroom-apparel-hood-open-face-l-white>: HTTP status code is not handled or not allowed 2026-01-25 05:55:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/suction-hose-wlh12572-1-1-4-inch-id-x-6-foot-long-hose>: HTTP status code is not handled or not allowed 2026-01-25 05:55:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s-4200-boom-stand>: HTTP status code is not handled or not allowed 2026-01-25 05:55:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/suction-hose-wlh10096-1-inch-id-x-8-foot-long-hose>: HTTP status code is not handled or not allowed 2026-01-25 05:55:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5scds-3716-screw-conveyor-shaft-3716-fits-ultimate-shaft-mount-reducer-size-5->: HTTP status code is not handled or not allowed 2026-01-25 05:55:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 72200 pages (at 97 pages/min), scraped 35412 items (at 23 items/min) 2026-01-25 05:55:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mgar300-port-adaptor-1-8-npt-regulator-filter-regulator>: HTTP status code is not handled or not allowed 2026-01-25 05:55:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-bolt-clamp-hc200-2-9-16-inch-nominal-dia-234-inch-min-dia-265-inch-max-dia>: HTTP status code is not handled or not allowed 2026-01-25 05:55:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ediusa-ballistic-vest-tested-to-level-iiia-ballistic-resistance-xxlarge>: HTTP status code is not handled or not allowed 2026-01-25 05:55:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smr8-151-shaft-mount-reducer-size-8-151-ratio-4-716-tapered-bore>: HTTP status code is not handled or not allowed 2026-01-25 05:55:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ediusa-pasgt-style-ballistic-helmet-level-iiia-ballistic-resistance-large-black>: HTTP status code is not handled or not allowed 2026-01-25 05:55:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flush-face-coupler-fm0606-3-8-inch-npt-port-male-plug>: HTTP status code is not handled or not allowed 2026-01-25 05:55:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bott-toolboard-cable-holder-for-perfo-panels-12-dia>: HTTP status code is not handled or not allowed 2026-01-25 05:55:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9oz-30-sateen-cotton-jacket-green-4xl>: HTTP status code is not handled or not allowed 2026-01-25 05:55:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-nut-45-elbow-h5356x16-1-inch-tube-od>: HTTP status code is not handled or not allowed 2026-01-25 05:55:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/upright-hydraulic-reservoir-ur50a-500-gal-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 05:55:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fr-cotton-leather-cape-sleeve-apron-w-black-anodized-snap-dark-brown-4xl>: HTTP status code is not handled or not allowed 2026-01-25 05:55:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wing-type-quick-detach-hydraulic-coupler-qdwc201-1-1-4-inch-npt-coupler-75-flow-gpm>: HTTP status code is not handled or not allowed 2026-01-25 05:55:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/techstar-sos-storage-bin-11-cu-ft-yellow>: HTTP status code is not handled or not allowed 2026-01-25 05:55:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sa-mirella-end-table-sanddune>: HTTP status code is not handled or not allowed 2026-01-25 05:55:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sa-mirella-conference-table-sitting-height-12-rectangle-southerntobacco>: HTTP status code is not handled or not allowed 2026-01-25 05:55:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tpi-workstation-fan-18-blade-diameter-18-hp-wall-mount>: HTTP status code is not handled or not allowed 2026-01-25 05:55:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sa-mirella-conference-table-sitting-height-16-rectangle-southerntobacco>: HTTP status code is not handled or not allowed 2026-01-25 05:55:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90-street-elbow-h3409x20-1-1-4-inch-x-1-1-4-inch-male-to-female>: HTTP status code is not handled or not allowed 2026-01-25 05:55:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bumper-pad-c40-1-x-7-foam-to-fit-30w-x-44l-infant-crib>: HTTP status code is not handled or not allowed 2026-01-25 05:55:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sa-mirella-wood-door-storage-cabinet-sanddune>: HTTP status code is not handled or not allowed 2026-01-25 05:55:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-long-sleeve-button-down-work-shirt-l-black>: HTTP status code is not handled or not allowed 2026-01-25 05:55:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/worldwide-electric-winl87-10-1-213-5tchelical-inline-speed-reducer213-5tc-input-flange10-1-ratio>: HTTP status code is not handled or not allowed 2026-01-25 05:55:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-sensing-spreader-valve-hv1030ls-spreader-valve-10-30-gpm-151-lpm-3000-psi-210-bar>: HTTP status code is not handled or not allowed 2026-01-25 05:55:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/snap-button-down-long-sleve-work-shirt-l-tall-navy>: HTTP status code is not handled or not allowed 2026-01-25 05:55:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/full-flow-control-gate-valve-hgv200-2-inch-valve-200-psi-wog-non-shock>: HTTP status code is not handled or not allowed 2026-01-25 05:55:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-shift-cylinder-as302-w-tubing-and-fittings>: HTTP status code is not handled or not allowed 2026-01-25 05:55:29 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:55:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/worldwide-electric-winl87-3-1-213-5tc-helical-inline-speed-reducer-213-5tc-input-flange-3-1-ratio>: HTTP status code is not handled or not allowed 2026-01-25 05:55:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ds-327100b-shrink-wrap-32w-x-100l-7mil-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:55:31 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:55:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clark-foam-products-1001089-foam-cube-polyether-charcoal-8-sq>: HTTP status code is not handled or not allowed 2026-01-25 05:55:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clark-foam-products-1001216-foam-sheet-lam-poly-char-1x24x48-in>: HTTP status code is not handled or not allowed 2026-01-25 05:55:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/taper-gage-4630-1-05-10mm-range-005mm-graduation>: HTTP status code is not handled or not allowed 2026-01-25 05:55:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-vis-full-zip-hooded-sweater-reflective-xl-yellow>: HTTP status code is not handled or not allowed 2026-01-25 05:55:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-vis-full-zip-hooded-sweater-reflective-m-tall-yellow>: HTTP status code is not handled or not allowed 2026-01-25 05:55:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-wide-professional-monitor-printer-stand-w-drawer>: HTTP status code is not handled or not allowed 2026-01-25 05:55:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-locker-accessory-slope-top-end-closure-15d-x-5h-729-parchment>: HTTP status code is not handled or not allowed 2026-01-25 05:55:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-visibility-athletic-performance-t-shirt-flame-resistant-6-oz-m-tall-yellow>: HTTP status code is not handled or not allowed 2026-01-25 05:55:38 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:55:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 05:55:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-rotary-base-reference-organizer-10-panel>: HTTP status code is not handled or not allowed 2026-01-25 05:55:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/coats_jackets/cold_weather_jackets_coats>: HTTP status code is not handled or not allowed 2026-01-25 05:55:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1000-series-chair-1000-cb-ecr-bl-esd-clean-room-vinyl-chrome-base-18-23h-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:55:42 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/roof_luggage_racks already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:55:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electrical-channel-el5-36-15a-6-outlets-36-long-work-surface-mounted-grey>: HTTP status code is not handled or not allowed 2026-01-25 05:55:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pack-rat-3-compart-drawer-white-48-l-x-30-w-x-12-12-h-337-3>: HTTP status code is not handled or not allowed 2026-01-25 05:55:44 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/work-benches/bench_tops_accessories/workbench_drawers_cabinets_pedestals already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:55:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-capacity-33-lbs-cylinders-heavy-duty-steel-gas-cylinder-cage-90w-x-30d-x-53h-white>: HTTP status code is not handled or not allowed 2026-01-25 05:55:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clearance-bar-7d-x-120l-yellow-wno-tape-no-graphics-htgrd7120yntng>: HTTP status code is not handled or not allowed 2026-01-25 05:55:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galvanized-steel-solid-shelving-add-on-189961>: HTTP status code is not handled or not allowed 2026-01-25 05:55:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pendant-mount-kit-series-b-27xst-27xl-121x-series>: HTTP status code is not handled or not allowed 2026-01-25 05:55:47 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:55:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 05:55:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fat-cap-orange-one-size-0063rhvoosa>: HTTP status code is not handled or not allowed 2026-01-25 05:55:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vented-plastic-shelving-42x24x86-nexelon-finish-331790N>: HTTP status code is not handled or not allowed 2026-01-25 05:55:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/post-kit-63-highsingle-797327>: HTTP status code is not handled or not allowed 2026-01-25 05:55:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/roof_luggage_racks>: HTTP status code is not handled or not allowed 2026-01-25 05:55:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/activity-tables-48-l-x-48-w-standard-height-round-white>: HTTP status code is not handled or not allowed 2026-01-25 05:55:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/work-benches/bench_tops_accessories/workbench_drawers_cabinets_pedestals>: HTTP status code is not handled or not allowed 2026-01-25 05:55:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hand-lever-pump-for-dispensing-oils-and-100-antifreeze>: HTTP status code is not handled or not allowed 2026-01-25 05:55:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-32w-x-33l-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:55:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ml2930-mc50-static-grounding-reel-manual-rewind-50-cable-le-21c-alligator-clip>: HTTP status code is not handled or not allowed 2026-01-25 05:55:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 05:55:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/55-0358-7-pc-interchangeable-esd-safe-metric-hex-driver-set-6-blades-1-handle>: HTTP status code is not handled or not allowed 2026-01-25 05:55:59 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:55:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lewiston-mailbox-post-fluted-base-ball-finial-support-brace-no-address-plates-white>: HTTP status code is not handled or not allowed 2026-01-25 05:56:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cardboard-single-wall-corrugated-box-32-x-32-x-12-200lb-test-ect-32-kraft-10-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:56:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-regular-slotted-single-wall-corrugated-box-13-x-13-x-13-275-lb-ect-44-25-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:56:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manchester-locking-security-option-with-decorative-scroll-door-manchester-faceplate-in-black>: HTTP status code is not handled or not allowed 2026-01-25 05:56:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mailbox-post-ornate-base-black-bayview-solar-lamp-w-3-address-plates-support-brace-black>: HTTP status code is not handled or not allowed 2026-01-25 05:56:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lewiston-e1-economy-mailbox-post-fluted-base-black-solar-lamp-black>: HTTP status code is not handled or not allowed 2026-01-25 05:56:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electric-powered-lift-drive-floor-crane-2500-lb-capacity-987777>: HTTP status code is not handled or not allowed 2026-01-25 05:56:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-literature-mailers-10-x-8-x-3-200lb-test-ect-32-b-50-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:56:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torque-multiplier-3200-ft-lbs-19-19-32-inch-long-w-nist-cert>: HTTP status code is not handled or not allowed 2026-01-25 05:56:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-42w-x-36l-black>: HTTP status code is not handled or not allowed 2026-01-25 05:56:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cardboard-single-wall-corrugated-box-28-x-18-x-6-200lb-test-ect-32-kraft-20-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:56:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 05:56:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 72289 pages (at 89 pages/min), scraped 35431 items (at 19 items/min) 2026-01-25 05:56:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/la-24-4f-70mm-stack-light-flashing-120v-acdc-led-yellow>: HTTP status code is not handled or not allowed 2026-01-25 05:56:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-50w-x-36l-green>: HTTP status code is not handled or not allowed 2026-01-25 05:56:10 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:56:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-inch-drive-deep-socket-8-mm-6-point>: HTTP status code is not handled or not allowed 2026-01-25 05:56:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-1-2-inch-drive-impact-socket-5-3-8-inch-6-point-7-3-4-inch-l>: HTTP status code is not handled or not allowed 2026-01-25 05:56:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/boost-sport-400-amp-lithium-jump-starter-gb20>: HTTP status code is not handled or not allowed 2026-01-25 05:56:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:56:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/j2416-oh-needle-bearing-drawn-cup-caged-oil-hole-bore-381mm>: HTTP status code is not handled or not allowed 2026-01-25 05:56:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-52w-x-30l-tan>: HTTP status code is not handled or not allowed 2026-01-25 05:56:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manchester-stucco-locking-column-mailbox-in-burnt-tuscan-w-decorative-scroll-door-in-bronze>: HTTP status code is not handled or not allowed 2026-01-25 05:56:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hk1612-needle-bearing-drawn-cup-caged-metric-bore-11989mm>: HTTP status code is not handled or not allowed 2026-01-25 05:56:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-48w-x-29l-green>: HTTP status code is not handled or not allowed 2026-01-25 05:56:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nta4860-needle-bearing-thrust-roller-assembly-bore-762mm>: HTTP status code is not handled or not allowed 2026-01-25 05:56:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-import-36-pc-letter-figure-steel-stamp-set>: HTTP status code is not handled or not allowed 2026-01-25 05:56:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/332-young-bros-general-purpose-36-pc-letter-figure-steel-stamp-set>: HTTP status code is not handled or not allowed 2026-01-25 05:56:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-48w-x-34l-green>: HTTP status code is not handled or not allowed 2026-01-25 05:56:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bright-hues-feather-assortment-325-pieces-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:56:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-camel-hair-paint-brushes-no-10-aluminum-ferrule-wood-handle-12-dozen>: HTTP status code is not handled or not allowed 2026-01-25 05:56:21 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 05:56:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nta3244-needle-bearing-thrust-roller-assembly-bore-508mm>: HTTP status code is not handled or not allowed 2026-01-25 05:56:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hk0908-needle-bearing-drawn-cup-caged-metric-bore-8992mm>: HTTP status code is not handled or not allowed 2026-01-25 05:56:22 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 05:56:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-combination-lock-cash-box-spr15508-115w-x-78h-x-33h-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:56:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b2028-oh-needle-bearing-drawn-cup-full-complement-oil-hole-bore-3175mm>: HTTP status code is not handled or not allowed 2026-01-25 05:56:24 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:56:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cem31112-75hp-1800rpm-56c-frame-3ph-208-230-460v-odp-c-face-rigid-premium-efficiency>: HTTP status code is not handled or not allowed 2026-01-25 05:56:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-33w-x-30l-black>: HTTP status code is not handled or not allowed 2026-01-25 05:56:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-52w-x-30l-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:56:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/em31154-15hp-1800rpm-56-frame-3ph-230-460v-odp-rigid-premium-efficiency>: HTTP status code is not handled or not allowed 2026-01-25 05:56:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vel11319-15hp-56c-frame-1800rpm-115-230v-odp-c-face-footless-premium-efficiency>: HTTP status code is not handled or not allowed 2026-01-25 05:56:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-54w-x-31l-black>: HTTP status code is not handled or not allowed 2026-01-25 05:56:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dixon-prang-crayons-master-pack-assorted-colors-800-box>: HTTP status code is not handled or not allowed 2026-01-25 05:56:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-60w-x-31l-navy>: HTTP status code is not handled or not allowed 2026-01-25 05:56:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-28w-x-30l-black>: HTTP status code is not handled or not allowed 2026-01-25 05:56:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tall-user-kit-for-single-display>: HTTP status code is not handled or not allowed 2026-01-25 05:56:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tb-1001-motor-terminal-conduit-box>: HTTP status code is not handled or not allowed 2026-01-25 05:56:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/miller-aircore-harness-quick-connect-buckle-green-ac-qcugn-436836>: HTTP status code is not handled or not allowed 2026-01-25 05:56:33 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:56:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 05:56:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hanging-feeder-hardware-kit-hg-kit>: HTTP status code is not handled or not allowed 2026-01-25 05:56:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cejn-pur-spiral-hose-kit-19-958-6060ms1-14-mnpt-20l-14-nipple-602291>: HTTP status code is not handled or not allowed 2026-01-25 05:56:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lifetime-heavy-duty-table-cart>: HTTP status code is not handled or not allowed 2026-01-25 05:56:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stock-tank-w-75b-auto-w-ser-75-gal>: HTTP status code is not handled or not allowed 2026-01-25 05:56:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/relaxed-fit-jeans-40w-x-30l-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:56:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-12-gauge-pegboard-shelf-cabinet-48w-x-24d-x-78h>: HTTP status code is not handled or not allowed 2026-01-25 05:56:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-28w-x-unhemmed-black>: HTTP status code is not handled or not allowed 2026-01-25 05:56:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-ghs401ap-oxidizer-ghs-label-vinyl-4-x-4-5-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:56:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-low-rise-fit-work-pants-52w-x-31l-brown>: HTTP status code is not handled or not allowed 2026-01-25 05:56:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ctl-bandsaw-blade-8-foot-3-3-4-inch-long-x-3-4-inch-wide-4-6-tpi-x-0035-thick>: HTTP status code is not handled or not allowed 2026-01-25 05:56:41 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:56:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-42w-x-34l-black>: HTTP status code is not handled or not allowed 2026-01-25 05:56:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-ghs2004alv2-flammable-ghs-label-vinyl-4-x-4-250-roll>: HTTP status code is not handled or not allowed 2026-01-25 05:56:43 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls landed on page that is not a product page. 2026-01-25 05:56:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-ghs202ap-explosive-ghs-label-vinyl-2-x-2-5-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:56:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-work-coveralls-46-short-navy>: HTTP status code is not handled or not allowed 2026-01-25 05:56:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-ghs2072alv2-dangerous-for-environment-ghs-label-vinyl-2-x-2-250-roll>: HTTP status code is not handled or not allowed 2026-01-25 05:56:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-ghs2082al2-health-hazard-ghs-label-paper-2-x-2-250-roll>: HTTP status code is not handled or not allowed 2026-01-25 05:56:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ctl-bandsaw-blade-12-foot-6-inch-long-x-1-inch-wide-6-10-tpi-x-0035-thick>: HTTP status code is not handled or not allowed 2026-01-25 05:56:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-30w-x-34l-dark-navy>: HTTP status code is not handled or not allowed 2026-01-25 05:56:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15383-3-pack-paring-knives-high-carbon-steel-stamped-white-handle-3-1-4->: HTTP status code is not handled or not allowed 2026-01-25 05:56:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/made-in-usa-logistic-e-strap-2x16-w-ratchet-for-e-track>: HTTP status code is not handled or not allowed 2026-01-25 05:56:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-westex-35w-x-30l-black>: HTTP status code is not handled or not allowed 2026-01-25 05:56:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-work-pants-reflective-material-flame-resistant-42w-x-34l-red>: HTTP status code is not handled or not allowed 2026-01-25 05:56:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 05:56:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/17160-bakers-spatula-high-carbon-steel-8-l>: HTTP status code is not handled or not allowed 2026-01-25 05:56:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/31602-wide-cook-s-knife-high-carbon-steel-stamped-white-handle-10-l>: HTTP status code is not handled or not allowed 2026-01-25 05:56:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-work-pants-reflective-material-flame-resistant-44w-x-32l-red>: HTTP status code is not handled or not allowed 2026-01-25 05:56:54 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:56:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-44w-x-unhemmed-navy>: HTTP status code is not handled or not allowed 2026-01-25 05:56:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/19770-hamburger-turner-high-carbon-steel-4-l-x-3-w>: HTTP status code is not handled or not allowed 2026-01-25 05:56:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/competitor-ventilated-three-tier-locker-adder-1-wide-12w-x-12d-x-24h-unassembled-black>: HTTP status code is not handled or not allowed 2026-01-25 05:56:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-double-tier-locker-3-wide-12w-x-15d-x-30h-assembled-black>: HTTP status code is not handled or not allowed 2026-01-25 05:56:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/r4i-145-2003-rotary-screw-air-compressor-3-phase-200-volts-5hp-120-gal>: HTTP status code is not handled or not allowed 2026-01-25 05:56:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-38w-x-34l-tan>: HTTP status code is not handled or not allowed 2026-01-25 05:56:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-double-tier-locker-starter-1-wide-12w-x-15d-x-30h-unassembled-black>: HTTP status code is not handled or not allowed 2026-01-25 05:56:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-work-coveralls-42-regular-green>: HTTP status code is not handled or not allowed 2026-01-25 05:57:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-pocket-cargo-pants-heavy-duty-twill-46w-x-32l-dark-navy>: HTTP status code is not handled or not allowed 2026-01-25 05:57:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-visibility-ripstop-cargo-pants-30w-x-36l-navy>: HTTP status code is not handled or not allowed 2026-01-25 05:57:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-visibility-ripstop-cargo-pants-40w-x-36l-navy>: HTTP status code is not handled or not allowed 2026-01-25 05:57:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-westex-56w-x-32l-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:57:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shd-102-spring-hangers-3-l-x-2-1-4-w-x-5-1-2-h>: HTTP status code is not handled or not allowed 2026-01-25 05:57:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a120k-v-belt-kevlar-1-2-x-122>: HTTP status code is not handled or not allowed 2026-01-25 05:57:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 05:57:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a134k-v-belt-kevlar-1-2-x-136>: HTTP status code is not handled or not allowed 2026-01-25 05:57:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3l380k-v-belt-kevlar-3-8-x-38>: HTTP status code is not handled or not allowed 2026-01-25 05:57:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-westex-31w-x-32l-navy>: HTTP status code is not handled or not allowed 2026-01-25 05:57:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a70k-v-belt-kevlar-1-2-x-72>: HTTP status code is not handled or not allowed 2026-01-25 05:57:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kerf-weatherstrip-70068-white-96-inch>: HTTP status code is not handled or not allowed 2026-01-25 05:57:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 72381 pages (at 92 pages/min), scraped 35440 items (at 9 items/min) 2026-01-25 05:57:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lf-4060-low-profile-compression-mount-double-deflection-22-lbs-max-load>: HTTP status code is not handled or not allowed 2026-01-25 05:57:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/en-14-hd-ss-forged-eye-nut-1-1-2-6-style-b-304-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 05:57:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/en-3-ss-drop-forged-eye-nut-3-8-16-style-a-304-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 05:57:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-visibility-ripstop-cargo-pants-28w-x-30l-navy>: HTTP status code is not handled or not allowed 2026-01-25 05:57:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-sheet-cloverleaf-57042-36-inchl-x-24-inchw-x-02-inchh-silver>: HTTP status code is not handled or not allowed 2026-01-25 05:57:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ppe-kit-ppe-spill-kit-16-piece-kit>: HTTP status code is not handled or not allowed 2026-01-25 05:57:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-westex-44w-x-30l-black>: HTTP status code is not handled or not allowed 2026-01-25 05:57:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stick-n-step-46602-4-inchd-x-16-inchw-grey>: HTTP status code is not handled or not allowed 2026-01-25 05:57:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zinsser-bulls-eye-1-2-3-water-base-primer-white-gallon-can-2001>: HTTP status code is not handled or not allowed 2026-01-25 05:57:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zinsser-odorless-oil-base-stain-blocker-gallon-can-3951>: HTTP status code is not handled or not allowed 2026-01-25 05:57:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-fit-work-pants-westex-46w-x-34l-black>: HTTP status code is not handled or not allowed 2026-01-25 05:57:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wood-grill-brush>: HTTP status code is not handled or not allowed 2026-01-25 05:57:17 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:57:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cargo-pants-with-double-reinforced-knees-flame-resistant-36w-x-30l-black>: HTTP status code is not handled or not allowed 2026-01-25 05:57:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-wire-shelving-add-on-54-inch-w-x-14-inch-d-x-74-inch-h>: HTTP status code is not handled or not allowed 2026-01-25 05:57:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/assy-es-s4242-ramp-trim-kit-fits-42-x-42-mat>: HTTP status code is not handled or not allowed 2026-01-25 05:57:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cesm-21cb-3648-safety-switch-mat-36-x-48-black>: HTTP status code is not handled or not allowed 2026-01-25 05:57:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regular-jeans-indura-denim-14-oz-46w-x-34l-flame-resistant-blue>: HTTP status code is not handled or not allowed 2026-01-25 05:57:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/indoor-portable-evaporative-air-cooler-cso71ae-15-pint>: HTTP status code is not handled or not allowed 2026-01-25 05:57:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-decor-satin-enamel-12-oz-aerosol-can-white-792195>: HTTP status code is not handled or not allowed 2026-01-25 05:57:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/utility-jeans-basket-weave-flame-resistant-8-oz-40w-x-34l-navy>: HTTP status code is not handled or not allowed 2026-01-25 05:57:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-24w-x-24d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 05:57:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/x-o-rust-oil-base-dtm-enamel-gloss-finish-bright-red-quart-371989>: HTTP status code is not handled or not allowed 2026-01-25 05:57:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/safety/protective_clothing_workwear/pants_shorts_overalls>: HTTP status code is not handled or not allowed 2026-01-25 05:57:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-work-pants-reflective-material-flame-resistant-50w-x-unhemmed-navy>: HTTP status code is not handled or not allowed 2026-01-25 05:57:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-work-pants-reflective-material-flame-resistant-56w-x-unhemmed-navy>: HTTP status code is not handled or not allowed 2026-01-25 05:57:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/color-decor-decorative-enamel-spray-10-oz-aerosol-can-white-satin-115397>: HTTP status code is not handled or not allowed 2026-01-25 05:57:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-9a-3ph-remote-start-terminals-start-stop-250-500v-3-1-4-2a>: HTTP status code is not handled or not allowed 2026-01-25 05:57:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-30w-x-14d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 05:57:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-9a-3ph-remote-start-terminals-start-stop-100-250v-3-1-4-2a>: HTTP status code is not handled or not allowed 2026-01-25 05:57:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/x-o-rust-anti-rust-enamel-satin-finish-satin-white-gallon-176838>: HTTP status code is not handled or not allowed 2026-01-25 05:57:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-48w-x-14d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 05:57:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-42w-x-14d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 05:57:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/x-o-rust-oil-base-dtm-enamel-gloss-finish-blue-quart-372052>: HTTP status code is not handled or not allowed 2026-01-25 05:57:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-epoxy-wire-shelving-add-on-48w-x-18d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 05:57:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/x-o-rust-oil-base-dtm-enamel-gloss-finish-pebble-gallon-802660>: HTTP status code is not handled or not allowed 2026-01-25 05:57:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-9a-1ph-remote-start-terminals-reset-button-100-250v-4-2-5-7a>: HTTP status code is not handled or not allowed 2026-01-25 05:57:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-9a-1ph-remote-start-start-stop-100-250v-3-1-4-2a>: HTTP status code is not handled or not allowed 2026-01-25 05:57:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-12w-x-36d-x-75h-with-28-8h-plastic-shelf-bins-red>: HTTP status code is not handled or not allowed 2026-01-25 05:57:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-18w-x-36d-x-75h-with-21-8h-plastic-shelf-bins-yellow>: HTTP status code is not handled or not allowed 2026-01-25 05:57:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-12a-3ph-direct-online-start-stop-100-250v-7-6-10a>: HTTP status code is not handled or not allowed 2026-01-25 05:57:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/itemizer-drawer-unit-lateral-49-in-48-in-6-3-4-in-318-3>: HTTP status code is not handled or not allowed 2026-01-25 05:57:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eco-premier-carpet-tile-22177614000-diagonal-18l-x-18w-x-14h-khaki-12pk>: HTTP status code is not handled or not allowed 2026-01-25 05:57:45 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_cabinets_drawers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:57:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-12a-3ph-direct-online-reset-button-250-500v-7-6-10a>: HTTP status code is not handled or not allowed 2026-01-25 05:57:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wesco-174-plastic-box-truck-12-bushel-red-272547-8-casters>: HTTP status code is not handled or not allowed 2026-01-25 05:57:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/door-organizer-tool-trays-21-3-4-in-20-1-4-in-9872-3-01>: HTTP status code is not handled or not allowed 2026-01-25 05:57:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/charger-115v>: HTTP status code is not handled or not allowed 2026-01-25 05:57:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-fastener-hfdf150-stalguardcoated-steel-anticorrosive-ss-screw-150pk>: HTTP status code is not handled or not allowed 2026-01-25 05:57:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-12a-1ph-remote-start-terminals-reset-button-100-250v-2-3-3-1a>: HTTP status code is not handled or not allowed 2026-01-25 05:57:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-16a-3ph-direct-online-start-stop-250-500v-7-6-10a>: HTTP status code is not handled or not allowed 2026-01-25 05:57:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-round-table-ladder-back-bar-ht-set-nepal-teak-laminate-table-black-vinyl-chair>: HTTP status code is not handled or not allowed 2026-01-25 05:57:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-16a-3ph-separate-coil-voltage-reset-button-250-500v-5-7-7-6a>: HTTP status code is not handled or not allowed 2026-01-25 05:57:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-16a-3ph-remote-start-reset-button-24-60v-4-2-5-7a>: HTTP status code is not handled or not allowed 2026-01-25 05:57:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/vehicle-fleet-maintenance/towing_cargo_management/van_storage_equipment/van_cabinets_drawers>: HTTP status code is not handled or not allowed 2026-01-25 05:57:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-16a-3ph-separate-coil-voltage-reset-button-250-500v-10-13a>: HTTP status code is not handled or not allowed 2026-01-25 05:57:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-16a-3ph-reset-button-24-60v-16-20a>: HTTP status code is not handled or not allowed 2026-01-25 05:57:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deep-groove-ball-bearings-metric-6303-2rs-2-rubber-seals-heavy-duty-17mm-bore-47mm-od>: HTTP status code is not handled or not allowed 2026-01-25 05:57:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-16a-1ph-remote-start-terminals-start-stop-100-250v-4-2-5-7a>: HTTP status code is not handled or not allowed 2026-01-25 05:57:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-16a-1ph-remote-start-start-stop-24-60v-7-6-10a>: HTTP status code is not handled or not allowed 2026-01-25 05:58:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elevating-legrests-for-bariatric-sentra-wheelchairs-1-pair-stdelr-tf>: HTTP status code is not handled or not allowed 2026-01-25 05:58:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deep-groove-ball-bearings-metric-6212-open-medium-duty-60mm-bore-110mm-od>: HTTP status code is not handled or not allowed 2026-01-25 05:58:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deep-groove-ball-bearings-metric-6208-2rs-2-rubber-seals-medium-duty-40mm-bore-80mm-od>: HTTP status code is not handled or not allowed 2026-01-25 05:58:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-26a-3ph-remote-start-start-stop-24-60v-24-29a>: HTTP status code is not handled or not allowed 2026-01-25 05:58:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deep-groove-ball-bearings-metric-6007-open-light-duty-35mm-bore-62mm-od>: HTTP status code is not handled or not allowed 2026-01-25 05:58:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-26a-3ph-remote-start-terminals-start-stop-100-250v-13-16a>: HTTP status code is not handled or not allowed 2026-01-25 05:58:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deep-groove-ball-bearings-metric-6312-zz-2-metal-shields-heavy-duty-60mm-bore-130mm-od>: HTTP status code is not handled or not allowed 2026-01-25 05:58:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deep-groove-ball-bearings-metric-6300-2rs-2-rubber-seals-heavy-duty-10mm-bore-35mm-od>: HTTP status code is not handled or not allowed 2026-01-25 05:58:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/847084-hex-cap-screw-5-16-18-x-1-3-4-carbon-steel-zinc-grade-5-pt-unc-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 05:58:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/754048-split-lock-washer-1-4-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 05:58:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rhino-12-red-vinyl-tape-white-print>: HTTP status code is not handled or not allowed 2026-01-25 05:58:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/858160-external-tooth-lock-nut-10-32-steel-zinc-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 05:58:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 72478 pages (at 97 pages/min), scraped 35468 items (at 28 items/min) 2026-01-25 05:58:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/898-wood-slanted-base-gun-storage-cabinet-12-long-guns>: HTTP status code is not handled or not allowed 2026-01-25 05:58:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-26a-1ph-remote-start-reset-button-24-60v-20-24a>: HTTP status code is not handled or not allowed 2026-01-25 05:58:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/349002-split-lock-washer-6-steel-zinc-pkg-of-2500>: HTTP status code is not handled or not allowed 2026-01-25 05:58:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/391020-flat-washer-10-18-8-stainless-steel-pkg-of-500>: HTTP status code is not handled or not allowed 2026-01-25 05:58:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-38a-3ph-remote-start-start-stop-24-60v-29-35a>: HTTP status code is not handled or not allowed 2026-01-25 05:58:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/863011-wing-nut-cold-forged-8-32-low-carbon-steel-zinc-coarse-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 05:58:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/d383p-osha-sign-danger-confined-space-do-not-enter-7-x-10-whiteredblack>: HTTP status code is not handled or not allowed 2026-01-25 05:58:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n155rb-osha-sign-notice-smoking-is-prohibited-except-in-designated-areas-10-x-14-whblbk>: HTTP status code is not handled or not allowed 2026-01-25 05:58:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dl2ap-dot-shipping-labels-flammable-gas-2-4-x-4-whitered-25-per-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:58:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precision-compression-spring-03-od-x-0045-x-0812-stainless-steel-pkg-of-5>: HTTP status code is not handled or not allowed 2026-01-25 05:58:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-52a-3ph-direct-online-start-stop-250-500v-30-40a>: HTTP status code is not handled or not allowed 2026-01-25 05:58:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-52a-1ph-separate-coil-voltage-start-stop-100-250v-30-40a>: HTTP status code is not handled or not allowed 2026-01-25 05:58:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/splotag11-25-tags-do-not-operate-equipment-lock-out-bilingual-6-x-3-whiteredblack-25pk>: HTTP status code is not handled or not allowed 2026-01-25 05:58:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precision-compression-spring-018-od-x-0022-x-0812-stainless-steel-pkg-of-5>: HTTP status code is not handled or not allowed 2026-01-25 05:58:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dcn21-nfpa-label-number-1-2-12-x-2-12-whiteblack-5pk>: HTTP status code is not handled or not allowed 2026-01-25 05:58:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m249r-sign-first-aid-7-x-10-whitegreen>: HTTP status code is not handled or not allowed 2026-01-25 05:58:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n324rb-osha-sign-notice-not-an-exit-10-x-14-whiteblueblack>: HTTP status code is not handled or not allowed 2026-01-25 05:58:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m750pb-hazardous-materials-sign-used-oil-10-x-14-whiteblack>: HTTP status code is not handled or not allowed 2026-01-25 05:58:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compression-spring-025-od-x-00286-x-2-486-rate-pkg-of-12>: HTTP status code is not handled or not allowed 2026-01-25 05:58:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fx126fr-fire-sign-fire-extinguisher-double-sided-12-x-4-whitered>: HTTP status code is not handled or not allowed 2026-01-25 05:58:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hms15p-hazardous-materials-systems-label-15-12-x-15-12-redyellowwhiteblue-5pk>: HTTP status code is not handled or not allowed 2026-01-25 05:58:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m28ap-fire-sign-fire-extinguisher-inside-3-x-5-whitered>: HTTP status code is not handled or not allowed 2026-01-25 05:58:27 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_seating/patio_dining_chairs already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:58:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clamp-mount-outlet-usb-charger>: HTTP status code is not handled or not allowed 2026-01-25 05:58:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c26p-osha-sign-caution-eye-protection-required-in-this-area-7-x-10-yellowblack>: HTTP status code is not handled or not allowed 2026-01-25 05:58:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vsm-abrasive-belt-100257-aluminum-oxide-3-8-x-13-50-grit>: HTTP status code is not handled or not allowed 2026-01-25 05:58:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vsm-abrasive-belt-100445-zirconia-alumina-37-x-60-60-grit>: HTTP status code is not handled or not allowed 2026-01-25 05:58:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-80a-3ph-direct-online-reset-button-100-250v-57-68a>: HTTP status code is not handled or not allowed 2026-01-25 05:58:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-80a-3ph-remote-start-terminals-start-stop-250-500v-48-60a>: HTTP status code is not handled or not allowed 2026-01-25 05:58:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nema-4x-enclosed-motor-starter-80a-3ph-direct-online-reset-button-100-250v-75-87a>: HTTP status code is not handled or not allowed 2026-01-25 05:58:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vsm-abrasive-belt-127060-zirconia-alumina-4-x-24-24-grit>: HTTP status code is not handled or not allowed 2026-01-25 05:58:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vsm-abrasive-belt-18589-silicon-carbide-6-x-48-80-grit>: HTTP status code is not handled or not allowed 2026-01-25 05:58:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vsm-abrasive-belt-22701-aluminum-oxide-52-x-75-150-grit>: HTTP status code is not handled or not allowed 2026-01-25 05:58:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_seating/patio_dining_chairs>: HTTP status code is not handled or not allowed 2026-01-25 05:58:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vsm-abrasive-belt-286225-ceramic-2-x-72-80-grit>: HTTP status code is not handled or not allowed 2026-01-25 05:58:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cl20r-rechargeable-clamp-light-1750-lumens-60m-beam>: HTTP status code is not handled or not allowed 2026-01-25 05:58:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vsm-abrasive-belt-31870-silicon-carbide-6-x-48-180-grit>: HTTP status code is not handled or not allowed 2026-01-25 05:58:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vsm-abrasive-belt-37593-aluminum-oxide-25-x-60-100-grit>: HTTP status code is not handled or not allowed 2026-01-25 05:58:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lab-clean-n10-ultrasonic-cleaning-solution-for-the-laboratory-7-1-ph-10-l>: HTTP status code is not handled or not allowed 2026-01-25 05:58:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/k-flex-for-right-ear-medium>: HTTP status code is not handled or not allowed 2026-01-25 05:58:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vsm-abrasive-belt-94116-aluminum-oxide-3-x-18-100-grit>: HTTP status code is not handled or not allowed 2026-01-25 05:58:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/titan-extreme-high-noise-headset-black>: HTTP status code is not handled or not allowed 2026-01-25 05:58:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-polypropylene-tagging-fasteners-801-standard-brown-1l-5000-pack>: HTTP status code is not handled or not allowed 2026-01-25 05:58:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-2352-bn-single-lever-shower-trim-kit-only-pvd-brushed-nickel>: HTTP status code is not handled or not allowed 2026-01-25 05:58:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hfc-72-72-damp-ceiling-fan-59136-matte-black>: HTTP status code is not handled or not allowed 2026-01-25 05:58:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pioneer-americana-2am401-two-handle-kitchen-faucet-with-spray-polished-chrome>: HTTP status code is not handled or not allowed 2026-01-25 05:58:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/professional-uhf-dual-wireless-microphone-system-wm1302>: HTTP status code is not handled or not allowed 2026-01-25 05:58:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/p-4420-bn-handheld-shower-set-pvd-brushed-nickel>: HTTP status code is not handled or not allowed 2026-01-25 05:58:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/75-osha-stairway-stair36o-9-75>: HTTP status code is not handled or not allowed 2026-01-25 05:58:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/43-osha-stairway-stair36o-5-43>: HTTP status code is not handled or not allowed 2026-01-25 05:58:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fao-first-aid-kit-50-person-ansi-compliant-class-b-plastic-case>: HTTP status code is not handled or not allowed 2026-01-25 05:58:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/47-osha-stairway-stair36o-5-47>: HTTP status code is not handled or not allowed 2026-01-25 05:58:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/141-ibc-stairway-stair36i-21-141>: HTTP status code is not handled or not allowed 2026-01-25 05:58:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/86-ibc-stairway-stair36i-13-86>: HTTP status code is not handled or not allowed 2026-01-25 05:58:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/143-ibc-stairway-stair36i-21-143>: HTTP status code is not handled or not allowed 2026-01-25 05:58:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-24-inch-w-x-24-inch-d-x-74-inch-h-2000-lbs-cap-4-open-and-1-solid-grid-shelves>: HTTP status code is not handled or not allowed 2026-01-25 05:58:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/101-osha-stairway-stair36o-12-101>: HTTP status code is not handled or not allowed 2026-01-25 05:58:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/111-osha-stairway-stair36o-13-111>: HTTP status code is not handled or not allowed 2026-01-25 05:58:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-42-inch-w-x-21-inch-d-x-86-inch-h-2000-lbs-cap-4-open-and-1-solid-grid-shelves>: HTTP status code is not handled or not allowed 2026-01-25 05:58:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keyless-rigid-coupling-c600e112-steel-1125d-x-2598d-118l-shaft>: HTTP status code is not handled or not allowed 2026-01-25 05:58:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keyless-rigid-coupling-c600e162-steel-1625d-x-378d-158l-shaft>: HTTP status code is not handled or not allowed 2026-01-25 05:58:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/95-ibc-stairway-stair36i-14-95>: HTTP status code is not handled or not allowed 2026-01-25 05:58:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/113-ibc-stairway-stair36i-17-113>: HTTP status code is not handled or not allowed 2026-01-25 05:58:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reducer-bushing-srb081020-galvanized-steel-12id-x-58od-114l>: HTTP status code is not handled or not allowed 2026-01-25 05:58:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-60-inch-w-x-21-inch-d-x-74-inch-h-2000-lbs-cap-4-open-1-solid-shelf-dunnage-stand>: HTTP status code is not handled or not allowed 2026-01-25 05:58:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reducer-bushing-srb050617-galvanized-steel-516id-x-38od-1116l>: HTTP status code is not handled or not allowed 2026-01-25 05:58:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-60-inch-w-x-24-inch-d-x-86-inch-h-2000-lbs-cap-3-open-1-solid-shelf-dunnage-stand>: HTTP status code is not handled or not allowed 2026-01-25 05:59:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pillow-block-ball-bearing-pbdcbl100-rigid-die-cast-zinc-housing-1-bore>: HTTP status code is not handled or not allowed 2026-01-25 05:59:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pillow-block-ball-bearing-pbphbl100-heavy-gauge-pressed-steel-housing-1-bore>: HTTP status code is not handled or not allowed 2026-01-25 05:59:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pillow-block-ball-bearing-pbphbl050-heavy-gauge-pressed-steel-housing-12-bore>: HTTP status code is not handled or not allowed 2026-01-25 05:59:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/45mm-locking-assembly-c405-series-c405m45x75-metric-m8-x-45>: HTTP status code is not handled or not allowed 2026-01-25 05:59:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-30-inch-w-x-21-inch-d-x-86-inch-h-2000-lbs-cap-5-open-grid-shelves>: HTTP status code is not handled or not allowed 2026-01-25 05:59:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-sided-self-supporting-tank-ls-3500-204-dia-x-33h-3500-gallon-cap-red>: HTTP status code is not handled or not allowed 2026-01-25 05:59:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/offset-handle-walking-cane-rtl10306-aluminum-foam-grip-black>: HTTP status code is not handled or not allowed 2026-01-25 05:59:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-aire-wall-decontamination-pool-aadp-33-36l-x-12w-x-29h-60-gallon-cap-yellow>: HTTP status code is not handled or not allowed 2026-01-25 05:59:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-aire-wall-decontamination-pool-aadp-33-36l-x-12w-x-29h-60-gallon-cap-orange>: HTTP status code is not handled or not allowed 2026-01-25 05:59:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-36-inch-w-x-18-inch-d-1000-lbs-cap-removable-solid-shelf-mat-polymer-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:59:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 72577 pages (at 99 pages/min), scraped 35490 items (at 22 items/min) 2026-01-25 05:59:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quick-adjust-wire-shelving-36-x-18-x-54-poly-z-brite>: HTTP status code is not handled or not allowed 2026-01-25 05:59:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-unit-42-inch-w-x-18-inch-d-x-74-inch-h-2000-lbs-cap-4-open-grid-shelves>: HTTP status code is not handled or not allowed 2026-01-25 05:59:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dpg83-91d-ansi-z87-1-converter-i-o-lens-a-f-polybag>: HTTP status code is not handled or not allowed 2026-01-25 05:59:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sz-300-medium-table-top-ultrasonic-cleaner-3-5-gallons>: HTTP status code is not handled or not allowed 2026-01-25 05:59:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-42-inch-w-x-24-inch-d-1000-lbs-cap-removable-solid-shelf-mat-polymer-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:59:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-mobile-work-center-flush-shelf-use-with-if-2436-5pytl>: HTTP status code is not handled or not allowed 2026-01-25 05:59:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electric-adjustable-height-workbench-laminate-safety-edge-48-in-w-x-30-in-d-gray>: HTTP status code is not handled or not allowed 2026-01-25 05:59:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/twin-window-fan-2-speed-white>: HTTP status code is not handled or not allowed 2026-01-25 05:59:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-modular-frl-wend-ports-c33454-gauge-poly-bowl-manual-ultra-fog-lubricator>: HTTP status code is not handled or not allowed 2026-01-25 05:59:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medina-63w-right-curved-desk-return-with-pencil-box-file-pedestal-gray-steel>: HTTP status code is not handled or not allowed 2026-01-25 05:59:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-modular-frl-wend-ports-c33352w-gauge-metal-bowl-wsight-manual-fog-lubricator>: HTTP status code is not handled or not allowed 2026-01-25 05:59:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/octagonal-sledge-hammer-1435g-4lb-head-10-hickory-handle>: HTTP status code is not handled or not allowed 2026-01-25 05:59:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48x24x79-metro-q-cart>: HTTP status code is not handled or not allowed 2026-01-25 05:59:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sensi-wrap-self-adherent-bandage-rolls-wo-rubber-latex-6w-x-5-yards-12-pcs>: HTTP status code is not handled or not allowed 2026-01-25 05:59:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/povidone-iodine-swabsticks-250-pcs>: HTTP status code is not handled or not allowed 2026-01-25 05:59:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torx-tamper-resistant-insert-bit-23026-1-4-drive-t25-tip>: HTTP status code is not handled or not allowed 2026-01-25 05:59:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-grade-phillips-power-bit-12004-2-1-4l-x-1-4-number2-tip-1-4-hexagon-drive>: HTTP status code is not handled or not allowed 2026-01-25 05:59:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/red-handled-screwdriver-9643r-slotted-tip-14-5-16l-10-x-5-16-round-shank>: HTTP status code is not handled or not allowed 2026-01-25 05:59:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ball-pein-hammer-1308p-12-3-8l-8-oz>: HTTP status code is not handled or not allowed 2026-01-25 05:59:26 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/air_compressors/single_stage_reciprocating_air_compressors already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 05:59:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forged-bolt-cutter-cop36-36-long>: HTTP status code is not handled or not allowed 2026-01-25 05:59:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medina-series-36-inch-storage-cabinet-with-wood-doors-textured-sea-salt>: HTTP status code is not handled or not allowed 2026-01-25 05:59:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u-sling-wo-head-support-large>: HTTP status code is not handled or not allowed 2026-01-25 05:59:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adhesive-fabric-bandages-sterile-1l-x-3w-2400-pcs>: HTTP status code is not handled or not allowed 2026-01-25 05:59:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-mobile-single-48w-x-15d-x-29-1-2h-gray-laminate-black-edge>: HTTP status code is not handled or not allowed 2026-01-25 05:59:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25-tub-mobile-cubbie-w-tubs-60w-x-15d-x-35-1-2h-gray-laminate-black-edge>: HTTP status code is not handled or not allowed 2026-01-25 05:59:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/siligentle-ag-silver-silicone-bordered-foam-dressing-6l-x-6w-50-pcs>: HTTP status code is not handled or not allowed 2026-01-25 05:59:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/combine-pads-sterile1pouch-8w-x-10l-360-pcs>: HTTP status code is not handled or not allowed 2026-01-25 05:59:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-tub-mobile-cubbie-w-clear-tubs-60w-x-15d-x-29-1-2h-birch-plywood>: HTTP status code is not handled or not allowed 2026-01-25 05:59:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tm229k-traffic-sign-roadwork-ahead-sign-30-x-30-orange>: HTTP status code is not handled or not allowed 2026-01-25 05:59:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tm197j-traffic-sign-reserved-parking-van-accessible-18-x-12-white>: HTTP status code is not handled or not allowed 2026-01-25 05:59:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/antimicrobial-gait-belt-w-metal-buckle-55-78l-x-1-1516w-multicolor-10-pcs>: HTTP status code is not handled or not allowed 2026-01-25 05:59:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unna-boot-bandages-w-calamine-3w-x-10-yards-12-pcs>: HTTP status code is not handled or not allowed 2026-01-25 05:59:36 [scrapy.core.scraper] (PID: 96) ERROR: Spider error processing (referer: None) Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/defer.py", line 346, in aiter_errback yield await it.__anext__() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 394, in __anext__ return await self.data.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/utils/python.py", line 375, in _async_chain async for o in as_async_generator(it): File "/usr/local/lib/python3.11/site-packages/scrapy/utils/asyncgen.py", line 21, in as_async_generator async for r in it: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/referer.py", line 384, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/urllength.py", line 62, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/usr/local/lib/python3.11/site-packages/scrapy/spidermiddlewares/depth.py", line 60, in process_spider_output_async async for r in result: File "/usr/local/lib/python3.11/site-packages/scrapy/core/spidermw.py", line 121, in process_async async for r in iterable: File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/spiders/__init__.py", line 160, in parse_product yield await page.to_item() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 81, in _to_item validation_item = self._validate_input() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/utils.py", line 205, in inner return cached_meth(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/web_poet/pages.py", line 140, in _validate_input validation_item = self.validate_input() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/scrapyd/eggs/catalog_extraction/1758126308.egg/catalog_extraction/pages/__init__.py", line 33, in validate_input raise NotProductPage(f"URL {self.url} landed on page that is not a product page.") scraping_utils.common.exceptions.NotProductPage: URL https://www.globalindustrial.com/c/motors_power_transmission/pneumatic_system_components/air_compressors/single_stage_reciprocating_air_compressors landed on page that is not a product page. 2026-01-25 05:59:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/combine-pads-sterile-1pouch-5w-x-9l-400-pcs>: HTTP status code is not handled or not allowed 2026-01-25 05:59:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cubbie-tote-tray-13-1-2l-x-8-5-8w-x-5-1-4h-black>: HTTP status code is not handled or not allowed 2026-01-25 05:59:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/folding-bedside-commode-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-25 05:59:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nmc-tm537j-traffic-sign-no-trucks-sign-24-x-24-white>: HTTP status code is not handled or not allowed 2026-01-25 05:59:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/offset-bariatric-cane-6-pairs>: HTTP status code is not handled or not allowed 2026-01-25 05:59:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/welded-storage-locker-w-center-shelf-mesh-sides-49w-x-27d-x-52h>: HTTP status code is not handled or not allowed 2026-01-25 05:59:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-visibility-storage-cabinet-w-2-adj-shelves-48w-x-32d-x-78h>: HTTP status code is not handled or not allowed 2026-01-25 05:59:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tool-storage-cabinet-w-pegboard-49w-x-33d-x-78h>: HTTP status code is not handled or not allowed 2026-01-25 05:59:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classic-whole-house-fan-qc-cl-3100-120v-3100cfm>: HTTP status code is not handled or not allowed 2026-01-25 05:59:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-large-capacity-refrigerator-for-pharmacies-abt-72b-72-cu-ft>: HTTP status code is not handled or not allowed 2026-01-25 05:59:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-large-capacity-refrigerator-for-pharmacies-abt-12b-12-cu-ft>: HTTP status code is not handled or not allowed 2026-01-25 05:59:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/massage-ball-9-cm-3-6-red-1-each>: HTTP status code is not handled or not allowed 2026-01-25 05:59:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jettrac-wall-mount-65l-black-s-retractable-belt-s-hook>: HTTP status code is not handled or not allowed 2026-01-25 05:59:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-drop-box-with-adjustable-chute-keyed-lock-mdl-170-12-x-6-x-16-beige>: HTTP status code is not handled or not allowed 2026-01-25 05:59:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sl-led-wall-sconce-w-rain-glass-panel-13-3-4-inch-tall-6-1-2-inch-wide-12w-black-finish-warm-white>: HTTP status code is not handled or not allowed 2026-01-25 05:59:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/amscope-powerful-led-dual-goose-neck-illuminator-6w>: HTTP status code is not handled or not allowed 2026-01-25 05:59:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/amscope-inspection-dissecting-zoom-power-stereo-microscope-7x-45x-magnification-w-64-led-light>: HTTP status code is not handled or not allowed 2026-01-25 05:59:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/endura-goatskin-glove-w-blended-aramid-liner-5-inch-split-cuff-ansi-a4-m>: HTTP status code is not handled or not allowed 2026-01-25 05:59:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eveleen-outdoor-30-square-bistro-table-black-synthetic-polymer-top-black-aluminum-frame>: HTTP status code is not handled or not allowed 2026-01-25 05:59:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/amscope-usb-microscope-camera-5mp-w-measurement-so-ware>: HTTP status code is not handled or not allowed 2026-01-25 05:59:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15in-replacement-roller-galvanized-steel-pack-of-3>: HTTP status code is not handled or not allowed 2026-01-25 05:59:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anatomical-chart-skull-sticky-back>: HTTP status code is not handled or not allowed 2026-01-25 05:59:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mc-spartan-4-way-smart-manifold-with-60-inch-high-pressure-hoses-standard-fittings>: HTTP status code is not handled or not allowed 2026-01-25 05:59:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mc-midi-tube-cutter-for-3-8-inch-to-1-1-8-inch-o-d-tubing>: HTTP status code is not handled or not allowed 2026-01-25 05:59:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mc-intellasense-iii-combustible-gas-leak-detector>: HTTP status code is not handled or not allowed 2026-01-25 05:59:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ktools-tradesman-pro-modular-piping-tool-pouch-with-belt-clip-black>: HTTP status code is not handled or not allowed 2026-01-25 05:59:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ktools-deluxe-tool-bag-black-canvas-13-pockets-18-inch>: HTTP status code is not handled or not allowed 2026-01-25 05:59:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/printing-calculator-cnmmp21dx-12-digit-fluorescent-screen-power-adapter-or-ac-supply-power>: HTTP status code is not handled or not allowed 2026-01-25 05:59:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/belkin-cord-concealer-blkf8b023-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:00:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ct-usa-metal-threaded-handle-dust-mop-frame-12-qty>: HTTP status code is not handled or not allowed 2026-01-25 06:00:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ct-usa-endurance-bed-1-0-81-inch-x-35-inch-blue-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:00:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usb-flash-drive-ver97088-8-gb-storage-capcity-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:00:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fl-casual-work-oxford-knit-black-12d>: HTTP status code is not handled or not allowed 2026-01-25 06:00:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/76055-angle-grinder-cup-brush-arbor-5-8-x-11-coarse-5-crimped>: HTTP status code is not handled or not allowed 2026-01-25 06:00:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inner-ring-for-shell-type-needle-roller-bearing-metric-20mm-bore-24mm-od-165mm-width>: HTTP status code is not handled or not allowed 2026-01-25 06:00:04 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/monitor_risers_stands already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 06:00:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fl-casual-work-three-quarter-smooth-leather-cognac-13d>: HTTP status code is not handled or not allowed 2026-01-25 06:00:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shell-type-needle-roller-bearing-inch-34-bore-1-od-750-width>: HTTP status code is not handled or not allowed 2026-01-25 06:00:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/75541-depressed-center-grinding-wheel-metal-grinding-4-x-1-8-arbor-5-8>: HTTP status code is not handled or not allowed 2026-01-25 06:00:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inner-ring-for-shell-type-needle-roller-bearing-inch-114-bore-112-od-2578mm-width>: HTTP status code is not handled or not allowed 2026-01-25 06:00:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fl-casual-work-three-quarter-nubuck-leather-black-7d>: HTTP status code is not handled or not allowed 2026-01-25 06:00:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 72673 pages (at 96 pages/min), scraped 35513 items (at 23 items/min) 2026-01-25 06:00:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbk-athletic-work-shoe-dual-layer-stretch-mesh-upper-black-dark-grey-7w>: HTTP status code is not handled or not allowed 2026-01-25 06:00:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/77120-drum-sander-3pc-refill-1-4-shank-3-4-x-1>: HTTP status code is not handled or not allowed 2026-01-25 06:00:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/75002-resin-fiber-disc-3-pak-7-8-arbor-60-grit-4-1-2>: HTTP status code is not handled or not allowed 2026-01-25 06:00:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbk-athletic-work-shoe-breathable-mesh-black-10w>: HTTP status code is not handled or not allowed 2026-01-25 06:00:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rp-works-ruched-bootie-full-grain-leather-black-9-5w>: HTTP status code is not handled or not allowed 2026-01-25 06:00:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/75004-resin-fiber-disc-3-pak-7-8-arbor-50-grit-4-1-2>: HTTP status code is not handled or not allowed 2026-01-25 06:00:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/79150-polishing-bonnet-nap-thickness-5-8-5>: HTTP status code is not handled or not allowed 2026-01-25 06:00:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office-school-supply/computers_electronics/computer_tablet_accessories/monitor_risers_stands>: HTTP status code is not handled or not allowed 2026-01-25 06:00:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbk-athletic-work-shoe-ultraknit-flexweave-grey-white-7m>: HTTP status code is not handled or not allowed 2026-01-25 06:00:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbk-athletic-work-shoe-dual-layer-stretch-mesh-upper-black-dark-grey-11m>: HTTP status code is not handled or not allowed 2026-01-25 06:00:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbk-athletic-work-shoe-dual-layer-stretch-mesh-upper-black-dark-grey-11w>: HTTP status code is not handled or not allowed 2026-01-25 06:00:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss316-66006x02-34x14-class-150-hex-bushing-stainless-steel-316>: HTTP status code is not handled or not allowed 2026-01-25 06:00:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ia-6-inch-waterproof-work-boot-waterproof-leather-brown-7-5m>: HTTP status code is not handled or not allowed 2026-01-25 06:00:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss316-64004-12-class-150-coupling-stainless-steel-316>: HTTP status code is not handled or not allowed 2026-01-25 06:00:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rp-works-two-eye-tie-work-sneaker-dual-layer-stretch-mesh-upper-blue-tan-11-5w>: HTTP status code is not handled or not allowed 2026-01-25 06:00:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-sided-compact-training-stairs-with-platform-55l-x-30w-x-40h>: HTTP status code is not handled or not allowed 2026-01-25 06:00:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buckhorn-cross-stack-jumbo-lug-34305gray-gray-25-l-x-16-w-x-8-1-2-h>: HTTP status code is not handled or not allowed 2026-01-25 06:00:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbk-athletic-work-shoe-breathable-mesh-black-8-5m>: HTTP status code is not handled or not allowed 2026-01-25 06:00:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/divider-board-for-parallel-bars-with-platform-10l>: HTTP status code is not handled or not allowed 2026-01-25 06:00:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glassless-mirror-with-mobile-caster-base-vertical-24w-x-72h>: HTTP status code is not handled or not allowed 2026-01-25 06:00:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3m-performance-yellow-masking-tape-301-18-mm-x-55-m>: HTTP status code is not handled or not allowed 2026-01-25 06:00:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vertical-swivel-sign-frame-50-1134sv-cl-14-x-22-for-7-posts-chrome>: HTTP status code is not handled or not allowed 2026-01-25 06:00:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f18933-0012-lab-aire-ii-single-sided-non-electric-benchtop-dryer-2-tier>: HTTP status code is not handled or not allowed 2026-01-25 06:00:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ia-6-inch-wedge-work-boot-leather-brown-7w>: HTTP status code is not handled or not allowed 2026-01-25 06:00:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h14712-0350-polypropylene-141-liter-drum-and-carboy-funnel-1pk>: HTTP status code is not handled or not allowed 2026-01-25 06:00:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ia-6-inch-waterproof-work-boot-lime-resistant-full-grain-leather-brown-7-5w>: HTTP status code is not handled or not allowed 2026-01-25 06:00:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h14629-0000-buchner-table-top-funnel-with-perforated-fixed-plate-24-id-13h>: HTTP status code is not handled or not allowed 2026-01-25 06:00:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vy-straight-leg-w-caster-black>: HTTP status code is not handled or not allowed 2026-01-25 06:00:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h50028-2001-portable-glove-box-system-with-2-gas-ports-gloves-and-clamping-rings>: HTTP status code is not handled or not allowed 2026-01-25 06:00:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h-b-b61801-0700-durac-precision-10001070-specific-gravity-hydrometer>: HTTP status code is not handled or not allowed 2026-01-25 06:00:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clearview-storage-box-with-latched-lid-17551706-32-qt-23-5-8l-x-16-3-8w-x-6-1-2h>: HTTP status code is not handled or not allowed 2026-01-25 06:00:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ml-lockout-station-premier-electrical-device-assortment-6-zen-padlocks-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:00:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pilot-kit-for-bakers-pride-m1054x>: HTTP status code is not handled or not allowed 2026-01-25 06:00:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u1482-4wm-premium-locker-4-person-wall-unit-12x18x14-3-4-ready-to-assemble-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:00:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u3518-2a-premium-locker-double-tier-15x21x36-6-doors-assembled-parchment>: HTTP status code is not handled or not allowed 2026-01-25 06:00:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commercial-products-hygen-12x12-dmf-charging-bucket-4-ct-plastic-white>: HTTP status code is not handled or not allowed 2026-01-25 06:00:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u1558-2-premium-locker-double-tier-15x15x36-2-doors-ready-to-assemble-parchment>: HTTP status code is not handled or not allowed 2026-01-25 06:00:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-chrome-2-tier-wire-shelving-add-on-unit-24w-x-14d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 06:00:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ironcat-insulated-top-grain-reverse-deerskin-mig-welding-gloves-gold-medium-all-leather>: HTTP status code is not handled or not allowed 2026-01-25 06:00:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uy1518-2a-maintenance-free-quiet-locker-double-tier-15x21x36-2-doors-assembled-parchment>: HTTP status code is not handled or not allowed 2026-01-25 06:00:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u1282-6a-me-black-tie-locker-six-tier-12x18x12-6-doors-assembled-black>: HTTP status code is not handled or not allowed 2026-01-25 06:00:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u3558-2hdv-heavy-duty-ventilated-locker-double-tier-15x15x36-6-doors-unassembled-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:00:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u1282-6me-black-tie-locker-six-tier-12x18x12-6-doors-unassembled-black>: HTTP status code is not handled or not allowed 2026-01-25 06:00:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u1518-1hdv-heavy-duty-ventilated-locker-single-tier-15x21x72-1-door-unassembled-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:00:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-chrome-2-tier-wire-shelving-starter-unit-48w-x-14d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 06:00:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bark-spud-with-handle-t-009-018-spud-hardwood-handle-18>: HTTP status code is not handled or not allowed 2026-01-25 06:00:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-cant-hook-handle-w-029-048-0327-hardwood-48>: HTTP status code is not handled or not allowed 2026-01-25 06:00:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-chrome-3-tier-wire-shelving-starter-unit-36w-x-24d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 06:00:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/np-two-piece-biocooler-fits-box-14-inch-l-x-14-inch-w-x-14-inch-h-white>: HTTP status code is not handled or not allowed 2026-01-25 06:00:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/np-two-piece-biocooler-fits-box-16-inch-l-x-16-inch-w-x-16-inch-h-white>: HTTP status code is not handled or not allowed 2026-01-25 06:00:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pathcade-barrier-system-w-hip-sheeting-2-side-72-inch-x-3-inch-x-38-inch-white>: HTTP status code is not handled or not allowed 2026-01-25 06:00:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dry-contact-input-relay-ribd01bdc-enclosed-time-delay-120vac-20a-spdt>: HTTP status code is not handled or not allowed 2026-01-25 06:00:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-gauge-straight-finish-brad-nail-1-1-2-length-galvanized-steel-pkg-of-50000>: HTTP status code is not handled or not allowed 2026-01-25 06:00:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sliding-door-handle-set-black-diecast-no-pull-croft>: HTTP status code is not handled or not allowed 2026-01-25 06:00:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pistol-case-qf540sbkl-watertight-5-gun-cap-15-x12-1-8-x12-1-8-black>: HTTP status code is not handled or not allowed 2026-01-25 06:00:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/octenol-mosquito-attractant-ma1000-6>: HTTP status code is not handled or not allowed 2026-01-25 06:00:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbwp-10-bushel-antimicrobial-vinyl-basket-truck-w-antimicrobial-bumper-navy>: HTTP status code is not handled or not allowed 2026-01-25 06:00:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/floor-marking-tape-blue-plus-shape-25pkg>: HTTP status code is not handled or not allowed 2026-01-25 06:00:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tuff-mark-tape-yellow-black-2w-x-100l-roll>: HTTP status code is not handled or not allowed 2026-01-25 06:00:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tt-esd-rubber-matting-mt2472-24x72x0080-green>: HTTP status code is not handled or not allowed 2026-01-25 06:00:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/horizon-mfg-tri-lingual-three-ring-sds-binder-3008-1-1-2w>: HTTP status code is not handled or not allowed 2026-01-25 06:00:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbwp-14-bushel-antimicrobial-vinyl-basket-truck-w-antimicrobial-bumper-navy>: HTTP status code is not handled or not allowed 2026-01-25 06:00:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/am4023u-over-ocular-eyepiece-digital-camera-36mm-inner-diameter-1-3mp-1x>: HTTP status code is not handled or not allowed 2026-01-25 06:00:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/horizon-mfg-boot-and-glove-dryer-1132-10-1-2l>: HTTP status code is not handled or not allowed 2026-01-25 06:01:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/horizon-mfg-boot-etractor-1111-9-3-4l>: HTTP status code is not handled or not allowed 2026-01-25 06:01:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/horizon-mfg-personal-protect-equipment-storage-rack-pvc-coated-2016-pvc-7-1-2l>: HTTP status code is not handled or not allowed 2026-01-25 06:01:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-hardwood-cutting-u-shank-shank-jigsaw-blade-u101bf10-tpi-5-piece>: HTTP status code is not handled or not allowed 2026-01-25 06:01:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corvette-jack-saddle-pad-4-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:01:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gs-filter-pipette-tip-1-20ul-universal-low-retention-graduated-50mm>: HTTP status code is not handled or not allowed 2026-01-25 06:01:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-safety-netting-for-96-in-bay-standard-attachment-330-lbs-black>: HTTP status code is not handled or not allowed 2026-01-25 06:01:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-safety-netting-for-96-in-bay-j-hook-attachment-330-lbs-black>: HTTP status code is not handled or not allowed 2026-01-25 06:01:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ajc-sbs-s12550-12v-55ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-25 06:01:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rocker-switch-125-250v-20a-black-for-southbend-1177541>: HTTP status code is not handled or not allowed 2026-01-25 06:01:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ajc-power-sonic-ps121000b-ps12000nb-12v-100ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-25 06:01:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/contactor-3-pole-30-40a-208-240v-for-cleveland-ke50750-3>: HTTP status code is not handled or not allowed 2026-01-25 06:01:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ajc-ultra-tech-ut1270-ut-1270-12v-7ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-25 06:01:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reach-in-freezer-1-door-23-cuft-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 06:01:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 72770 pages (at 97 pages/min), scraped 35534 items (at 21 items/min) 2026-01-25 06:01:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2x2x6-round-end-150-gallon-poly-stock-tank>: HTTP status code is not handled or not allowed 2026-01-25 06:01:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reach-in-refrigerator-1-door-23-cuft-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 06:01:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ajc-deka-8g30h-12v-100ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-25 06:01:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hn-smartlink-four-leg-chair-19-1-2-inch-x-19-5-8-inch-x-31-inch-onyx-seat-onyx-base-4-carton>: HTTP status code is not handled or not allowed 2026-01-25 06:01:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/freestanding-office-partition-panel-with-partial-window-60-1-4-w-x-60-h-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:01:12 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_chair_table_sets/conversation_sets already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 06:01:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-freestanding-office-partition-panel-with-partial-window-60-1-4-w-x-61-1-2-h-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:01:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/duk-stellar-aluminum-5-piece-sectional-with-tables-and-cushions>: HTTP status code is not handled or not allowed 2026-01-25 06:01:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/r2141-w-3-siphon-jet-floor-mounted-toiletw-back-spud-white-enviro-finish>: HTTP status code is not handled or not allowed 2026-01-25 06:01:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ajc-power-patrol-sla1152-12v-26ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-25 06:01:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thhn-wire-w-non-metallic-connectors-1-2-x-4>: HTTP status code is not handled or not allowed 2026-01-25 06:01:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-x-2-signature-wheel-with-roller-bearing-for-1-2-axle-700-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:01:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/run-capacitor-55-plus-7-5-mfd-440v-round>: HTTP status code is not handled or not allowed 2026-01-25 06:01:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tradepro-filter-drier-liquid-line-3-8-ods-8ci-sweat>: HTTP status code is not handled or not allowed 2026-01-25 06:01:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carrier-weathermaker-rooftop-gas-heat-electric-cool-unit-5-ton-3-ph>: HTTP status code is not handled or not allowed 2026-01-25 06:01:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blower-motor-1050-rpm-230v-1-2-to-1-3-hp>: HTTP status code is not handled or not allowed 2026-01-25 06:01:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/office/outdoor_furniture/outdoor_chair_table_sets/conversation_sets>: HTTP status code is not handled or not allowed 2026-01-25 06:01:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-6-step-tilt-roll-ladder-serrated-grating-hl6nsg>: HTTP status code is not handled or not allowed 2026-01-25 06:01:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dti-coverall-bound-seams-elastic-wrists-ankles-white-xl-25-qty>: HTTP status code is not handled or not allowed 2026-01-25 06:01:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dti-coverall-bound-seams-elastic-wrists-ankles-white-3x-25-qty>: HTTP status code is not handled or not allowed 2026-01-25 06:01:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/122018-wall-mount-nameplate-sign-holder-w-adhesive-tape-8-5-x-2-5-10-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:01:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trigger-sprayer-chemical-resistant-28mm-yellow-24-case-gjo85145>: HTTP status code is not handled or not allowed 2026-01-25 06:01:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lnr-xtd-200-waste-oil-heater-pkg-6-inch-chimney-roof-kit-215-gallon-tank-200000-btu>: HTTP status code is not handled or not allowed 2026-01-25 06:01:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lnr-xt-250-waste-oil-heater-pkg-8-inch-chimney-wall-kit-215-gallon-tank-250000-btu>: HTTP status code is not handled or not allowed 2026-01-25 06:01:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/608a-05-model-608a-with-flow-rate-of-05-gpm>: HTTP status code is not handled or not allowed 2026-01-25 06:01:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wern-roofing-kit-in-back-pack-h113002-a210400-l232050-and-back-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:01:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1041-long-foot-pedal-for-valves-103-104-105-106>: HTTP status code is not handled or not allowed 2026-01-25 06:01:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wern-aerial-lift-kit-h312002-r330010-in-duffle-bag>: HTTP status code is not handled or not allowed 2026-01-25 06:01:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/300624-sloped-chrome-wire-basket-9-high-metal>: HTTP status code is not handled or not allowed 2026-01-25 06:01:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/56f1-3-person-floor-mount-wash-station-with-single-foot-pedal-valves>: HTTP status code is not handled or not allowed 2026-01-25 06:01:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/252301-single-pocket-bi-fold-size-countertop-brochure-holder-6-25-x-7-75-10-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:01:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/252306-4-tier-tri-fold-size-countertop-brochure-holder-4-5-x-13-25-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:01:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wern-cable-choker-8-feet-l-o-ring-5-16-inch-vinyl-coated-cable>: HTTP status code is not handled or not allowed 2026-01-25 06:01:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wern-aerial-srl-kit-h412002-r530011-a111006-in-back-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:01:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/28-wall-clock-black-metal>: HTTP status code is not handled or not allowed 2026-01-25 06:01:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/556118-large-bucket-for-pegboard-slatwall-7-5-x-9-clear>: HTTP status code is not handled or not allowed 2026-01-25 06:01:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700350-pur-pegboard-powerwing-13-75-x-44-purple>: HTTP status code is not handled or not allowed 2026-01-25 06:01:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hs-2l-16-wide-hand-sink-low-lead-compliant>: HTTP status code is not handled or not allowed 2026-01-25 06:01:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-408l-royal-series-4-center-raised-deck-mount-faucet-8-spout-low-lead>: HTTP status code is not handled or not allowed 2026-01-25 06:01:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/701414-blk-interlocking-pegboard-countertop-display-14-x-20-black-solid>: HTTP status code is not handled or not allowed 2026-01-25 06:01:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gm-scrim-reinforced-exam-gown-w-glued-shoulders-30-inch-x-42-inch-blue-50-case>: HTTP status code is not handled or not allowed 2026-01-25 06:01:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gm-exam-gown-30-inch-x-42-inch-blue-50-case-231>: HTTP status code is not handled or not allowed 2026-01-25 06:01:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/21-118l-swing-to-rigid-adapter-low-lead>: HTTP status code is not handled or not allowed 2026-01-25 06:01:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/252922-50pk-countertop-tri-fold-brochure-holder-w-card-pocket>: HTTP status code is not handled or not allowed 2026-01-25 06:01:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-706l-royal-series-8-space-saver-pre-rinse-w-faucet-6-spout-wall-mount-low-lead>: HTTP status code is not handled or not allowed 2026-01-25 06:01:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/21-330l-low-lead-heavy-duty-cold-valve-repair-kit-with-built-in-check-valve>: HTTP status code is not handled or not allowed 2026-01-25 06:01:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/701465-pnk-4-sided-interlocking-pegboard-floor-display-w-wheels-14-x-60-pink>: HTTP status code is not handled or not allowed 2026-01-25 06:01:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-i-beam-mount-fan-w-drop-cord-switch-2-speed-silver>: HTTP status code is not handled or not allowed 2026-01-25 06:01:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/703387-clr-4-sided-interlocking-pegboard-display-8-x-40-clear-opaque>: HTTP status code is not handled or not allowed 2026-01-25 06:01:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/belleville-disc-spring-075-od-x-038-id-x-0028-thick-x-0051-oah-1074-carbon-steel>: HTTP status code is not handled or not allowed 2026-01-25 06:01:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/703388-blk-4-sided-interlocking-pegboard-display-w-wheels-8-x-40-black-solid>: HTTP status code is not handled or not allowed 2026-01-25 06:01:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/az-ph-rectangluar-bar-height-tile-top-fire-pit-with-wind-screen>: HTTP status code is not handled or not allowed 2026-01-25 06:01:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keystock-1-4-x-1-4-x-1-ft-316-stainless-steel-plain-undersize-astm-a484>: HTTP status code is not handled or not allowed 2026-01-25 06:01:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keystock-3-16-x-1-4-x-1-ft-carbon-steel-plain-oversize-astm-a29>: HTTP status code is not handled or not allowed 2026-01-25 06:01:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keystock-5-16-x-5-16-x-3-ft-316-stainless-steel-plain-undersize-astm-a484>: HTTP status code is not handled or not allowed 2026-01-25 06:01:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/703385-pnk-interlocking-pegboard-counterop-display-8-x-20-pink>: HTTP status code is not handled or not allowed 2026-01-25 06:01:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/igg-magnetic-base-stand-w-176-lbs-holding-power>: HTTP status code is not handled or not allowed 2026-01-25 06:01:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inplant-office-vinyl-cov-drywall-12x16-4-wall-3-lgt-4-dplx-1-dr-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:01:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inplant-office-vinyl-cov-drywall-12x16-4-wall-3-lgt-4-dplx-1-dr-white>: HTTP status code is not handled or not allowed 2026-01-25 06:01:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inplant-office-vinyl-cov-drywall-10x20-4-wall-3-lgt-4-dplx-1-dr-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:01:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/igg-tri-point-internal-bore-gauge-ip54-0-236-0-315-inch-0-001mm-0-00005-accuracy-0-00015-inch>: HTTP status code is not handled or not allowed 2026-01-25 06:02:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inplant-office-pntd-steel-drywall-12x24-4-wall-4-lgt-5-dplx-1-dr-wht>: HTTP status code is not handled or not allowed 2026-01-25 06:02:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hygrade-heavy-duty-super-cover-poly-tarp-sth2040-10-mil-silverblack-20l-x-40w>: HTTP status code is not handled or not allowed 2026-01-25 06:02:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hygrade-blue-poly-tarp-mt3060-5-mil-30l-x-60w>: HTTP status code is not handled or not allowed 2026-01-25 06:02:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hygrade-blue-poly-tarp-mt4060-5-mil-40l-x-60w>: HTTP status code is not handled or not allowed 2026-01-25 06:02:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inplant-office-pntd-steel-drywall-10x12-2-wall-3-lgt-2-dplx-1-dr-gry>: HTTP status code is not handled or not allowed 2026-01-25 06:02:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-blue-chemical-resistant-gloves-ansell-88-356-unsupported-unlined-size-8>: HTTP status code is not handled or not allowed 2026-01-25 06:02:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inplant-office-vinyl-cov-drywall-10x12-4-wall-3-lgt-2-dplx-1-dr-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:02:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inplant-office-vinyl-cov-hdbd-16x20-2-wall-5-lgts-6-dplx-1-door-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:02:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reciprocating-saw-dcs380b-20vmax-0-3000-spm-1-1-8l-stroke-tool-only>: HTTP status code is not handled or not allowed 2026-01-25 06:02:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inplant-office-vinyl-cov-drywall-12x20-2-wall-4-lgt-5-dplx-1-dr-white>: HTTP status code is not handled or not allowed 2026-01-25 06:02:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electric-office-partition-panel-with-partial-window-48-1-4w-x-76h-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:02:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aware-wear-s721-non-ansi-female-vest-61912-pink-x-large>: HTTP status code is not handled or not allowed 2026-01-25 06:02:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inplant-office-vinyl-cov-drywall-12x20-4-wall-4-lgt-5-dplx-1-dr-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:02:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 72866 pages (at 96 pages/min), scraped 35557 items (at 23 items/min) 2026-01-25 06:02:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/framed-double-sided-changeable-letterboard-36h-x-24w>: HTTP status code is not handled or not allowed 2026-01-25 06:02:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jaw-assembly-cpl-00039>: HTTP status code is not handled or not allowed 2026-01-25 06:02:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-green-wire-ledge-48-x-4>: HTTP status code is not handled or not allowed 2026-01-25 06:02:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conductive-hinged-boxes-w-foam-4-3-8l-x-3-3-8w-x-7-10h>: HTTP status code is not handled or not allowed 2026-01-25 06:02:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-plastic-barrier-chain-1-inch-x-25-feet-l-army-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:02:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/circuit-board-shipping-and-storage-box-w-foam-7-3-16l-x-6-3-8w-x-1-3-4h>: HTTP status code is not handled or not allowed 2026-01-25 06:02:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mounting-kit-with-18-w-louver-36-w-whiteboard-60-w-workbench-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:02:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-plastic-barrier-chain-2-inch-x-100-feet-l-crimson>: HTTP status code is not handled or not allowed 2026-01-25 06:02:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-heavy-duty-plastic-barrier-chain-2-inch-x-500-feet-l-army-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:02:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastek-esd-open-bin-box-6w-x-24d-x-4h>: HTTP status code is not handled or not allowed 2026-01-25 06:02:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-reflective-plastic-barrier-chain-2-inch-x-100-feet-l-army-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:02:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-step-aluminum-rolling-ladder-16inw-ribbed-tread-28ind-top-step-w-o-handrails>: HTTP status code is not handled or not allowed 2026-01-25 06:02:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conductive-reel-storage-container-5-slots-5-3-4l-x-8-1-4w-x-7-1-4h>: HTTP status code is not handled or not allowed 2026-01-25 06:02:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esd-storage-container-17-7-8l-x-10-1-8w-x-2-5-8h>: HTTP status code is not handled or not allowed 2026-01-25 06:02:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esd-in-plant-handle-container-w-fixed-dividers-lid-cell-size-5-7-8-x-5-3-8-x-2-3-4>: HTTP status code is not handled or not allowed 2026-01-25 06:02:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-series-swivel-plate-caster-408199559-brk4-rubber-total-lock-brake-8dia-600-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 06:02:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esd-in-plant-handle-container-w-fixed-dividers-lid-cell-size-15-1-4-x-20-1-8-x-1>: HTTP status code is not handled or not allowed 2026-01-25 06:02:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-series-swivel-plate-caster-406109839-brk7-polyolefin-top-lock-brake-6dia-700-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 06:02:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-series-swivel-plate-caster-404109339-brk7-phenolic-top-lock-brake-4dia-800-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 06:02:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-series-swivel-plate-caster-4061099497-brk7-polyurethane-with-brake-6dia-1200-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 06:02:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-series-rigid-plate-caster-404108459-rubber-on-polyolefin-4dia-400-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 06:02:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-series-rigid-plate-caster-408198929-ss-polyurethane-on-polyolefin-8dia-1000-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 06:02:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wash-bottle-117160009-ldpe-500ml-capacity-write-on-4-pk-natural-cap-wide-mouth>: HTTP status code is not handled or not allowed 2026-01-25 06:02:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wash-bottle-116180016-ldpe-500ml-capacity-12-pk-natural-cap-narrow-mouth>: HTTP status code is not handled or not allowed 2026-01-25 06:02:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-bar-table-60-inches-x-60-inches-x-29-inches-with-medium-oak-top-and-dual-pedestal-black-cross-base>: HTTP status code is not handled or not allowed 2026-01-25 06:02:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp100-radar-sign-4-cell-battery-power-with-dolly-white-sign-128876>: HTTP status code is not handled or not allowed 2026-01-25 06:02:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/automotive-stackable-plastic-pallet-with-lip-48-x-45-x-6>: HTTP status code is not handled or not allowed 2026-01-25 06:02:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-10-x-3-1-2-carriage-bolt-square-neck-grade-8-full-unc-steel-plain-pkg-of-5>: HTTP status code is not handled or not allowed 2026-01-25 06:02:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-putty-blueberry-firm-4-ounce>: HTTP status code is not handled or not allowed 2026-01-25 06:02:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-putty-5-pound-set-of-6-6-colors>: HTTP status code is not handled or not allowed 2026-01-25 06:02:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-exercise-putty-tan-xx-soft-3-ounce>: HTTP status code is not handled or not allowed 2026-01-25 06:02:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nol-1-18mm-heavy-duty-rubber-grip-snap-off-blade-utility-knife>: HTTP status code is not handled or not allowed 2026-01-25 06:02:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-wide-designer-wood-locker-21368-single-tier-3-wide-15x18x72-cherry-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 06:02:34 [HeadersSpooferDownloaderMiddleware] (PID: 96) WARNING: Request https://www.globalindustrial.com/c/storage/lockers/standard_lockers/wood_standard_lockers already has headers. They will be preserved, but that may lead to fingerprint inconsistency. If the headers are necessary, consider disabling SPOOF_FULL_HEADERS. 2026-01-25 06:02:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-wide-designer-wood-locker-27368-z-style-3-wide-15x18x72-gray-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 06:02:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/side-panel-33330-for-15d-designer-wood-locker-without-sloping-hood-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:02:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cl-s40-20d-20in-stand-on-automatic-scrubber-140-ah-maint-free-batteries-56104486>: HTTP status code is not handled or not allowed 2026-01-25 06:02:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b1c18x21x-utility-sink-1compartment-w-18l-x-21w-bowl-12-deep>: HTTP status code is not handled or not allowed 2026-01-25 06:02:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14006003-3-prong-tool-holder-for-perfo-panels-6-long>: HTTP status code is not handled or not allowed 2026-01-25 06:02:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acr1717rh-full-size-pharmaceutical-all-refrigerator-right-hand-door-swing-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 06:02:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slim-white-pe-foam-roller-half-round-3-dia-x-36l>: HTTP status code is not handled or not allowed 2026-01-25 06:02:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-pe-foam-roller-round-6-dia-x-12l>: HTTP status code is not handled or not allowed 2026-01-25 06:02:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-band-hand-exerciser-additional-latex-free-bands-only-blue-25-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:02:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exerciser-5-piece-set-medium>: HTTP status code is not handled or not allowed 2026-01-25 06:02:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hand-exerciser-small-red-light>: HTTP status code is not handled or not allowed 2026-01-25 06:02:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-spring-hand-grip-1-each>: HTTP status code is not handled or not allowed 2026-01-25 06:02:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-series-inflatable-exercise-ball-75-cm-30-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:02:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inflatable-exercise-ball-set-orange-55-cm-22-ball-and-pump-in-retail-box>: HTTP status code is not handled or not allowed 2026-01-25 06:02:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-series-inflatable-exercise-ball-45-cm-18-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:02:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/c/storage/lockers/standard_lockers/wood_standard_lockers>: HTTP status code is not handled or not allowed 2026-01-25 06:02:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inflatable-reciprocal-stepper-cushion-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:02:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-mobile-floor-rack-for-inflatable-exercise-balls-62l-x-20w-x-72h-3-shelves>: HTTP status code is not handled or not allowed 2026-01-25 06:02:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foam-roller-massager-22-long-black-orange>: HTTP status code is not handled or not allowed 2026-01-25 06:02:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-ton-cap-aluminum-gantry-crane-8-ft-span-adj-height-6-6-min-9-0-max>: HTTP status code is not handled or not allowed 2026-01-25 06:02:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-ton-10-span-301-series-steel-wall-mounted-wall-bracket-jib-crane-tie-rod-design>: HTTP status code is not handled or not allowed 2026-01-25 06:02:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11-stp-steel-safe-roll-lad-wgt-act-lock-24-w-serr-stp-org-wa113214g-o>: HTTP status code is not handled or not allowed 2026-01-25 06:02:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-stp-steel-safe-roll-lad-wgt-act-lock-16-w-perf-stp-org-wa082414p-o>: HTTP status code is not handled or not allowed 2026-01-25 06:02:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fs-942-pvc-floor-sink-full-grate-dome-strainer-2-solvent-weld-outlet-for-sch-40-pvc-pipe>: HTTP status code is not handled or not allowed 2026-01-25 06:02:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdpe-ground-protection-mat-4-x-8-black-am48>: HTTP status code is not handled or not allowed 2026-01-25 06:02:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mounted-coat-rack-3-double-prong-hook-rail-nickel-mahogany>: HTTP status code is not handled or not allowed 2026-01-25 06:02:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/legacy-bracket-bktm02x20x20less-2w-x-20d-x-20h>: HTTP status code is not handled or not allowed 2026-01-25 06:02:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/125-x-13-heavy-duty-airlaid-14-fold-wipe-75case>: HTTP status code is not handled or not allowed 2026-01-25 06:02:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9in-x-3-4in-100-polyester-roller-covers-6-pack-6-case-9ap075-6pk>: HTTP status code is not handled or not allowed 2026-01-25 06:02:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-piece-dralon-woven-high-density-foam-mini-roller-paint-kit-6-case-612-wv6fq>: HTTP status code is not handled or not allowed 2026-01-25 06:02:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6in-x-1-4in-dralon-woven-mini-roller-cover-w-24in-frame-12-case-624-wv025>: HTTP status code is not handled or not allowed 2026-01-25 06:02:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyethylene-cart-11-bushel-4-wheel-10-casters>: HTTP status code is not handled or not allowed 2026-01-25 06:02:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-plain-trolley-4000-lbs-capacity-plain>: HTTP status code is not handled or not allowed 2026-01-25 06:02:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-geared-trolley-6000-lbs-capacity-geared>: HTTP status code is not handled or not allowed 2026-01-25 06:02:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/builders-hoist-bracket-for-obh500ng-and-obh1000ng>: HTTP status code is not handled or not allowed 2026-01-25 06:02:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inline-mixed-flow-duct-fan-6-galvanized-steel-291-cfm-energy-star>: HTTP status code is not handled or not allowed 2026-01-25 06:02:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/small-benchtop-muffle-furnace-with-a1-controller-1-3l-120v>: HTTP status code is not handled or not allowed 2026-01-25 06:02:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-8-7x7-galvanized-aircraft-cable>: HTTP status code is not handled or not allowed 2026-01-25 06:02:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eagle-professional-grade-3-air-cut-off-tool-ega510>: HTTP status code is not handled or not allowed 2026-01-25 06:02:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a5-adjustable-arms-doral-integra-series>: HTTP status code is not handled or not allowed 2026-01-25 06:02:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/400-x-expanding-time-card-rack>: HTTP status code is not handled or not allowed 2026-01-25 06:02:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/riser-straight-with-carpet-96l-x-18w-x-24h-black>: HTTP status code is not handled or not allowed 2026-01-25 06:03:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10in-chenille-microfiber-washing-tool-blue-sw19170>: HTTP status code is not handled or not allowed 2026-01-25 06:03:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10in-genuine-sheet-skin-replacement-bonnet-for-sw19140-yellow-sw19155>: HTTP status code is not handled or not allowed 2026-01-25 06:03:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trailer-tire-570-8-6-ply-on-8x375-5-45-wheel>: HTTP status code is not handled or not allowed 2026-01-25 06:03:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/125-ml-flask-clip-for-precision-shaking-water-baths>: HTTP status code is not handled or not allowed 2026-01-25 06:03:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sewn-liquid-bag-filter-nylon-mesh-12-dia-x-18-l-800-micron-draw-string-pkg-qty-50>: HTTP status code is not handled or not allowed 2026-01-25 06:03:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-20-x-1-1-2-security-machine-screw-flat-torx-head-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 06:03:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-32-x-1-security-cap-screw-flat-socket-head-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 06:03:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-13-x-1-1-2-security-cap-screw-button-socket-head-18-8-stainless-steel-pkg-of-10>: HTTP status code is not handled or not allowed 2026-01-25 06:03:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/welded-liquid-bag-filter-polypropylene-felt-7-1-8-d-x-16-1-2-l-50-micron-plastic-flange-pkg-50>: HTTP status code is not handled or not allowed 2026-01-25 06:03:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inch-metric-hex-l-wrench-dbl-pack-12199-12137>: HTTP status code is not handled or not allowed 2026-01-25 06:03:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m6-x-1-0-x-25mm-security-machine-screw-button-torx-head-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 06:03:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-8-star-tip-screwdrivers-t6-t25-34532>: HTTP status code is not handled or not allowed 2026-01-25 06:03:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-16-tamperproof-hex-key-double-ended>: HTTP status code is not handled or not allowed 2026-01-25 06:03:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-32-x-1-1-2-security-machine-screw-button-torx-head-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 06:03:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/office-partition-panel-with-partial-window-and-raceway-48-1-4w-x-64h-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:03:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 72964 pages (at 98 pages/min), scraped 35565 items (at 8 items/min) 2026-01-25 06:03:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-24-x-1-security-machine-screw-button-torx-head-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 06:03:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-office-partition-panel-with-partial-window-and-raceway-36-1-4w-x-77-1-2h-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:03:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-28-x-1-2-security-machine-screw-button-torx-head-18-8-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 06:03:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/infinity-locker-six-tier-12x18x12-6-door-assembled-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:03:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-freestanding-office-partition-panel-with-partial-window-48-1-4-w-x-43-1-2-h-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:03:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-32-x-1-1-4-spanner-machine-screw-oval-head-brass-plain-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 06:03:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-40-tamperproof-torx-power-bit>: HTTP status code is not handled or not allowed 2026-01-25 06:03:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-control-1-8-tube-x-1-8-swift-fit-flow-out-screw-adjustment>: HTTP status code is not handled or not allowed 2026-01-25 06:03:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guard-pair-gu39016r36-galvanized-steel-90-degree-curve-16bf-36inside-radius>: HTTP status code is not handled or not allowed 2026-01-25 06:03:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guard-pair-gu305-for-5-conveyors-galvanized-steel-2-1-2h-x-1w-x-12ga>: HTTP status code is not handled or not allowed 2026-01-25 06:03:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/needle-valve-10mm-tube-x-3-8-metal-release-collet-knob-adjustmet>: HTTP status code is not handled or not allowed 2026-01-25 06:03:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/male-hose-adapter-w-barb-20mm-x-1-2-taper>: HTTP status code is not handled or not allowed 2026-01-25 06:03:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-straight-roller-conveyor-5f05t06b51-51bf-2-5-8-roller-dia-6-axle-centers>: HTTP status code is not handled or not allowed 2026-01-25 06:03:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0559101-1-2-tempering-control-valve>: HTTP status code is not handled or not allowed 2026-01-25 06:03:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-control-4mm-tube-x-m5-metal-release-collet-flow-out-screw-adjustment>: HTTP status code is not handled or not allowed 2026-01-25 06:03:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heraeus-pico-21-ventilated-microcentrifuge-75002416-120v-60hz>: HTTP status code is not handled or not allowed 2026-01-25 06:03:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultraspill-berm-plus-orange>: HTTP status code is not handled or not allowed 2026-01-25 06:03:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelving-30w-x-24d-x-34h-2-shelf-starter-chrome>: HTTP status code is not handled or not allowed 2026-01-25 06:03:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelving-30w-x-24d-x-63h-4-shelf-add-on-chrome>: HTTP status code is not handled or not allowed 2026-01-25 06:03:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-xtex-blanket-and-tethers-for-ultraoil-blanket>: HTTP status code is not handled or not allowed 2026-01-25 06:03:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelving-72w-x-24d-x-63h-4-shelf-add-on-chrome>: HTTP status code is not handled or not allowed 2026-01-25 06:03:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-wire-shelving-36w-x-14d-x-63h-4-shelf-add-on-chrome>: HTTP status code is not handled or not allowed 2026-01-25 06:03:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slide-seal-bag-8w-7l>: HTTP status code is not handled or not allowed 2026-01-25 06:03:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-equipment-cover-clear-on-roll-28w-56l>: HTTP status code is not handled or not allowed 2026-01-25 06:03:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-epoxy-wire-shelving-189221gz-54w-x-24d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 06:03:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/advanced-shop-wiper-450-8-5-x-10-blue-200-bucket-2-buckets-case>: HTTP status code is not handled or not allowed 2026-01-25 06:03:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-hand-towel-roll-7-88-inch-x-800-ft-natural-6-rolls-case>: HTTP status code is not handled or not allowed 2026-01-25 06:03:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-step-nestable-plastic-step-stand-blue-25-w-x-25-d-x-10-h-nst-1-bl>: HTTP status code is not handled or not allowed 2026-01-25 06:03:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reese-s-sticks-wafer-bar-1-5-oz-20-count-gr>: HTTP status code is not handled or not allowed 2026-01-25 06:03:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/karcher-1-107-380-0-2500psi-2-4gpm-gas-pressure-washer>: HTTP status code is not handled or not allowed 2026-01-25 06:03:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0-5ml-0-8ml-paddles-for-tube-revolver-88881001-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:03:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-wall-reclosable-seal-top-specimen-trans-bag-purple-tint-12w-15l>: HTTP status code is not handled or not allowed 2026-01-25 06:03:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-medical-face-mask-3-ply-w-earloops-astm-level-3-blue-50-box>: HTTP status code is not handled or not allowed 2026-01-25 06:03:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-digit-calculator-12123a-2-color-printing-7-1-4-x-8-1-4-x-2-1-2-white>: HTTP status code is not handled or not allowed 2026-01-25 06:03:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bar-for-stacking-for-digital-bottle-tube-roller-88881003-4-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:03:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-96-w-x-36-d-x-120-h-starter-3-shelf-levels-steel-deck-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:03:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-rollers-for-digital-bottle-tube-roller-88881003-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:03:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-96-w-x-24-d-x-72-h-add-on-3-shelf-levels-steel-deck-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:03:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pegboard-panels-powdercoat-green-16-x-32-2-pc>: HTTP status code is not handled or not allowed 2026-01-25 06:03:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-readybuilt-locker-36-w-x-12-d-x-78-h-725-hallowell-gray-6-tier-3-wide-assembled>: HTTP status code is not handled or not allowed 2026-01-25 06:03:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-round-bed-risers-black-plastic-fits-up-to-3-diameter-post-300-lb-weight-capacity-4-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:03:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fort-knox-locker-half-height-door-and-6-drawers-24-w-x-24-d-x-78-h-blk-body-red-drs-all-welded>: HTTP status code is not handled or not allowed 2026-01-25 06:03:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smooth-platform-kit-4-x-36-x-18>: HTTP status code is not handled or not allowed 2026-01-25 06:03:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-coil-thread-repair-kit-for-metal-m8-125-15d>: HTTP status code is not handled or not allowed 2026-01-25 06:03:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-value-max-locker-12-w-x-15-d-x-78-h-725-hallowell-gray-single-tier-1-wide-assembled>: HTTP status code is not handled or not allowed 2026-01-25 06:03:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/block-heater-28-x-10-mm-dia-tubes>: HTTP status code is not handled or not allowed 2026-01-25 06:03:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kahlo-36-round-multipurpose-breakroom-tapered-leg-table-grey-black>: HTTP status code is not handled or not allowed 2026-01-25 06:03:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drill-point-screw-12-14-x-1-hex-washer-head-pkg-of-400-21341>: HTTP status code is not handled or not allowed 2026-01-25 06:03:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/screw-w-wings-10-16-x-1-716-phillips-flat-head-pkg-of-300-21381>: HTTP status code is not handled or not allowed 2026-01-25 06:03:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/earth-cork-board-oak-frame-48w-x-36h-sb0720001233>: HTTP status code is not handled or not allowed 2026-01-25 06:03:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-96-w-x-48-d-x-120-h-add-on-3-shelf-levels-wood-deck-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:03:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-48-w-x-24-d-x-120-h-starter-3-shelf-levels-wood-deck-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:03:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-value-max-locker-12-w-x-12-d-x-78-h-729-parchment-triple-tier-1-wide-assembled>: HTTP status code is not handled or not allowed 2026-01-25 06:03:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-48-w-x-48-d-x-84-h-starter-3-shelf-levels-wood-deck-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:03:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dixon-740-modeling-clay-assortment-14-lb-each-bluegreenredyellow-1-lb>: HTTP status code is not handled or not allowed 2026-01-25 06:03:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/westcott-15553-straight-kleenearth-soft-handle-scissors-7-length-bluegray>: HTTP status code is not handled or not allowed 2026-01-25 06:03:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/automotive-exterior-rust-remover-5-gal>: HTTP status code is not handled or not allowed 2026-01-25 06:03:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chenille-kraft-3907-01-felt-sheet-pack-rectangular-9-x-12-assorted-colors-12pack>: HTTP status code is not handled or not allowed 2026-01-25 06:03:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-48-w-x-48-d-x-120-h-starter-3-shelf-levels-wood-deck-sand>: HTTP status code is not handled or not allowed 2026-01-25 06:03:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/biocane-34-canister-and-cane-system-34-8-liters>: HTTP status code is not handled or not allowed 2026-01-25 06:03:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/withdrawal-device-for-thermo-20-liquid-nitrogen-transfer-vessel>: HTTP status code is not handled or not allowed 2026-01-25 06:03:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lseriesdigital-lab-scalepoint-precision>: HTTP status code is not handled or not allowed 2026-01-25 06:03:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010131-14-16-ss-terminals>: HTTP status code is not handled or not allowed 2026-01-25 06:03:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2171139-o-ring-1-5-16-inch-od-for-server-products>: HTTP status code is not handled or not allowed 2026-01-25 06:03:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/command-17003-general-purpose-hooks-5-lb-capacity-plastic-white>: HTTP status code is not handled or not allowed 2026-01-25 06:03:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bates-9841000-tally-i-hand-model-tally-counter-registers-0-9999-chrome>: HTTP status code is not handled or not allowed 2026-01-25 06:03:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/command-17091clrcabp-clear-hooks-strips-plastic-medium-50-hooks>: HTTP status code is not handled or not allowed 2026-01-25 06:03:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25-5702-30isw-fa-iii-sar-with-v-100-valve-and-welding-helmet-9100-air>: HTTP status code is not handled or not allowed 2026-01-25 06:03:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/263173-fryer-skimmer>: HTTP status code is not handled or not allowed 2026-01-25 06:03:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/endust-259000-antistatic-premoistened-wipes-for-electronics-cloth-5-12-x-7-70tub>: HTTP status code is not handled or not allowed 2026-01-25 06:03:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kimberly-clark-75130-scott-shop-towels-roll-10-25-x-11-blue-55roll-30carton>: HTTP status code is not handled or not allowed 2026-01-25 06:03:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ribbed-3-step-24-alum-rolling-ladder-10-top-step-spring-load-a3s30>: HTTP status code is not handled or not allowed 2026-01-25 06:03:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-96-w-x-48-d-x-72-h-add-on-3-shelf-levels-steel-deck-medium-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:03:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/681298-agitator-motor>: HTTP status code is not handled or not allowed 2026-01-25 06:03:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ribbed-2-step-18-alum-rolling-ladder-10-top-step-spring-load-a2sh>: HTTP status code is not handled or not allowed 2026-01-25 06:03:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/read-right-rr1209-onestep-screen-cleaner-5-x-5-24box>: HTTP status code is not handled or not allowed 2026-01-25 06:03:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011482-relay-compressor-120v>: HTTP status code is not handled or not allowed 2026-01-25 06:03:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/21-compartment-small-steel-scoop-storage-box>: HTTP status code is not handled or not allowed 2026-01-25 06:03:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sign-exit-and-exit-please-do-not-enter-double-sided-7-inch-by-11-inch>: HTTP status code is not handled or not allowed 2026-01-25 06:03:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/441137-transformer>: HTTP status code is not handled or not allowed 2026-01-25 06:03:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/264652-extension-fryer-drain-9-5-8-inch-x1-1-2-inch>: HTTP status code is not handled or not allowed 2026-01-25 06:03:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pl-c-lamp-cfq26wg24q827-26w-2700k>: HTTP status code is not handled or not allowed 2026-01-25 06:03:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/draw-shave-tmb-13dc-curved-13-steel-blade-with-wood-handles>: HTTP status code is not handled or not allowed 2026-01-25 06:04:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1751134-casters-4-inch-wheel-9-inch-lift-4-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:04:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-portable-air-conditioner-2-5-ton-w-cold-air-nozzles-29-000-btu-230v>: HTTP status code is not handled or not allowed 2026-01-25 06:04:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lockout-tagout-hasps-1-opening-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:04:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-96-w-x-36-d-x-84-h-add-on-3-shelf-levels-steel-deck-medium-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:04:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/champion-sports-viper4-viper-soccer-ball-size-4-white>: HTTP status code is not handled or not allowed 2026-01-25 06:04:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tennesse-state-flag-3x5-nylon>: HTTP status code is not handled or not allowed 2026-01-25 06:04:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-96-w-x-36-d-x-84-h-starter-3-shelf-levels-wire-deck-medium-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:04:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alera-aleva353624my-valencia-series-reversible-returnbridge-shell-35w-x-23-58d-mahogany>: HTTP status code is not handled or not allowed 2026-01-25 06:04:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alera-aleva257224mc-valencia-series-credenza-shell-70-78w-x-23-58d-x-29-12h-medium-cherry>: HTTP status code is not handled or not allowed 2026-01-25 06:04:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/111392-tailpiece-2-inch>: HTTP status code is not handled or not allowed 2026-01-25 06:04:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1111093-washer-3-4-inch-od-hose-ts-for-ts-brass-bronze-works>: HTTP status code is not handled or not allowed 2026-01-25 06:04:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1111173-button-index-blue-wristblade-for-ts-brass-bronze-works>: HTTP status code is not handled or not allowed 2026-01-25 06:04:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-pharmacy-vaccine-refrigerator-single-swing-solid-door-23-cu-ft-ph-abt-s23s>: HTTP status code is not handled or not allowed 2026-01-25 06:04:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/381443-fuse-for-cleveland-range>: HTTP status code is not handled or not allowed 2026-01-25 06:04:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-pharmacy-vaccine-refrigerator-single-swing-glass-door-12-cu-ft-ph-abt-s12g>: HTTP status code is not handled or not allowed 2026-01-25 06:04:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 73062 pages (at 98 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:04:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-inch-x-50-foot-pvc-lay-flat-discharge-hose-coupled-with-cxe-aluminum-cam-and-groove-fittings>: HTTP status code is not handled or not allowed 2026-01-25 06:04:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2361005-latch-assembly-magnetic-mech-for-star-manufacturing>: HTTP status code is not handled or not allowed 2026-01-25 06:04:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-ton-air-manual-service-jack-hw93667>: HTTP status code is not handled or not allowed 2026-01-25 06:04:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/100-ton-air-hydraulic-truck-jack-23991>: HTTP status code is not handled or not allowed 2026-01-25 06:04:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chateau-series-mahogany-laminate-desking-59x295x30-table>: HTTP status code is not handled or not allowed 2026-01-25 06:04:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-72-w-x-48-d-x-96-h-add-on-3-shelf-levels-wood-deck-medium-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:04:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-inch-x-20-foot-fertilizer-solution-suction-discharge-hose-assembly-w-plated-steel-king-nipple>: HTTP status code is not handled or not allowed 2026-01-25 06:04:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/charcoal-steel-desk-series-stack-on-hutch-48>: HTTP status code is not handled or not allowed 2026-01-25 06:04:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2631082-o-ring-wash-arm-2-inch-od-for-hobart>: HTTP status code is not handled or not allowed 2026-01-25 06:04:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-storage-rack-96-w-x-36-d-x-84-h-add-on-3-shelf-levels-wire-deck-medium-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:04:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-ton-adjustable-load-sling-hw93803>: HTTP status code is not handled or not allowed 2026-01-25 06:04:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-40-14w-cabinet-6-drawer-42-compart-light-gray-keyed-alike>: HTTP status code is not handled or not allowed 2026-01-25 06:04:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-ton-in-line-bottle-jack-10120>: HTTP status code is not handled or not allowed 2026-01-25 06:04:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-quiet-ceiling-exhaust-fan-w-ec-motor-and-speed-control-200-cfm-2-0-sones-120v>: HTTP status code is not handled or not allowed 2026-01-25 06:04:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-ton-service-jack-hw93657>: HTTP status code is not handled or not allowed 2026-01-25 06:04:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-ton-tripod-underhoist-stand-33020>: HTTP status code is not handled or not allowed 2026-01-25 06:04:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1761499-blender-tempest-64-oz-poly-for-hamilton-beach>: HTTP status code is not handled or not allowed 2026-01-25 06:04:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-20-x-3-inch-plain-finish-low-carbon-fully-threaded-stud>: HTTP status code is not handled or not allowed 2026-01-25 06:04:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-free-exercise-band-pear-level-0-6-yard-roll-box>: HTTP status code is not handled or not allowed 2026-01-25 06:04:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-band-loop-18-red>: HTTP status code is not handled or not allowed 2026-01-25 06:04:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25-ton-arbor-plates-pair-ap25>: HTTP status code is not handled or not allowed 2026-01-25 06:04:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-band-loop-8-green>: HTTP status code is not handled or not allowed 2026-01-25 06:04:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8013063-cold-control>: HTTP status code is not handled or not allowed 2026-01-25 06:04:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-free-exercise-band-black-25-yard-roll-box>: HTTP status code is not handled or not allowed 2026-01-25 06:04:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-40-14w-cabinet-12-drawer-177-compart-bright-blue-no-lock>: HTTP status code is not handled or not allowed 2026-01-25 06:04:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/band-aid-5570-antibiotic-adhesive-bandages-assorted-sizes-20box>: HTTP status code is not handled or not allowed 2026-01-25 06:04:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011822-grommet-for-bunn-o-matic>: HTTP status code is not handled or not allowed 2026-01-25 06:04:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-28-14w-mobile-cabinet7-drawer72-compart-classic-blue-individual-lock>: HTTP status code is not handled or not allowed 2026-01-25 06:04:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/handles-for-exercise-band-tubing-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 06:04:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-ae21b-black-epoxy-safety-push-handle-21w>: HTTP status code is not handled or not allowed 2026-01-25 06:04:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/expander-3-piece-set-red-green-blue-10-0046>: HTTP status code is not handled or not allowed 2026-01-25 06:04:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1021039-stopper-1-inch-rubber-for-standard-keil>: HTTP status code is not handled or not allowed 2026-01-25 06:04:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/braided-ground-strap-207110-025-1-0-gauge-25-pcs>: HTTP status code is not handled or not allowed 2026-01-25 06:04:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1591116-brush-hand-blue-w-scraper>: HTTP status code is not handled or not allowed 2026-01-25 06:04:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-40-14w-drawer-cabinet-6-drawer-72-compart-bright-blue-no-lock>: HTTP status code is not handled or not allowed 2026-01-25 06:04:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-40-14w-cabinet-9-drawer-120-compart-bright-blue-keyed-alike>: HTTP status code is not handled or not allowed 2026-01-25 06:04:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foam-padded-sports-handle-for-exercise-band-tubing-10-each>: HTTP status code is not handled or not allowed 2026-01-25 06:04:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-ah14z-poly-z-brite-utility-cart-handle-14w>: HTTP status code is not handled or not allowed 2026-01-25 06:04:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-al424s-stainless-steel-wire-ledge-24l-x-4h>: HTTP status code is not handled or not allowed 2026-01-25 06:04:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-band-handles-with-d-ring-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 06:04:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/negative-straight-clamp-connector-3420-005n-2-0-gauge-5-pcs>: HTTP status code is not handled or not allowed 2026-01-25 06:04:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/681154-fan-motor>: HTTP status code is not handled or not allowed 2026-01-25 06:04:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ladiess-traditional-length-white-65-polyester-35-cotton-poplin-lab-coat-size-44>: HTTP status code is not handled or not allowed 2026-01-25 06:04:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fkr-400-16-portable-direct-heat-sealer-with-temperature-control-10mm-seal-width>: HTTP status code is not handled or not allowed 2026-01-25 06:04:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-tin-plated-copper-lugs-5952-050dl-4-gauge-50-pcs>: HTTP status code is not handled or not allowed 2026-01-25 06:04:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/263850-screw-for-bj-t-stat>: HTTP status code is not handled or not allowed 2026-01-25 06:04:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7161016-screw-1-way-sheet-mtl-10-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 06:04:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010722-fryer-burner-for-imperial-cooking-equipment>: HTTP status code is not handled or not allowed 2026-01-25 06:04:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unisex-ciel-blue-cotton-polyester-fashion-blend-reversible-scrub-pants-size-xl>: HTTP status code is not handled or not allowed 2026-01-25 06:04:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/761008-flat-beater30qt-for-hobart>: HTTP status code is not handled or not allowed 2026-01-25 06:04:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1506030-dispenser-napkin-counter-stand>: HTTP status code is not handled or not allowed 2026-01-25 06:04:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unisex-navy-cotton-polyester-fashion-blend-reversible-set-in-sleeve-scrub-shirt-size-2xl>: HTTP status code is not handled or not allowed 2026-01-25 06:04:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-nylon-cable-ties-502112-100-141-long-100-pcs>: HTTP status code is not handled or not allowed 2026-01-25 06:04:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-free-exercise-band-6-yard-roll-box-5-color-set-10-6118>: HTTP status code is not handled or not allowed 2026-01-25 06:04:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-inplant-office-w-1-window-class-a-8-w-x-10-d-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:04:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ironwood-2-compartments-wardrobe-cabinet-black-grill-door-oiled-cherry-color>: HTTP status code is not handled or not allowed 2026-01-25 06:04:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blue-florescent-cable-ties-502205-100-85-100-pcs>: HTTP status code is not handled or not allowed 2026-01-25 06:04:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-powder-exercise-band-50-yard-rolls-5-color-set>: HTTP status code is not handled or not allowed 2026-01-25 06:04:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ironwood-3-compartment-solid-door-wood-storage-cabinet-maple-color>: HTTP status code is not handled or not allowed 2026-01-25 06:04:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-band-yellow-48-l-band>: HTTP status code is not handled or not allowed 2026-01-25 06:04:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1061180-aerator-faucet-05gpm-55-64thd>: HTTP status code is not handled or not allowed 2026-01-25 06:04:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-band-blue-50-yard-roll>: HTTP status code is not handled or not allowed 2026-01-25 06:04:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1031103-boot-toggle-switch-rubber-red-for-ultrafryer>: HTTP status code is not handled or not allowed 2026-01-25 06:04:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/starter-ground-magnalugs-6420-005h-1-2-5-pcs>: HTTP status code is not handled or not allowed 2026-01-25 06:04:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/red-dual-post-insulator-terminal-protectors-5713-025r-4-0-gauge-max-25-pcs>: HTTP status code is not handled or not allowed 2026-01-25 06:04:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conntek-20591-25-20-amp-generator-power-extension-cord-with-nema-l14-20p-to-l14-20r>: HTTP status code is not handled or not allowed 2026-01-25 06:04:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/77141-24-double-blade-ultra-hygiene-squeegee-pink>: HTTP status code is not handled or not allowed 2026-01-25 06:04:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1031104-boot-toggle-switch-rubber-gray-for-ultrafryer>: HTTP status code is not handled or not allowed 2026-01-25 06:04:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fireplace-grate-bg7-277m-27w-3-4-steel-6-bars>: HTTP status code is not handled or not allowed 2026-01-25 06:04:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leviton-26720-100-gx23-gx23-2-base-2-pin-compact-fluorescent-lampholder>: HTTP status code is not handled or not allowed 2026-01-25 06:04:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1241336-motor-ec-16w-ccw-1800-rpm-for-silver-king>: HTTP status code is not handled or not allowed 2026-01-25 06:04:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leviton-23452-a-slide-on-lamp-support-clip-for-horiz-2g11-base-twin-tube-fluorescent-lampholder>: HTTP status code is not handled or not allowed 2026-01-25 06:04:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/321509-o-ring-5-16-inch-id-x-1-16-inch-width>: HTTP status code is not handled or not allowed 2026-01-25 06:04:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/71602-24-single-blade-ultra-hygiene-squeegee-green>: HTTP status code is not handled or not allowed 2026-01-25 06:04:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-band-loop-exerciser-3-size-set-10-15-30-silver>: HTTP status code is not handled or not allowed 2026-01-25 06:04:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-free-exercise-band-black-6-yard-roll-box>: HTTP status code is not handled or not allowed 2026-01-25 06:04:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/262339-palnut-mounting-bracket>: HTTP status code is not handled or not allowed 2026-01-25 06:04:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2721269-orifice-burner-41-for-nieco-corp>: HTTP status code is not handled or not allowed 2026-01-25 06:04:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/263796-cap-end-round-tubing>: HTTP status code is not handled or not allowed 2026-01-25 06:04:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exercise-band-loop-exerciser-3-size-set-10-15-30-red>: HTTP status code is not handled or not allowed 2026-01-25 06:04:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/56861-3-gallon-bucket-pink>: HTTP status code is not handled or not allowed 2026-01-25 06:05:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slush-boots-size-14-rubber-yellow-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 06:05:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/69626-4-scraper-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:05:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s-1590-af-sentinel-mark-ll-shower-valve-combination-w-lever-handle>: HTTP status code is not handled or not allowed 2026-01-25 06:05:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/56601-dustpan-pink>: HTTP status code is not handled or not allowed 2026-01-25 06:05:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-free-exercise-band-yellow-50-yard-roll-box>: HTTP status code is not handled or not allowed 2026-01-25 06:05:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/561558-deck-mount-faucet>: HTTP status code is not handled or not allowed 2026-01-25 06:05:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/55103-handheld-cleaning-pad-holder-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:05:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/55005-cleaning-pad-holder-white>: HTTP status code is not handled or not allowed 2026-01-25 06:05:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/huot-toolscoot-hsk-100a-cnc-holders>: HTTP status code is not handled or not allowed 2026-01-25 06:05:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/53685-216-1-6-tube-brush-for-flex-rod-medium-white>: HTTP status code is not handled or not allowed 2026-01-25 06:05:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/huot-tap-dispenser-multiple-type>: HTTP status code is not handled or not allowed 2026-01-25 06:05:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fieldpro-purofort-full-safety-boots-steel-toe-14h-size-6-black>: HTTP status code is not handled or not allowed 2026-01-25 06:05:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-clothes-hook-bumper>: HTTP status code is not handled or not allowed 2026-01-25 06:05:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-free-exercise-band-5-band-light-yellow-red-green-retail-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:05:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rigpro174-purofort-full-safety-boots-steel-toe-12h-size-5-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:05:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 73158 pages (at 96 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:05:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/321006-gas-connector3-4-inch-mpt-x-36-inch-for-dormont-mfg>: HTTP status code is not handled or not allowed 2026-01-25 06:05:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-audio-visual-instrument-cart-32-x-24-x-40-three-shelves>: HTTP status code is not handled or not allowed 2026-01-25 06:05:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-audio-visual-instrument-cart-32-x-24-x-48-three-shelves>: HTTP status code is not handled or not allowed 2026-01-25 06:05:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-audio-visual-instrument-cart-32-x-24-x-54-three-shelves>: HTTP status code is not handled or not allowed 2026-01-25 06:05:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latex-free-exercise-band-5-strip-red-1-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:05:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/53913-216-3-5-pipe-brush-for-flex-rod-medium-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:05:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/seapro-purofort-non-safety-boots-metal-midsole-size-14-sea-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:05:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-tempered-glass-channel-frame-mirror-18-w-x-30-h>: HTTP status code is not handled or not allowed 2026-01-25 06:05:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metmax-acifort-full-safety-boots-w-metatarsal--ankle-protection-16h-size-5-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:05:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1361009-relay-24v-for-franke>: HTTP status code is not handled or not allowed 2026-01-25 06:05:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drum-tray-dolly-1>: HTTP status code is not handled or not allowed 2026-01-25 06:05:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-cylinder-truck-4-wheel-w-hoist-ring-10-1-2-pneumatic-wheels-1000-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:05:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/64408-small-hand-brush-stiff-purple>: HTTP status code is not handled or not allowed 2026-01-25 06:05:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1831148-damper-kit-2-sheets-4-springs-for-roundup-food-equipment>: HTTP status code is not handled or not allowed 2026-01-25 06:05:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-bracket-24w-x-4d-x-2h-2-cylinder-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:05:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-matrixseries-surface-mounted-seat-cover-dispenser>: HTTP status code is not handled or not allowed 2026-01-25 06:05:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/342015-heating-element-120v-400w-for-bevles>: HTTP status code is not handled or not allowed 2026-01-25 06:05:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-shelf-liner-42w-x-18d-white>: HTTP status code is not handled or not allowed 2026-01-25 06:05:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2051103-knob-timer-1-3-4-inch-od-187-inch-d-inch-for-hobart>: HTTP status code is not handled or not allowed 2026-01-25 06:05:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/342010-heating-element-kit208v-8kw-for-frymaster>: HTTP status code is not handled or not allowed 2026-01-25 06:05:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/53604-216-0-6-drain-cleaning-brush-stiff-red>: HTTP status code is not handled or not allowed 2026-01-25 06:05:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-duty-trailer-stabilizing-jack-stand>: HTTP status code is not handled or not allowed 2026-01-25 06:05:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1901388-funnel-brew-yellow-square-for-bunn-o-matic>: HTTP status code is not handled or not allowed 2026-01-25 06:05:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/77524-16-foam-blade-squeegee-red>: HTTP status code is not handled or not allowed 2026-01-25 06:05:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ada-rotating-pedestal-grill-with-3-1-2-dia-post-300-sq-cooking-surface>: HTTP status code is not handled or not allowed 2026-01-25 06:05:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-black-cable-tie>: HTTP status code is not handled or not allowed 2026-01-25 06:05:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/synthetic-winch-line-with-stainless-steel-thimble-716-dia-x-150l>: HTTP status code is not handled or not allowed 2026-01-25 06:05:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/52-bollard-flex-concrete-installation-white-cover-blue-tapes>: HTTP status code is not handled or not allowed 2026-01-25 06:05:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1901120-handle-nsf-for-bunn-o-matic>: HTTP status code is not handled or not allowed 2026-01-25 06:05:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8014215-terminal-block-kit-3-pole-ce-for-hatco-corp>: HTTP status code is not handled or not allowed 2026-01-25 06:05:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wood-deck-narrow-aisle-high-end-platform-truck-60-l-x-16-w>: HTTP status code is not handled or not allowed 2026-01-25 06:05:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/synthetic-winch-line-with-stainless-steel-thimble-58-dia-x-100l>: HTTP status code is not handled or not allowed 2026-01-25 06:05:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/52-bollard-flex-natural-ground-installation-white-cover-red-tapes>: HTTP status code is not handled or not allowed 2026-01-25 06:05:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5381-90-4-216-3-5-one-piece-pipe-brush-medium-stiff-red>: HTTP status code is not handled or not allowed 2026-01-25 06:05:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bobrick-channel-frame-mirror-24-w-x-30-h>: HTTP status code is not handled or not allowed 2026-01-25 06:05:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/45879-bench-brush-soft-black>: HTTP status code is not handled or not allowed 2026-01-25 06:05:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blue-hopper-front-bulk-truck-58-cu>: HTTP status code is not handled or not allowed 2026-01-25 06:05:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-shelf-liner-36w-x-24d-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:05:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gray-hopper-front-security-bulk-truck-48-cu>: HTTP status code is not handled or not allowed 2026-01-25 06:05:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-hopper-front-security-bulk-truck-58-cu>: HTTP status code is not handled or not allowed 2026-01-25 06:05:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fiberglass-handle-red-hfr>: HTTP status code is not handled or not allowed 2026-01-25 06:05:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/29644-67-ultra-hygiene-handle-red>: HTTP status code is not handled or not allowed 2026-01-25 06:05:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-x-18-x-60-phenolic-locker-single-tier-dove-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:05:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/synthetic-winch-line-with-stainless-steel-thimble-38-dia-x-100l>: HTTP status code is not handled or not allowed 2026-01-25 06:05:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/38905-large-hand-brush-stiff-white>: HTTP status code is not handled or not allowed 2026-01-25 06:05:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-x-15-x-60-phenolic-locker-double-tier-graphite-grafix>: HTTP status code is not handled or not allowed 2026-01-25 06:05:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/85-1077-insulated-set-screw-wire-connector-medium>: HTTP status code is not handled or not allowed 2026-01-25 06:05:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/line-voltage-thermostat-double-pole-heat-only-aet9dwts>: HTTP status code is not handled or not allowed 2026-01-25 06:05:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-w-x-30-d-production-workbench-esd-laminate-safety-edge-complete-bench-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:05:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-polyester-filter-14-w-x-14-h-x-1-d>: HTTP status code is not handled or not allowed 2026-01-25 06:05:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelate-wire-shelf-utility-cart-with-brakes-48x24-3-shelves-800-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:05:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/851285-paper-test-qt10-hydrion>: HTTP status code is not handled or not allowed 2026-01-25 06:05:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hp-cf210x-hp-131x-high-yield-toner-2400-page-yield-black>: HTTP status code is not handled or not allowed 2026-01-25 06:05:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/44-h-x-72-w-so-nylon-column-protector-black-cover-red-tapes>: HTTP status code is not handled or not allowed 2026-01-25 06:05:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brother-tn210m-toner-1400-page-yield-magenta>: HTTP status code is not handled or not allowed 2026-01-25 06:05:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-wire-shelf-truck-48x18x80-1200-pound-capacity-1>: HTTP status code is not handled or not allowed 2026-01-25 06:05:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-shelf-liner-18w-x-12d-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:05:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-wire-shelf-truck-60x24x80-1200-pound-capacity-with-brakes-1>: HTTP status code is not handled or not allowed 2026-01-25 06:05:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/31796-16-small-particle-push-broom-soft-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:05:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lexmark-50f1000-toner1500-page-yield-black>: HTTP status code is not handled or not allowed 2026-01-25 06:05:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/29626-59-ultra-hygiene-handle-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:05:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-w-x-36-d-production-workbench-maple-butcher-block-safety-edge-complete-bench-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:05:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2301048-dial-thermostat-flat-right-for-groen>: HTTP status code is not handled or not allowed 2026-01-25 06:05:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dinex-base-for-stsii-turbo-temp-induction-chargers-12-cs-dark-blue-dx821050>: HTTP status code is not handled or not allowed 2026-01-25 06:05:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/221615-knob>: HTTP status code is not handled or not allowed 2026-01-25 06:05:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dinex-wax-base-for-model-4731129-9-dxtmp1097a>: HTTP status code is not handled or not allowed 2026-01-25 06:05:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fir-plywood-premium-grade-crate-premium57-22-23-collapsible-inside-57l-x-22w-x-23>: HTTP status code is not handled or not allowed 2026-01-25 06:05:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1061027-adaptor-inlet-1-2-inch-nps-m-x2-inch>: HTTP status code is not handled or not allowed 2026-01-25 06:05:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/battery-powered-li-straddle-stacker-truck-with-fixed-legs-2000-lb-capacity-6>: HTTP status code is not handled or not allowed 2026-01-25 06:05:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fir-plywood-premium-grade-crate-premium76-34-23-collapsible-inside-76l-x-34w-x-23>: HTTP status code is not handled or not allowed 2026-01-25 06:05:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/collapsible-wire-mesh-steel-container-40-1-2x34-1-2x26-zinc-galv>: HTTP status code is not handled or not allowed 2026-01-25 06:05:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hydraulic-stacker-li-truck-1000-lb-capacity-with-24-x-24-platform-3>: HTTP status code is not handled or not allowed 2026-01-25 06:05:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/presto-stationary-li-table-foot-operated-2>: HTTP status code is not handled or not allowed 2026-01-25 06:05:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dinex-filigree-recycled-pattern-pocketfold-napkins-17-x-17-1200-cuss>: HTTP status code is not handled or not allowed 2026-01-25 06:05:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/56896-1-5-gallon-bucket-lid-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:05:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dinex-fenwick-insulated-bowl-5-oz-48-cs-cranberry-dx520061>: HTTP status code is not handled or not allowed 2026-01-25 06:05:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metal-access-door-24-x-24>: HTTP status code is not handled or not allowed 2026-01-25 06:05:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-access-door-22-x-22>: HTTP status code is not handled or not allowed 2026-01-25 06:05:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011694-bulb-230v-25w-for-moffat>: HTTP status code is not handled or not allowed 2026-01-25 06:05:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/outdoor-snow-ice-melting-heated-walkway-mat-3-x-10-120-volts>: HTTP status code is not handled or not allowed 2026-01-25 06:05:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fir-plywood-premium-grade-crate-premium28-22-29-collapsible-inside-28l-x-22w-x-29>: HTTP status code is not handled or not allowed 2026-01-25 06:05:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dinex-fenwick-insulated-mug-8-oz-48-cs-onyx-dx500003>: HTTP status code is not handled or not allowed 2026-01-25 06:06:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dinex-fenwick-insulated-bowl-9-oz-48-cs-cranberry-dx530061>: HTTP status code is not handled or not allowed 2026-01-25 06:06:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010227-knob-kit-for-garland-range>: HTTP status code is not handled or not allowed 2026-01-25 06:06:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96-x-30-shop-top-work-bench-adjustable-height-1-1-2-top>: HTTP status code is not handled or not allowed 2026-01-25 06:06:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/-spraybar-assembly-for-apex153-2000-portable-evaporative-coolers>: HTTP status code is not handled or not allowed 2026-01-25 06:06:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/321447-door-gasket-per-ft>: HTTP status code is not handled or not allowed 2026-01-25 06:06:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carson-optical-od-10-clip-flip-magnifier>: HTTP status code is not handled or not allowed 2026-01-25 06:06:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96-x-30-shop-top-work-bench-fixed-height-1-1-2-top>: HTTP status code is not handled or not allowed 2026-01-25 06:06:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dinex-turnbury-insulated-pedestal-base-mug-8-oz-48-cs-cranberry-dx300061>: HTTP status code is not handled or not allowed 2026-01-25 06:06:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bott-singlespring-clips-for-perfo-panels-package-of-5-34w>: HTTP status code is not handled or not allowed 2026-01-25 06:06:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pure-lye-1-pound>: HTTP status code is not handled or not allowed 2026-01-25 06:06:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/41865-long-handle-scrubbing-brush-stiff-white>: HTTP status code is not handled or not allowed 2026-01-25 06:06:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nickel-safe-1-gallon>: HTTP status code is not handled or not allowed 2026-01-25 06:06:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-36-plastic-safety-edge-work-bench-adjustable-height-1-5-8-top>: HTTP status code is not handled or not allowed 2026-01-25 06:06:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acrylic-sign-blue-line-forms-here>: HTTP status code is not handled or not allowed 2026-01-25 06:06:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/263663-nut-compression-5-16-inch-10-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:06:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 73255 pages (at 97 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:06:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/29155-13-push-broom-extra-stiff-white>: HTTP status code is not handled or not allowed 2026-01-25 06:06:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/small-cube-ice-machine-525lb>: HTTP status code is not handled or not allowed 2026-01-25 06:06:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011123-brush-fryer-basket-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:06:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-unit-black-yellow-24-black-yellow-belt>: HTTP status code is not handled or not allowed 2026-01-25 06:06:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dinex-turnbury-insulated-pedestal-based-bowl-9-oz-48-cs-onyx-dx330003>: HTTP status code is not handled or not allowed 2026-01-25 06:06:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storability-vertical-hang-rail-w-mounting-hardware>: HTTP status code is not handled or not allowed 2026-01-25 06:06:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hand-held-weighted-ball-yellow-1-kg-2-2-lb>: HTTP status code is not handled or not allowed 2026-01-25 06:06:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2902502-gba80500-80v-pro-series-50ah-battery>: HTTP status code is not handled or not allowed 2026-01-25 06:06:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forkli-able-shipping-container-with-lid-44-x-29-7>: HTTP status code is not handled or not allowed 2026-01-25 06:06:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-shopping-cart-corral-1>: HTTP status code is not handled or not allowed 2026-01-25 06:06:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conntek-20411-050-50-sjtw-12-3-outdoor-extension-cord-with-3-outlet-nema-5-15p-r>: HTTP status code is not handled or not allowed 2026-01-25 06:06:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forkli-able-shipping-container-with-lid-51-x-19-7>: HTTP status code is not handled or not allowed 2026-01-25 06:06:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bar-wall-rack-for-exercise-weight-bars-10-hook-rack>: HTTP status code is not handled or not allowed 2026-01-25 06:06:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-ss-nsf-wall-shelf-22>: HTTP status code is not handled or not allowed 2026-01-25 06:06:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/americana-climbing-peak-safety-helmet-type-1-4-point-mega-ratchet-suspension-orange>: HTTP status code is not handled or not allowed 2026-01-25 06:06:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-ss-nsf-wall-shelf-26>: HTTP status code is not handled or not allowed 2026-01-25 06:06:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hw-h6062a1000-humidipro-digital-humidity-control-h6062a1000>: HTTP status code is not handled or not allowed 2026-01-25 06:06:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-utility-cart-3-shelves-36x24>: HTTP status code is not handled or not allowed 2026-01-25 06:06:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-ss-nsf-wall-shelf-23>: HTTP status code is not handled or not allowed 2026-01-25 06:06:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cfc10025-socket-cap-screw>: HTTP status code is not handled or not allowed 2026-01-25 06:06:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industries-handrail-bracket-for-15-tubing-polished-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 06:06:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-storage-cabinet-36x22x46-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:06:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/63-h-nexel-chrome-wire-shelving-36-w-x-14-d>: HTTP status code is not handled or not allowed 2026-01-25 06:06:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-cu-yd-self-dumping-steel-hopper-with-bump-release-4000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:06:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-force-rug-5-x-8-60-inch-x-92-inch>: HTTP status code is not handled or not allowed 2026-01-25 06:06:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1221041-cylinder-lock>: HTTP status code is not handled or not allowed 2026-01-25 06:06:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1191097-foot-gry-plst-f-1-1-2-inch-od-rd>: HTTP status code is not handled or not allowed 2026-01-25 06:06:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/water-filter-manifold-designed-for-ice-makers-producing-up-to-1-000-lbs-w-special-scale-inhibitor>: HTTP status code is not handled or not allowed 2026-01-25 06:06:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/86-h-nexel-chrome-wire-shelving-60-w-x-36-d>: HTTP status code is not handled or not allowed 2026-01-25 06:06:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cmc04016-machine-screw>: HTTP status code is not handled or not allowed 2026-01-25 06:06:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industries-flush-tee-fitting-side-outlet-for-15-tubing-satin-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 06:06:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ice-maker-soft-chewable-ice-crystals-approx-1053-lb-production-chewable-ice-crystals>: HTTP status code is not handled or not allowed 2026-01-25 06:06:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poly-oxford-insulated-pull-on-safety-pants-3xl-black>: HTTP status code is not handled or not allowed 2026-01-25 06:06:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a323-portable-dissolved-oxygen-meter-kit-stara3235>: HTTP status code is not handled or not allowed 2026-01-25 06:06:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slotted-turner-black>: HTTP status code is not handled or not allowed 2026-01-25 06:06:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-cube-ice-maker-approx-334-lb-production-full-size-cube>: HTTP status code is not handled or not allowed 2026-01-25 06:06:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-oz-solid-oval-spoodle-black>: HTTP status code is not handled or not allowed 2026-01-25 06:06:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/120-l-wheel-alignment-guides-pair-of-2>: HTTP status code is not handled or not allowed 2026-01-25 06:06:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ripstop-insulated-safety-bib-overall-3xl-black>: HTTP status code is not handled or not allowed 2026-01-25 06:06:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cdc06070-socket-cap-screw>: HTTP status code is not handled or not allowed 2026-01-25 06:06:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stacking-bin-16-1-2-x-18-x-11-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:06:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-quart-mix-bowl-stand>: HTTP status code is not handled or not allowed 2026-01-25 06:06:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-handle-medicine-ball-18-lb-9-dia-23-cm-green>: HTTP status code is not handled or not allowed 2026-01-25 06:06:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelite-vented-plastic-mat-shelf-truck-48l-x-18w-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 06:06:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poly-oxford-insulated-safety-bib-overall-xl-fluorescent-green>: HTTP status code is not handled or not allowed 2026-01-25 06:06:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-handle-medicine-ball-18-lb-9-diameter-gold>: HTTP status code is not handled or not allowed 2026-01-25 06:06:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelite-vented-plastic-mat-shelf-truck-72l-x-18w-x-60h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 06:06:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bin-nestable-4-1-8-w-x-23-5-8-d-x-4-hgreen>: HTTP status code is not handled or not allowed 2026-01-25 06:06:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plyometric-ball-rack-vertical-5-ball-capacity-12l-x-12w-x-58h>: HTTP status code is not handled or not allowed 2026-01-25 06:06:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bin-nestable-8-3-8-w-x-11-5-8-d-x-4-h-beige>: HTTP status code is not handled or not allowed 2026-01-25 06:06:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/firm-medicine-ball-30-lb-11-diameter-gold>: HTTP status code is not handled or not allowed 2026-01-25 06:06:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bin-nestable-11-1-8-w-x-17-7-8-d-x-4-h-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:06:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelite-vented-plastic-mat-shelf-truck-60l-x-24w-x-60h-polyurethane-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 06:06:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelite-vented-plastic-mat-shelf-truck-72l-x-24w-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 06:06:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bsc12-hex-jam-nut>: HTTP status code is not handled or not allowed 2026-01-25 06:06:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insulated-flex-safety-pants-xl-fluorescent-orange>: HTTP status code is not handled or not allowed 2026-01-25 06:06:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esd-lid-for-stacking-box-25-3-4l-x-17-3-4w>: HTTP status code is not handled or not allowed 2026-01-25 06:06:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/561546-wall-mount-faucet-economy-w-6-inch-spout>: HTTP status code is not handled or not allowed 2026-01-25 06:06:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cfc05010-socket-cap-screw>: HTTP status code is not handled or not allowed 2026-01-25 06:06:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/relaxed-fit-flex-twill-cargo-pants-34l-inseam-x-32-waist-black>: HTTP status code is not handled or not allowed 2026-01-25 06:06:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adaptor-plate-5>: HTTP status code is not handled or not allowed 2026-01-25 06:06:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-in-nickel-chrome-plated-guard>: HTTP status code is not handled or not allowed 2026-01-25 06:06:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelite-vented-plastic-mat-shelf-truck-48l-x-18w-x-69h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 06:06:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-w-machinery-wire-fence-partition-panel>: HTTP status code is not handled or not allowed 2026-01-25 06:06:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bmj06020-socket-cap-screw>: HTTP status code is not handled or not allowed 2026-01-25 06:06:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-in-cast-aluminum-propeller-5-8-in-bore>: HTTP status code is not handled or not allowed 2026-01-25 06:06:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelite-vented-plastic-mat-shelf-truck-48l-x-18w-x-69h-polyurethane-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 06:06:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/relaxed-fit-flex-twill-cargo-pants-30l-inseam-x-42-waist-black>: HTTP status code is not handled or not allowed 2026-01-25 06:06:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelite-vented-plastic-mat-shelf-truck-42l-x-18w-x-80h-polyurethane-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 06:06:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/servewell-double-deck-overshelf-1>: HTTP status code is not handled or not allowed 2026-01-25 06:06:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industries-flange-wall-for-1-tubing-satin-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 06:06:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11203-6-internal-double-leg-shock-lanyard-steel-rebar-hook>: HTTP status code is not handled or not allowed 2026-01-25 06:06:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bishamon-self-leveling-work-positioner-2>: HTTP status code is not handled or not allowed 2026-01-25 06:06:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-shelf-for-12-gauge-heavy-duty-cabinet-42-wx24-d>: HTTP status code is not handled or not allowed 2026-01-25 06:06:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-l-power-supply-with-mounting-rail-beige>: HTTP status code is not handled or not allowed 2026-01-25 06:06:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009786-hp-wheel-only-caster-for-henny-penny>: HTTP status code is not handled or not allowed 2026-01-25 06:06:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-shelf-for-12-gauge-heavy-duty-cabinet-36-wx18-d>: HTTP status code is not handled or not allowed 2026-01-25 06:06:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/Barrel-Key-Switch-Single-Pole-20A-120-277V-AC-Rotary-Locking>: HTTP status code is not handled or not allowed 2026-01-25 06:06:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1000g-weight>: HTTP status code is not handled or not allowed 2026-01-25 06:06:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kennedy-10-hook-1>: HTTP status code is not handled or not allowed 2026-01-25 06:07:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/660lb-counting-scale>: HTTP status code is not handled or not allowed 2026-01-25 06:07:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-file-folders-15-cut-assorted-one-ply-top-tab-letter-manila-100box>: HTTP status code is not handled or not allowed 2026-01-25 06:07:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2231307-plate-ejector-std-trk2>: HTTP status code is not handled or not allowed 2026-01-25 06:07:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kennedy-6-drawer-mechanic-brown>: HTTP status code is not handled or not allowed 2026-01-25 06:07:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010171-screw-on-gasket-1825x-2125>: HTTP status code is not handled or not allowed 2026-01-25 06:07:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ripstop-safety-rain-jacket-csa-class-1-level-2-xl-black>: HTTP status code is not handled or not allowed 2026-01-25 06:07:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pull-on-ripstop-safety-rain-pants-csa-class-3-level-2-3xl-fluorescent-green>: HTTP status code is not handled or not allowed 2026-01-25 06:07:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insulated-safety-coverall-5xl-navy>: HTTP status code is not handled or not allowed 2026-01-25 06:07:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kennedy-7-drawer-roller-friction-red-2>: HTTP status code is not handled or not allowed 2026-01-25 06:07:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/relaxed-fit-flex-twill-cargo-pants-30l-inseam-x-28-waist-black>: HTTP status code is not handled or not allowed 2026-01-25 06:07:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glass-panel-lift-beam-economy-multiple-length-22876-10000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:07:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/54-h-nexel-chrome-wire-shelving-add-on-48-w-x-14-d>: HTTP status code is not handled or not allowed 2026-01-25 06:07:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011824-eterna-cartridge-new-stlye-rtc-for-ts-brass-bronze-works>: HTTP status code is not handled or not allowed 2026-01-25 06:07:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smead-lockit-two-pocket-folder-leatherette-stock-11-x-8-12-dark-blue-25box>: HTTP status code is not handled or not allowed 2026-01-25 06:07:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/55-oz-airhorn-with-power-pack-10-425>: HTTP status code is not handled or not allowed 2026-01-25 06:07:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 73350 pages (at 95 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:07:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/relaxed-fit-flex-twill-cargo-pants-32l-inseam-x-38-waist-black>: HTTP status code is not handled or not allowed 2026-01-25 06:07:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/relaxed-fit-flex-twill-cargo-pants-32l-inseam-x-50-waist-navy>: HTTP status code is not handled or not allowed 2026-01-25 06:07:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/through-thread-plastic-hand-knobs-m6x10-thread-32mm-knob-dia-20mm-knob-height>: HTTP status code is not handled or not allowed 2026-01-25 06:07:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sourcing-reed-switch-1-amp-tie-rod-3m-cbl>: HTTP status code is not handled or not allowed 2026-01-25 06:07:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/360-stretch-waist-flex-twill-cargo-pants-30l-inseam-x-28-29-waist-black>: HTTP status code is not handled or not allowed 2026-01-25 06:07:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/54-h-nexel-chrome-wire-shelving-60-w-x-36-d>: HTTP status code is not handled or not allowed 2026-01-25 06:07:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hubbel-industrial-single-outlet-16-3-35>: HTTP status code is not handled or not allowed 2026-01-25 06:07:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ms1-angle-mount-for-4-bore-nfpa-cylinder>: HTTP status code is not handled or not allowed 2026-01-25 06:07:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/63-h-nexel-chrome-wire-shelving-add-on-54-w-x-24-d>: HTTP status code is not handled or not allowed 2026-01-25 06:07:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/74-h-nexel-chrome-wire-shelving-30-w-x-18-d>: HTTP status code is not handled or not allowed 2026-01-25 06:07:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hubbel-commercial-14-3-40>: HTTP status code is not handled or not allowed 2026-01-25 06:07:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smead-3-expansion-folder-two-fasteners-end-tab-letter-gray-green-25box>: HTTP status code is not handled or not allowed 2026-01-25 06:07:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smead-file-folders-13-cut-top-tab-letter-bright-assorted-colors-100box>: HTTP status code is not handled or not allowed 2026-01-25 06:07:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smead-3-12-accordion-expansion-end-tab-file-pocket-straight-tab-lgl-manila-25bx>: HTTP status code is not handled or not allowed 2026-01-25 06:07:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14765-5-gallon-safety-can-polyethylene-red>: HTTP status code is not handled or not allowed 2026-01-25 06:07:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/86-h-nexel-chrome-wire-shelving-24-w-x-18-d>: HTTP status code is not handled or not allowed 2026-01-25 06:07:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clamps-wilton14714>: HTTP status code is not handled or not allowed 2026-01-25 06:07:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/86-h-nexel-chrome-wire-shelving-add-on-42-w-x-18-d>: HTTP status code is not handled or not allowed 2026-01-25 06:07:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-125mm-bore-x-160mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 06:07:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smead-2-accordion-expansion-wallet-with-cord-letter-redrope>: HTTP status code is not handled or not allowed 2026-01-25 06:07:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transparent-vinyl-welding-curtain-6w-x-10h-orange>: HTTP status code is not handled or not allowed 2026-01-25 06:07:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-100mm-bore-x-30mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 06:07:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/relaxed-fit-flex-twill-cargo-pants-30l-inseam-x-46-waist-navy>: HTTP status code is not handled or not allowed 2026-01-25 06:07:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/264062-grinder-plate-1-4-inch>: HTTP status code is not handled or not allowed 2026-01-25 06:07:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clamps-columbian-41407>: HTTP status code is not handled or not allowed 2026-01-25 06:07:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/63-h-nexel-chrome-wire-shelving-add-on-60-w-x-18-d>: HTTP status code is not handled or not allowed 2026-01-25 06:07:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silicone-coated-fiberglass-welding-blanket-6w-x-6h-red>: HTTP status code is not handled or not allowed 2026-01-25 06:07:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jam-nut-m27-x-2-ch41-sp12-for-iso-cylinders>: HTTP status code is not handled or not allowed 2026-01-25 06:07:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/relaxed-fit-flex-twill-cargo-pants-without-hem-42-waist-black>: HTTP status code is not handled or not allowed 2026-01-25 06:07:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vermiculite-coated-fiberglass-welding-blanket-8w-x-8h-black>: HTTP status code is not handled or not allowed 2026-01-25 06:07:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/350017-jmd-15-milling-drilling-machine-w-r-8-taper-1hp-115-230v-1-phase>: HTTP status code is not handled or not allowed 2026-01-25 06:07:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/type-r-class-2-heavy-duty-surveyor-safety-vest-zipper-m-green-sv59z-2zgd-m>: HTTP status code is not handled or not allowed 2026-01-25 06:07:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012916-knob-assembly-t-stat-200-550-deg-f-for-garland-range>: HTTP status code is not handled or not allowed 2026-01-25 06:07:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/riser-tapered-carpet-66l-x-18w-x-16h-grey>: HTTP status code is not handled or not allowed 2026-01-25 06:07:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vises-wilton-63302>: HTTP status code is not handled or not allowed 2026-01-25 06:07:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vises-wilton-63300>: HTTP status code is not handled or not allowed 2026-01-25 06:07:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-poly-oxford-5-in-1-safety-jacket-3xlt-fluorescent-orange>: HTTP status code is not handled or not allowed 2026-01-25 06:07:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/type-r-class-2-heavy-duty-surveyor-safety-vest-snap-m-orange-sv59-2zod-m>: HTTP status code is not handled or not allowed 2026-01-25 06:07:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/terry-hand-mitts-for-paraffin-treatment-6-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:07:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stage-pie-set-carpet-48w-stage-units-8h-16h-24h-grey>: HTTP status code is not handled or not allowed 2026-01-25 06:07:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-125mm-bore-x-300mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 06:07:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rg11-u-14awg-flooded-outdoor-direct-burial-500-ft-spool-black>: HTTP status code is not handled or not allowed 2026-01-25 06:07:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/type-r-class-2-heavy-duty-surveyor-safety-vest-zipper-xl-orange-sv59z-2zod-xl>: HTTP status code is not handled or not allowed 2026-01-25 06:07:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cp1212-wall-mounted-panel-10-187-x-10-187-fits-12-x-12-box-white>: HTTP status code is not handled or not allowed 2026-01-25 06:07:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-poly-oxford-5-in-1-safety-jacket-m-fluorescent-green>: HTTP status code is not handled or not allowed 2026-01-25 06:07:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-wire-shelf-36x24>: HTTP status code is not handled or not allowed 2026-01-25 06:07:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kit-clevis-bracket-mount-al-63-for-iso-15552-cylinders>: HTTP status code is not handled or not allowed 2026-01-25 06:07:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/23awg-4-pr-utp-cat6e-550-mhz-cmr-1000-ft-box-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:07:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/moist-heat-pack-cover-foam-filled-terry-half-size-12-x-15-12-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:07:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-duty-flat-heating-blanket-md0520-120v-21l-x-6w>: HTTP status code is not handled or not allowed 2026-01-25 06:07:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-ripstop-fleece-lined-safety-jacket-l-fluorescent-green>: HTTP status code is not handled or not allowed 2026-01-25 06:07:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/28-1676-sight-gauge-glass-tube-red-and-white-stripe-5-8-inch-x-12-inch>: HTTP status code is not handled or not allowed 2026-01-25 06:07:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-male-connector-60000-05-02-516-tube-x-18-swift-fit-univ-thread-ss>: HTTP status code is not handled or not allowed 2026-01-25 06:07:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-ripstop-fleece-lined-safety-jacket-xl-fluorescent-orange>: HTTP status code is not handled or not allowed 2026-01-25 06:07:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1502510-pump-ultra-9-inch-dip-tube-2-pk-for-san-jamar>: HTTP status code is not handled or not allowed 2026-01-25 06:07:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/moist-heat-pack-half-size-5-x-12-11-1313>: HTTP status code is not handled or not allowed 2026-01-25 06:07:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cepnc-b-custom-molded-earplug-black-neckcord-w-screws-each>: HTTP status code is not handled or not allowed 2026-01-25 06:07:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inline-needle-valve-82830-06-38-tube-nickel-plated-brass>: HTTP status code is not handled or not allowed 2026-01-25 06:07:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8ft-long-box-pleat-skirting-16h-stage-green>: HTTP status code is not handled or not allowed 2026-01-25 06:07:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2221323-lid-container-mx1000xt-1100xt-for-waring-products>: HTTP status code is not handled or not allowed 2026-01-25 06:07:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-male-89000-53-02-532-tube-x-18-swift-fit-univ-thread-metallic-collet>: HTTP status code is not handled or not allowed 2026-01-25 06:07:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-stage-carpet-96l-x-36w-x-32h-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:07:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paraffin-bath-refill-36-lb-blocks-lavender-fragrance>: HTTP status code is not handled or not allowed 2026-01-25 06:07:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-male-50000n-6-12-6mm-tube-x-12-swift-fit-univ-thread>: HTTP status code is not handled or not allowed 2026-01-25 06:07:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bar-mop-towel-32-oz-16x19>: HTTP status code is not handled or not allowed 2026-01-25 06:07:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/snap-sleeves-18-pairs-per-package>: HTTP status code is not handled or not allowed 2026-01-25 06:07:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-32mm-bore-x-1300mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 06:07:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speedy-board-dock-board-2>: HTTP status code is not handled or not allowed 2026-01-25 06:07:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-control-89953-04-02-screw-adj-14-tube-x-18-swift-fit-metallic-collet>: HTTP status code is not handled or not allowed 2026-01-25 06:07:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rapi-kool-cold-paddle-64-oz-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:07:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-additional-shelf-level-boltless-43>: HTTP status code is not handled or not allowed 2026-01-25 06:07:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galvanized-garbage-can-lid-3>: HTTP status code is not handled or not allowed 2026-01-25 06:07:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galvanized-garbage-can-lid-6>: HTTP status code is not handled or not allowed 2026-01-25 06:07:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-quilted-safety-freezer-jacket-lt-fluorescent-orange>: HTTP status code is not handled or not allowed 2026-01-25 06:07:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-40mm-bore-x-185mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 06:07:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-coupler-80621-06-14-x-38-male-nptf>: HTTP status code is not handled or not allowed 2026-01-25 06:07:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insulated-cold-gear-overall-5xl-black>: HTTP status code is not handled or not allowed 2026-01-25 06:07:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-coupler-80621-08-14-x-12-male-nptf>: HTTP status code is not handled or not allowed 2026-01-25 06:07:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-water-leak-locator-system>: HTTP status code is not handled or not allowed 2026-01-25 06:07:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/264615-burner-shield-weldamnet-for-duke-manufacturing>: HTTP status code is not handled or not allowed 2026-01-25 06:07:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-z-brite-wire-shelving-60-w-x-24-d-x-74-h>: HTTP status code is not handled or not allowed 2026-01-25 06:07:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-z-brite-wire-shelving-30-w-x-18-d-x-86-h>: HTTP status code is not handled or not allowed 2026-01-25 06:07:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/female-cross-82620n-04-14-nptf-nickel-plated-brass>: HTTP status code is not handled or not allowed 2026-01-25 06:08:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-red-water-proof-temperature-sensor>: HTTP status code is not handled or not allowed 2026-01-25 06:08:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-quilted-safety-freezer-jacket-s-fluorescent-green>: HTTP status code is not handled or not allowed 2026-01-25 06:08:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-control-89968-08-06-knob-adj-12-tube-x-38-swift-fit-metallic-collet>: HTTP status code is not handled or not allowed 2026-01-25 06:08:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-50mm-bore-x-580mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 06:08:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-run-tee-55222-6-14-6mm-tube-x-14-swift-fit-univ-thread-composite-body>: HTTP status code is not handled or not allowed 2026-01-25 06:08:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-record-storage-rack-gray-48-wx-48-d-x-84-h-with-polyethylene-file-boxes>: HTTP status code is not handled or not allowed 2026-01-25 06:08:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-z-brite-wire-shelving-add-on-42-w-x-24-d-x-63-h>: HTTP status code is not handled or not allowed 2026-01-25 06:08:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-quilted-safety-freezer-jacket-2xl-black>: HTTP status code is not handled or not allowed 2026-01-25 06:08:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-black-urethane-reusable-cold-pack-circular-10-diameter>: HTTP status code is not handled or not allowed 2026-01-25 06:08:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-control-55915-5-14-knob-adj-5mm-tube-x-14-swift-fit-composite-body>: HTTP status code is not handled or not allowed 2026-01-25 06:08:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/851094-silicone-sealant>: HTTP status code is not handled or not allowed 2026-01-25 06:08:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-outdoor-ashtray-1-gallon>: HTTP status code is not handled or not allowed 2026-01-25 06:08:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-ram-valve-hose-adapter-case>: HTTP status code is not handled or not allowed 2026-01-25 06:08:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-additional-level-wide-span-rack-96x36-wood-deck-800-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:08:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2uc-2-u-cutter>: HTTP status code is not handled or not allowed 2026-01-25 06:08:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 73448 pages (at 98 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:08:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-50mm-bore-x-55mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 06:08:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-z-brite-wire-shelving-add-on-36-w-x-24-d-x-86-h>: HTTP status code is not handled or not allowed 2026-01-25 06:08:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-control-89953-08-08-screw-adj-12-tube-x-12-swift-fit-metallic-collet>: HTTP status code is not handled or not allowed 2026-01-25 06:08:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vollrath-lobsternut-cracker-forged-ss>: HTTP status code is not handled or not allowed 2026-01-25 06:08:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-run-tee-55222-12-14-12mm-tube-x-14-swift-fit-univ-thread-composite-body>: HTTP status code is not handled or not allowed 2026-01-25 06:08:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wraps-2-1-2-x-18-blue-package-of-3>: HTTP status code is not handled or not allowed 2026-01-25 06:08:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-extra-shelving-48x18x60-with-4-shelves-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 06:08:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0005-inch-steel-shim-stock-8-inch-x-12-inch-flat-sheets-pack-of-5>: HTTP status code is not handled or not allowed 2026-01-25 06:08:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8mm-id-x-10mm-od-x-025mm-die-button-shim-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-25 06:08:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/instant-reusable-hot-compress-standard-5-x-15-12-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:08:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-male-connector-60020-4-m5-4mm-tube-x-m5-bspp-thread-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 06:08:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-control-55905-12-38-knob-adj-12mm-tube-x-38-swift-fit-composite-body>: HTTP status code is not handled or not allowed 2026-01-25 06:08:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1132-inch-id-x-1316-inch-od-x-0006-inch-steel-arbor-shim-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 06:08:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-inch-id-x-2-34-inch-od-x-00015-inch-steel-arbor-spacer-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 06:08:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/075-inch-id-x-0999-inch-od-x-0015-inch-die-button-shim-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-25 06:08:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-6432-single-acting-magnetic-cylinder-8mm-bore-x-10mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 06:08:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eagle-paint-ink-safety-cabinet-manual-close-60-gallon>: HTTP status code is not handled or not allowed 2026-01-25 06:08:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-control-89958-04-04-knob-adj-14-tube-x-14-swift-fit-metallic-collet>: HTTP status code is not handled or not allowed 2026-01-25 06:08:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crane-hoist-drum-dumper-55-gallon-steel-drum>: HTTP status code is not handled or not allowed 2026-01-25 06:08:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-heavy-duty-shelving-36x18x60-with-4-shelves-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 06:08:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eagle-drum-safety-cabinet-manual-close-110-gallon>: HTTP status code is not handled or not allowed 2026-01-25 06:08:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-15552-double-acting-magnetic-cylinder-32mm-bore-x-280mm-stroke>: HTTP status code is not handled or not allowed 2026-01-25 06:08:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/85-1013-nickel-plated-female-quick-disconnect-1-4-inch-tab-wire-gauge-10-100-box>: HTTP status code is not handled or not allowed 2026-01-25 06:08:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hoover-vacuum-replacement-bags-3-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:08:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-heavy-duty-shelving-36x12x72-with-5-shelves-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 06:08:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/moto-vest-4-pockets-cottonpolyester-4xl-black>: HTTP status code is not handled or not allowed 2026-01-25 06:08:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eagle-type-i-stainless-safety-can-25-gallons>: HTTP status code is not handled or not allowed 2026-01-25 06:08:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10600-universal-standing-seam-roof-clamp-2-way-galvanized-steel-130-310-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:08:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/american-std-showerhead>: HTTP status code is not handled or not allowed 2026-01-25 06:08:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-step-roll-fold-steel-rolling-ladder-grip-strut>: HTTP status code is not handled or not allowed 2026-01-25 06:08:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/diablo-11070-8-self-retracting-lifeline-leading-edge-galvanized-steel-swivel-snap-hook>: HTTP status code is not handled or not allowed 2026-01-25 06:08:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/light-deburring-wheel-8-inch-x-1-2-inch-x-3-inch-fin-7s-grit-silicon-carbide>: HTTP status code is not handled or not allowed 2026-01-25 06:08:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/00230-bolt-hole-anchor-galvanized-steel-130-420-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:08:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cyclone-21035-construction-harness-quick-connect-chest-legs-tongue-buckle-waist-xl>: HTTP status code is not handled or not allowed 2026-01-25 06:08:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fibre-disc-tr-982c-2-inch-dia-80-grit-ceramic-grain>: HTTP status code is not handled or not allowed 2026-01-25 06:08:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clean-and-finish-disc-6-inch-x-1-2-inch-vfn-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-25 06:08:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-extra-heavy-duty-boltless-shelving-add-on-48-in-w-x-12-in-d-x-72-in-h>: HTTP status code is not handled or not allowed 2026-01-25 06:08:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/61141-2-x-4-stair-mount-powder-coated-steel-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:08:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1412134-belt-pak-f-table-turner-wipes>: HTTP status code is not handled or not allowed 2026-01-25 06:08:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plus-cut-and-polish-disc-d5-4-inch-x-1-1-4-in-med-grit-alum>: HTTP status code is not handled or not allowed 2026-01-25 06:08:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-record-storage-rack-gray-72x30x84>: HTTP status code is not handled or not allowed 2026-01-25 06:08:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009629-20a-sgl-pole-breaker-bolt-on>: HTTP status code is not handled or not allowed 2026-01-25 06:08:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plug-8610-5-5mm-nylon>: HTTP status code is not handled or not allowed 2026-01-25 06:08:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/union-connnector-85040-02-18-tube-composite-body>: HTTP status code is not handled or not allowed 2026-01-25 06:08:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/union-connnector-85040-04-14-tube-composite-body>: HTTP status code is not handled or not allowed 2026-01-25 06:08:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/container-dolly-f12-c-26000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:08:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-extra-shelving-36x12x96-gray-with-5-shelves-1500-lb-cap-per-shelf>: HTTP status code is not handled or not allowed 2026-01-25 06:08:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-additional-shelf-with-laminated-deck-72x36-gray-2>: HTTP status code is not handled or not allowed 2026-01-25 06:08:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-additional-shelf-with-laminated-deck-72x24-gray-1>: HTTP status code is not handled or not allowed 2026-01-25 06:08:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disc-pad-face-plate-ribbed-81732-5-inch-extra-hard-red>: HTTP status code is not handled or not allowed 2026-01-25 06:08:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-bulk-rack-shelf-without-deck-48x24-gray-1>: HTTP status code is not handled or not allowed 2026-01-25 06:08:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-20-ga-48-wx30-dx73-h-open-clip-style-5-shelf>: HTTP status code is not handled or not allowed 2026-01-25 06:08:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0020-inch-steel-shim-stock-12-inch-x-120-inch-roll>: HTTP status code is not handled or not allowed 2026-01-25 06:08:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-elbow-85110-02-02-18-tube-x-18-swift-fit-univ-thread-composite-body>: HTTP status code is not handled or not allowed 2026-01-25 06:08:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/93281-6-traffic-cone-chain-kit-w-reflective-collar-28-cone-height-hdpe-pvc-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:08:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/locking-plug-30a-125-250v-l14-30pb-w>: HTTP status code is not handled or not allowed 2026-01-25 06:08:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jet-tech-undercounter-high-temperature-cup-glass-washer>: HTTP status code is not handled or not allowed 2026-01-25 06:08:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultrasonic-thickness-gauge-kit-with-5-step-calibration-block-0004-resolution>: HTTP status code is not handled or not allowed 2026-01-25 06:08:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-145-tefc-1ph-3450rpm-c145k34fb12ee>: HTTP status code is not handled or not allowed 2026-01-25 06:08:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50107-plastic-barrier-chain-on-a-reel-hdpe-2-x125-8-51mm-gold>: HTTP status code is not handled or not allowed 2026-01-25 06:08:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-20-ga-48-wx30-dx97-h-open-clip-style-5-shelf>: HTTP status code is not handled or not allowed 2026-01-25 06:08:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0031-inch-steel-shim-stock-6-inch-x-18-inch-flat-sheets-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 06:08:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-20-ga-36-wx12-dx97-h-open-clip-style-7-shelf>: HTTP status code is not handled or not allowed 2026-01-25 06:08:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/union-elbow-55130-5-5mm-tube-composite-body>: HTTP status code is not handled or not allowed 2026-01-25 06:08:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1201079-caster-stem-5-inch-gry>: HTTP status code is not handled or not allowed 2026-01-25 06:08:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0005-inch-brass-shim-stock-6-inch-x-180-inch-roll>: HTTP status code is not handled or not allowed 2026-01-25 06:08:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-56h-tefc-1ph-3450rpm-p6c34fb23hh>: HTTP status code is not handled or not allowed 2026-01-25 06:08:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-seat-with-arm-rests-headrest--low-profile-suspension-vinyl-black>: HTTP status code is not handled or not allowed 2026-01-25 06:08:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30010-500-plastic-barrier-chain-hdpe-1-5-x500-6-38mm-brown>: HTTP status code is not handled or not allowed 2026-01-25 06:08:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deposit-vault-mailbox-and-parcel-drop-dvcs0020-free-standing-rear-access-black>: HTTP status code is not handled or not allowed 2026-01-25 06:08:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b16h-part-ss-worm-gear-hose-clamp-1116-inch-1-12-inch-clamping-diameter-box-of-10>: HTTP status code is not handled or not allowed 2026-01-25 06:08:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50030-100-alternating-plastic-barrier-chain-hdpe-2-x100-8-51mm-yellow-pink>: HTTP status code is not handled or not allowed 2026-01-25 06:08:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30066-100-plastic-barrier-chain-hdpe-1-5-x100-6-38mm-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:08:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/female-elbow-89105-53-04-532-tube-x-14-nptf-female-metallic-release-collet>: HTTP status code is not handled or not allowed 2026-01-25 06:08:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m6-x-1-0-x-20mm-socket-head-cap-screw-304-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 06:08:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50109-plastic-barrier-chain-on-a-reel-hdpe-2-x125-8-51mm-gold>: HTTP status code is not handled or not allowed 2026-01-25 06:08:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-male-elbow-60115-4-m5-4mm-tube-x-m5-thread-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 06:08:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5031104-lower-door-hinge-for-hoshizaki-of-america>: HTTP status code is not handled or not allowed 2026-01-25 06:08:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50036-100-alternating-plastic-barrier-chain-hdpe-2-x100-8-51mm-black-red>: HTTP status code is not handled or not allowed 2026-01-25 06:08:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m4-x-0-7-x-12mm-cup-point-socket-set-screw-304-stainless-steel-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 06:08:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-tourniquet-for-smartcompliance-general-business-cabinet-1-x-18>: HTTP status code is not handled or not allowed 2026-01-25 06:08:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-56h-tenv-1ph-1075rpm-a6p11nr1gg>: HTTP status code is not handled or not allowed 2026-01-25 06:08:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/51024-25-heavy-duty-plastic-barrier-chain-hdpe-2-x25-54mm-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:08:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-branch-tee-55210-10-14-10mm-tube-x-14-swift-fit-composite-body>: HTTP status code is not handled or not allowed 2026-01-25 06:09:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storage-cabinet-wardrobe-storage-1>: HTTP status code is not handled or not allowed 2026-01-25 06:09:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bantam-switch-self-retracting-lifeline-single-leg-steel-snap-hook-6l>: HTTP status code is not handled or not allowed 2026-01-25 06:09:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/341706-booster-element-208v-5000w-for-hubbell-the-electric-heater-co>: HTTP status code is not handled or not allowed 2026-01-25 06:09:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30008-100-plastic-barrier-chain-hdpe-1-5-x100-6-38mm-silver>: HTTP status code is not handled or not allowed 2026-01-25 06:09:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tote-box-accessory-short-divider-6>: HTTP status code is not handled or not allowed 2026-01-25 06:09:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/apex-rechargeable-led-headlamp-550-lumens-120m-beam-distance-black>: HTTP status code is not handled or not allowed 2026-01-25 06:09:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/apex-industrial-led-headlamp-650-lumens-160m-beam-distance-black>: HTTP status code is not handled or not allowed 2026-01-25 06:09:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nut-eyebolt-5-8-11-x-12l-1-3-8-id-2-1-2-od-shoulder-pattern-304-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 06:09:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-1482-stainless-steel-phillips-truss-head-10-24-x-1-inch-machine-screw-100-box>: HTTP status code is not handled or not allowed 2026-01-25 06:09:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m8-x-1-25-x-35mm-flat-head-socket-cap-screw-304-stainless-steel-pkg-of-50>: HTTP status code is not handled or not allowed 2026-01-25 06:09:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/union-tee-85230-53-532-tube-composite-body>: HTTP status code is not handled or not allowed 2026-01-25 06:09:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2661109-caster-358-non-lock-for-taylor-freezer>: HTTP status code is not handled or not allowed 2026-01-25 06:09:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/superscrape-mat-3x10>: HTTP status code is not handled or not allowed 2026-01-25 06:09:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m3-x-0-5-x-20mm-machine-screw-phillips-flat-head-304-stainless-steel-din-965-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 06:09:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/union-y-85310-06-38-tube-composite-body>: HTTP status code is not handled or not allowed 2026-01-25 06:09:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-x-18-slim-style-enclosed-letterboard-satin-frame>: HTTP status code is not handled or not allowed 2026-01-25 06:09:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 73548 pages (at 100 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:09:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30354-10-s-hook-acetal-copolymer-1-5-green-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 06:09:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m5-x-0-8-x-12mm-machine-screw-phillips-flat-head-304-stainless-steel-din-965-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 06:09:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m2-x-0-4-x-8mm-machine-screw-phillips-pan-head-304-stainless-steel-din-7985-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 06:09:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/741302-door-gasket-25-5-8-inch-x-27-inch>: HTTP status code is not handled or not allowed 2026-01-25 06:09:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd52h-916-inch-band-heavy-duty-3-pc-part-ss-hose-clamp-10-pcs>: HTTP status code is not handled or not allowed 2026-01-25 06:09:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/comfy-hand-wrist-orthosis-pediatric-small-with-one-cover>: HTTP status code is not handled or not allowed 2026-01-25 06:09:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-flame-resistant-hi-vis-t-shirt-3xl-tall-fluorescent-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:09:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48x36-outdoor-combo-board-corkboard-white-dry-erase-board>: HTTP status code is not handled or not allowed 2026-01-25 06:09:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dart-portion-cup-lids-for-12-1oz-containers-clear-125sleeve>: HTTP status code is not handled or not allowed 2026-01-25 06:09:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50707-10-master-link-acetal-copolymer-2-gold-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 06:09:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-flame-resistant-hi-vis-t-shirt-5xl-tall-fluorescent-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:09:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1481113-hinge-assembly-door-btm-rh-for-true-manufacturing>: HTTP status code is not handled or not allowed 2026-01-25 06:09:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ankle-brace-02d-sport-stirrup-ankle-right>: HTTP status code is not handled or not allowed 2026-01-25 06:09:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cs20h-collared-screw-worm-gear-hose-clamp-34-inch-1-34-inch-clamping-diam-10-pcs>: HTTP status code is not handled or not allowed 2026-01-25 06:09:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-cu-yd-yellow-self-dumping-forkli-hopper-with-heavy-gauge-base>: HTTP status code is not handled or not allowed 2026-01-25 06:09:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-48-dp-1ph-1725rpm-a4c17dj74bb>: HTTP status code is not handled or not allowed 2026-01-25 06:09:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b116hsp-300-series-ss-hose-clamp-5-34-inch-7-34-inch-clamping-diameter-box-of-10>: HTTP status code is not handled or not allowed 2026-01-25 06:09:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-42-tefc-1ph-1725rpm-c42c17fb1cc>: HTTP status code is not handled or not allowed 2026-01-25 06:09:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dart-conex-polypropylene-portion-container-clear-1oz-125bag>: HTTP status code is not handled or not allowed 2026-01-25 06:09:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/superior-graphite-33215-slip-plate3-1-gallon-can-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-25 06:09:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-tefc-1ph-3500rpm-c184k34fc2aa>: HTTP status code is not handled or not allowed 2026-01-25 06:09:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/recycle-42-square-container-mixed-open-top-brown>: HTTP status code is not handled or not allowed 2026-01-25 06:09:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-tecgen-flame-resistant-work-shirt-3xl-short-navy>: HTTP status code is not handled or not allowed 2026-01-25 06:09:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/recycle-42-square-container-mixed-open-top-beige>: HTTP status code is not handled or not allowed 2026-01-25 06:09:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/l-4525-123-3a-cord-reel-single-outlet>: HTTP status code is not handled or not allowed 2026-01-25 06:09:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-tecgen-flame-resistant-work-shirt-4xl-short-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:09:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-rubbermaid-brute-20-gallon-trash-container-2>: HTTP status code is not handled or not allowed 2026-01-25 06:09:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-tecgen-flame-resistant-work-shirt-s-short-navy>: HTTP status code is not handled or not allowed 2026-01-25 06:09:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bucket-wringer-combo-with-down-pressure-1>: HTTP status code is not handled or not allowed 2026-01-25 06:09:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insulated-ibc-steel-tote-heater-th250-250-gallon-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:09:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dixie-pathways-paper-hot-cups-12-oz-500carton>: HTTP status code is not handled or not allowed 2026-01-25 06:09:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-partition-flexible-seven-panel-1>: HTTP status code is not handled or not allowed 2026-01-25 06:09:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/boardwalk-full-length-polystyrene-cutlery-fork-white-100box>: HTTP status code is not handled or not allowed 2026-01-25 06:09:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chair-on-the-go-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:09:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galvanized-steel-guardrail-kit-12-zip-bases-11-rz-guardrail-10-l>: HTTP status code is not handled or not allowed 2026-01-25 06:09:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-unicoil-hose-bender-for-58-inch-id-095-inch-max-od-hose>: HTTP status code is not handled or not allowed 2026-01-25 06:09:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-heater-high-capacity-194-3000225015001125w-240208v-white>: HTTP status code is not handled or not allowed 2026-01-25 06:09:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-inch-x-4-inch-x-0125-inch-stainless-steel-slotted-shim-pack-of-5>: HTTP status code is not handled or not allowed 2026-01-25 06:09:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0010-inch-brown-plastic-color-coded-shim-stock-5-inch-x-20-inch-flat-sheet>: HTTP status code is not handled or not allowed 2026-01-25 06:09:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clean-room-stool-vinyl-imperial-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:09:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hefty-easy-grip-disposable-plastic-party-cups-9-oz-red-50pack>: HTTP status code is not handled or not allowed 2026-01-25 06:09:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s9155-1w-led-decorative-diamond-cut-bulb-medium-base-clear-2700k>: HTTP status code is not handled or not allowed 2026-01-25 06:09:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-flame-resistant-dh-air153-work-shirt-xl-tall-khaki>: HTTP status code is not handled or not allowed 2026-01-25 06:09:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1201194-caster-stem-4-inch-7-16-14-w-brake>: HTTP status code is not handled or not allowed 2026-01-25 06:09:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-flame-resistant-dh-air153-work-shirt-2xl-regular-khaki>: HTTP status code is not handled or not allowed 2026-01-25 06:09:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-dock-plate-8067-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:09:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kimberly-clark-professional-scott-full-fold-dispenser-napkins-1-ply-13-x-12-white-375pack>: HTTP status code is not handled or not allowed 2026-01-25 06:09:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/boardwalk-paper-food-baskets-3lb-capacity-redwhite>: HTTP status code is not handled or not allowed 2026-01-25 06:09:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hammer-drill-bits-irwin-322033>: HTTP status code is not handled or not allowed 2026-01-25 06:09:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-inch-x-2-inch-x-0025-inch-white-plastic-color-coded-slotted-shim-pack-of-20>: HTTP status code is not handled or not allowed 2026-01-25 06:09:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thread-repair-kit-for-metal-thin-wall-m4-0-7-x-5-16-18-ez-310-m4>: HTTP status code is not handled or not allowed 2026-01-25 06:09:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blue-plastic-box-truck-16-bushel-medium-duty>: HTTP status code is not handled or not allowed 2026-01-25 06:09:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-countertop-auto-spray-soap-sanitizer-dispenser-white-chrome>: HTTP status code is not handled or not allowed 2026-01-25 06:09:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-inch-x-3-inch-x-0002-inch-stainless-steel-slotted-shim-pack-of-20>: HTTP status code is not handled or not allowed 2026-01-25 06:09:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s9496-65w-led-mr16-40-beam-spread-gu53-base-3000k-12v>: HTTP status code is not handled or not allowed 2026-01-25 06:09:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s9381-45w-led-mr16-gu10-base-40-beam-spread-5000k-120v>: HTTP status code is not handled or not allowed 2026-01-25 06:09:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gray-plastic-box-truck-18-bushel>: HTTP status code is not handled or not allowed 2026-01-25 06:09:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/red-plastic-box-truck-18-bushel>: HTTP status code is not handled or not allowed 2026-01-25 06:09:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/851223-pad-grease-universal16x16>: HTTP status code is not handled or not allowed 2026-01-25 06:09:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s9173-14w-led-s14-ceramic-orange-medium-base>: HTTP status code is not handled or not allowed 2026-01-25 06:09:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-electric-air-compressor-2-hp-8-gallon-capacity-horizontal-4-cfm>: HTTP status code is not handled or not allowed 2026-01-25 06:09:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/size-c-4-inch-x-4-inch-stainless-steel-slotted-shim-260-piece-full-assortment>: HTTP status code is not handled or not allowed 2026-01-25 06:09:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/type-309-stainless-steel-tool-wrap-width-12-inch-length-100-thickness-0002-inch>: HTTP status code is not handled or not allowed 2026-01-25 06:09:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vci-flat-bags-26-x-66-6-mil-clear-50-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:09:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-5200mah-mobile-power-bank-usb-battery-charge-with-led-flashlight>: HTTP status code is not handled or not allowed 2026-01-25 06:09:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0029-inch-diameter-music-wire-1-pound-coil>: HTTP status code is not handled or not allowed 2026-01-25 06:09:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vci-resealable-zip-closure-flat-bags-18-x-24-4-mil-blue-250-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:09:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s9158-14w-led-g16-12-globe-medium-base-clear-2700k>: HTTP status code is not handled or not allowed 2026-01-25 06:09:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-gantry-crane-10-fixed-height-4000-lb-capacity-1>: HTTP status code is not handled or not allowed 2026-01-25 06:09:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rotom-460B-460-552-mfd-125-volts-start-round>: HTTP status code is not handled or not allowed 2026-01-25 06:09:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bin-nestable-clear-7>: HTTP status code is not handled or not allowed 2026-01-25 06:09:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0007-inch-diameter-music-wire-14-pound-coil>: HTTP status code is not handled or not allowed 2026-01-25 06:09:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2061224-blade-whisk-mixer-mp450-2-pk-for-robot-coupe>: HTTP status code is not handled or not allowed 2026-01-25 06:09:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0002-inch-steel-shim-stock-6-inch-x-100-inch-roll>: HTTP status code is not handled or not allowed 2026-01-25 06:09:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-pocket-card-display>: HTTP status code is not handled or not allowed 2026-01-25 06:09:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exo2-migr-03-mmotor-impact-glove>: HTTP status code is not handled or not allowed 2026-01-25 06:09:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0029-inch-diameter-stainless-steel-wire-1-pound-coil>: HTTP status code is not handled or not allowed 2026-01-25 06:09:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011520-fan-motor-assembly-evap-115v>: HTTP status code is not handled or not allowed 2026-01-25 06:09:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-12-inch-id-x-2-18-inch-od-x-0062-inch-steel-arbor-shim-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 06:09:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1591167-brush-kit-phu>: HTTP status code is not handled or not allowed 2026-01-25 06:09:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-otr-dual-lock-pin-style-jack-stand-25-ton-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:09:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/Luxor-Mobile-Bin-Cart-with-Four-6H-Totes-MBS-SR-4L-Gray-Black-19-1-2L-x-17W-x-37-1-4H>: HTTP status code is not handled or not allowed 2026-01-25 06:09:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-tray-truck-24-x-33-x-66-with-14-tray-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:09:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-inch-plastic-classroom-chair-set-of-five-navy>: HTTP status code is not handled or not allowed 2026-01-25 06:10:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/battery-powered-high-precision-torque-wrench-kit-1-drive-500-2000-ftlb-10-38l>: HTTP status code is not handled or not allowed 2026-01-25 06:10:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lightning-lift153-super-duty-floor-jack-3-12-ton-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:10:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pv-1000w-12v-permanent-mount-inverter-pv1000>: HTTP status code is not handled or not allowed 2026-01-25 06:10:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blue-armor-2000-standard-harness-quick-connect-legs-s>: HTTP status code is not handled or not allowed 2026-01-25 06:10:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-dock-board-with-aluminum-curbs-40>: HTTP status code is not handled or not allowed 2026-01-25 06:10:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-expansion-anchor-3-8-x-2-zamac-alloy-unc-pkg-of-50-9520>: HTTP status code is not handled or not allowed 2026-01-25 06:10:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reusable-concrete-anchor-3-4-x-3-deep-drill-hole>: HTTP status code is not handled or not allowed 2026-01-25 06:10:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-2-single-air-actuated-valve-pilot-spring-return-g1-8-ports>: HTTP status code is not handled or not allowed 2026-01-25 06:10:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-normally-closed-single-air-actuated-valve-pilot-spr-return-g1-2-ports>: HTTP status code is not handled or not allowed 2026-01-25 06:10:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hub-buddy-replacement-cleaning-disc-for-model-37330-hub-buddy-xl-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 06:10:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-double-air-actuated-valve-pilot-1-8-nptf-ports>: HTTP status code is not handled or not allowed 2026-01-25 06:10:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-panel-divider-7-ft-4-inch-high-x-13-ft-1-in-long-fabric-summer-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:10:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-panel-divider-6-ft-high-x-5-ft-9-inch-long-fabric-greystone>: HTTP status code is not handled or not allowed 2026-01-25 06:10:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/263185-burner-divider-for-blodgett-oven>: HTTP status code is not handled or not allowed 2026-01-25 06:10:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/741248-drawer-gasket7-inch-x-28-13-16-inch-for-beverage-air>: HTTP status code is not handled or not allowed 2026-01-25 06:10:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 73647 pages (at 99 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:10:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/13-panel-divider-6-ft-8-inch-high-x-24-ft-1-inch-long-fabric-stone>: HTTP status code is not handled or not allowed 2026-01-25 06:10:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/263394-kason-10036000002-hasp-padlock-flush-for-kason>: HTTP status code is not handled or not allowed 2026-01-25 06:10:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1421384-brush-tank-30-inch-l-5-inch-od-for-carlisle-foodservice-products>: HTTP status code is not handled or not allowed 2026-01-25 06:10:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/06545-1-4-x-4-x-8-plastic-masonry-shim-4814-125pc>: HTTP status code is not handled or not allowed 2026-01-25 06:10:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/06495-1-4-x-2-x-4-plastic-masonry-shim-4214-504pc>: HTTP status code is not handled or not allowed 2026-01-25 06:10:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lab-pack-open-head-poly-drum-plastic-lever-lock-55-gallon-cap-23-34-dia-black>: HTTP status code is not handled or not allowed 2026-01-25 06:10:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keyless-porcelain-socket-w-rim-double-snap-in-clip>: HTTP status code is not handled or not allowed 2026-01-25 06:10:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/06105-1-4-x-3-x-4-plastic-horseshoe-shim-54-250pc>: HTTP status code is not handled or not allowed 2026-01-25 06:10:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electric-hydraulic-scissor-ground-lift-tilt-table-52-x-52-4000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:10:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-foot-8-inch-high-dry-erase-overlay>: HTTP status code is not handled or not allowed 2026-01-25 06:10:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluorescent-starter-holder-snap-in-mounting-spring>: HTTP status code is not handled or not allowed 2026-01-25 06:10:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/06445-1-8-x-2-x-2-plastic-masonry-shim-2218-1020pc>: HTTP status code is not handled or not allowed 2026-01-25 06:10:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/06055-3-8-x-2-5-16-x-3-plastic-horseshoe-shim-solid-23s-150pc>: HTTP status code is not handled or not allowed 2026-01-25 06:10:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/csweu2-led-wet-location-emergency-unit-w-adjustable-heads>: HTTP status code is not handled or not allowed 2026-01-25 06:10:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/panel-leg-workbench-with-fixed-height-legs-5>: HTTP status code is not handled or not allowed 2026-01-25 06:10:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stanley-ct10x-heavy-duty-staple-gun-cable-tacker>: HTTP status code is not handled or not allowed 2026-01-25 06:10:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/new-haven-6-ash-wood-picnic-table-surface-mount-brown>: HTTP status code is not handled or not allowed 2026-01-25 06:10:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/316ss-hose-adapters-tubeto-barbed-hose-for-quick-clamp-fittings-for-2-tubex-2-barbed-hose-id>: HTTP status code is not handled or not allowed 2026-01-25 06:10:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8x8x8-corrugated-boxes>: HTTP status code is not handled or not allowed 2026-01-25 06:10:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/push-to-connect-tube-fitting-nylon-90-degree-elbow-adapter-10mm-tube-od-x-14-npt-male>: HTTP status code is not handled or not allowed 2026-01-25 06:10:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011356-drawer-rail-lefttop-btm-for-turbo-air-refrigeration>: HTTP status code is not handled or not allowed 2026-01-25 06:10:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/madison-8-outdoor-bench-with-back-surface-mount-brown>: HTTP status code is not handled or not allowed 2026-01-25 06:10:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tilt-out-storage-bin-3-compartments-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:10:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washdown-motor-56-tefc-1ph-1725rpm-c6c17wc6gg>: HTTP status code is not handled or not allowed 2026-01-25 06:10:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-way-canopy-switch>: HTTP status code is not handled or not allowed 2026-01-25 06:10:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-plastic-shelf-truck-24-w-x-18-d-x-32-1-2-h-3-shelves>: HTTP status code is not handled or not allowed 2026-01-25 06:10:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-overhead-light-beige>: HTTP status code is not handled or not allowed 2026-01-25 06:10:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-external-tooth-lock-washer-267-256-i-d-steel-zinc-grade-2-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:10:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washdown-motor-56-tenv-3ph-1725-1425rpm-cz6t17vc54aa>: HTTP status code is not handled or not allowed 2026-01-25 06:10:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/phenolic-cfl-lampholder-w-uno-ring-g24q-1-gx24q-1-60hz-0-15a-13w-277v>: HTTP status code is not handled or not allowed 2026-01-25 06:10:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/new-haven-4-ash-wood-flat-bench-without-back-brown>: HTTP status code is not handled or not allowed 2026-01-25 06:10:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washdown-motor-56-tefc-1ph-1725rpm-c6c17wb1ll>: HTTP status code is not handled or not allowed 2026-01-25 06:10:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1031228-support-bsk-1925-inch-sq-w-defltr-for-ultrafryer>: HTTP status code is not handled or not allowed 2026-01-25 06:10:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/madison-8-outdoor-bench-without-back-surface-mount-brown>: HTTP status code is not handled or not allowed 2026-01-25 06:10:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-fender-washer-525-545-i-d-051-084-thick-steel-zinc-grade-2-pkg-of-50>: HTTP status code is not handled or not allowed 2026-01-25 06:10:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/250ml-bottle-top-filter-0-22um-pes-filter-75mm-diameter-sterile-24-case>: HTTP status code is not handled or not allowed 2026-01-25 06:10:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-0ml-micro-centrifuge-tube-resealable-bag-sterile-5000-case>: HTTP status code is not handled or not allowed 2026-01-25 06:10:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-round-table-with-4-imme-armless-chairs-moonbeam-seatdesigner-white-top>: HTTP status code is not handled or not allowed 2026-01-25 06:10:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-ft-cord-set-18-2-spt-1-brown>: HTTP status code is not handled or not allowed 2026-01-25 06:10:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/viton-x-profile-o-ring-dash-213-pack-of-5>: HTTP status code is not handled or not allowed 2026-01-25 06:10:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fowler-6-inch-economy-digital-caliper>: HTTP status code is not handled or not allowed 2026-01-25 06:10:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/infinity-locker-single-tier-15x18x72-1-door-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:10:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-16-flat-washer-uss-3-8-i-d-064-104-thick-steel-zinc-grade-2-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 06:10:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-square-table-with-2-imme-armless-barstools-moonbeam-seatstudio-teak-top>: HTTP status code is not handled or not allowed 2026-01-25 06:10:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-length-squeeze-disher-47391-gray-3-7-oz-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:10:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fowler-001-inch-agd-indicator-black>: HTTP status code is not handled or not allowed 2026-01-25 06:10:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pes-syringe-filter-0-22um-13mm-sterile-75-case>: HTTP status code is not handled or not allowed 2026-01-25 06:10:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/end-panel-24-x-97>: HTTP status code is not handled or not allowed 2026-01-25 06:10:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fowler-6-inch-150mm-outside-digital-spring-caliper>: HTTP status code is not handled or not allowed 2026-01-25 06:10:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pack-of-100-buna-n-square-profile-o-rings-dash-011>: HTTP status code is not handled or not allowed 2026-01-25 06:10:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/recycling-container-gray-15-gallon>: HTTP status code is not handled or not allowed 2026-01-25 06:10:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1ul-inoculating-loop-individual-pack-sterile-white-218mm-600-case>: HTTP status code is not handled or not allowed 2026-01-25 06:10:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fishing-retractable-gear-tether-0trg-341-xtreme-duty-36extention-black-carabiner>: HTTP status code is not handled or not allowed 2026-01-25 06:10:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10ml-serological-pipet-bulk-packed-in-bags-sterile-500-case>: HTTP status code is not handled or not allowed 2026-01-25 06:10:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adam-bench-weighing-scale-33lb>: HTTP status code is not handled or not allowed 2026-01-25 06:10:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-shelf-level-boltless-5>: HTTP status code is not handled or not allowed 2026-01-25 06:10:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/soft-fda-silicone-rubber-cord-0139-cross-section-10-ft-length>: HTTP status code is not handled or not allowed 2026-01-25 06:10:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-shelf-level-boltless-wood-deck-4>: HTTP status code is not handled or not allowed 2026-01-25 06:10:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bin-nestable-blue-2>: HTTP status code is not handled or not allowed 2026-01-25 06:10:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pack-of-10-buna-n-square-profile-o-rings-dash-326>: HTTP status code is not handled or not allowed 2026-01-25 06:10:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fiberglass-compact-safety-cage-ladder-6-10-type-1aa-19506-244>: HTTP status code is not handled or not allowed 2026-01-25 06:10:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pack-of-10-viton-square-profile-o-rings-dash-018>: HTTP status code is not handled or not allowed 2026-01-25 06:10:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/true-hepa-filter-290-300-dx95-air-purifiers>: HTTP status code is not handled or not allowed 2026-01-25 06:10:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-red-pin-heavy-duty-dock-boards-17>: HTTP status code is not handled or not allowed 2026-01-25 06:10:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-velocity-multiuse-ext-ladder-17-type-1a-15417-001>: HTTP status code is not handled or not allowed 2026-01-25 06:10:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deck-mounted-sensor-faucet-with-trim-plate-1-gpm-chrome>: HTTP status code is not handled or not allowed 2026-01-25 06:10:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-round-table-with-4-imme-armless-chairs-light-gray-seatmaple-top>: HTTP status code is not handled or not allowed 2026-01-25 06:10:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-9-drawer-standard-width-cabinet-blue-no-lock>: HTTP status code is not handled or not allowed 2026-01-25 06:10:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/304-stainless-steel-caps-for-quick-clamp-fittings-for-1-and-1-12-tube>: HTTP status code is not handled or not allowed 2026-01-25 06:10:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-red-pin-heavy-duty-dock-boards-18>: HTTP status code is not handled or not allowed 2026-01-25 06:10:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sec-alum-63-crossover-truckbox-singlelid-lowpro-wedge-mat-blk-79211104>: HTTP status code is not handled or not allowed 2026-01-25 06:10:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3m-3-upper-taping-head-2>: HTTP status code is not handled or not allowed 2026-01-25 06:10:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-10-drawer-shallow-depth-cabinet-blue-keyed-alike>: HTTP status code is not handled or not allowed 2026-01-25 06:10:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/304ss-straight-adapters-tubeto-fnptfor-quick-clamp-fittings-for-12-tubeodx-12-npt-female>: HTTP status code is not handled or not allowed 2026-01-25 06:10:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-face-mask-required-wall-sign-green>: HTTP status code is not handled or not allowed 2026-01-25 06:10:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cdn-proaccurate-insta-read-cooking-thermometer-display-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:10:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traex-wide-mouth-squeeze-bottle-kits-22016-13-16-oz-clear>: HTTP status code is not handled or not allowed 2026-01-25 06:10:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/task-conference-chair-high-back-19-22h-seat-black>: HTTP status code is not handled or not allowed 2026-01-25 06:10:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/distribution-container-with-hinged-lid-21-1-2x15-1-4x12-1-2>: HTTP status code is not handled or not allowed 2026-01-25 06:10:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6ta03o-zinc-die-cast-adjustable-lever-w-steel-components-3-8-16-tapped-thread>: HTTP status code is not handled or not allowed 2026-01-25 06:10:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-plastic-tote-lid-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:10:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-single-shaft-open-blower-115v-1625rpm-130hp>: HTTP status code is not handled or not allowed 2026-01-25 06:10:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-capacity-33-lbs-cylinders-heavy-duty-steel-gas-cylinder-cage-30w-x-30d-x-38h-white>: HTTP status code is not handled or not allowed 2026-01-25 06:10:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012764-svc-kit-i1-upr-jet-plate-for-turbochef>: HTTP status code is not handled or not allowed 2026-01-25 06:11:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-58-stock-460v-1075rpm-34-12hp>: HTTP status code is not handled or not allowed 2026-01-25 06:11:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/recycled-monthly-wall-calendar-blue-and-red-20-x-30-2015-aagpm428>: HTTP status code is not handled or not allowed 2026-01-25 06:11:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-steel-shelving-20-ga-36-wx24-dx97-h-closed-clip-5-shelves-starter>: HTTP status code is not handled or not allowed 2026-01-25 06:11:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-steel-shelving-20-ga-36-wx30-dx97-h-closed-clip-5-shelves-starter>: HTTP status code is not handled or not allowed 2026-01-25 06:11:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-unassembled-deluxe-storage-cabinet-36w-x-24d-x-78-3-4-h-medium-grey>: HTTP status code is not handled or not allowed 2026-01-25 06:11:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/264666-lock-washer-for-groen>: HTTP status code is not handled or not allowed 2026-01-25 06:11:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/direct-replacement-rheem-ruud-208-230v-825rpm-12hp>: HTTP status code is not handled or not allowed 2026-01-25 06:11:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-safety-view-locker-single-tier-12x18x72-3-doors-assembled-parchment>: HTTP status code is not handled or not allowed 2026-01-25 06:11:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-2-double-solenoid-valve-pilot-g-1-4-12v-dc-3w-coil-black-connection>: HTTP status code is not handled or not allowed 2026-01-25 06:11:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c-thru-48-jumbo-welded-storage-cabinet-48w-x-24d-x-78h-medium-grey>: HTTP status code is not handled or not allowed 2026-01-25 06:11:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-metal-pegboard-black-48-x-32-x-3-4>: HTTP status code is not handled or not allowed 2026-01-25 06:11:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-open-cntr-g-1-8-dbl-solenoid-valve-pilot-spring-cntred-24v-dc-3w-coil-black-conn>: HTTP status code is not handled or not allowed 2026-01-25 06:11:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/281011-stone-set-for-bizerba>: HTTP status code is not handled or not allowed 2026-01-25 06:11:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-cal-flame-resistant-coveralls-xl-ln-gray-tcg02150870>: HTTP status code is not handled or not allowed 2026-01-25 06:11:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-closed-single-solenoid-valve-pilot-spr-return-1-4-nptf-220v-ac-5va-coil-blk-conn>: HTTP status code is not handled or not allowed 2026-01-25 06:11:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-dbl-solenoid-vlv-ext-pilot-open-cntr-spr-cntred-g1-8-24v-ac-5va-coil-led-conn>: HTTP status code is not handled or not allowed 2026-01-25 06:11:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 73747 pages (at 100 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:11:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lightweight-long-sleeve-fr-t-shirt-m-desert-sand-df2-cm-446ls-ds-md>: HTTP status code is not handled or not allowed 2026-01-25 06:11:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-16-18-x-2-1-2-security-machine-screw-round-one-way-head-steel-zinc-ft-unc-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:11:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-work-shirt-m-gray-tcg01150216>: HTTP status code is not handled or not allowed 2026-01-25 06:11:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/34-h-computer-workstation>: HTTP status code is not handled or not allowed 2026-01-25 06:11:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kitchen-pegboard-pack-storage-organization-kit-gray-white-32-x-32-x-6>: HTTP status code is not handled or not allowed 2026-01-25 06:11:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-dbl-solenoid-valve-pilot-spr-cntred-closed-cntr-g1-8-220v-ac-5va-coil-led-conn>: HTTP status code is not handled or not allowed 2026-01-25 06:11:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-rack-slotted-beam-pair-4-channel-96x42>: HTTP status code is not handled or not allowed 2026-01-25 06:11:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14ab-x-2-security-sheet-metal-screw-flat-hex-socket-head-302hq-18-8-stainless-steel-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:11:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quick-adjust-wire-shelving-36x18x54>: HTTP status code is not handled or not allowed 2026-01-25 06:11:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009765-hp-bw-finger-ring-for-henny-penny>: HTTP status code is not handled or not allowed 2026-01-25 06:11:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c-thru-deluxe-welded-storage-cabinet-36w-x-24d-x-78h-putty>: HTTP status code is not handled or not allowed 2026-01-25 06:11:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-20-x-3-4-security-machine-screw-round-one-way-head-18-8-stainless-steel-ft-unc-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:11:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-20-x-1-security-machine-screw-pan-spanner-head-18-8-stainless-steel-ft-unc-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:11:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-starter-60inw-x-36ind-x-72inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 06:11:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-heavy-duty-boltless-shelving-starter-60inw-x-36ind-x-96inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 06:11:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quick-adjust-wire-shelf-truck-60x24x60-1200-pound-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:11:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergonomic-shell-chair-right-handed-tablet-arm-blue-5-per-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:11:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/label-holder-3-4-in-48-in-clear-strip>: HTTP status code is not handled or not allowed 2026-01-25 06:11:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-32-x-1-1-2-security-machine-screw-round-one-way-head-steel-zinc-ft-unc-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:11:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-double-solenoid-valve-ext-pilot-g-1-4-12v-dc-3w-coil-led-connection>: HTTP status code is not handled or not allowed 2026-01-25 06:11:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-heavy-duty-boltless-shelving-add-on-72inw-x-30ind-x-84inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 06:11:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-32-x-1-security-machine-screw-truss-spanner-head-18-8-stainless-steel-ft-unf-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:11:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/label-holder-3-in-6-in-magnetic-write-on>: HTTP status code is not handled or not allowed 2026-01-25 06:11:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pegboard-standard-tool-storage-kit-gray-white-48-x-32-x-9>: HTTP status code is not handled or not allowed 2026-01-25 06:11:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6ab-x-1-security-sheet-metal-screw-button-hex-socket-head-steel-black-oxide-usa-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:11:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quick-adjust-shelf-72x18-with-4-hooks>: HTTP status code is not handled or not allowed 2026-01-25 06:11:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-heavy-duty-boltless-shelving-add-on-48inw-x-36ind-x-60inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 06:11:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010881-pc-board-assembly-for-waring-products>: HTTP status code is not handled or not allowed 2026-01-25 06:11:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-2-double-solenoid-valve-ext-pilot-1-4-nptf-110v-ac-5va-coil-led-connection>: HTTP status code is not handled or not allowed 2026-01-25 06:11:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/label-holder-in-c-in-design-1-in-3-in-self-adheres>: HTTP status code is not handled or not allowed 2026-01-25 06:11:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/label-holders-2-in-4-in-clear-slot-aluminum-ex-trusion>: HTTP status code is not handled or not allowed 2026-01-25 06:11:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pegboard-hobby-craft-organizer-storage-kit-white-32-x-32-x-9>: HTTP status code is not handled or not allowed 2026-01-25 06:11:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-shelf-truck-48x24x80-1200-pound-capacity-with-brakes>: HTTP status code is not handled or not allowed 2026-01-25 06:11:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-32-x-1-security-machine-screw-button-torx-head-alloy-steel-black-oxide-ft-unc-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:11:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pegboard-hobby-craft-organizer-storage-kit-blue-black-32-x-32-x-9>: HTTP status code is not handled or not allowed 2026-01-25 06:11:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/neoprene-foam-sheet-no-adhesive-1-8-x-36-x-36>: HTTP status code is not handled or not allowed 2026-01-25 06:11:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26-upright-frame-48-l-adjustable-panel-truck-sold-pairs>: HTTP status code is not handled or not allowed 2026-01-25 06:11:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pegboard-garden-tool-board-organizer-white-red-48-x-32-x-9>: HTTP status code is not handled or not allowed 2026-01-25 06:11:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-end-wood-shelf-truck-48-x-24-1200-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:11:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/belaire-spout-ladle-2-oz-capacity-12-long>: HTTP status code is not handled or not allowed 2026-01-25 06:11:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2561150-clip-cotter-hairpin-1-inch-l-for-silver-king>: HTTP status code is not handled or not allowed 2026-01-25 06:11:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-strip-curtain-bulk-roll-scratch-resistant-ribbed-clear-12-x-150-x-110>: HTTP status code is not handled or not allowed 2026-01-25 06:11:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-swing-panel-pegboard-24w-x-1-12d-x-48h-white-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-25 06:11:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/laser-ink-jet-insert-letter-pref-1-in-8-in-500-per-package>: HTTP status code is not handled or not allowed 2026-01-25 06:11:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-32-x-3-4-security-machine-screw-button-torx-head-alloy-steel-black-oxide-ft-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:11:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/china-cap-strainer-10-deep-cone-stainless>: HTTP status code is not handled or not allowed 2026-01-25 06:11:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-rod-hook-for-18--14-pegboard-1w-x-4d-x-2h-silver-pack-of-3>: HTTP status code is not handled or not allowed 2026-01-25 06:11:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-head-chuck-dial-gauge-max-160-psi-s-933>: HTTP status code is not handled or not allowed 2026-01-25 06:11:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pizza-cutter-2-5-8-stainless-steel-wheel>: HTTP status code is not handled or not allowed 2026-01-25 06:11:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metal-man-big-window-auto-darkening-welding-helmet-variable-shade-control-patriotic>: HTTP status code is not handled or not allowed 2026-01-25 06:11:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magliner-aluminum-platform-truck-with-diamond-deck-48-x-30-1400-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:11:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2901035-lid-steelvac-blk-m-fvp>: HTTP status code is not handled or not allowed 2026-01-25 06:11:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-bright-tea-co-matcha-latte-green-tea-053-oz-pouch-pack-of-72>: HTTP status code is not handled or not allowed 2026-01-25 06:11:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aero-three-bowl-ss-sink-18-x-18-with-right-left-side-drainboard>: HTTP status code is not handled or not allowed 2026-01-25 06:11:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14ab-x-1-security-sheet-metal-screw-button-torx-head-302hq-stainless-steel-ft-usa-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:11:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spice-womens-safety-work-boots-steel-toe-7h-size-9w-auburn>: HTTP status code is not handled or not allowed 2026-01-25 06:11:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-king-sk3000-pallet-rack-72w-x-36d-x-120h-cap-per-level-add-on-unit>: HTTP status code is not handled or not allowed 2026-01-25 06:11:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mirage-serving-tong-8>: HTTP status code is not handled or not allowed 2026-01-25 06:11:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/locboard-square-hole-pegboard-24w-x-916d-x-24h-silver-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-25 06:11:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/surface-mount-square-remote-fixture-6v-9w-6x4>: HTTP status code is not handled or not allowed 2026-01-25 06:11:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60w-x-12d-plastic-laminate-cantilever-shelf-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:11:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-king-sk3000-pallet-rack-120w-x-36d-x-144h-3320-lbs-cap-per-level-starter-unit>: HTTP status code is not handled or not allowed 2026-01-25 06:11:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-king-sk3000-pallet-rack-108w-x-36d-x-144h-4040-lbs-cap-per-level-starter-unit>: HTTP status code is not handled or not allowed 2026-01-25 06:11:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nyc-battery-unit-12v-50w>: HTTP status code is not handled or not allowed 2026-01-25 06:11:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-king-sk3000-pallet-rack-96w-x-36d-x-144h-4900-lbs-cap-per-level-starter-unit>: HTTP status code is not handled or not allowed 2026-01-25 06:11:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010925-keyway-lower-shaft-for-hobart>: HTTP status code is not handled or not allowed 2026-01-25 06:11:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-king-sk3000-pallet-rack-108w-x-48d-x-120h-4040-lbs-cap-per-level-add-on-unit>: HTTP status code is not handled or not allowed 2026-01-25 06:11:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6690-cooling-arm-sleeves-lime-l-12284>: HTTP status code is not handled or not allowed 2026-01-25 06:11:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/49-inch-width-x-19-1-2-inch-height-snow-plow-for-dixie-chopper-zee2-mowers>: HTTP status code is not handled or not allowed 2026-01-25 06:11:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/omega-womens-safety-work-boots-steel-toe-6h-size-10-black>: HTTP status code is not handled or not allowed 2026-01-25 06:11:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clearview-3-16-polycarbonate-industrial-work-station-partition-48-w-x-72-h>: HTTP status code is not handled or not allowed 2026-01-25 06:11:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/square-edge-work-bench-top-shop-top-60-w-x-36-d-x-1-1-2-thick>: HTTP status code is not handled or not allowed 2026-01-25 06:11:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-16-x-3-4-security-machine-screw-button-torx-head-302hq-stainless-steel-ft-unc-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:11:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/230ltr-wide-soft-cap-leather-knee-pads-brown-18232>: HTTP status code is not handled or not allowed 2026-01-25 06:11:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-16-x-1-1-2-security-machine-screw-button-torx-head-302hq-stainless-steel-ft-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:11:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gravity-womens-safety-work-boots-steel-toe-4h-size-75w-black>: HTTP status code is not handled or not allowed 2026-01-25 06:11:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/valence-womens-safety-work-boots-steel-toe-7h-size-6-chestnut-brown>: HTTP status code is not handled or not allowed 2026-01-25 06:11:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-2-inch-carbon-steel-preformed-band-it-jr-clamp-with-5-8-inch-band>: HTTP status code is not handled or not allowed 2026-01-25 06:11:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/surveyor-vest-c2l2-solid-lime-4xl-5xl>: HTTP status code is not handled or not allowed 2026-01-25 06:11:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-vis-sweatshirt-detachable-hood-zipper-closure-2-pockets-fl-lime-small>: HTTP status code is not handled or not allowed 2026-01-25 06:11:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8014221-lock-solenoid-assembly-120v-brew-basket-for-fetco>: HTTP status code is not handled or not allowed 2026-01-25 06:11:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/badger-insulated-fleece-lined-boots-plain-toe-midsole-deep-lug-17-h-blk-size-6>: HTTP status code is not handled or not allowed 2026-01-25 06:11:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iron-eagle-174-jacket-green-inner-cuffs-storm-fly-front-hood-snaps-med>: HTTP status code is not handled or not allowed 2026-01-25 06:11:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spac-trac-6-unit-storage-shelving-chrome-36x18x74>: HTTP status code is not handled or not allowed 2026-01-25 06:12:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-inch-carbon-steel-center-punch-preformed-band-it-clamp-with-5-8-inch-band>: HTTP status code is not handled or not allowed 2026-01-25 06:12:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spac-trac-4-unit-storage-shelving-chrome-48x24x74>: HTTP status code is not handled or not allowed 2026-01-25 06:12:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iron-eagle-174-jacket-gold-inner-cuffs-storm-fly-front-hood-snaps-3xl>: HTTP status code is not handled or not allowed 2026-01-25 06:12:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spac-trac-5-unit-storage-shelving-chrome-48x21x74>: HTTP status code is not handled or not allowed 2026-01-25 06:12:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-t-shirt-short-sleeve-1-pocket-fl-lime-xl>: HTTP status code is not handled or not allowed 2026-01-25 06:12:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reflective-t-shirt-long-sleeve-1-pocket-silver-tape-type-r-class-3-fl-lime-4xl>: HTTP status code is not handled or not allowed 2026-01-25 06:12:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reflective-t-shirt-long-sleeve-1-pocket-silver-tape-type-r-class-3-fl-lime-2xl>: HTTP status code is not handled or not allowed 2026-01-25 06:12:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-liner-1-16-clear-acrylic-48-w-x-12-d>: HTTP status code is not handled or not allowed 2026-01-25 06:12:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-sink-2-stations-with-knee-valve-operatefaucets-straight-legs-36-l-x20-wx>: HTTP status code is not handled or not allowed 2026-01-25 06:12:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-516-1-light-die-cast-bulk-head-frosted-diffuser-semi-gloss-white-10-5w-x-9-125h>: HTTP status code is not handled or not allowed 2026-01-25 06:12:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/omega-womens-safety-work-boots-steel-toe-6h-size-6-bourbon-brown>: HTTP status code is not handled or not allowed 2026-01-25 06:12:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/77-532-1-light-ceiling-mount-white-glass-globe-6w-x-7-25h>: HTTP status code is not handled or not allowed 2026-01-25 06:12:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iron-eagle-174-overall-gold-knee-patch-pockets-loto-straps-4xl>: HTTP status code is not handled or not allowed 2026-01-25 06:12:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/281976-salamander-skin>: HTTP status code is not handled or not allowed 2026-01-25 06:12:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/square-edge-work-bench-top-maple-72-w-x-30-d-x-2-1-4-thick>: HTTP status code is not handled or not allowed 2026-01-25 06:12:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iprotect-safety-glasses-erb-safety-17510-clear>: HTTP status code is not handled or not allowed 2026-01-25 06:12:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 73847 pages (at 100 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:12:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-4962-boxwood-1-light-outdoor-wall-clear-beveled-rustic-bronze-8w-x-15-75h>: HTTP status code is not handled or not allowed 2026-01-25 06:12:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hard-hat-erb-safety-americana-19769-pink>: HTTP status code is not handled or not allowed 2026-01-25 06:12:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-coat-rack-with-bars-wall-mount-48-w-x-24-d-x-6-h>: HTTP status code is not handled or not allowed 2026-01-25 06:12:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/77-702-1-light-flood-light-w-adjust-swivel-gray-4-5w-x-8h>: HTTP status code is not handled or not allowed 2026-01-25 06:12:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/valence-womens-safety-work-boots-steel-toe-7h-size-7-chestnut-brown>: HTTP status code is not handled or not allowed 2026-01-25 06:12:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aware-wear-ansi-class-2-economy-mesh-vest-61457-orange>: HTTP status code is not handled or not allowed 2026-01-25 06:12:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2661013-brush-white-bristle-1-2-inch-x3-inch-for-taylor-freezer>: HTTP status code is not handled or not allowed 2026-01-25 06:12:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-shelving-48x18x60-with-4-shelves-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 06:12:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/icon-153-overall-fluorescent-lime-black-snap-fly-front-5xl>: HTTP status code is not handled or not allowed 2026-01-25 06:12:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/badger-neoprene-boots-plain-toe-upper-rubber-sole-steel-shank-15-h-blk-size-4>: HTTP status code is not handled or not allowed 2026-01-25 06:12:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-wire-chip-basket-40-liter-9-980-849-0>: HTTP status code is not handled or not allowed 2026-01-25 06:12:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-528-1-light-die-cast-bulk-head-frosted-diffuser-semi-gloss-white-8-5w-x-12-5h>: HTTP status code is not handled or not allowed 2026-01-25 06:12:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/badger-insulated-fleece-lined-boots-plain-toe-midsole-deep-lug-17-h-blk-size-11>: HTTP status code is not handled or not allowed 2026-01-25 06:12:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ivr-l-100-18-2-tc-dp-industrial-vacuum-26-4-gallons-9-988-913-0>: HTTP status code is not handled or not allowed 2026-01-25 06:12:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-steel-wheel-riser-120-x-24-x-12>: HTTP status code is not handled or not allowed 2026-01-25 06:12:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/46-in-round-solid-top-portable-picnic-table-with-4-seats-perforated-blue-top>: HTTP status code is not handled or not allowed 2026-01-25 06:12:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2721323-bracket-clutch-restraint-for-nieco-corp>: HTTP status code is not handled or not allowed 2026-01-25 06:12:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/surface-mount-post-27-1-2-h-and-base-plate-package-granite>: HTTP status code is not handled or not allowed 2026-01-25 06:12:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2661046-clip-scraper-blade-8-3-4-inch-for-taylor-freezer>: HTTP status code is not handled or not allowed 2026-01-25 06:12:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-36x18x84-with-5-shelves-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 06:12:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-drum-gripper-attachment-for-rough-terrain-electric-powered-pallet-jack-trucks>: HTTP status code is not handled or not allowed 2026-01-25 06:12:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paulson-htampshield-kit-for-slotted-cap-atpv-12-cal-grey-anti-fog-lens-amp1-12-hts>: HTTP status code is not handled or not allowed 2026-01-25 06:12:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/76-602-2-light-ceiling-flush-mount-medium-white-mushroom-old-bronze-9-5w-x-6h>: HTTP status code is not handled or not allowed 2026-01-25 06:12:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plc-03387-2oz-natural-hdpe-cylinder-bottle-with-24-410-white-pe-flip-top-cap-case-of-48>: HTTP status code is not handled or not allowed 2026-01-25 06:12:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adder-unit-letter-25-inch-30-3-4-inch-92-1-2-inch-8-opening-bone-white>: HTTP status code is not handled or not allowed 2026-01-25 06:12:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-high-letter-end-panel-kit-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:12:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12in-wall-spacer-w-hardware-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-25 06:12:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/421814-low-pressure-switch>: HTTP status code is not handled or not allowed 2026-01-25 06:12:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/letter-depth-flat-shelf-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:12:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hss-tmx-t-shank-gp-drills-118-16-0mm-diameter-x-218mm-oal-number-2-mt>: HTTP status code is not handled or not allowed 2026-01-25 06:12:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/legal-depth-flat-shelf-bone-white>: HTTP status code is not handled or not allowed 2026-01-25 06:12:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-expansion-stem-caster-3-dia-element-derlin-bearings>: HTTP status code is not handled or not allowed 2026-01-25 06:12:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-stem-caster-4-dia-thermo-pro-grip-ring-with-top-lock-2-h-stem>: HTTP status code is not handled or not allowed 2026-01-25 06:12:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-boltless-shelving-starter-36inw-x-18ind-x-48inh-no-deck>: HTTP status code is not handled or not allowed 2026-01-25 06:12:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deep-cleaning-ribbed-6-foot-wide-cut-length-mat-green>: HTTP status code is not handled or not allowed 2026-01-25 06:12:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-boltless-shelving-starter-48inw-x-18ind-x-60inh-no-deck>: HTTP status code is not handled or not allowed 2026-01-25 06:12:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wooden-chicken-coop-with-ramp-nesting-box-wire-mesh-run-and-waterproof-roof>: HTTP status code is not handled or not allowed 2026-01-25 06:12:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-3262-2-light-ceiling-flush-mount-frosted-white-13-125w-x-5-375h>: HTTP status code is not handled or not allowed 2026-01-25 06:12:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-x-24-18-gauge-stainless-steel-workbench>: HTTP status code is not handled or not allowed 2026-01-25 06:12:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-swivel-casters-2-w-brakes-96-workbenches>: HTTP status code is not handled or not allowed 2026-01-25 06:12:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/job-site-box-18-1-4-x-8-removable-storage-tray>: HTTP status code is not handled or not allowed 2026-01-25 06:12:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/k161-c-frame-dia-120v>: HTTP status code is not handled or not allowed 2026-01-25 06:12:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-high-letter-starter-empty-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:12:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-356-empire-6-light-chandelier-alabaster-bell-brushed-nickel-26w-x-14h>: HTTP status code is not handled or not allowed 2026-01-25 06:12:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-vinyl-padded-stool-backless-black>: HTTP status code is not handled or not allowed 2026-01-25 06:12:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-high-letter-adder-empty-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:12:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1381224-switch-magnetic-door-alarm>: HTTP status code is not handled or not allowed 2026-01-25 06:12:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2031148-switch-power-for-globe-food-equipment>: HTTP status code is not handled or not allowed 2026-01-25 06:12:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/421583-potentiometer>: HTTP status code is not handled or not allowed 2026-01-25 06:12:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hss-tmx-jewelers-saw-4-inch-dia-x-020-inch-face-x-1-2-inch-hole-x-310-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:12:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proforce-2-gallon-oil-free-pancake-air-compressor-with-kit>: HTTP status code is not handled or not allowed 2026-01-25 06:12:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-245-1-light-ceiling-mount-white-ball-brushed-nickel-6w-x-7-25h>: HTTP status code is not handled or not allowed 2026-01-25 06:12:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/in-2217-manhole-collar-fits-24-26-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:12:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-boltless-shelving-add-on-36inw-x-12ind-x-96inh-no-deck>: HTTP status code is not handled or not allowed 2026-01-25 06:12:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1381212-thermometer-k-type-352-for-cooper-atkins>: HTTP status code is not handled or not allowed 2026-01-25 06:12:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-air-contractor-9-gallon-portable-air-compressor-with-9-hp-honda-gas-engine>: HTTP status code is not handled or not allowed 2026-01-25 06:12:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-8482-stainless-steel-mobile-scissor-li-table-32-x-19-1000-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 06:12:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/62-106-frame-1-module-pendant-frosted-glass-brushed-nickel-5-625w-x-42h>: HTTP status code is not handled or not allowed 2026-01-25 06:12:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hss-tmx-interchangeable-pilot-counterbore-taper-shank-hss-5-flute-2-1-2-mt5>: HTTP status code is not handled or not allowed 2026-01-25 06:12:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hss-tmx-interchangeable-pilot-counterbore-taper-shank-hss-5-flute-1-9-16-mt4>: HTTP status code is not handled or not allowed 2026-01-25 06:12:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-boltless-shelving-add-on-36inw-x-12ind-x-72inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 06:12:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8763-ultra-containment-wall-2-feet-high-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:12:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-style-6-ball-coupler-1-4-x-1-4-male-steel-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 06:12:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rcl-5-neoprene-leveling-mount-glide-cup-leveling-stud-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:12:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-boltless-shelving-add-on-48inw-x-18ind-x-84inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 06:12:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-back-conductive-fabric-chair-w-aluminum-base-esd-casters-light-burgundy>: HTTP status code is not handled or not allowed 2026-01-25 06:12:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-copper-press-x-press-w-coupling-with-stop>: HTTP status code is not handled or not allowed 2026-01-25 06:12:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-boltless-shelving-add-on-36inw-x-18ind-x-96inh-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 06:12:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-maroon-x-stripping-pad-10-per-case>: HTTP status code is not handled or not allowed 2026-01-25 06:12:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-24-w-10-step-steel-rolling-ladder-21-d-top-step>: HTTP status code is not handled or not allowed 2026-01-25 06:12:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-296-3-light-vanity-wall-polished-chrome-18w-x-4-75h>: HTTP status code is not handled or not allowed 2026-01-25 06:12:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/266300-roller-w-tire-1-5-16od-1-4-20>: HTTP status code is not handled or not allowed 2026-01-25 06:12:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cgw-412x5811-t29-zpce-zirconia-reg-60-grit-flap-disc>: HTTP status code is not handled or not allowed 2026-01-25 06:12:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-propelled-electric-powered-pallet-jack-truck-4500-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:12:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gd10-gd2-sack-filter-1471100500>: HTTP status code is not handled or not allowed 2026-01-25 06:12:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-ultra-drain-rectangular-seal-36-x-58>: HTTP status code is not handled or not allowed 2026-01-25 06:12:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-step-24-w-perforated-all-directional-steel-ladder>: HTTP status code is not handled or not allowed 2026-01-25 06:12:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-boltless-shelving-add-on-48inw-x-24ind-x-96inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 06:12:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-step-24-w-perforated-all-directional-steel-ladder>: HTTP status code is not handled or not allowed 2026-01-25 06:12:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-awg-3-cond-45-ft-15-amp-single-outlet-with-cord-435-lbs>: HTTP status code is not handled or not allowed 2026-01-25 06:12:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-step-24-w-perforated-all-directional-steel-ladder-1>: HTTP status code is not handled or not allowed 2026-01-25 06:12:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/od-630-906-hose-roller-guide-nylon-roller>: HTTP status code is not handled or not allowed 2026-01-25 06:13:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e-series-lever-dolly-e-84-po-boxed-84-handle-polyurethane-wheels-5000-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 06:13:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/meteorlite-5-high-profile-strobe-light-12-80-volts-permanent-mount-clear>: HTTP status code is not handled or not allowed 2026-01-25 06:13:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8009715-goodyear-ax54-belt>: HTTP status code is not handled or not allowed 2026-01-25 06:13:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyurethane-mold-on-wheel-812-sx-12-dia-3w-3000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:13:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-35-300-psi-hose-assemblies>: HTTP status code is not handled or not allowed 2026-01-25 06:13:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-boltless-shelving-starter-48inw-x-12ind-x-60inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 06:13:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-24v-20a-battery-charger-641327>: HTTP status code is not handled or not allowed 2026-01-25 06:13:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/21-31-regular-duty-rigid-caster-31-3-hr-solid-rubber-hard-3-dia-250-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:13:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/office-chair-mat-for-carpet-45w-x-53l-with-25-x-12-lip-straight-edge-ind-pkg>: HTTP status code is not handled or not allowed 2026-01-25 06:13:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-shelf-boltless-shelving-starter-36inw-x-18ind-x-72inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 06:13:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2271145-two-timer-display-assembly-for-henny-penny>: HTTP status code is not handled or not allowed 2026-01-25 06:13:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/office-chair-mat-for-hard-floor-46w-x-60l-straight-edge-ind-pkg>: HTTP status code is not handled or not allowed 2026-01-25 06:13:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cyclone-hand-chain-hoist-on-plain-trolley-1-4-ton-15-ft-lift>: HTTP status code is not handled or not allowed 2026-01-25 06:13:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polycarbonate-divider-with-square-corners-60-w-x-60-l-opal>: HTTP status code is not handled or not allowed 2026-01-25 06:13:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-doors-cell-phone-locker-22x16x26-black-with-combo-locks>: HTTP status code is not handled or not allowed 2026-01-25 06:13:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-boltless-shelving-starter-48inw-x-24ind-x-84inh-laminate-deck>: HTTP status code is not handled or not allowed 2026-01-25 06:13:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sb-smh-connector-2-0-wire-gauge-350-amp-red>: HTTP status code is not handled or not allowed 2026-01-25 06:13:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 73946 pages (at 99 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:13:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011744-switch-kit>: HTTP status code is not handled or not allowed 2026-01-25 06:13:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/configurable-multi-platform-access-system-2-step-27-h-45-deg-stairway-ladder-mpassw3>: HTTP status code is not handled or not allowed 2026-01-25 06:13:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012390-adapter-7-16-inch-t-x-1-4-inch-p-3-8t-inch>: HTTP status code is not handled or not allowed 2026-01-25 06:13:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hurricane-360-hand-chain-hoist-3-ton-20-ft-lift>: HTTP status code is not handled or not allowed 2026-01-25 06:13:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coffing-lsb-ratchet-lever-hoist-3-ton-20-ft-lift>: HTTP status code is not handled or not allowed 2026-01-25 06:13:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/461502-board-control-for-manitowoc-machines>: HTTP status code is not handled or not allowed 2026-01-25 06:13:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/461506-t-stat-d1-d18>: HTTP status code is not handled or not allowed 2026-01-25 06:13:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-cabinet-with-removable-bins-46>: HTTP status code is not handled or not allowed 2026-01-25 06:13:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-cabinet-with-removable-bins-51>: HTTP status code is not handled or not allowed 2026-01-25 06:13:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cyclone-hand-chain-hoist-on-low-headroom-geared-trolley-2-ton-15-ft-lift>: HTTP status code is not handled or not allowed 2026-01-25 06:13:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/266224-roller-drawer-slide-metal>: HTTP status code is not handled or not allowed 2026-01-25 06:13:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-3-shelf-boltless-shelving-starter-36inw-x-18ind-x-36inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 06:13:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-3-shelf-boltless-shelving-starter-36inw-x-12ind-x-48inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 06:13:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-3-shelf-boltless-shelving-starter-48inw-x-12ind-x-48inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 06:13:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cyclone-hand-chain-hoist-on-geared-trolley-4-ton-10-ft-lift>: HTTP status code is not handled or not allowed 2026-01-25 06:13:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safe-heavy-duty-key-safe-cabinet-17-w-x-9-1-2-d-x-22-h-electronic-lock-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:13:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-cabinet-with-removable-bins-32>: HTTP status code is not handled or not allowed 2026-01-25 06:13:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safe-burglary-fire-safe-21-3-4-w-x-25-d-x-22-1-2-h-digital-lock-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:13:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-underbody-tuck-box-14-12-18-silver-1705149>: HTTP status code is not handled or not allowed 2026-01-25 06:13:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-lift-beam-low-headroom-multiple-length-2000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:13:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flock-lined-x-large-nitrile-glove-18-mil-size-10>: HTTP status code is not handled or not allowed 2026-01-25 06:13:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cyclone-hand-chain-hoist-on-geared-trolley-2-ton-10-ft-lift>: HTTP status code is not handled or not allowed 2026-01-25 06:13:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-4-shelf-boltless-shelving-starter-36inw-x-12ind-x-96inh-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 06:13:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-safety-view-locker-five-tier-12x12x12-5-doors-unassembled-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:13:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-x-30-metal-shelf-for-security-truck-red>: HTTP status code is not handled or not allowed 2026-01-25 06:13:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/35130-fast-cut-thin-cutting-wheel-4-1-2-inch-x-0045-inch-x-7-8-inch-type-1-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-25 06:13:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/75t3q-cl-75w-halogen-w-double-ended-base-120v>: HTTP status code is not handled or not allowed 2026-01-25 06:13:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8013607-probe-kit-25-inch-lvx20x-eeg16x-2xx-for-henny-penny>: HTTP status code is not handled or not allowed 2026-01-25 06:13:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medium-duty-service-door-single-panel-beige-3ft-x-7ft-with-kickplate>: HTTP status code is not handled or not allowed 2026-01-25 06:13:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-bicycle-small-front-wire-basket>: HTTP status code is not handled or not allowed 2026-01-25 06:13:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-42-inch-high-post-chrome-4-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:13:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-54-inch-high-post-Poly-Z-Brite-4-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:13:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cathodic-descaler-for-market-forge-91-1337>: HTTP status code is not handled or not allowed 2026-01-25 06:13:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-68-inch-high-post-nexelon-4-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:13:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-14-inch-high-post-poly-green-4-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:13:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kent-champion-28-vacuum-bag>: HTTP status code is not handled or not allowed 2026-01-25 06:13:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/water-filter-manifold-for-ice-makers-producing-up-to-1-000-lbs>: HTTP status code is not handled or not allowed 2026-01-25 06:13:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/100t4-g9-cl-100w-halogen-w-double-loop-base>: HTTP status code is not handled or not allowed 2026-01-25 06:13:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/or-schedule-36x48-magnetic-steel-whiteboard-with-aluminum-frame>: HTTP status code is not handled or not allowed 2026-01-25 06:13:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/75a21-f-12v-75w-incandescent-w-medium-base>: HTTP status code is not handled or not allowed 2026-01-25 06:13:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f32t8-830-env-32w-fluorescent-w-medium-bi-pin-base-warm>: HTTP status code is not handled or not allowed 2026-01-25 06:13:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-security-computer-cabinet-blue-1>: HTTP status code is not handled or not allowed 2026-01-25 06:13:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dispense-rite-countertop-lid-straw-condiment-organizer-wide>: HTTP status code is not handled or not allowed 2026-01-25 06:13:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-density-flat-bag-17in-x-21in-1-mil-pkg-qty-1000>: HTTP status code is not handled or not allowed 2026-01-25 06:13:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-ldpe-pallet-cover-51in-x-97in-2-mil-pallet-size-51-x-49-x-97-pkg-qty-50>: HTTP status code is not handled or not allowed 2026-01-25 06:13:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-8-to-8-bearing-splitter-57bs6>: HTTP status code is not handled or not allowed 2026-01-25 06:13:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-row-national-rep-tip-n-roll-bleacher-12ft-l-sgl-footboard>: HTTP status code is not handled or not allowed 2026-01-25 06:13:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-track-seal-top-bag-2-1-2in-x-3in-4-mil-pkg-qty-1000>: HTTP status code is not handled or not allowed 2026-01-25 06:13:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-gauge-heavy-duty-60-w-x-24-d-x-84-h-cabinet>: HTTP status code is not handled or not allowed 2026-01-25 06:13:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-deluxe-literature-mailers-10l-x-4w-x-4h-kraft-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-25 06:13:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/741468-gasket-beaded-sponge-10>: HTTP status code is not handled or not allowed 2026-01-25 06:13:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/legal-size-fireproof-file-cabinet-21-w-x-25-d-x-53-h-putty>: HTTP status code is not handled or not allowed 2026-01-25 06:13:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-deluxe-literature-mailers-16l-x-12w-x-6h-kraft-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-25 06:13:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waste-oil-drain-8-gallon-plastic>: HTTP status code is not handled or not allowed 2026-01-25 06:13:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jib-crane-wall-mounted-under-braced-2-000-lb-capacity-1>: HTTP status code is not handled or not allowed 2026-01-25 06:13:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/engine-stand-1-000-capacity-lbs-19-1-2-w-front-33-1-4-w-rear-36-l>: HTTP status code is not handled or not allowed 2026-01-25 06:13:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/datum-tekstak-laptop-storage-locker-8-tier-electronic-lock-7>: HTTP status code is not handled or not allowed 2026-01-25 06:13:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torque-stick-13-16-3-4-drive-475-ft-lbs-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:13:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2441089-board-control-qcse-for-star-manufacturing>: HTTP status code is not handled or not allowed 2026-01-25 06:13:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2691048-door-assembly-slide-top-36-inch-w-for-perlick-refrigeration>: HTTP status code is not handled or not allowed 2026-01-25 06:13:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-lid-24-gallon-black>: HTTP status code is not handled or not allowed 2026-01-25 06:13:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50t4-cl-50w-halogen-w-bi-pin-base-12v>: HTTP status code is not handled or not allowed 2026-01-25 06:13:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-lid-36-gallon-black>: HTTP status code is not handled or not allowed 2026-01-25 06:13:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cardboard-corrugated-boxes-29l-x-24w-x-24h-kraft-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 06:13:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/461673-thermostat-assembly-for-alto-shaam>: HTTP status code is not handled or not allowed 2026-01-25 06:13:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/light-duty-floor-crane-1000-lb-capacity-10>: HTTP status code is not handled or not allowed 2026-01-25 06:13:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/461705-interface-board-for-frymaster>: HTTP status code is not handled or not allowed 2026-01-25 06:13:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tn-motor-filter-for-tn-cv30-1-cv38-1-cv48-2>: HTTP status code is not handled or not allowed 2026-01-25 06:13:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-cal-wide-rib-corrugated-rubber-floor-mat-1-8-thick-x-4-x-6-utility-runner>: HTTP status code is not handled or not allowed 2026-01-25 06:13:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cardboard-corrugated-boxes-48l-x-24w-x-8h-kraft-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 06:13:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drivable-mini-scissor-lift-dmsl-19c>: HTTP status code is not handled or not allowed 2026-01-25 06:13:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-w-x-30-d-mobile-workbench-stainless-steel-square-edge-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:13:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/import-48-collet-capacity-r-8-collet-rack>: HTTP status code is not handled or not allowed 2026-01-25 06:13:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-w-x-36-d-shop-top-square-edge-workbench-adjustable-height-black>: HTTP status code is not handled or not allowed 2026-01-25 06:13:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-w-x-30-d-esd-square-edge-workbench-adjustable-height-black>: HTTP status code is not handled or not allowed 2026-01-25 06:13:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wheel-semi-steel-8x2-with-5-8-axle>: HTTP status code is not handled or not allowed 2026-01-25 06:13:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/apc-smart-ups-rt-surta48xlbp>: HTTP status code is not handled or not allowed 2026-01-25 06:13:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cross-beam-adapter>: HTTP status code is not handled or not allowed 2026-01-25 06:13:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cft8w-4p-830-18w-compact-fluorescent-w-gx24vq-2-base-warm>: HTTP status code is not handled or not allowed 2026-01-25 06:13:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torque-limiting-extension-3-4-drive-350-ft-lbs>: HTTP status code is not handled or not allowed 2026-01-25 06:13:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10500-series-laminate-wood-desk-left-return-harvest>: HTTP status code is not handled or not allowed 2026-01-25 06:13:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/office-desk-with-6-drawers-72-x-24-cherry>: HTTP status code is not handled or not allowed 2026-01-25 06:13:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kx40fr-3m-dc-40w-halogen-w-dc-bay-base>: HTTP status code is not handled or not allowed 2026-01-25 06:13:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/apc-smart-ups-700va-su700x93>: HTTP status code is not handled or not allowed 2026-01-25 06:13:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pocket-aneroid-sphygmomanometer-with-adult-cuff-and-case>: HTTP status code is not handled or not allowed 2026-01-25 06:14:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-1132-on-off-lighted-rocker-switch-16a-250v>: HTTP status code is not handled or not allowed 2026-01-25 06:14:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprague-rappaport-stethoscope-22-tubing>: HTTP status code is not handled or not allowed 2026-01-25 06:14:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-mailers-5l-x-2w-x-2h-kraft-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-25 06:14:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/create-a-capbump-cap-insert-with-foam-for-6-panel-ball-cap-19402>: HTTP status code is not handled or not allowed 2026-01-25 06:14:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-mechanics-gloves-hi-viz-pink-s-28858>: HTTP status code is not handled or not allowed 2026-01-25 06:14:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5t2-50-5w-compact-fluorescent-w-medium-base-natural-light>: HTTP status code is not handled or not allowed 2026-01-25 06:14:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-20-26t4-27-12-w-compact-fluorescent-w-medium-base-soft-white>: HTTP status code is not handled or not allowed 2026-01-25 06:14:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bottle-jack-12-ton-short-body>: HTTP status code is not handled or not allowed 2026-01-25 06:14:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bin-nestable-8-3-8-w-x-23-5-8-d-x-4-h-red-2>: HTTP status code is not handled or not allowed 2026-01-25 06:14:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bin-nestable-8-3-8-w-x-23-5-8-d-x-4-h-yellow-2>: HTTP status code is not handled or not allowed 2026-01-25 06:14:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/separable-roller-follower-metric-nast25zz-double-shielded-25-mm-bore-52-mm-od>: HTTP status code is not handled or not allowed 2026-01-25 06:14:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-153-cartridge-filter-16-gallon-wet-dry-vacuum>: HTTP status code is not handled or not allowed 2026-01-25 06:14:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roller-follower-full-comp-inch-cry28vuur-crowned-od-double-sealed-1-34-od>: HTTP status code is not handled or not allowed 2026-01-25 06:14:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/purell-sani-wipe-twlet-indv-wrap-1000>: HTTP status code is not handled or not allowed 2026-01-25 06:14:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 74045 pages (at 99 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:14:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/antibac-gld-soap-btl-fresh-81-l>: HTTP status code is not handled or not allowed 2026-01-25 06:14:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/75-ton-ironworker-3-phase-460v-iw75-3p460>: HTTP status code is not handled or not allowed 2026-01-25 06:14:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/prem-met-air-fresh-66-oz-arsl-citrus-12>: HTTP status code is not handled or not allowed 2026-01-25 06:14:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grease-needle-adapter-fine-point>: HTTP status code is not handled or not allowed 2026-01-25 06:14:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jlt-1889-0600-6-digital-caliper>: HTTP status code is not handled or not allowed 2026-01-25 06:14:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dh-cr-3-small-cable-cover-3925x525x075-channel-15x05>: HTTP status code is not handled or not allowed 2026-01-25 06:14:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp10p-3-m219-poly-self-priming-mag-drive-pump-1hp-208-230-460v-3-phase-55-gpm>: HTTP status code is not handled or not allowed 2026-01-25 06:14:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fc22t5-835-22w-fluorescent-w-2gx13-base-neutral-white>: HTTP status code is not handled or not allowed 2026-01-25 06:14:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-container-forkli-able-with-lid-4>: HTTP status code is not handled or not allowed 2026-01-25 06:14:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/big-joe-174-cb-22-fully-powered-counterbalanced-li-truck-2200-lb-104-li>: HTTP status code is not handled or not allowed 2026-01-25 06:14:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/switchboard-non-conductive-diamond-mat-3-wide-full-75-ft-roll-black-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:14:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a1212ch-continuous-hinge-wclamps-type-12-1200x1200x600-steelgray>: HTTP status code is not handled or not allowed 2026-01-25 06:14:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cf42dt-e-in-841-42w-compact-fluorescent-w-gx24vq-4-base-cool-white>: HTTP status code is not handled or not allowed 2026-01-25 06:14:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ase12x12x6nk-pull-box-screw-cover-1200x1200x600-steelgray>: HTTP status code is not handled or not allowed 2026-01-25 06:14:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tote-box-dividable-14>: HTTP status code is not handled or not allowed 2026-01-25 06:14:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-shelf-high-capacity-boltless-shelving-steel-deck-add-on-72inw-x-18ind-x-72inh>: HTTP status code is not handled or not allowed 2026-01-25 06:14:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/65-ton-ironworker-3-phase-380v-50-hz-iw65-3p380>: HTTP status code is not handled or not allowed 2026-01-25 06:14:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/266284-bracket-hinge-w-closer-top>: HTTP status code is not handled or not allowed 2026-01-25 06:14:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foot-pedal-wd-cabinets-203268yw>: HTTP status code is not handled or not allowed 2026-01-25 06:14:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flexvolt-174-20v-60v-max-power-tool-battery-12-0ah-lithium-ion>: HTTP status code is not handled or not allowed 2026-01-25 06:14:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lever-color-safe-bleach-cn-vnd-1002-oz>: HTTP status code is not handled or not allowed 2026-01-25 06:14:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tote-box-dividable-35>: HTTP status code is not handled or not allowed 2026-01-25 06:14:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2661107-freezer-door-assembly-358-359-758-772-for-taylor-freezer>: HTTP status code is not handled or not allowed 2026-01-25 06:14:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ind-qlty-steel-wool-hand-pad-000-fine-126>: HTTP status code is not handled or not allowed 2026-01-25 06:14:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ind-qlty-steel-wool-hand-pad-2-med-coarse-126>: HTTP status code is not handled or not allowed 2026-01-25 06:14:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ind-qlty-steel-wool-reel-3-coarse-65-lb>: HTTP status code is not handled or not allowed 2026-01-25 06:14:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/541201-dual-solenoid-valve-120v-for-southbend-range>: HTTP status code is not handled or not allowed 2026-01-25 06:14:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dsc-214m-flush-access-door-for-all-surf-stud-36wx36h-mp3636s>: HTTP status code is not handled or not allowed 2026-01-25 06:14:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1171268-cartridge-waterfilter-cfs8000-s-for-cuno-inc>: HTTP status code is not handled or not allowed 2026-01-25 06:14:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cut-resistant-gloves-micro-foam-nitrile-coated-ansi-a4-l-graypurple>: HTTP status code is not handled or not allowed 2026-01-25 06:14:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/x5t3-1-4-5w-miniature-sealed-beams-w-mini-wedgebase-12v>: HTTP status code is not handled or not allowed 2026-01-25 06:14:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/counter-weight-drum-handler-manual-powered-78-h>: HTTP status code is not handled or not allowed 2026-01-25 06:14:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-ft-goat-hay-rack-galvanized>: HTTP status code is not handled or not allowed 2026-01-25 06:14:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7600-multi-gas-vapor-smart-cartridges>: HTTP status code is not handled or not allowed 2026-01-25 06:14:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5031119-digital-controller-refrigerator-for-hoshizaki-of-america>: HTTP status code is not handled or not allowed 2026-01-25 06:14:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7100-organic-vapor-cartridges>: HTTP status code is not handled or not allowed 2026-01-25 06:14:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/protective-eye-shield-kit-clear>: HTTP status code is not handled or not allowed 2026-01-25 06:14:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slit-loom-5>: HTTP status code is not handled or not allowed 2026-01-25 06:14:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hid-retrofit-lamp-115w-5000k-180-paddle>: HTTP status code is not handled or not allowed 2026-01-25 06:14:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011835-thermocouple-48-inch>: HTTP status code is not handled or not allowed 2026-01-25 06:14:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cut-resistant-gloves-micro-foam-nitrile-coated-ansi-a5-l-hi-vis-lime>: HTTP status code is not handled or not allowed 2026-01-25 06:14:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/krp-150fr-fire-rated-access-door-for-wall-ceil-paddle-handle-22wx36h-krpp3622ph>: HTTP status code is not handled or not allowed 2026-01-25 06:14:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spiral-cut-loom-1>: HTTP status code is not handled or not allowed 2026-01-25 06:14:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-cantilever-rack-single-sided-add-on-medium-duty-48-w-x-33-d-x-8-h-with-lip>: HTTP status code is not handled or not allowed 2026-01-25 06:14:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drywall-recessed-door-w-factory-installed-drywall-lock-24wx24h-rdwpd2424l>: HTTP status code is not handled or not allowed 2026-01-25 06:14:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011165-cuno-rplmnt-cartridge-scale-to-chlor-for-cuno-inc>: HTTP status code is not handled or not allowed 2026-01-25 06:14:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6413-5w-miniature-sealed-beams-w-festoon-base>: HTTP status code is not handled or not allowed 2026-01-25 06:14:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/l-i-hss-straight-flute-chucking-reamer-2615>: HTTP status code is not handled or not allowed 2026-01-25 06:14:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scratch-resistant-strip-door-curtain-12-w-x-7-h>: HTTP status code is not handled or not allowed 2026-01-25 06:14:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/325-ultra-winterliner-one-size-blueblack>: HTTP status code is not handled or not allowed 2026-01-25 06:14:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scratch-resistant-strip-door-curtain-10-w-x-8-h>: HTTP status code is not handled or not allowed 2026-01-25 06:14:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tripower-classical-section-molded-notch-vbelt-gates-bx46>: HTTP status code is not handled or not allowed 2026-01-25 06:14:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tripower-classical-section-molded-notch-vbelt-gates-ax49>: HTTP status code is not handled or not allowed 2026-01-25 06:14:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1681384-wire-assembly-ignitionmodule-for-frymaster>: HTTP status code is not handled or not allowed 2026-01-25 06:14:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scratch-resistant-strip-door-curtain-6-w-x-12-h>: HTTP status code is not handled or not allowed 2026-01-25 06:14:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scratch-resistant-strip-door-curtain-12-w-x-12-h>: HTTP status code is not handled or not allowed 2026-01-25 06:14:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25a19-f-25w-incandescent-w-medium-base-120v>: HTTP status code is not handled or not allowed 2026-01-25 06:14:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010487-thermostat-fdo>: HTTP status code is not handled or not allowed 2026-01-25 06:14:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25g16-1-2-25w-incandescent-w-candelabra-base-120v>: HTTP status code is not handled or not allowed 2026-01-25 06:14:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1331442-cleaner-burner-port-tip-set>: HTTP status code is not handled or not allowed 2026-01-25 06:14:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibco-silent-pneumatic-turbine-vibrator-mlt-13>: HTTP status code is not handled or not allowed 2026-01-25 06:14:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hazmat-pool-with-drain-74-x-74-x-8-5901-ye-d>: HTTP status code is not handled or not allowed 2026-01-25 06:14:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-step-orange-extra-heavy-duty-steel-rolling-ladder-expanded-metal-tread>: HTTP status code is not handled or not allowed 2026-01-25 06:14:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uhmw-polyethylene-plastic-bar-1-8-thick-x-2-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 06:14:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibco-silent-pneumatic-turbine-vibrator-vs-160al>: HTTP status code is not handled or not allowed 2026-01-25 06:14:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/x-large-dripillow-berm-w-pad-38-x-42-5680-ye>: HTTP status code is not handled or not allowed 2026-01-25 06:14:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hipower-ii-classical-section-wrapped-vbelt-gates-b136>: HTTP status code is not handled or not allowed 2026-01-25 06:14:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibco-12-volt-electric-vibrator-dc-700>: HTTP status code is not handled or not allowed 2026-01-25 06:14:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-48x18x96-with-5-shelves-no-deck>: HTTP status code is not handled or not allowed 2026-01-25 06:14:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pl-u-bend-fluorescent-bulb-g23-base-9w-530-lumens-3500k-neutral-white-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 06:14:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vibco-small-impact-electric-vibrator-spr-20>: HTTP status code is not handled or not allowed 2026-01-25 06:14:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012181-ignition-module>: HTTP status code is not handled or not allowed 2026-01-25 06:14:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uhmw-polyethylene-plastic-bar-1-4-thick-x-4-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 06:14:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/generac-7225-guardian-14kw-120-240v-1-phase-air-cooled-standby-generator-ng-lp-wifi-enabled>: HTTP status code is not handled or not allowed 2026-01-25 06:14:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15ca8-15w-incandescent-w-candelabra-base-120v>: HTTP status code is not handled or not allowed 2026-01-25 06:14:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/918-series-airtight-watertight-case-with-foam-918-1003-16-7-8-l-x-12-7-8-w-x-9-5-16-h-orange>: HTTP status code is not handled or not allowed 2026-01-25 06:14:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cantilever-rack-inclined-arm-10>: HTTP status code is not handled or not allowed 2026-01-25 06:14:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plt-fluorescent-bulb-gx24q4-base-42w-3200-lumens-6500k-daylight-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 06:14:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cantilever-rack-straight-arm-with-2-lip-26>: HTTP status code is not handled or not allowed 2026-01-25 06:14:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/38-1277-duplex-receptacle-nema-5-20r>: HTTP status code is not handled or not allowed 2026-01-25 06:14:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/respirator-cartridge-type-half-mask-2>: HTTP status code is not handled or not allowed 2026-01-25 06:15:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/250-1-4-diameter-vinyl-coated-5-16-diameter-7x19-galvanized-aircraft-cable>: HTTP status code is not handled or not allowed 2026-01-25 06:15:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/381449-wire-harness>: HTTP status code is not handled or not allowed 2026-01-25 06:15:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/misty-bolex-23-percent-hydrochloric-acid-bowl-cleaner-wintergreen>: HTTP status code is not handled or not allowed 2026-01-25 06:15:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/250-3-4-dia-6x36-extra-improved-plow-steel-bright-wire-rope-independent-wire-rope-center>: HTTP status code is not handled or not allowed 2026-01-25 06:15:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ivory-dish-detergent-liquid-classic-scent-24-oz-bottle>: HTTP status code is not handled or not allowed 2026-01-25 06:15:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/381283-locking-plug>: HTTP status code is not handled or not allowed 2026-01-25 06:15:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unisan-clip-on-dust-mop-handle-double-lacquered-wood-1516-diameter-60>: HTTP status code is not handled or not allowed 2026-01-25 06:15:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/38-1292-type-so-power-cord-3-wire-10-gauge-600v-30a-25>: HTTP status code is not handled or not allowed 2026-01-25 06:15:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ushio-f54t5ho-830-46in85cri>: HTTP status code is not handled or not allowed 2026-01-25 06:15:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-1-thick-x-1-1-2-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 06:15:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milton-1864-14-twist-lock-universal-coupler-2-fnpt-10-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:15:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/65-manhandler-hoist-3-16-galvanized-steel-wire-rope-snap-hook-350-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:15:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medium-duty-appliance-extension-cord-143-gauge-15a-125v-12l-beigegray>: HTTP status code is not handled or not allowed 2026-01-25 06:15:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ushio-enh-jcr120v-250w>: HTTP status code is not handled or not allowed 2026-01-25 06:15:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milton-s-99705w-m-style-industrial-safety-coupler-1-4-fnpt>: HTTP status code is not handled or not allowed 2026-01-25 06:15:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/north-7583p100l-organic-vapor-acid-gas-cartridge-p100-filter-2-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:15:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 74142 pages (at 97 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:15:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milton-s-1838w-h-style-industrial-plug-3-8-fnpt>: HTTP status code is not handled or not allowed 2026-01-25 06:15:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metro-aluminum-foil-roll-lighter-gauge-standard-18-x-500-ft-silver-3>: HTTP status code is not handled or not allowed 2026-01-25 06:15:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/north-single-use-disposable-safety-pack-includes-masks-gloves-wipes>: HTTP status code is not handled or not allowed 2026-01-25 06:15:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-1-thick-x-4-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 06:15:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-3-4-thick-x-4-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 06:15:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-a15-light-bulb-e17-base-6w-480-lumens-2700k-soft-white-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 06:15:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-1-8-thick-x-4-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 06:15:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rapidwire-base-plate-loose-partition>: HTTP status code is not handled or not allowed 2026-01-25 06:15:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-g2sc-112-ss>: HTTP status code is not handled or not allowed 2026-01-25 06:15:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010192-sensor-flame-pilot>: HTTP status code is not handled or not allowed 2026-01-25 06:15:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/881324-filter-drier-liquid-line-for-emerson-radio-corp>: HTTP status code is not handled or not allowed 2026-01-25 06:15:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-in-comfort-cooler-belt-drive-whole-house-fan-1-3-hp>: HTTP status code is not handled or not allowed 2026-01-25 06:15:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-in-comfort-cooler-belt-drive-whole-house-fan-1-2-hp>: HTTP status code is not handled or not allowed 2026-01-25 06:15:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8012267-valve-gas-reg-nat3-4-x-3-4-w-intermitt-for-cleveland-range>: HTTP status code is not handled or not allowed 2026-01-25 06:15:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polycarbonate-plastic-sheet-3-8-thick-x-48-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 06:15:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/turn-signal-flasher-standard-ignition-efl67>: HTTP status code is not handled or not allowed 2026-01-25 06:15:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/840-style-woven-wire-double-hinged-door-6w-x-7h-12-5-1-4-overall-height>: HTTP status code is not handled or not allowed 2026-01-25 06:15:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-gsc-093-ss>: HTTP status code is not handled or not allowed 2026-01-25 06:15:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-par16-recessed-floodlight-bulb-e26-base-6w-500-lumens-50k-daylight-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 06:15:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-622-hand-chain-hoist-3-ton-cap-15ft-lift>: HTTP status code is not handled or not allowed 2026-01-25 06:15:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/76-1196-7-gpm-scale-and-sediment-reduction-filter-cartridge>: HTTP status code is not handled or not allowed 2026-01-25 06:15:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8248z-class-2-twotone-surveyors-vest-orange-24065>: HTTP status code is not handled or not allowed 2026-01-25 06:15:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fifo-flow-rack-m36-4-steel-shelves-36-27-37-w-x-31-49-71-d-x-73-3-13-h-485-lb-cap-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:15:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2531369-wire-hi-temp-10-ga-tan-100>: HTTP status code is not handled or not allowed 2026-01-25 06:15:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-g25-globe-light-bulb-e26-base-6w-520-lumens-2700k-soft-white-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 06:15:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-rack-tear-drop-row-spacer-12-depth>: HTTP status code is not handled or not allowed 2026-01-25 06:15:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72x30-esd-safety-edge-mobile-power-apron-production-bench-tan-32313639>: HTTP status code is not handled or not allowed 2026-01-25 06:15:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/import-26pc-transfer-punch-set-letters-a-z>: HTTP status code is not handled or not allowed 2026-01-25 06:15:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1781080-probe-water-level-w-hex-fit-for-wilbur-curtis-co>: HTTP status code is not handled or not allowed 2026-01-25 06:15:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-g2sc-168-ss>: HTTP status code is not handled or not allowed 2026-01-25 06:15:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/photoluminescent-fire-extinguisher-rigid-pvc-sign-non-adhesive>: HTTP status code is not handled or not allowed 2026-01-25 06:15:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-g16-frosted-globe-light-bulb-e26-base-5w-350-lumens-27k-warm-white-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 06:15:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-one-piece-clamping-collar-gm1c-60-b>: HTTP status code is not handled or not allowed 2026-01-25 06:15:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dorian-tool-super-quick-change-tool-post-sd35cxa>: HTTP status code is not handled or not allowed 2026-01-25 06:15:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nsf-fabricated-3-compart-sink-24l-x-24w-bowl-9h-splash-24l-left-right-drainboards-18-ga>: HTTP status code is not handled or not allowed 2026-01-25 06:15:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bark-spud-with-handle-tmw-08-24-l-21-5-8-wood-handle-green>: HTTP status code is not handled or not allowed 2026-01-25 06:15:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nsf-fabricated-3-compart-sink-16l-x-20w-bowl-9h-splash-18l-left-right-drainboards-18-ga>: HTTP status code is not handled or not allowed 2026-01-25 06:15:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-set-screw-collar-gmc-03-b>: HTTP status code is not handled or not allowed 2026-01-25 06:15:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grip-24-w-10-step-steel-rolling-ladder-20-d-top-step>: HTTP status code is not handled or not allowed 2026-01-25 06:15:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/38-1305-96-inch-heavy-duty-sjtow-appliance-power-cord-125v-14-gauge-wire>: HTTP status code is not handled or not allowed 2026-01-25 06:15:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/800-1-4-ratchet-red-composite>: HTTP status code is not handled or not allowed 2026-01-25 06:15:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6275-inside-corner-die-grinder>: HTTP status code is not handled or not allowed 2026-01-25 06:15:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-presentation-easel-69-maximum-height-metal-black>: HTTP status code is not handled or not allowed 2026-01-25 06:15:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-trash-can-dome-lid-dolly-44-gallon-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:15:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vitan-linear-globe-pendant-light-fixture-g9-base-15w-satin-nickel>: HTTP status code is not handled or not allowed 2026-01-25 06:15:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rockportrk6640-mens-more-energy-6-lace-to-toe-waterproof-work-boot-brown-size-9-m>: HTTP status code is not handled or not allowed 2026-01-25 06:15:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72x30-esd-square-edge-power-apron-production-bench-blue-32313816>: HTTP status code is not handled or not allowed 2026-01-25 06:15:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-a19-light-bulb-e26-base-6w-480-lumens-6500k-daylight-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-25 06:15:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tap-and-drill-sets-hss-in-metal-index-18-pc-set-national-fine>: HTTP status code is not handled or not allowed 2026-01-25 06:15:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ulti-mate-garage-2-0-series-5-piece-tall-cabinet-set-177-5-x-21-x-80-black>: HTTP status code is not handled or not allowed 2026-01-25 06:15:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-16-h3-taper-chamfer-hss-tmx-hand-tap-ground-rh-4-flutes>: HTTP status code is not handled or not allowed 2026-01-25 06:15:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reebokrb1910-mens-classic-skateboard-shoe-black-size-9-m>: HTTP status code is not handled or not allowed 2026-01-25 06:15:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-ft-pll-light-bulb-4-pin-2g11-base-17w-2350-lumens-30k-warm-white-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 06:15:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reebokrb1910-mens-classic-skateboard-shoe-black-size-105-w>: HTTP status code is not handled or not allowed 2026-01-25 06:15:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/faultless-rigid-plate-caster-6-polyurethane-wheel>: HTTP status code is not handled or not allowed 2026-01-25 06:15:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/747-business-full-strip-desk-stapler-20-sheet-capacity-steel-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:15:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ulti-mate-garage-2-0-series-3-piece-cabinet-set-83w-x-21d-x-80h-black>: HTTP status code is not handled or not allowed 2026-01-25 06:15:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-par30s-flood-light-bulb-e26-base-9w-850-lumens-3000k-warm-white-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 06:15:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-w-x-30-d-shop-top-square-edge-with-drawer-riser>: HTTP status code is not handled or not allowed 2026-01-25 06:15:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-30-steel-square-edge-with-drawer-riser>: HTTP status code is not handled or not allowed 2026-01-25 06:15:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-36-steel-square-edge-with-drawer>: HTTP status code is not handled or not allowed 2026-01-25 06:15:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-t12-filament-light-bulb-e26-base-3w-350-lumens-2700k-warm-white-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 06:15:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-suffocation-warning-poly-bags-12-w-x-15-l-1-mil-clear-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:15:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/561594-vacuum-breaker-for-insinger>: HTTP status code is not handled or not allowed 2026-01-25 06:15:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sts10b442-10-hp-oil-less-scroll-compressor-120-gallon-horizontal-116-psi-3-phase-460v>: HTTP status code is not handled or not allowed 2026-01-25 06:15:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/venture-patio-umbrella-w-silver-pole-132w-sunbrella-1a-spectrum-aztec-fabric>: HTTP status code is not handled or not allowed 2026-01-25 06:15:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8011032-ts-safety-valve>: HTTP status code is not handled or not allowed 2026-01-25 06:15:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/venture-patio-umbrella-with-silver-pole-132w-sunbrella-1a-spa-fabric>: HTTP status code is not handled or not allowed 2026-01-25 06:15:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sts151262-5-hp-oil-less-scroll-compressor-60-gallon-horizontal-116-psi-1-phase-208-230v>: HTTP status code is not handled or not allowed 2026-01-25 06:15:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-g2sc-187-b>: HTTP status code is not handled or not allowed 2026-01-25 06:15:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hg-2-hss-2-flute-bright-square-single-end-mill-1-8-x-3-8-x-3-8-x-2-5-16>: HTTP status code is not handled or not allowed 2026-01-25 06:15:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cap-bracket-strap-with-10-high-temp-face-shield-cb6-hd-im22-l6f>: HTTP status code is not handled or not allowed 2026-01-25 06:15:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/venture-patio-umbrella-with-silver-pole-132w-sunbrella-1a-forest-green-fabric>: HTTP status code is not handled or not allowed 2026-01-25 06:15:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-valve-5-port-2-pos-dbl-solenoid-1-4-nptf-240vac>: HTTP status code is not handled or not allowed 2026-01-25 06:15:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sam-splint-orange-76001>: HTTP status code is not handled or not allowed 2026-01-25 06:15:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-mesh-deck-48-wx12-d>: HTTP status code is not handled or not allowed 2026-01-25 06:15:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bca-367218-db-bookcase-all-welded-36-wx18-dx72-h-denim-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:15:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-valve-solenoid-1-4-nptf-3-port-2-pos-normally-clsd-or-normally-open-12vac-piped-exh>: HTTP status code is not handled or not allowed 2026-01-25 06:15:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-vertical-file-legal-size-4-drawer-desert-putty-18w-x-26-1-2d-x-52h>: HTTP status code is not handled or not allowed 2026-01-25 06:15:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/made-in-usa-w-1-water-hardening-drill-rod-1-2>: HTTP status code is not handled or not allowed 2026-01-25 06:15:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/outdoor-24in-x-40in-cocktail-table-aluminum-volcanic-black-sunset>: HTTP status code is not handled or not allowed 2026-01-25 06:15:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-piloted-valve-5-port-2-pos-sngl-air-ctrl-1-2-npt-cv-279>: HTTP status code is not handled or not allowed 2026-01-25 06:16:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aah-48rb-wr-storage-cabinet-all-welded-48-wx18-dx72-h-wine-red>: HTTP status code is not handled or not allowed 2026-01-25 06:16:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-step-steel-safety-rolling-ladder-w-weight-actuated-lock-16-w-perforated-step-orange-w-cal-osha-handrail-cal-wa052414p-o>: HTTP status code is not handled or not allowed 2026-01-25 06:16:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mountain-organic-natural-icemelter-44-lb-bag-200-20043>: HTTP status code is not handled or not allowed 2026-01-25 06:16:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paint-ink-cabinet-with-manual-close-double-door-40-gallon>: HTTP status code is not handled or not allowed 2026-01-25 06:16:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bca-368418-ts-bookcase-all-welded-36-wx18-dx84-h-tropic-sand>: HTTP status code is not handled or not allowed 2026-01-25 06:16:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-coupling-gscc-100-100>: HTTP status code is not handled or not allowed 2026-01-25 06:16:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acid-corrosive-cabinet-with-manual-close-double-door-30-gallon>: HTTP status code is not handled or not allowed 2026-01-25 06:16:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/linkit-450-series-portable-dirt-and-aggregate-conveyor-18l-x-18w>: HTTP status code is not handled or not allowed 2026-01-25 06:16:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-step-steel-stock-picking-ladder-serrated-tread-w-cal-osha-handrail-cal-spl-6-g>: HTTP status code is not handled or not allowed 2026-01-25 06:16:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8010231-thermostat-kit-for-montague-oven>: HTTP status code is not handled or not allowed 2026-01-25 06:16:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-linear-bar-vanity-light-fixture-3540w-4050-lumens-47-1316l-brushed-nickel>: HTTP status code is not handled or not allowed 2026-01-25 06:16:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mount-bracket-assembly-kit-for-use-w-m3v1-valve>: HTTP status code is not handled or not allowed 2026-01-25 06:16:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bca-363018-w-bookcase-all-welded-36-wx18-dx30-h-white>: HTTP status code is not handled or not allowed 2026-01-25 06:16:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rb8850-stealth-8-boot-with-side-zipper-composite-toe-men-s-sz-11-5-m-wide-coyote>: HTTP status code is not handled or not allowed 2026-01-25 06:16:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnetic-label-holders-6-w-x-75-h-12-per-package>: HTTP status code is not handled or not allowed 2026-01-25 06:16:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heritage-backless-bench-recycled-plastic-6-ft-black-frame-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:16:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 74241 pages (at 99 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:16:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-valve-5-port-2-pos-sngl-solenoid-1-4-nptf-24vac>: HTTP status code is not handled or not allowed 2026-01-25 06:16:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/newport-patio-umbrella-with-silver-pole-108w-sunbrella-1a-seaside-fabric>: HTTP status code is not handled or not allowed 2026-01-25 06:16:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/newport-patio-umbrella-with-silver-pole-108w-sunbrella-1a-dolce-oasis-fabric>: HTTP status code is not handled or not allowed 2026-01-25 06:16:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grip-24-w-6-step-steel-rolling-ladder-21-d-top-step-w-handrails-orange-w-cal-osha-handrail-cal-fsh62621g-o>: HTTP status code is not handled or not allowed 2026-01-25 06:16:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/globe-style-outdoor-vanity-light-fixture-60w-6l-black>: HTTP status code is not handled or not allowed 2026-01-25 06:16:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regulator-mini-type-1-8-npt>: HTTP status code is not handled or not allowed 2026-01-25 06:16:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-valve-5-port-2-pos-man-valve-1-8-nptf-port-fingertip-lever-actr-air-spring-ret>: HTTP status code is not handled or not allowed 2026-01-25 06:16:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-w-x-24-h-x-4-d-mini-pleated-merv-15-air-filter-box-style-global-industrial>: HTTP status code is not handled or not allowed 2026-01-25 06:16:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hazard-communication-in-construction-environments-safety-dvd-program>: HTTP status code is not handled or not allowed 2026-01-25 06:16:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-dolly-27-l-x-16-w-with-open-deck-steel-wheels>: HTTP status code is not handled or not allowed 2026-01-25 06:16:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25-w-x-14-h-x-1-d-pleated-merv-8-standard-capacity-air-filter-global-industrial>: HTTP status code is not handled or not allowed 2026-01-25 06:16:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-24-w-8-step-steel-rolling-ladder-14-d-top-step-w-handrails-lock-step-orange-w-cal-osha-handrail-cal-fs083214p-o>: HTTP status code is not handled or not allowed 2026-01-25 06:16:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zip-a-duct-16in-gray-straight-section-with-vents-667-cfm>: HTTP status code is not handled or not allowed 2026-01-25 06:16:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hardwood-platform-truck-60-x-30-1400-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:16:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/supported-scaffolding-safety-in-construction-environments-dvd-kit>: HTTP status code is not handled or not allowed 2026-01-25 06:16:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/venture-patio-umbrella-w-silver-pole-132w-sunbrella-1a-heather-beige-fabric>: HTTP status code is not handled or not allowed 2026-01-25 06:16:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hardwood-platform-truck-60-x-30-2000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:16:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-w-x-24-h-x-18-d-6-pocket-merv-16-air-filter-global-industrial>: HTTP status code is not handled or not allowed 2026-01-25 06:16:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/820-2-horizontal-acting-toggle-clamp-vertical-mounting-base-stainless-steel-size-230>: HTTP status code is not handled or not allowed 2026-01-25 06:16:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergotron-100-series-pivot-single>: HTTP status code is not handled or not allowed 2026-01-25 06:16:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bca-365218-ts-bookcase-all-welded-36-wx18-dx52-h-tropic-sand>: HTTP status code is not handled or not allowed 2026-01-25 06:16:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fr-tacoma-heavyweight-zip-front-hoodie-lgt>: HTTP status code is not handled or not allowed 2026-01-25 06:16:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/graphic-signs-caution-forkli-traffic-area-vinyl-10-w-x-14-h>: HTTP status code is not handled or not allowed 2026-01-25 06:16:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zip-a-duct-36in-gray-inlet-section>: HTTP status code is not handled or not allowed 2026-01-25 06:16:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergotron-lx-dual-stacking-arm>: HTTP status code is not handled or not allowed 2026-01-25 06:16:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rvz53fr-type-r-class-3-flame-resistant-vest-l-hi-vis-lime>: HTTP status code is not handled or not allowed 2026-01-25 06:16:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oshas-forklift-powered-industrial-truck-safety-compliance-manual>: HTTP status code is not handled or not allowed 2026-01-25 06:16:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/graphic-signs-danger-eye-protection-required-vinyl-10-w-x-14-h-2>: HTTP status code is not handled or not allowed 2026-01-25 06:16:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gondolashelving-closedbasefront36-black-796516>: HTTP status code is not handled or not allowed 2026-01-25 06:16:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fr-tacoma-heavyweight-zip-front-hoodie-3x>: HTTP status code is not handled or not allowed 2026-01-25 06:16:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aah-30rb-h-grn-storage-cabinet-all-welded-30-wx18-dx72-h-hunter-green>: HTTP status code is not handled or not allowed 2026-01-25 06:16:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e-z-install-flexible-water-line-kit>: HTTP status code is not handled or not allowed 2026-01-25 06:16:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-deskside-green-recycling-container-28-1-8-qt>: HTTP status code is not handled or not allowed 2026-01-25 06:16:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/valor-4000-v41pwe15t-water-resistant-scale-30-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:16:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aah-48rbmag3-db-storage-cabinet-all-welded-48-wx24-dx78-h-black>: HTTP status code is not handled or not allowed 2026-01-25 06:16:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-folding-bath-bench>: HTTP status code is not handled or not allowed 2026-01-25 06:16:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-step-stock-picking-ladder-perf-step>: HTTP status code is not handled or not allowed 2026-01-25 06:16:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/orange-folding-pole-stretcher>: HTTP status code is not handled or not allowed 2026-01-25 06:16:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/residential-heat-pump-unit-15-seer2-35-ton-208230v>: HTTP status code is not handled or not allowed 2026-01-25 06:16:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss-hose-braid-w-150-fixed-x-floating-fl-6-x-24>: HTTP status code is not handled or not allowed 2026-01-25 06:16:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss-hose-braid-w-grooved-nipples-2-5-x-24>: HTTP status code is not handled or not allowed 2026-01-25 06:16:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ss-hose-braid-w-grooved-nipples-4-x-36>: HTTP status code is not handled or not allowed 2026-01-25 06:16:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-step-extra-heavy-duty-steel-rolling-safety-ladder-expanded-metal-tread-w-cal-osha-handrail-cal-ml063221x>: HTTP status code is not handled or not allowed 2026-01-25 06:16:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e-z-walker-caddy>: HTTP status code is not handled or not allowed 2026-01-25 06:16:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aah-48rbmag3-r-storage-cabinet-all-welded-48-wx24-dx78-h-red>: HTTP status code is not handled or not allowed 2026-01-25 06:16:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-dual-access-fiberglass-step-ladder-2>: HTTP status code is not handled or not allowed 2026-01-25 06:16:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1000163-tongue-buckle-belt-restraint-310-lbs-medium>: HTTP status code is not handled or not allowed 2026-01-25 06:16:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/combat-application-tourniquet-c-a-t>: HTTP status code is not handled or not allowed 2026-01-25 06:16:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1341050-pro-shock-absorbing-lanyard-6l-130-310-cap-lbs>: HTTP status code is not handled or not allowed 2026-01-25 06:16:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-folding-security-gate-7-1-2-x-6-1-2>: HTTP status code is not handled or not allowed 2026-01-25 06:16:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-w-x-18-h-x-2-d-pleated-merv-8-standard-capacity-air-filter-global-industrial>: HTTP status code is not handled or not allowed 2026-01-25 06:16:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1001220-rescue-wristlets-18l-x-1w-310-cap-lbs>: HTTP status code is not handled or not allowed 2026-01-25 06:16:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-rollator-fold-up-and-removable-back-support-padded-seat-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:16:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acessory-kit-mh45-b4b-noise-cancelling-speaker-microphone>: HTTP status code is not handled or not allowed 2026-01-25 06:16:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/infinity-locker-single-tier-15x18x72-1-door-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:16:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1202753-rope-lifeline-30l-310-cap-lbs>: HTTP status code is not handled or not allowed 2026-01-25 06:16:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clp1060-bluetooth-swivel-earpiece>: HTTP status code is not handled or not allowed 2026-01-25 06:16:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1001090-soft-sling-seat-310-cap-lbs-universal>: HTTP status code is not handled or not allowed 2026-01-25 06:16:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-blue-streak-flip-back-detachable-desk-arms-and-swing-away-foot-rest>: HTTP status code is not handled or not allowed 2026-01-25 06:16:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-w-x-14-h-x-1-d-carbon-pleated-merv-6-standard-performance-air-filter-global-industrial>: HTTP status code is not handled or not allowed 2026-01-25 06:16:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/28-inch-sentra-ec-heavy-duty-extra-wide-detachable-full-arm>: HTTP status code is not handled or not allowed 2026-01-25 06:16:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-locker-sloping-top-12-w-x-15-d-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:16:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2190001-pro-standing-seam-roof-anchor-fixed-420-cap-lbs>: HTTP status code is not handled or not allowed 2026-01-25 06:16:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/handheld-radio-battery-lithium-ion-high-capacity-2000mah>: HTTP status code is not handled or not allowed 2026-01-25 06:16:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rdx-whip-uhf-antenna-4-watt-for-use-rdu4100-and-rdu4160d-portable-radios>: HTTP status code is not handled or not allowed 2026-01-25 06:16:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bc200-spring-loaded-belt-clip>: HTTP status code is not handled or not allowed 2026-01-25 06:16:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/patient-lift-sling-mesh-polyester>: HTTP status code is not handled or not allowed 2026-01-25 06:16:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s602083-wire-rope-sling-1-x-20-eye-and-eye-9800-11200-22400-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-25 06:16:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/full-body-patient-lift-sling-solid-x-large>: HTTP status code is not handled or not allowed 2026-01-25 06:16:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/valencia-series-hutch-with-doors-70-58wx15dx35-12h-mahogany>: HTTP status code is not handled or not allowed 2026-01-25 06:16:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/axle-bracket-kit-with-one-3-4-diameter-48-long-axle>: HTTP status code is not handled or not allowed 2026-01-25 06:16:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-squeegee-front-for-nobles-tennant-610147-nobles-tennant-630291>: HTTP status code is not handled or not allowed 2026-01-25 06:16:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/good-sense-liquid-odor-counteractant-apple-scent-845-oz-bottle-2case>: HTTP status code is not handled or not allowed 2026-01-25 06:16:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-pilaster-w-shoe-10w-x-82h-sand-beach>: HTTP status code is not handled or not allowed 2026-01-25 06:16:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s201016-3-poly-web-sling-endless-6720-8400-16800-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-25 06:16:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bradley-wash-fountain-circular-raising-vent-17>: HTTP status code is not handled or not allowed 2026-01-25 06:16:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1801-1-2-hss-heavy-duty-steam-oxide-135-split-point-jobber-length-drill>: HTTP status code is not handled or not allowed 2026-01-25 06:16:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s602021-wire-rope-sling-3-4-x-12-eye-and-eye-7000-8000-16000-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-25 06:16:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-closed-linear-bearing-sws8-12id-125l>: HTTP status code is not handled or not allowed 2026-01-25 06:16:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bradley-wash-fountain-corner-raising-vent-23>: HTTP status code is not handled or not allowed 2026-01-25 06:16:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-piece-standard-hand-evaluation-set-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:16:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stationary-chest-2>: HTTP status code is not handled or not allowed 2026-01-25 06:16:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s201031-20-poly-web-sling-endless-10560-13200-26400-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-25 06:16:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-inward-swing-partition-door-w-hardware-26w-dark-khaki>: HTTP status code is not handled or not allowed 2026-01-25 06:16:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e-z-adjust-boltless-shelf-truck-with-wood-shelves-1>: HTTP status code is not handled or not allowed 2026-01-25 06:16:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wireless-manual-muscle-tester-with-clinical-fet-data-collection-software-packages>: HTTP status code is not handled or not allowed 2026-01-25 06:16:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hydraulic-hand-dynamometer-200-lb-capacity-black>: HTTP status code is not handled or not allowed 2026-01-25 06:16:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ball-transfer-table-with-3-centers-6400-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:17:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hydraulic-pinch-gauge-50-lb-dial-gauge-and-analog-output-signal-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:17:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyester-strapping-4825773t29-1-2-inches-x-0-025-inches-x-2900-feet-green-16-inches-x-3-inches-core>: HTTP status code is not handled or not allowed 2026-01-25 06:17:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/empty-sign-frame-with-2-clear-acrylic-11-in-x14-in-with-adapter>: HTTP status code is not handled or not allowed 2026-01-25 06:17:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/back-leg-chest-dynamometer-with-standard-platform-child-165-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:17:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-w-x-6-h-workbench-stringer-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:17:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/49-24-0146-led-work-light-12v>: HTTP status code is not handled or not allowed 2026-01-25 06:17:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-pilaster-w-shoe-24w-x-82h-satin>: HTTP status code is not handled or not allowed 2026-01-25 06:17:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-pilaster-w-shoe-12w-x-82h-satin>: HTTP status code is not handled or not allowed 2026-01-25 06:17:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1838-1-2-hss-heavy-duty-bright-118-point-multi-purpose-carbide-tipped-masonry-drill>: HTTP status code is not handled or not allowed 2026-01-25 06:17:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/i2p2-injury-and-illness-prevention-programs-employee-booklet>: HTTP status code is not handled or not allowed 2026-01-25 06:17:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18u-phantom-class-open-frame-swing-out-rack-1915-3-800-18>: HTTP status code is not handled or not allowed 2026-01-25 06:17:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-squeegee-rear-a-for-factory-cat-25-754l>: HTTP status code is not handled or not allowed 2026-01-25 06:17:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/infinity-8482-locker-3-tier-12-x-15-x-24-3-door-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:17:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pa-950-series-tri-pack-heavy-duty-closer-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 06:17:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-outward-swing-partition-door-w-hardware-24w-royal-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:17:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lever-keyed-lock-entry-lever-trim-clutch-dull-chrome>: HTTP status code is not handled or not allowed 2026-01-25 06:17:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 74344 pages (at 103 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:17:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rim-reversible-panic-device-for-doors-25-36-wide-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 06:17:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/digital-door-lock-m210-mechanical-keyless-deadbolt-jet-black>: HTTP status code is not handled or not allowed 2026-01-25 06:17:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b151120-4-single-leg-chain-sling-w-sling-hook>: HTTP status code is not handled or not allowed 2026-01-25 06:17:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-tier-2-door-digital-locker-12inwx12indx78inh-tan-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 06:17:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b151056-4-single-leg-chain-sling-w-sling-grab-hook>: HTTP status code is not handled or not allowed 2026-01-25 06:17:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-signs-danger-no-smoking-vinyl-7-h-x-10-w>: HTTP status code is not handled or not allowed 2026-01-25 06:17:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-wall-exhaust-shutter-rear-flange-51-eas-mf-51>: HTTP status code is not handled or not allowed 2026-01-25 06:17:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16963-propress-3-4-jaw-assembly-for-1-2-1-1-4-copper-tubing>: HTTP status code is not handled or not allowed 2026-01-25 06:17:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-industry-standard-clamping-couplings-climax-2iscc-087-087-s>: HTTP status code is not handled or not allowed 2026-01-25 06:17:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-pilaster-w-shoe-7w-x-82h-white>: HTTP status code is not handled or not allowed 2026-01-25 06:17:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1878-1-2-hss-heavy-duty-black-gold-135-split-point-3-flatted-shank-jobber-length-drill>: HTTP status code is not handled or not allowed 2026-01-25 06:17:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1877-1-1-2-black-gold-118-point-3-flat-1-2-reduced-shank-silver-deming-drill>: HTTP status code is not handled or not allowed 2026-01-25 06:17:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1889-1-2-6in-oal-hss-heavy-duty-bright-118-point-fast-helix-carbide-tipped-masonry-drill>: HTTP status code is not handled or not allowed 2026-01-25 06:17:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-climax-2c-062>: HTTP status code is not handled or not allowed 2026-01-25 06:17:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/donut-bumper>: HTTP status code is not handled or not allowed 2026-01-25 06:17:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-squeegee-set-urethane-for-nobles-tennant-391378>: HTTP status code is not handled or not allowed 2026-01-25 06:17:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sk2000-174-boltless-pallet-rack-4-1-16-x-96-step-beam-pair>: HTTP status code is not handled or not allowed 2026-01-25 06:17:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-heated-fleece-hoodie-jacket-with-detachable-hood-xl-charcoal-mix>: HTTP status code is not handled or not allowed 2026-01-25 06:17:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sk3000-174-structural-channel-pallet-rack-4-x-96-channel-beam-pair>: HTTP status code is not handled or not allowed 2026-01-25 06:17:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hanging-rail-48>: HTTP status code is not handled or not allowed 2026-01-25 06:17:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1877-7-8-black-gold-118-point-3-flat-1-2-reduced-shank-silver-deming-drill>: HTTP status code is not handled or not allowed 2026-01-25 06:17:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7130-03-liftbak-back-support-large>: HTTP status code is not handled or not allowed 2026-01-25 06:17:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ledge-21-l-x-4-h-wire-shelves>: HTTP status code is not handled or not allowed 2026-01-25 06:17:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s152039-8-poly-web-sling-eye-and-eye-5120-6400-12800-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-25 06:17:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-hss-staggered-tooth-side-milling-cutter-8-1116-1-12>: HTTP status code is not handled or not allowed 2026-01-25 06:17:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u-shaped-reception-station-with-window-88w-x-44d-x-44h-gray-counter-blue-panel>: HTTP status code is not handled or not allowed 2026-01-25 06:17:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-safety-pant-30-waist-x-32l-inseam-navy>: HTTP status code is not handled or not allowed 2026-01-25 06:17:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1889-1-4-12in-oal-hss-heavy-duty-bright-118-point-fast-helix-carbide-tipped-masonry-drill>: HTTP status code is not handled or not allowed 2026-01-25 06:17:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cut-resistant-staryarn-polyurethane-latex-glove-m-1-pair-gsp-19ym>: HTTP status code is not handled or not allowed 2026-01-25 06:17:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/starter-panel-to-wallpanel-to-pilaster-bracket-kit-for-steel-partitions-zamac>: HTTP status code is not handled or not allowed 2026-01-25 06:17:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1880-9-32-cobalt-heavy-duty-straw-135-left-hand-mechanics-length-drill>: HTTP status code is not handled or not allowed 2026-01-25 06:17:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-locker-triple-tier-3-wide-12-3-4w-x-18d-x-24-5-16h-tan-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 06:17:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polymer-partition-panel-wo-brackets-5412w-azure>: HTTP status code is not handled or not allowed 2026-01-25 06:17:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-ldpe-carboy-with-spigot-25-liter-1-each>: HTTP status code is not handled or not allowed 2026-01-25 06:17:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-climax-1c-081-z>: HTTP status code is not handled or not allowed 2026-01-25 06:17:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polymer-pilaster-w-shoe-5w-x-82h-azure>: HTTP status code is not handled or not allowed 2026-01-25 06:17:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-climax-2c-012-s>: HTTP status code is not handled or not allowed 2026-01-25 06:17:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-climax-1c-075-z>: HTTP status code is not handled or not allowed 2026-01-25 06:17:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/three-sided-frame-24x30-nexelon>: HTTP status code is not handled or not allowed 2026-01-25 06:17:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polymer-partition-panel-wo-brackets-5412w-black-confetti>: HTTP status code is not handled or not allowed 2026-01-25 06:17:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pekalon-round-brush-set-medium-hard-bristles-for-sgv-6-5-2-863-009-0>: HTTP status code is not handled or not allowed 2026-01-25 06:17:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/narrow-mouth-hdpe-bottles-with-closure-15ml-case-of-72>: HTTP status code is not handled or not allowed 2026-01-25 06:17:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-squeegee-rear-a-for-factory-cat-33-754u>: HTTP status code is not handled or not allowed 2026-01-25 06:17:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dunnage-shelf-18x60-2>: HTTP status code is not handled or not allowed 2026-01-25 06:17:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polymer-pilaster-w-shoe-5w-x-82h-black-confetti>: HTTP status code is not handled or not allowed 2026-01-25 06:17:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wide-mouth-hdpe-economy-bottles-with-closure-125ml-case-of-72>: HTTP status code is not handled or not allowed 2026-01-25 06:17:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/l-shaped-reception-station-with-window-and-raceway-80w-x-80d-x-46h-cherry-counter-blue-panel>: HTTP status code is not handled or not allowed 2026-01-25 06:17:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1899-3-16-hss-general-purpose-steam-oxide-118-point-jobber-length-drill>: HTTP status code is not handled or not allowed 2026-01-25 06:17:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/side-chair-vinyl-gray-chrome-frame-pack-of-two>: HTTP status code is not handled or not allowed 2026-01-25 06:17:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-lightweight-long-sleeve-flame-resistant-safety-shirt-l-navy-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:17:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-climax-1c-156>: HTTP status code is not handled or not allowed 2026-01-25 06:17:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluorinated-hdpe-carboy-with-closure-20l-case-of-4>: HTTP status code is not handled or not allowed 2026-01-25 06:17:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-drain-hose-w-drain-cap-full-assembly-for-nobles-tennant-1014268>: HTTP status code is not handled or not allowed 2026-01-25 06:17:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flame-resistant-safety-pant-38-waist-x-32l-inseam-navy>: HTTP status code is not handled or not allowed 2026-01-25 06:17:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ptsn-12-havana-tumbler-12-oz-8h>: HTTP status code is not handled or not allowed 2026-01-25 06:17:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacment-squeegee-set-for-nacecare-900518>: HTTP status code is not handled or not allowed 2026-01-25 06:17:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/157-3-4-hss-general-purpose-steam-oxide-118-point-screw-machine-drill>: HTTP status code is not handled or not allowed 2026-01-25 06:17:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polymer-outward-swing-partition-door-36w-azure>: HTTP status code is not handled or not allowed 2026-01-25 06:17:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cafetorium-plastic-stack-chair-charcoal-8800-series>: HTTP status code is not handled or not allowed 2026-01-25 06:17:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10220-1ww-heavy-duty-bariatric-two-button-walker-with-5-dual-wheels>: HTTP status code is not handled or not allowed 2026-01-25 06:17:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-flame-resistant-heavyweight-pullover-hoodie-l-black>: HTTP status code is not handled or not allowed 2026-01-25 06:17:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-stainless-steel-blade-safety-knife-10-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:17:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-lightweight-short-sleeve-flame-resistant-safety-shirt-xl-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:17:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-x-6-x-36-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-25 06:17:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polymer-partition-panel-wo-brackets-5912w-black-confetti>: HTTP status code is not handled or not allowed 2026-01-25 06:17:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/27019-8140ba-type-o-class-1-breakaway-sash-orange-3xl>: HTTP status code is not handled or not allowed 2026-01-25 06:17:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12202-6220-charge-pack-clear-lxl>: HTTP status code is not handled or not allowed 2026-01-25 06:17:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-x-8-x-4-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-25 06:17:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-squeegee-front-for-factory-cat-33-755u>: HTTP status code is not handled or not allowed 2026-01-25 06:17:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polymer-pilaster-w-shoe-5w-x-82h-olive>: HTTP status code is not handled or not allowed 2026-01-25 06:17:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-partition-panel-wo-brackets-5712w-khaki>: HTTP status code is not handled or not allowed 2026-01-25 06:17:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-vis-flame-resistant-safety-cargo-pant-44-waist-x-34l-inseam-navy>: HTTP status code is not handled or not allowed 2026-01-25 06:17:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/150asp-tn-21-hss-heavy-duty-tin-135-split-point-jobber-length-drill>: HTTP status code is not handled or not allowed 2026-01-25 06:17:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-g-10-fr-4-garolite-rod-1-1-2-diameter-x-3-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 06:17:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pwbn-9v-oval-woven-basket>: HTTP status code is not handled or not allowed 2026-01-25 06:17:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-pilaster-w-shoe-6w-x-82h-royal-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:17:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bi-level-refrigerated-drinking-fountain-filtered>: HTTP status code is not handled or not allowed 2026-01-25 06:17:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trapeze-disposable-dusting-sheets-8-x-125-ft-white-250-sheetsroll>: HTTP status code is not handled or not allowed 2026-01-25 06:17:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40903-xl-dog-life-vest-yellow-x-large>: HTTP status code is not handled or not allowed 2026-01-25 06:17:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-1c-series-climax-h1c-318>: HTTP status code is not handled or not allowed 2026-01-25 06:17:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polymer-pilaster-w-shoe-4w-x-82h-black>: HTTP status code is not handled or not allowed 2026-01-25 06:17:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-1c-series-climax-h1c-393>: HTTP status code is not handled or not allowed 2026-01-25 06:18:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polymer-pilaster-w-shoe-6w-x-82h-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:18:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-boxes-12-x-12-x-6-200lb-test-ect-32-25-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:18:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scf-6r-round-mesh-skimmer-6-1-2d>: HTTP status code is not handled or not allowed 2026-01-25 06:18:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slotted-cap-with-staz-on-suspension-green>: HTTP status code is not handled or not allowed 2026-01-25 06:18:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-hp-concentrated-multi-surface-cleaner-citrus-scent-5000-ml-rtd-bottle>: HTTP status code is not handled or not allowed 2026-01-25 06:18:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-3-4-x-12-3-4-x-13-1-2-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-25 06:18:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pl-57b-cover-for-pl-7b>: HTTP status code is not handled or not allowed 2026-01-25 06:18:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pl-5k-dish-box>: HTTP status code is not handled or not allowed 2026-01-25 06:18:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pwd2-powerswap-nucleus-powerdock-dual-station>: HTTP status code is not handled or not allowed 2026-01-25 06:18:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-dual-grip-shovel-14-in-blade-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:18:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ratchet-trailer-stabilizing-jack-100-10000-lb-static-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:18:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-x-11-x-6-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-25 06:18:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ratchet-trailer-stabilizing-jack-125000-lb-static-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:18:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scrp-16-grill-scraper-with-5>: HTTP status code is not handled or not allowed 2026-01-25 06:18:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milwaukee-packout-48-22-8315-15-tote>: HTTP status code is not handled or not allowed 2026-01-25 06:18:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milwaukee-packout-48-22-8435-compact-organizer>: HTTP status code is not handled or not allowed 2026-01-25 06:18:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polymer-outward-swing-partition-door-36w-moss>: HTTP status code is not handled or not allowed 2026-01-25 06:18:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thermoplastic-waterfall-utility-pump-4-10-hp>: HTTP status code is not handled or not allowed 2026-01-25 06:18:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 74444 pages (at 100 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:18:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labelwriter-cleaning-cards-10box>: HTTP status code is not handled or not allowed 2026-01-25 06:18:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-strength-alkaline-drain-cleaner-32-oz-bottle>: HTTP status code is not handled or not allowed 2026-01-25 06:18:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/skullerz-8955-lightweight-bump-cap-long-brim-lime>: HTTP status code is not handled or not allowed 2026-01-25 06:18:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slotted-full-brim-hat-with-1-touch-suspension-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:18:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-with-keyway-2c-kw-series-climax-2c-075-kw>: HTTP status code is not handled or not allowed 2026-01-25 06:18:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-couplings-recessed-screw-climax-cc-137-100>: HTTP status code is not handled or not allowed 2026-01-25 06:18:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metal-frame-for-caps-10-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:18:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tmt-mt2-dial-meat-thermometer>: HTTP status code is not handled or not allowed 2026-01-25 06:18:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-industry-standard-clamping-couplings-climax-2iscc-200-200-s>: HTTP status code is not handled or not allowed 2026-01-25 06:18:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-153-auto-floor-scrubber-20-cleaning-path-two-115-amp-batteries>: HTTP status code is not handled or not allowed 2026-01-25 06:18:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-linen-le-phenolic-sheet-0-093-thick-12-x-48>: HTTP status code is not handled or not allowed 2026-01-25 06:18:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-climax-1c-268-z>: HTTP status code is not handled or not allowed 2026-01-25 06:18:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-laminate-pilaster-w-shoe-7w-x-82h-canyon-zephyr>: HTTP status code is not handled or not allowed 2026-01-25 06:18:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/skullerz-8955-lightweight-bump-cap-short-brim-black>: HTTP status code is not handled or not allowed 2026-01-25 06:18:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/canadian-freedom-series-slotted-protective-hat-camouflage>: HTTP status code is not handled or not allowed 2026-01-25 06:18:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-climax-2c-200-s>: HTTP status code is not handled or not allowed 2026-01-25 06:18:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sugar-free-solo-shots-powdered-drink-blue-lemonade-169-fl-oz-10-bags-of-50-packets>: HTTP status code is not handled or not allowed 2026-01-25 06:18:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22-x-6-x-6-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-25 06:18:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-faced-weld-one-side-bumper-22-1101-10-1-4-w-x-6-d-x-20-h>: HTTP status code is not handled or not allowed 2026-01-25 06:18:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/laminated-bumper-24-3988-16-w-x-10-d-x-10-h>: HTTP status code is not handled or not allowed 2026-01-25 06:18:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-laminate-pilaster-w-shoe-12w-x-82h-canyon-zephyr>: HTTP status code is not handled or not allowed 2026-01-25 06:18:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sf-5-wide-mouth-funnel-5d-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 06:18:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-couplings-recessed-screw-climax-cc-087-087>: HTTP status code is not handled or not allowed 2026-01-25 06:18:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silver-duct-tape-1-88-x-35-yards>: HTTP status code is not handled or not allowed 2026-01-25 06:18:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-corner-medical-shelf-cabinet-10w-x-10d-x-1-34h-silver>: HTTP status code is not handled or not allowed 2026-01-25 06:18:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-glue-1-75-ounces>: HTTP status code is not handled or not allowed 2026-01-25 06:18:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rx7230st-rx-series-option-al-72-w-x-30-d-x-1-h-stainless-steel-top>: HTTP status code is not handled or not allowed 2026-01-25 06:18:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medical-storage-wall-cabinet-with-2-shelves-keyed-the-same-11-18w-x-4-12d-x-15h>: HTTP status code is not handled or not allowed 2026-01-25 06:18:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stack-pack-worktop-21-1316l-x-15-34w-x-2-18h-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:18:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-climax-2c-237>: HTTP status code is not handled or not allowed 2026-01-25 06:18:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/preventer-small-edition-battery-fire-containment-bag-kit-14w-x-1d-x-10h-orange>: HTTP status code is not handled or not allowed 2026-01-25 06:18:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-x-24-x-20-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-25 06:18:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-x-35-x-40-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-25 06:18:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glass-board-frosted-36-x-24>: HTTP status code is not handled or not allowed 2026-01-25 06:18:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-utility-pressure-gauge-1-8-npt-dry-fillable-0-160-psi-ctr-back-mount-102d-208f>: HTTP status code is not handled or not allowed 2026-01-25 06:18:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kf-series-24-impulse-heat-foot-sealer-standing-operation-5mm-seal-width>: HTTP status code is not handled or not allowed 2026-01-25 06:18:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelf-18wx18dx78h-blue-all-welded>: HTTP status code is not handled or not allowed 2026-01-25 06:18:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-canvas-ce-phenolic-sheet-0-250-thick-48-x-96>: HTTP status code is not handled or not allowed 2026-01-25 06:18:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/phenolic-black-core-pilaster-w-shoe-8w-x-82h-almond>: HTTP status code is not handled or not allowed 2026-01-25 06:18:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rfm-4b-fluted-ramekins-4-oz-melamine-bone>: HTTP status code is not handled or not allowed 2026-01-25 06:18:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-5-pressure-gauge-1-8-npt-glycerine-filled-0-30-psi-ctr-back-mount-ss-202l-158c>: HTTP status code is not handled or not allowed 2026-01-25 06:18:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kf-series-8-direct-heat-foot-sealer-standing-operation-10mm-meshed-seal-width>: HTTP status code is not handled or not allowed 2026-01-25 06:18:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-x-14-x-14-12-10-8-multi-depth-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-25 06:18:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emax-erv0100001-10-hp-rotary-screw-compressor-tankless-145-psi-45-cfm-1ph-208-230v>: HTTP status code is not handled or not allowed 2026-01-25 06:18:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ptm-18r-round-potato-masher>: HTTP status code is not handled or not allowed 2026-01-25 06:18:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelf-vented-sides-24wx18dx78h-gray-all-welded>: HTTP status code is not handled or not allowed 2026-01-25 06:18:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-laminate-outward-swing-door-w-hardware-26w-almond>: HTTP status code is not handled or not allowed 2026-01-25 06:18:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pipe-inspection-camera-10-in-display-200-ft-cable-distance-counter>: HTTP status code is not handled or not allowed 2026-01-25 06:18:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelf-vented-sides-24wx24dx78h-gray-all-welded>: HTTP status code is not handled or not allowed 2026-01-25 06:18:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powered-stair-climbing-hand-truck-375-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:18:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergodyne-skullerz-8961-single-reflective-hard-hat-safety-helmet-sticker-kit-white>: HTTP status code is not handled or not allowed 2026-01-25 06:18:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-shovel-square-blade>: HTTP status code is not handled or not allowed 2026-01-25 06:18:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-1-2-x-12-kra-crimped-end-tubes-case>: HTTP status code is not handled or not allowed 2026-01-25 06:18:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelf-vented-sides-24wx21dx72h-gray-all-welded>: HTTP status code is not handled or not allowed 2026-01-25 06:18:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/integra-office-chair-high-back-articulating-tilt-18-23h-seat-black>: HTTP status code is not handled or not allowed 2026-01-25 06:18:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/back-guard-rails-for-4-level-transport-risers-black>: HTTP status code is not handled or not allowed 2026-01-25 06:18:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lexington-extended-height-task-chair-mid-back-no-tilt-21-28-12h-seat-navyblack>: HTTP status code is not handled or not allowed 2026-01-25 06:18:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-x-42-kra-mailing-tubes-with-caps-case>: HTTP status code is not handled or not allowed 2026-01-25 06:18:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/equipment-stand-w-adjustable-undershelf-30-x-30-24-high-stainless-top-legsand-undershelf>: HTTP status code is not handled or not allowed 2026-01-25 06:18:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-table-mounted-double-deck-rear-turn-up15-wide-84-long-18-gauge-430-ss>: HTTP status code is not handled or not allowed 2026-01-25 06:18:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelf-24wx18dx76h-red-all-welded>: HTTP status code is not handled or not allowed 2026-01-25 06:18:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/straight-soil-dishtable-left-to-right-107-long-16304-ss>: HTTP status code is not handled or not allowed 2026-01-25 06:18:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1872-whole-house-dehumidifier-130-pint-with-casters>: HTTP status code is not handled or not allowed 2026-01-25 06:18:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/work-table-24-wide-top-24-long-ss-legsside-rear-ss-crossrails-16-gauge>: HTTP status code is not handled or not allowed 2026-01-25 06:18:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-paper-phenolic-sheet-1-250-x-36-000-x-48-000>: HTTP status code is not handled or not allowed 2026-01-25 06:18:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-black-executive-service-cart-39-x-21-x-38-300-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:18:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kitchen-bath-adhesive-caulk-5-5-oz-white-7079818001>: HTTP status code is not handled or not allowed 2026-01-25 06:18:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stretch-wrap-with-extended-core-handle-cast-120-gauge-3w-x-700l-clear-pack-of-18>: HTTP status code is not handled or not allowed 2026-01-25 06:18:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qf-pipe-12-dia-galvanized-steel>: HTTP status code is not handled or not allowed 2026-01-25 06:18:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/painter-acrylic-latex-caulk-10-1-oz-white-7079818670>: HTTP status code is not handled or not allowed 2026-01-25 06:18:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-x-8-x-8-white-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-25 06:18:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-paper-phenolic-sheet-1-000-thick-48-x-48>: HTTP status code is not handled or not allowed 2026-01-25 06:18:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vinyl-adhesive-caulk-10-oz-transluscent-5113600006>: HTTP status code is not handled or not allowed 2026-01-25 06:18:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelfsecurity-boxvented24wx21dx78hrdall-welded>: HTTP status code is not handled or not allowed 2026-01-25 06:18:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3m-safety-walk-slip-resistant-conformable-tapes-treads-510-bk-4-in-x-60-ft1-case>: HTTP status code is not handled or not allowed 2026-01-25 06:18:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stretch-wrap-with-extended-core-handle-cast-70-gauge-30w-x-1000l-clear-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-25 06:18:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scotch-brite-light-duty-scrubbing-pad-9030-3-5-in-x-5-in-40-case>: HTTP status code is not handled or not allowed 2026-01-25 06:18:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/qf-elbow-90-degree-1-5-clr-8-dia-galvanized-steel>: HTTP status code is not handled or not allowed 2026-01-25 06:18:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-set-screw-collar-climax-mc-35>: HTTP status code is not handled or not allowed 2026-01-25 06:18:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergodyne-skullerz-half-frame-safety-sunglasses-anti-fog-scratch-orange-lens-clear-frame>: HTTP status code is not handled or not allowed 2026-01-25 06:18:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-50-basis-weight-kra-paper>: HTTP status code is not handled or not allowed 2026-01-25 06:18:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0345r-iron-tuff-low-bib-overall-short-sage-large>: HTTP status code is not handled or not allowed 2026-01-25 06:18:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wsecurity-bxfootlockervented24x18x78blall-welded>: HTTP status code is not handled or not allowed 2026-01-25 06:19:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0385r-iron-tuff-high-bib-overall-short-navy-3xl>: HTTP status code is not handled or not allowed 2026-01-25 06:19:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reborn-collection-tote-bag-with-integrated-shoe-pocket-black>: HTTP status code is not handled or not allowed 2026-01-25 06:19:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sg-59bmd-af-helium-18-ultra-light-safety-glasses-metal-detect-blue-lens>: HTTP status code is not handled or not allowed 2026-01-25 06:19:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0385r-iron-tuff-high-bib-overall-regular-sage-xl>: HTTP status code is not handled or not allowed 2026-01-25 06:19:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rx-500c-2-5-rx-500-magnifier-safety-glasses-2-5-magnifier-clear-lens>: HTTP status code is not handled or not allowed 2026-01-25 06:19:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-led-dock-light-40w-4900-lumens-5000k-on-off-switch-with-60-l-arm-w-fan>: HTTP status code is not handled or not allowed 2026-01-25 06:19:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/diamond-star153-anti-fatigue-mat-1116-thick-2w-x-3l-blackyellow>: HTTP status code is not handled or not allowed 2026-01-25 06:19:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bsot-15-solid-basting-spoon-15l-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 06:19:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wsecurity-bxfootlockervented24x24x78blall-welded>: HTTP status code is not handled or not allowed 2026-01-25 06:19:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wsecurity-box-footlocker-24wx18dx76hblall-welded>: HTTP status code is not handled or not allowed 2026-01-25 06:19:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-poly-bag-rolls-20x30-2-mil-500-rl>: HTTP status code is not handled or not allowed 2026-01-25 06:19:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-sided-bulk-trucks-lip-slat-sides-3600-lb-cap-24-60-poly-wheels>: HTTP status code is not handled or not allowed 2026-01-25 06:19:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0345r-iron-tuff-low-bib-overall-regular-navy-4xl>: HTTP status code is not handled or not allowed 2026-01-25 06:19:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-poly-bag-rolls-12x18-4-mil-500-rl>: HTTP status code is not handled or not allowed 2026-01-25 06:19:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/restore-10-floor-restorer-spray-buff-neutral-scent-gallon-bottle-4-case-nl170-g4>: HTTP status code is not handled or not allowed 2026-01-25 06:19:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-fixed-shade-welding-lens-frame>: HTTP status code is not handled or not allowed 2026-01-25 06:19:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 74543 pages (at 99 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:19:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0685r-denim-high-bib-overall-regular-black-small>: HTTP status code is not handled or not allowed 2026-01-25 06:19:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0345r-iron-tuff-low-bib-overall-regular-sage-2xl>: HTTP status code is not handled or not allowed 2026-01-25 06:19:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0385r-iron-tuff-high-bib-overall-tall-navy-medium>: HTTP status code is not handled or not allowed 2026-01-25 06:19:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-sided-bulk-truck-slat-sides-3-shlvs-lip-3600-lbs-cap-30-48-poly-wheels>: HTTP status code is not handled or not allowed 2026-01-25 06:19:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0385r-iron-tuff-high-bib-overall-tall-navy-xl>: HTTP status code is not handled or not allowed 2026-01-25 06:19:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelf-24wx24dx72h-blue-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 06:19:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-coupling-climax-rc-087-s>: HTTP status code is not handled or not allowed 2026-01-25 06:19:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40rb-stainless-steel-scoop-32-oz>: HTTP status code is not handled or not allowed 2026-01-25 06:19:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0345r-iron-tuff-low-bib-overall-regular-sage-large>: HTTP status code is not handled or not allowed 2026-01-25 06:19:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-one-piece-industry-standard-clamping-couplings-climax-miscc-09-09-s>: HTTP status code is not handled or not allowed 2026-01-25 06:19:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4003-stainless-steel-scoop-32-oz>: HTTP status code is not handled or not allowed 2026-01-25 06:19:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergodyne-skullerz-8953-hard-shell-bump-cap-sweatband-replacement-black-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 06:19:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vanguard-efi-series-gas-engine-vertical-shaft-28-hp>: HTTP status code is not handled or not allowed 2026-01-25 06:19:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-door-cell-phone-locker-cp12-091572-dlgy4-wide-whasp-36w-x-15d-x-72-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:19:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-couplings-recessed-screw-w-keyway-cc-series-climax-2cc-125-125-kw>: HTTP status code is not handled or not allowed 2026-01-25 06:19:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-coupling-climax-rc-100-s>: HTTP status code is not handled or not allowed 2026-01-25 06:19:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/132c-ice-logger-boot-regular-tan-6>: HTTP status code is not handled or not allowed 2026-01-25 06:19:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wsecurity-bxfootlockervented24x24x72blall-welded>: HTTP status code is not handled or not allowed 2026-01-25 06:19:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-flow-reducer>: HTTP status code is not handled or not allowed 2026-01-25 06:19:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-plastic-trash-container-lid-garbage-can-lid-20-gallon-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:19:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-nova-3-head-support>: HTTP status code is not handled or not allowed 2026-01-25 06:19:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kira-b-50-autonomous-floor-scrubber-22-cleaning-width-gray-full-kit>: HTTP status code is not handled or not allowed 2026-01-25 06:19:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0359-iron-tuff-arctic-jacket-regular-navy-3xl>: HTTP status code is not handled or not allowed 2026-01-25 06:19:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shield-electrolyte-freeze-pop-reduced-sugar-case-36>: HTTP status code is not handled or not allowed 2026-01-25 06:19:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-climax-2c-187-z>: HTTP status code is not handled or not allowed 2026-01-25 06:19:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-framed-stadium-locker-wshelf-security-box-18wx18dx76h-red-all-welded>: HTTP status code is not handled or not allowed 2026-01-25 06:19:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/max-lift-standard-scissor-lift-table-foot-operated-4000-lb-capacity-48l-x-24w>: HTTP status code is not handled or not allowed 2026-01-25 06:19:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelf-24x42-nexelon>: HTTP status code is not handled or not allowed 2026-01-25 06:19:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-9-misc-products-tagboard-placard>: HTTP status code is not handled or not allowed 2026-01-25 06:19:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-8-corrosive-rigid-plastic-placard>: HTTP status code is not handled or not allowed 2026-01-25 06:19:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0344r-iron-tuff-minus-50-suit-short-sage-4xl>: HTTP status code is not handled or not allowed 2026-01-25 06:19:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-threaded-clamping-collar-recessed-screw-climax-tc-112-07>: HTTP status code is not handled or not allowed 2026-01-25 06:19:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poly-z-brite-wire-shelf-24x60>: HTTP status code is not handled or not allowed 2026-01-25 06:19:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-cold-air-spinner>: HTTP status code is not handled or not allowed 2026-01-25 06:19:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dxs2153-sae-standard-impact-socket-1316-drive-black>: HTTP status code is not handled or not allowed 2026-01-25 06:19:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-climax-1c-031-a>: HTTP status code is not handled or not allowed 2026-01-25 06:19:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/azar-252378-three-tier-brochure-holder-9-1-4-w-x-6-d-x-13-1-4-h>: HTTP status code is not handled or not allowed 2026-01-25 06:19:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/max-lift-standard-scissor-lift-table-foot-operated-4000-lb-capacity-72l-x-36w>: HTTP status code is not handled or not allowed 2026-01-25 06:19:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slimline-square-container-35-gallon-greystone-ssc35-00gst>: HTTP status code is not handled or not allowed 2026-01-25 06:19:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/maxicut-ultra-micro-foam-nitrile-coated-palm-fingers-gloves-ansi-a4-s-gray-6-dozencase>: HTTP status code is not handled or not allowed 2026-01-25 06:19:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/maxicut-ultra-micro-foam-nitrile-coated-palm-fingers-gloves-ansi-a4-l-gray-6-dozencase>: HTTP status code is not handled or not allowed 2026-01-25 06:19:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/azar-199622-nameplate-w-adhesive-back-11-w-x-1-8-d-x-6-h>: HTTP status code is not handled or not allowed 2026-01-25 06:19:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0381r-iron-tuff-minus-50-suit-short-navy-medium>: HTTP status code is not handled or not allowed 2026-01-25 06:19:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oe-fine-wire-single-platinum-spark-plug-6737-vm-auto>: HTTP status code is not handled or not allowed 2026-01-25 06:19:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dolly-20-32-44-55-gallon-rubbermaid-round-trash-container>: HTTP status code is not handled or not allowed 2026-01-25 06:19:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oe-fine-wire-double-iridium-spark-plug-9698-vm-auto>: HTTP status code is not handled or not allowed 2026-01-25 06:19:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bash-sledge-hammer-10-lb-head-16l-unbreakable-steel-core-handle>: HTTP status code is not handled or not allowed 2026-01-25 06:19:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-inch-h-green-metal-counter-height-stool-with-vertical-slat-back>: HTTP status code is not handled or not allowed 2026-01-25 06:19:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aviation-wheel-chock-w-locking-rope-polyurethane-14l-x-5-12w-x-3-12h>: HTTP status code is not handled or not allowed 2026-01-25 06:19:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-inch-h-black-metal-barstool-with-vertical-slat-back>: HTTP status code is not handled or not allowed 2026-01-25 06:19:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-nova-2000-helmet-liner-large>: HTTP status code is not handled or not allowed 2026-01-25 06:19:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-inch-h-black-antique-gold-metal-barstool-with-vertical-slat-back>: HTTP status code is not handled or not allowed 2026-01-25 06:19:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-floor-stripper-concentrate-gallon-bottle-4-bottles-case-1045769>: HTTP status code is not handled or not allowed 2026-01-25 06:19:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-cu-yd-yellow-self-dumping-forkli-hopper>: HTTP status code is not handled or not allowed 2026-01-25 06:19:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-l4-battery-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:19:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-set-screw-collar-climax-mc-05>: HTTP status code is not handled or not allowed 2026-01-25 06:19:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/optional-door-with-acrylic-window-fold-out-cabinet-black>: HTTP status code is not handled or not allowed 2026-01-25 06:19:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-set-screw-collar-climax-mc-05-s>: HTTP status code is not handled or not allowed 2026-01-25 06:19:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-set-screw-collar-climax-mc-09-s>: HTTP status code is not handled or not allowed 2026-01-25 06:19:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-constant-flow-valve>: HTTP status code is not handled or not allowed 2026-01-25 06:19:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/23-gallon-square-rubbermaid-waste-receptacle-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:19:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0342r-iron-tuff-jackoat-regular-navy-medium>: HTTP status code is not handled or not allowed 2026-01-25 06:19:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-two-piece-clamping-collar-climax-m2c-07-s>: HTTP status code is not handled or not allowed 2026-01-25 06:19:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dxs2153-metric-standard-impact-socket-19mm-drive-black>: HTTP status code is not handled or not allowed 2026-01-25 06:19:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-zlink-tlink-air-inlet-adapter>: HTTP status code is not handled or not allowed 2026-01-25 06:19:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wheel-axle-size-1-2-5>: HTTP status code is not handled or not allowed 2026-01-25 06:19:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/j-4-jiggers-steel-1-oz-x-1-1-2-oz>: HTTP status code is not handled or not allowed 2026-01-25 06:19:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ten-inch-brass-interlocking-stencil-gothic-style-numbers-thirteen-piece-kit>: HTTP status code is not handled or not allowed 2026-01-25 06:19:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-industry-standard-clamping-couplings-w-keyway-climax-iscc-137-137skw>: HTTP status code is not handled or not allowed 2026-01-25 06:19:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-industry-standard-clamping-couplings-climax-iscc-150-100-s>: HTTP status code is not handled or not allowed 2026-01-25 06:19:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-industry-standard-clamping-couplings-climax-iscc-150-150-s>: HTTP status code is not handled or not allowed 2026-01-25 06:19:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/half-inch-brass-interlocking-stencil-letters-and-numbers-ninty-two-piece-set>: HTTP status code is not handled or not allowed 2026-01-25 06:19:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/azar-700911-bowl-for-pegboard-slatwall-8-w-x-13-1-4-d-x-7-1-2-h>: HTTP status code is not handled or not allowed 2026-01-25 06:19:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-inch-brass-interlocking-stencil-letters-and-numbers-seventy-seven-piece-set>: HTTP status code is not handled or not allowed 2026-01-25 06:19:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/azar-700026-horizontal-sign-holder-11-7-8-w-x-3-16-d-x-6-h>: HTTP status code is not handled or not allowed 2026-01-25 06:19:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lifting-wall-mount-base-for-500-lb-steel-davit-crane>: HTTP status code is not handled or not allowed 2026-01-25 06:19:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lifting-3-8-x-45-synthetic-wire-cable-assembly>: HTTP status code is not handled or not allowed 2026-01-25 06:19:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ts100-015-electric-chain-hoist-1-ton-capacity-15-lift-267-fpm-230v>: HTTP status code is not handled or not allowed 2026-01-25 06:19:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-locker-triple-tier-12x18x24-9-doors-assembled-parchment>: HTTP status code is not handled or not allowed 2026-01-25 06:19:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/msb-08f-fine-mesh-8-diameter-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 06:19:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-nova-3-cool-air-tube-nylon-cape>: HTTP status code is not handled or not allowed 2026-01-25 06:19:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-x-36-corrugated-sheets>: HTTP status code is not handled or not allowed 2026-01-25 06:19:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lifting-manual-chain-hoist-w-overload-protection-5-ton-capacity-15-lift>: HTTP status code is not handled or not allowed 2026-01-25 06:19:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-inch-x-5-yards-self-adhesive-bandage-non-latex-tan-pack-of-8>: HTTP status code is not handled or not allowed 2026-01-25 06:19:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sanding-belt-aluminum-oxide-180-grit-48l-x-2w-pack-of-3>: HTTP status code is not handled or not allowed 2026-01-25 06:19:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-20-floor-machine>: HTTP status code is not handled or not allowed 2026-01-25 06:19:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-inch-x-5-yards-self-adhesive-bandage-non-latex-tan-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-25 06:19:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-storage-cabinet-36x18x72-gray-1>: HTTP status code is not handled or not allowed 2026-01-25 06:20:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pf-16-funnel-16d>: HTTP status code is not handled or not allowed 2026-01-25 06:20:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-inch-x-5-yards-self-adhesive-bandage-latex-pink-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 06:20:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-diamond-deck-truck-48-x-24-1200-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:20:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cvc627mt-hd-tvi-2mp-waterproof-marine-camera-36mm-fixed-lens-white>: HTTP status code is not handled or not allowed 2026-01-25 06:20:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-inch-x-5-yards-self-adhesive-bandage-non-latex-kelly-green-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-25 06:20:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-nova-3-hot-tube-xxl-blast-jacket>: HTTP status code is not handled or not allowed 2026-01-25 06:20:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dxs2153-sae-deep-impact-socket-1-14-drive-black>: HTTP status code is not handled or not allowed 2026-01-25 06:20:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/active-shooter-trauma-essential-medical-kit>: HTTP status code is not handled or not allowed 2026-01-25 06:20:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-industry-standard-clamping-couplings-climax-iscc-037-037>: HTTP status code is not handled or not allowed 2026-01-25 06:20:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-low-flow-industrial-spray-nozzle-safety-orange-brass-adapter>: HTTP status code is not handled or not allowed 2026-01-25 06:20:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fw-s500-electric-food-warmer-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 06:20:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fw-s600-electric-food-cooker-warmer-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 06:20:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n12-swivel-hose-adapter-brass>: HTTP status code is not handled or not allowed 2026-01-25 06:20:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/235282-clear-glass-crimp-neck-serum-vials-with-20mm-neck-finish-10ml-case-of-800>: HTTP status code is not handled or not allowed 2026-01-25 06:20:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glc-00984-clear-glass-screw-thread-vials-with-green-caps-2-dram-7-5ml-case-of-144>: HTTP status code is not handled or not allowed 2026-01-25 06:20:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-recessed-screw-climax-h1c-175>: HTTP status code is not handled or not allowed 2026-01-25 06:20:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 74647 pages (at 104 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:20:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-inch-x-5-yards-self-adhesive-bandage-non-latex-bright-pink-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-25 06:20:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50-all-weather-extension-cord-14awg-15a-125v-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:20:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stretch-wrap-18-x-1500-dispenser-white>: HTTP status code is not handled or not allowed 2026-01-25 06:20:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1170-1-replacement-desiccant-charges-1-quart>: HTTP status code is not handled or not allowed 2026-01-25 06:20:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-by-16-stage-package-24-inches-height-blue-carpet-box-pleat-black-skirting>: HTTP status code is not handled or not allowed 2026-01-25 06:20:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/68-ada-rectangular-picnic-table-polished-tan-river-rock-top-gray-limestone-leg>: HTTP status code is not handled or not allowed 2026-01-25 06:20:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-inch-x-5-yards-self-adhesive-bandage-non-latex-bright-pink>: HTTP status code is not handled or not allowed 2026-01-25 06:20:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/r8-collet-chuck-steel-er32>: HTTP status code is not handled or not allowed 2026-01-25 06:20:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-inch-x-5-yards-self-adhesive-bandage-non-latex-true-red-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-25 06:20:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-r-m55-csa-roll-bender-60-hz-1ph-220v>: HTTP status code is not handled or not allowed 2026-01-25 06:20:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ssdb-20-double-boiler-w-cover>: HTTP status code is not handled or not allowed 2026-01-25 06:20:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-recessed-screw-climax-h1c-087-s>: HTTP status code is not handled or not allowed 2026-01-25 06:20:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-green-3-tier-wire-shelving-starter-unit-30-w-x-21-d-x-34-h>: HTTP status code is not handled or not allowed 2026-01-25 06:20:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-sh-120250-hd-heavy-duty-hydraulic-shear-120l-15-hp-3ph-220v>: HTTP status code is not handled or not allowed 2026-01-25 06:20:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-green-2-tier-wire-shelving-add-on-unit-72-w-x-18-d-x-34-h>: HTTP status code is not handled or not allowed 2026-01-25 06:20:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-inch-x-5-yards-self-adhesive-bandage-non-latex-yellow-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-25 06:20:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-outdoor-steel-picnic-bench-with-backrest-perforated-metal-red>: HTTP status code is not handled or not allowed 2026-01-25 06:20:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/armless-chair-with-antimicrobial-upholstery-black>: HTTP status code is not handled or not allowed 2026-01-25 06:20:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-bp-11210cnc-120-2-axis-programmable-hydraulic-press-brake-3ph-220v>: HTTP status code is not handled or not allowed 2026-01-25 06:20:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/combination-metal-storage-cabinet-36x18x72-black>: HTTP status code is not handled or not allowed 2026-01-25 06:20:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1451113-fisher-conveyor-chain-hc>: HTTP status code is not handled or not allowed 2026-01-25 06:20:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ulti-mate-garage-2-0-series-9-piece-cabinet-set-141w-x-21d-x-36-5h-black-mdf>: HTTP status code is not handled or not allowed 2026-01-25 06:20:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1452150-flink-conveyor-chain-lmc-5h-20096>: HTTP status code is not handled or not allowed 2026-01-25 06:20:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/imperial-shelving-starter-48x12x88-7-openings-sand>: HTTP status code is not handled or not allowed 2026-01-25 06:20:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manual-4-way-3-way-valve-dc-power-unit-pu5007-18-qt-poly-reservoir-250-inch-nptf-outlet>: HTTP status code is not handled or not allowed 2026-01-25 06:20:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-sheet-with-acrylic-adhesive-70a-12-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 06:20:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1108-deluxe-filterregulator-metal-bowl-12-npt-250-psi-40-micron>: HTTP status code is not handled or not allowed 2026-01-25 06:20:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modesto-30-in-round-planter-neptune-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:20:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-removable-safety-bollard-4-dia-x-36-h-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:20:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-green-2-tier-wire-shelving-add-on-unit-60-w-x-18-d-x-34-h>: HTTP status code is not handled or not allowed 2026-01-25 06:20:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-military-marine-grade-anti-slip-tape-6-w-x-30-l-black-2-pack-roll>: HTTP status code is not handled or not allowed 2026-01-25 06:20:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g-106-paneled-shakers-w-mushroom-tops>: HTTP status code is not handled or not allowed 2026-01-25 06:20:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-garage-light-fixture-24-x-9-80w-120v-277v-black>: HTTP status code is not handled or not allowed 2026-01-25 06:20:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-strip-no-adhesive-60a-18-thick-x-2-wide-x-5-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 06:20:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/twps-7-bakery-spatulas-7-3-4l-blade-white-polypropylene-handle>: HTTP status code is not handled or not allowed 2026-01-25 06:20:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/d132rb-osha-sign-danger-construction-area-10-x-14-whiteredblack>: HTTP status code is not handled or not allowed 2026-01-25 06:20:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/23174-cut-off-wheel-type-1-saitech-4-1-2-inch-x-045-inch-x-7-8-inch-ceramic-alum-oxide>: HTTP status code is not handled or not allowed 2026-01-25 06:20:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-5-tier-shelving-starter-unit-solid-galvanized-steel-24w-x-24d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 06:20:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-bsv-18vs-220-v2-18-vertical-bandsaw-1-hp-1-phase-220v>: HTTP status code is not handled or not allowed 2026-01-25 06:20:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-3-tier-shelving-starter-unit-solid-galvanized-steel-36w-x-24d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 06:20:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-roll-no-adhesive-40a-12-thick-x-36-wide-x-10-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 06:20:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/38073-straight-cartridge-roll-1-2-inch-x-1-1-2-inch-x-1-8-inch-60-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-25 06:20:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42028-spiral-band-1-inch-x-1-inch-60-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-25 06:20:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coper-notcher-for-55t-ironworkers>: HTTP status code is not handled or not allowed 2026-01-25 06:20:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-green-2-tier-wire-shelving-starter-unit-60-w-x-14-d-x-34-h>: HTTP status code is not handled or not allowed 2026-01-25 06:20:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-recessed-screw-climax-h1c-018-s>: HTTP status code is not handled or not allowed 2026-01-25 06:20:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-temperature-epdm-rubber-sheet-no-adhesive-60a-116-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 06:20:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/viton-rubber-roll-with-high-temp-adhesive-75a-116-thick-x-36-wide-x-10-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 06:20:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-cord-spring-rewind-reels-fluorescent-tube-light-100-cord-16-awg>: HTTP status code is not handled or not allowed 2026-01-25 06:20:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spring-rewind-hose-reel-for-air-water-3-8-i-d-50-hose-300-psi>: HTTP status code is not handled or not allowed 2026-01-25 06:20:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oversize-round-punch-die-set-1-732>: HTTP status code is not handled or not allowed 2026-01-25 06:20:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-threaded-clamping-collar-climax-istc-100-14-s>: HTTP status code is not handled or not allowed 2026-01-25 06:20:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/35107-psa-disc-6-inch-dia-320-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-25 06:20:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-x-8-core-machine-grade-strapping-18000-l-x-1-4-w-x-0-024-thick-clear>: HTTP status code is not handled or not allowed 2026-01-25 06:20:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/35024-psa-disc-2-inch-dia-60-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-25 06:20:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-5-swivel-and-rigid-plate-caster-kit-with-hardware-includes-2-swivel-casters-2-rigid-casters-and-4-screws-16-bolts-16-flat-washers-and-16-nut-bolts>: HTTP status code is not handled or not allowed 2026-01-25 06:20:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-punch-die-set-2932>: HTTP status code is not handled or not allowed 2026-01-25 06:20:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60523-sanding-belt-1a-x-2-inch-x-60-inch-60-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-25 06:20:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esw-ins-11-qt-stainless-steel-insert-fits-esw-66>: HTTP status code is not handled or not allowed 2026-01-25 06:20:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/117-3-250-challenger-hand-crank-hose-reel-3-8-i-d-250-hose-capacity-less-hose4000-psi>: HTTP status code is not handled or not allowed 2026-01-25 06:20:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classroom-connector-school-to-home-folders-assorted-6-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:20:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-39-welding-positioner-2200-lb-capacity-1-ph-220v>: HTTP status code is not handled or not allowed 2026-01-25 06:20:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/epdm-rubber-sheet-with-acrylic-adhesive-60a-116-thick-x-36-wide-x-36-long>: HTTP status code is not handled or not allowed 2026-01-25 06:20:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-compartment-box-for-rubbermaid-trademaster-carts>: HTTP status code is not handled or not allowed 2026-01-25 06:20:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-rubber-tool-grip-for-rubbmaid-hygen-microfiber-cleaning-cart>: HTTP status code is not handled or not allowed 2026-01-25 06:20:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bam-12-bain-marie-12-qt>: HTTP status code is not handled or not allowed 2026-01-25 06:20:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-roll-no-adhesive-60a-12-thick-x-36-wide-x-4-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 06:20:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/apzs-14-pizza-screen>: HTTP status code is not handled or not allowed 2026-01-25 06:20:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-avant-caster-with-threaded-swivel-stem-1-1-2h-stem-220-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:20:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g-2140-galvanized-alloy-shackle-bta-1-1-2-30t-wll>: HTTP status code is not handled or not allowed 2026-01-25 06:20:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-1358-g100-chain-clevis-grab-hook-3-8-8800-lbs-wll>: HTTP status code is not handled or not allowed 2026-01-25 06:20:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/84256-water-proof-sanding-sheets-9-inch-x-11-inch-320-grit-silicon-carbide>: HTTP status code is not handled or not allowed 2026-01-25 06:20:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-roll-no-adhesive-70a-14-thick-x-36-wide-x-8-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 06:20:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-oxide-zirconium-oxide-belt-2-x-72-3650180-grit-pack-of-3>: HTTP status code is not handled or not allowed 2026-01-25 06:20:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22570-depressed-center-wheel-t27-4-1-2-inch-x-045-inch-x-5-8-11-inch-ceramic-alum->: HTTP status code is not handled or not allowed 2026-01-25 06:20:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/l-320an-s-c-alloy-hook-eye-w-latch-11t-wll>: HTTP status code is not handled or not allowed 2026-01-25 06:20:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50133-ao-fiber-disc-7-inch-x-7-8-inch-24-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-25 06:20:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-recessed-screw-climax-h1c-137-a>: HTTP status code is not handled or not allowed 2026-01-25 06:20:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-clamping-collar-recessed-screw-climax-h1c-137-s>: HTTP status code is not handled or not allowed 2026-01-25 06:20:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0002-08-windsor-dinner-knife>: HTTP status code is not handled or not allowed 2026-01-25 06:20:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-sheet-with-acrylic-adhesive-70a-116-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 06:20:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/38322-full-taper-cartridge-roll-1-2-inch-x-1-1-2-inch-x-1-8-inch-120-grit-aluminum-oxide>: HTTP status code is not handled or not allowed 2026-01-25 06:21:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dolly-24-3-8-x-14-7-8x8-tote-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:21:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tempco-immersion-heater-tank-1000-120v-304-ss>: HTTP status code is not handled or not allowed 2026-01-25 06:21:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g-209-galvanized-carbon-shackle-spa-3-8-1t-wll>: HTTP status code is not handled or not allowed 2026-01-25 06:21:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-inner-air-filter-for-iafs-1700-iafs-2400>: HTTP status code is not handled or not allowed 2026-01-25 06:21:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dillon-ed-junior-dynamometer-25000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:21:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/new-age-garden-raised-garden-bed-or-sand-box-48-x-48-x-8>: HTTP status code is not handled or not allowed 2026-01-25 06:21:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-piece-magnetic-wall-container-set-white>: HTTP status code is not handled or not allowed 2026-01-25 06:21:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tall-magnetic-wall-container-white>: HTTP status code is not handled or not allowed 2026-01-25 06:21:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-punch-assembly-for-40t-100t-2013-and-older>: HTTP status code is not handled or not allowed 2026-01-25 06:21:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/set-screw-collar-climax-c-143>: HTTP status code is not handled or not allowed 2026-01-25 06:21:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-woven-file-box-brown>: HTTP status code is not handled or not allowed 2026-01-25 06:21:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mounted-cabinet-with-polycarbonate-door-avalanche-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:21:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-d-ring-vinyl-view-binder-1-capacity-white>: HTTP status code is not handled or not allowed 2026-01-25 06:21:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hr-125-hoist-ring-swivel-3-4-10-x-2-25-thread-proj-0-95-5000-lbs-wll>: HTTP status code is not handled or not allowed 2026-01-25 06:21:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/comfort-grip-deluxe-plus-d-ring-view-binder-3-capacity-8-1-2-x-11-black>: HTTP status code is not handled or not allowed 2026-01-25 06:21:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-roll-no-adhesive-60a-38-thick-x-36-wide-x-6-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 06:21:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 74745 pages (at 98 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:21:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-d-ring-vinyl-view-binder-3-capacity-white>: HTTP status code is not handled or not allowed 2026-01-25 06:21:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-piece-clamping-collar-recessed-screw-climax-h2c-081-s>: HTTP status code is not handled or not allowed 2026-01-25 06:21:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-step-24-w-perforated-safety-angle-ladder>: HTTP status code is not handled or not allowed 2026-01-25 06:21:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/canonls82z-minidesk-calculator-8-digit-lcd>: HTTP status code is not handled or not allowed 2026-01-25 06:21:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ms-80s-tax-and-currency-calculator-8-digit-lcd>: HTTP status code is not handled or not allowed 2026-01-25 06:21:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c-1202-sliding-door-handle-set-white>: HTTP status code is not handled or not allowed 2026-01-25 06:21:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-step-16-w-grip-safety-angle-ladder>: HTTP status code is not handled or not allowed 2026-01-25 06:21:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gn-251-6-setting-bolts-steel-w-retaining-magnet-zinc-plated-m8-1-2-mag-dia-1-l>: HTTP status code is not handled or not allowed 2026-01-25 06:21:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/din-6319-spherical-washers-steel-seat-type-blackened-m6-1-16-t-1-4-i-d>: HTTP status code is not handled or not allowed 2026-01-25 06:21:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/028-oring-aflas-138id-x-112od-8083-duro-round-black-pkg-100>: HTTP status code is not handled or not allowed 2026-01-25 06:21:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/din-6319-spherical-washers-steel-dished-type-blackened-m6-1-8-t-1-4-i-d>: HTTP status code is not handled or not allowed 2026-01-25 06:21:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-high-capacity-bulk-rack-without-decking-60x18x72-starter>: HTTP status code is not handled or not allowed 2026-01-25 06:21:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leeson-10-lb-ft-coupler-brake-56c-143-5tc-nema2-ip23-230-460v-1ph-aluminum-stearns>: HTTP status code is not handled or not allowed 2026-01-25 06:21:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-high-capacity-bulk-rack-with-wire-decking-96x18x72-add-on>: HTTP status code is not handled or not allowed 2026-01-25 06:21:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-strip-with-acrylic-adhesive-70a-18-thick-x-4-wide-x-5-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 06:21:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/329-quad-ring-xring-2id-x-238od-70-duro-round-black-pkg-100>: HTTP status code is not handled or not allowed 2026-01-25 06:21:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-2-shelf-cart-poly-green-24l-x-24w-x-43h-pneumatic-rigid-casters>: HTTP status code is not handled or not allowed 2026-01-25 06:21:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fo-603-tip-out-bin-carts-3-drawers-40-lb-cap-6-516l-x-23-1116w-x-9-78h-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:21:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cortech-usa-seg453075p-seg-max-correc-l-mattress-4-5-x-30-x-75-w-integrated-pillows-1-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:21:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/capital-locker-double-tier-12x18x36-2-door-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:21:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n-7367-pocket-door-privacy-lock-with-pull-3-34-inch-satin-nickel>: HTTP status code is not handled or not allowed 2026-01-25 06:21:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-mini-tarp-hook-stake-orange>: HTTP status code is not handled or not allowed 2026-01-25 06:21:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leeson-dc-controls-scr-series-pwm-series-nema4x-non-reversing>: HTTP status code is not handled or not allowed 2026-01-25 06:21:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metal-lathe-7-5-hp-3-phase-220v-pl-1860e-1-0>: HTTP status code is not handled or not allowed 2026-01-25 06:21:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hammer-case-assembly-for-dxs-2236-series-air-impact-wrench>: HTTP status code is not handled or not allowed 2026-01-25 06:21:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vertical-milling-machine-3-hp-3-phase-220v-vm-949-3>: HTTP status code is not handled or not allowed 2026-01-25 06:21:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hammer-case-gasket-for-dxs-2236-series-air-impact-wrench>: HTTP status code is not handled or not allowed 2026-01-25 06:21:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leeson-single-phase-general-purpose-motor-5hp-1740rpm-184-tefc-208-230v-60hz-manual-rigid>: HTTP status code is not handled or not allowed 2026-01-25 06:21:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-mobile-table-24-x-36>: HTTP status code is not handled or not allowed 2026-01-25 06:21:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hydraulic-metal-shear-3-hp-3-phase-220v-sh-6014>: HTTP status code is not handled or not allowed 2026-01-25 06:21:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-sheet-with-high-temp-adhesive-50a-132-thick-x-36-wide-x-36-long>: HTTP status code is not handled or not allowed 2026-01-25 06:21:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-auto-floor-scrubber-18-cleaning-path-two-115-amp-batteries>: HTTP status code is not handled or not allowed 2026-01-25 06:21:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-sided-shelf-truck-3-shelves-30-x-60>: HTTP status code is not handled or not allowed 2026-01-25 06:21:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-sheet-with-high-temp-adhesive-60a-38-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 06:21:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-sided-shelf-truck-4-shelves-30-x-48>: HTTP status code is not handled or not allowed 2026-01-25 06:21:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/38107-18-long-arm-hex-end-l-wrench>: HTTP status code is not handled or not allowed 2026-01-25 06:21:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paramount-locker-single-tier-15x18x72-1-door-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:21:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leeson-single-phase-farm-ag-motor-1hp-1725rpm-tefc-60hz-manual-round-feed-auger>: HTTP status code is not handled or not allowed 2026-01-25 06:21:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-bar-pipe-truck-36-x-60>: HTTP status code is not handled or not allowed 2026-01-25 06:21:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paramount-locker-double-tier-12x18x36-6-door-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:21:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-shelf-wagon-truck-24-x-48-10-inch-pneumatic-wheels>: HTTP status code is not handled or not allowed 2026-01-25 06:21:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5lb-co2-fire-extinguisher-wall-mount-type-b-c>: HTTP status code is not handled or not allowed 2026-01-25 06:21:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leeson-single-phase-explosion-proof-motor-1-5hp-3450rpm-56-epfc-60hz-automatic-1sf-round>: HTTP status code is not handled or not allowed 2026-01-25 06:21:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-labels-with-trueblock-technology-3-1-3-x-4-white-600-box>: HTTP status code is not handled or not allowed 2026-01-25 06:21:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-mesh-tarp-7-1-2-ft-x-12-ft-3011035>: HTTP status code is not handled or not allowed 2026-01-25 06:21:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-roll-no-adhesive-70a-14-thick-x-36-wide-x-7-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 06:21:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leeson-single-phase-general-purpose-motor-3hp-3450rpm-145-tefc-230v-60hz-1sf-rigid-c>: HTTP status code is not handled or not allowed 2026-01-25 06:21:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-mesh-tarp-7-1-2-ft-x-20-ft-3009095>: HTTP status code is not handled or not allowed 2026-01-25 06:21:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-x-30-shop-top-pedestal-workbench-w-3-drawers-cabinet>: HTTP status code is not handled or not allowed 2026-01-25 06:21:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tyvek-400-sfr-coverall-open-wrists--ankle-storm-flap-serged-seams-m-white-25pk>: HTTP status code is not handled or not allowed 2026-01-25 06:21:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chain-cover-for-dtg-tarp-crank-5544621>: HTTP status code is not handled or not allowed 2026-01-25 06:21:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leeson-single-phase-explosion-proof-motor-3-4hp-3450rpm-56-epfc-60hz-automatic-1-0sf-rigid>: HTTP status code is not handled or not allowed 2026-01-25 06:21:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-mesh-panel-2x8>: HTTP status code is not handled or not allowed 2026-01-25 06:21:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-roll-no-adhesive-70a-18-thick-x-36-wide-x-7-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 06:21:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-mesh-hinged-door-10x4>: HTTP status code is not handled or not allowed 2026-01-25 06:21:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-seal-mailer-0-6-x-10-golden-brown>: HTTP status code is not handled or not allowed 2026-01-25 06:21:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-sided-bulk-truck-mesh-sides-30-x-48-8-x-2-polyurethane-wheels>: HTTP status code is not handled or not allowed 2026-01-25 06:21:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elephantwall-topple-barrier-16-ft--h-x-8-ft-l>: HTTP status code is not handled or not allowed 2026-01-25 06:21:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leeson-single-phase-general-purpose-motor-1-2hp-1725rpm-56-tefc-115-208-230v-60hz-manu-round>: HTTP status code is not handled or not allowed 2026-01-25 06:21:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-sheet-no-adhesive-60a-332-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 06:21:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrislynn-m12x15x360-free-running-helical-inserts-bulk-pack-of-50-inserts>: HTTP status code is not handled or not allowed 2026-01-25 06:21:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/basic-large-moving-boxes-21l-x-17w-x-17h-kraft-blue-5-ctn>: HTTP status code is not handled or not allowed 2026-01-25 06:21:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-mesh-partition-security-room-20x20x8-without-roof-2-sides>: HTTP status code is not handled or not allowed 2026-01-25 06:21:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nomad-construction-site-tent-direct-fired-heater-liquid-propane-190000-btu>: HTTP status code is not handled or not allowed 2026-01-25 06:21:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-general-security-laminated-padlocks-no-2ka>: HTTP status code is not handled or not allowed 2026-01-25 06:21:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rapidroll-wheeled-model-main-base-70-7060>: HTTP status code is not handled or not allowed 2026-01-25 06:21:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/313-carton-sealing-tape-2-x-110-yds-25-mil-clear-6pack>: HTTP status code is not handled or not allowed 2026-01-25 06:21:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/126-quiet-carton-sealing-tape-2-x-110-yds-26-mil-clear-6pack>: HTTP status code is not handled or not allowed 2026-01-25 06:21:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3073-carton-sealing-tape-2-x-110-yds-26-mil-clear-36pack>: HTTP status code is not handled or not allowed 2026-01-25 06:21:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/office-moving-kit-kraftwhite>: HTTP status code is not handled or not allowed 2026-01-25 06:21:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wagon-truck-flush-deck-36-x-72-pneumatic-wheels-3000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:21:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hanging-bracket-kit-for-high-intensity-overhead-heaters>: HTTP status code is not handled or not allowed 2026-01-25 06:21:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/work-height-platform-truck-30-x-60-mold-on-rubber-wheels>: HTTP status code is not handled or not allowed 2026-01-25 06:21:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-general-security-laminated-padlocks-no-31lh>: HTTP status code is not handled or not allowed 2026-01-25 06:21:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-sheet-no-adhesive-60a-132-thick-x-12-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 06:21:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/400-acrylic-carton-sealing-tape-2-x-55-yds-2-mil-clear-36pack>: HTTP status code is not handled or not allowed 2026-01-25 06:21:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caster-steer-trailer-36-x-72>: HTTP status code is not handled or not allowed 2026-01-25 06:21:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-wheeler-wagon-truck-flush-deck-36-x-60>: HTTP status code is not handled or not allowed 2026-01-25 06:21:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-deck-shelf-truck-flush-top-sloped-handle-30-x-48>: HTTP status code is not handled or not allowed 2026-01-25 06:21:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-high-security-steel-solid-body-padlocks-no-6230kalh>: HTTP status code is not handled or not allowed 2026-01-25 06:21:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iteq230-circuit-breaker-type-qp-2-pole-30a>: HTTP status code is not handled or not allowed 2026-01-25 06:21:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-general-security-laminated-padlocks-no-8ka>: HTTP status code is not handled or not allowed 2026-01-25 06:21:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-general-security-laminated-padlocks-no-4ka>: HTTP status code is not handled or not allowed 2026-01-25 06:21:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paper-hot-cups-12-oz-white-1000-carton>: HTTP status code is not handled or not allowed 2026-01-25 06:22:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s75022-class-2-short-sleeve-tshirt-fluorescent-yellow-lg>: HTTP status code is not handled or not allowed 2026-01-25 06:22:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/viton-rubber-sheet-with-high-temp-adhesive-75a-18-thick-x-36-wide-x-36-long>: HTTP status code is not handled or not allowed 2026-01-25 06:22:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-roll-no-adhesive-50a-316-thick-x-36-wide-x-7-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 06:22:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-general-security-laminated-padlocks-no-21lj>: HTTP status code is not handled or not allowed 2026-01-25 06:22:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carton-sealing-tape-mixed-merchandise-2-x-110-yds-22-mil-redwhite-6pack>: HTTP status code is not handled or not allowed 2026-01-25 06:22:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ited140-classified-circuit-breaker-type-qd-replacement-for-square-d-type-qo-1-pole-40a>: HTTP status code is not handled or not allowed 2026-01-25 06:22:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-general-security-laminated-padlocks-no-4kab>: HTTP status code is not handled or not allowed 2026-01-25 06:22:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-shelving-open-shelving-36x12x39-20>: HTTP status code is not handled or not allowed 2026-01-25 06:22:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-dia-round-hatch-net-high-vis-orange-web-free-placement-stainless-brackets-snap-hooks>: HTTP status code is not handled or not allowed 2026-01-25 06:22:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-platform-truck-24-x-36-polyurethane-wheels-3600-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:22:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/p1x1-14-c45-steel-black-oxide-split-taper-bushing-1-14-inch-bore>: HTTP status code is not handled or not allowed 2026-01-25 06:22:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-sheet-with-acrylic-adhesive-60a-116-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 06:22:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-shelving-closed-shelving-36x12x75-11>: HTTP status code is not handled or not allowed 2026-01-25 06:22:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-part-carbonless-inventory-tag-2500-2999-8-6-14l-x-3-18w-500pack>: HTTP status code is not handled or not allowed 2026-01-25 06:22:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-strip-with-acrylic-adhesive-70a-116-thick-x-2-wide-x-5-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 06:22:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 74844 pages (at 99 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:22:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-strip-with-acrylic-adhesive-70a-132-thick-x-4-wide-x-5-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 06:22:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carton-sealing-tape-caution-if-seal-is-broken-2-x-110-yds-redwhite-36pk>: HTTP status code is not handled or not allowed 2026-01-25 06:22:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mid-size-stainless-steel-spray-nozzle-safety-orange>: HTTP status code is not handled or not allowed 2026-01-25 06:22:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-end-platform-truck-24-x-48>: HTTP status code is not handled or not allowed 2026-01-25 06:22:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/colored-carton-sealing-tape-3-x-55-yds-22-mil-black-6pack>: HTTP status code is not handled or not allowed 2026-01-25 06:22:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carton-sealing-tape-if-seal-is-broken-3-x-110-yds-22-mil-redwhite-6pk>: HTTP status code is not handled or not allowed 2026-01-25 06:22:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluidity-03f-hung-diaphragm-solenoid-valve-2-2-nc-nbr-seal-3-4-nptf-18-mm-24v-dc>: HTTP status code is not handled or not allowed 2026-01-25 06:22:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-x-4-hatch-net-high-vis-orange-webbing-free-placement-stainless-brackets-snap-hooks>: HTTP status code is not handled or not allowed 2026-01-25 06:22:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluidity-02f-direct-acting-solenoid-valve-2-2-nc-fkm-seal-1-2-nptf-3-mm-12v-dc>: HTTP status code is not handled or not allowed 2026-01-25 06:22:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-x-5-hatch-net-high-vis-orange-webbing-free-placement-stainless-brackets-snap-hooks>: HTTP status code is not handled or not allowed 2026-01-25 06:22:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-retractable-cone-bar-orange-with-reflective-tape-5-to-8>: HTTP status code is not handled or not allowed 2026-01-25 06:22:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-roll-with-acrylic-adhesive-60a-18-thick-x-36-wide-x-10-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 06:22:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-roll-with-acrylic-adhesive-50a-14-thick-x-36-wide-x-10-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 06:22:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2300-hitop-work-rubber-overshoes-black-cleated-outsole-2xl>: HTTP status code is not handled or not allowed 2026-01-25 06:22:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-sheet-with-acrylic-adhesive-70a-132-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 06:22:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluidity-03f-hung-diaphragm-solenoid-valve-2-2-nc-fkm-seal-1-4-nptf-10-mm-12v-dc>: HTTP status code is not handled or not allowed 2026-01-25 06:22:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-sheet-with-acrylic-adhesive-60a-316-thick-x-36-wide-x-36-long>: HTTP status code is not handled or not allowed 2026-01-25 06:22:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hytrol-5-ft-28-inch-w-1-9-inch-dia-galvanized-rollers-25-inch-between-rail-6-inch-rollers-centers>: HTTP status code is not handled or not allowed 2026-01-25 06:22:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paramount-locker-single-tier-12x12x60-1-door-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:22:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hytrol-5-ft-18-inch-w-1-9-inch-dia-galvanized-rollers-15-inch-between-rail-4-5-inch-rollers-centers>: HTTP status code is not handled or not allowed 2026-01-25 06:22:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/170-acrylic-carton-sealing-tape-2-x-55-yds-18-mil-tan-6pack>: HTTP status code is not handled or not allowed 2026-01-25 06:22:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paramount-locker-single-tier-12x12x60-3-door-assembled-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:22:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paramount-locker-single-tier-12x15x60-3-door-assembled-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:22:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hytrol-2-ft-ball-transfer-table-21-inch-between-frame-4-inch-ball-center>: HTTP status code is not handled or not allowed 2026-01-25 06:22:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-af-blend-sheet-0-375-thick-x-24-000-w-x-48-000-l>: HTTP status code is not handled or not allowed 2026-01-25 06:22:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-strip-with-acrylic-adhesive-70a-12-thick-x-6-wide-x-5-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 06:22:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-strip-with-acrylic-adhesive-70a-12-thick-x-2-wide-x-5-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 06:22:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/375-carton-sealing-tape-2-x-55-yds-31-mil-tan-6pack>: HTTP status code is not handled or not allowed 2026-01-25 06:22:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paramount-locker-single-tier-15x18x72-3-door-assembled-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:22:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700-hot-melt-tape-2-x-55-yds-19-mil-tan-6pack>: HTTP status code is not handled or not allowed 2026-01-25 06:22:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paramount-locker-double-tier-12x12x36-2-door-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:22:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paramount-locker-double-tier-12x12x36-2-door-assembled-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:22:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-bunan-rubber-sheet-no-adhesive-70a-332-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 06:22:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paramount-locker-double-tier-12x18x36-2-door-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:22:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluidity-03f-hung-diaphragm-solenoid-valve-2-2-nc-epdm-seal-1-4-nptf-10-mm-24v-dc>: HTTP status code is not handled or not allowed 2026-01-25 06:22:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-300-nipple-1-8-npt-female-aro-210-standard>: HTTP status code is not handled or not allowed 2026-01-25 06:22:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fda-neoprene-rubber-strip-with-acrylic-adhesive-60a-14-thick-x-1-wide-x-10-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 06:22:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-310-nipple-1-8-female-npt-industrial-interchange-1-4>: HTTP status code is not handled or not allowed 2026-01-25 06:22:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40xl037-6a4-aluminum-clear-anodized-40-tooth-2-546-pitch-finished-bore-pulley>: HTTP status code is not handled or not allowed 2026-01-25 06:22:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-sheet-no-adhesive-60a-316-thick-x-36-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 06:22:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o56107-durascrim-snap-fly-front-overall-yellow-5xl>: HTTP status code is not handled or not allowed 2026-01-25 06:22:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o56107-durascrim-snap-fly-front-overall-yellow-lg>: HTTP status code is not handled or not allowed 2026-01-25 06:22:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vinyl-light-duty-ribbed-stair-tread-square-nose-124063-x-72-light-brown>: HTTP status code is not handled or not allowed 2026-01-25 06:22:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-diamond-stair-tread-square-nose-1231-x-72-black>: HTTP status code is not handled or not allowed 2026-01-25 06:22:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-roll-no-adhesive-40a-14-thick-x-36-wide-x-7-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 06:22:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/advantage-pro-self-propelled-electric-pallet-jack-truck-27x48-24v-batt-3300-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 06:22:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/duct-adaptor-single-outlet-for-heat-wagon-oil-indirect-fired-heater-110k-btu-ductable>: HTTP status code is not handled or not allowed 2026-01-25 06:22:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/advantage-pro-max-self-propelled-electric-pallet-jack-truck-27w-x-48l-4400-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 06:22:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-circle-tag-3-dia-blue-100pack>: HTTP status code is not handled or not allowed 2026-01-25 06:22:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-delrin-homopolymer-rod-6-000-dia>: HTTP status code is not handled or not allowed 2026-01-25 06:22:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esd-office-chair-medium-height-value-line-fabric-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:22:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-tag-pre-strung-5-4-34l-x-2-38w-dark-blue-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 06:22:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-deck-bulk-rack-shelf-96x24-2>: HTTP status code is not handled or not allowed 2026-01-25 06:22:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-in-super-air-knife-only-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 06:22:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elasglow-glow-in-the-dark-drop-over-four-inches-wide-by-one-inch-high-red-ed1010-r-glow>: HTTP status code is not handled or not allowed 2026-01-25 06:22:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-tag-pre-strung-8-6-14l-x-3-18w-dark-blue-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 06:22:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4x5-8-t29-z3-reg-60-gritflap-disc>: HTTP status code is not handled or not allowed 2026-01-25 06:22:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-sheet-no-adhesive-60a-116-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 06:22:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-roll-no-adhesive-50a-14-thick-x-36-wide-x-5-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 06:22:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0404-m6x1-0-d5-4-flute-bright-taper-plug-and-bottoming-hand-tap-set>: HTTP status code is not handled or not allowed 2026-01-25 06:22:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gn471-crank-handles-aluminum-revolving-handle-through-round-bore-1-2-6-3-long>: HTTP status code is not handled or not allowed 2026-01-25 06:22:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-roll-no-adhesive-50a-18-thick-x-36-wide-x-8-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 06:22:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cherne-4655110>: HTTP status code is not handled or not allowed 2026-01-25 06:22:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gn471-1-crank-handles-zinc-die-cast-revolving-handle-thru-square-bore-1-2-3-15-l>: HTTP status code is not handled or not allowed 2026-01-25 06:22:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/queueway-8-foot-red-velour-rope-polished-brass-ends>: HTTP status code is not handled or not allowed 2026-01-25 06:22:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1150-wintertuff-ice-traction-spikes-black-studded-outsole-lg>: HTTP status code is not handled or not allowed 2026-01-25 06:22:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slatwall-white-shelves-3-4-x10-x47-3-4-finished-on-2-sides-and-3-edges>: HTTP status code is not handled or not allowed 2026-01-25 06:22:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-delrin-homopolymer-rod-2-500-dia>: HTTP status code is not handled or not allowed 2026-01-25 06:22:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-shelf-bench-rack-with-128-bins-4-wide-2>: HTTP status code is not handled or not allowed 2026-01-25 06:22:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2x1x1-4-aluminum-oxide-40-grit-flap-wheel>: HTTP status code is not handled or not allowed 2026-01-25 06:22:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s-hook-2-l-x-0-177-thick-carbon-steel-zinc-clear-usa-shook-006>: HTTP status code is not handled or not allowed 2026-01-25 06:22:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s-hook-1-1-16-l-x-0-080-thick-stainless-steel-plain-usa-shks-001>: HTTP status code is not handled or not allowed 2026-01-25 06:22:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-ketron-peek-rod-q-0-500-dia>: HTTP status code is not handled or not allowed 2026-01-25 06:22:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-chipboard-pads-14l-x-8-12w-kraft-575pack>: HTTP status code is not handled or not allowed 2026-01-25 06:22:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylatron-nsm-rod-5-000-dia>: HTTP status code is not handled or not allowed 2026-01-25 06:22:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/l-shaped-electric-height-adjustable-desk-72-w-24-d-gray-w-black-base>: HTTP status code is not handled or not allowed 2026-01-25 06:22:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-1-pneumatic-grease-pump-400-lb>: HTTP status code is not handled or not allowed 2026-01-25 06:22:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/13-shelf-steel-shelving-with-81-akro-mils-shelf-bins-2>: HTTP status code is not handled or not allowed 2026-01-25 06:22:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3m-s-b-6x1-2-2s-fne048011-13719>: HTTP status code is not handled or not allowed 2026-01-25 06:22:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/902-black-original-ink-cartridge-300-page-yield>: HTTP status code is not handled or not allowed 2026-01-25 06:22:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-fda-silicone-rubber-sheet-no-adhesive-40a-12-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 06:22:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-tag-2-3-14l-x-1-58w--fluorescent-pink-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 06:23:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mb924b-neoprene-steel-toe-snugleg-boots-brick-redbrown-size-11>: HTTP status code is not handled or not allowed 2026-01-25 06:23:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26x-high-yield-black-original-laserjet-toner-cartridge-9000-page-yield>: HTTP status code is not handled or not allowed 2026-01-25 06:23:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/used-oil-storage-system-180-gallon>: HTTP status code is not handled or not allowed 2026-01-25 06:23:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perm-liner-truck-18-bu-green-vinyl-wire-base-all-swivel>: HTTP status code is not handled or not allowed 2026-01-25 06:23:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bench-rack-with-22-akrobins-36x20-4>: HTTP status code is not handled or not allowed 2026-01-25 06:23:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylatron-nsm-sheet-0-250-thick-x-24-w-x-48-l>: HTTP status code is not handled or not allowed 2026-01-25 06:23:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perm-liner-truck-16-bu-gray-vinyl-wire-base-all-swivel>: HTTP status code is not handled or not allowed 2026-01-25 06:23:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-locker-bench-hardwood-top-w-steel-tube-pedestals-bolt-down-60-x-9-1-2-x-17>: HTTP status code is not handled or not allowed 2026-01-25 06:23:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-strength-neoprene-rubber-sheet-no-adhesive-60a-12-thick-x-12-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 06:23:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perm-liner-truck-12-bu-green-vinyl-wire-base-all-swivel>: HTTP status code is not handled or not allowed 2026-01-25 06:23:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22w-120v-warm-white-led-downlight-bronze-texture>: HTTP status code is not handled or not allowed 2026-01-25 06:23:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-oil-filled-cast-nylon-rod-3-5-dia>: HTTP status code is not handled or not allowed 2026-01-25 06:23:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corner-bumper-kit-black-set-of-four-4-fits-royal-basket-brand-vinyl-and-canvas-basket-trucks>: HTTP status code is not handled or not allowed 2026-01-25 06:23:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-step-industrial-access-stairway-ladder-perforated>: HTTP status code is not handled or not allowed 2026-01-25 06:23:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fda-neoprene-rubber-sheet-no-adhesive-60a-116-thick-x-6-wide-x-6-long>: HTTP status code is not handled or not allowed 2026-01-25 06:23:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/textured-neoprene-rubber-sheet-with-acrylic-adhesive-50a-14-thick-x-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 06:23:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 74942 pages (at 98 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:23:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tygon-b44-4x-tube-aaa05085-3-000-id-x-3-500-od-x-20-l>: HTTP status code is not handled or not allowed 2026-01-25 06:23:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/del-extraction-arm-6-h>: HTTP status code is not handled or not allowed 2026-01-25 06:23:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perm-liner-truck-12-bu-red-vinyl-wood-base-2-rg-2-sw>: HTTP status code is not handled or not allowed 2026-01-25 06:23:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/del-cone-nozzle-7-3-4-l-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:23:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/expandable-add-on-rack-96x36x84-gray-with-3-level-wire-deck-1100lb-cap-per-level>: HTTP status code is not handled or not allowed 2026-01-25 06:23:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-7-gauge-steel-workbench-drawer-adjustable-height-30-x-60>: HTTP status code is not handled or not allowed 2026-01-25 06:23:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-7-gauge-steel-workbench-adjustable-height-36-x-72>: HTTP status code is not handled or not allowed 2026-01-25 06:23:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tyvek174-shipping-tag-4-14l-x-2-18w-blue-100pack>: HTTP status code is not handled or not allowed 2026-01-25 06:23:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gorbel-ld-wall-mounted-tool-solutions-jib-crane-10-ft-span-150-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:23:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/del-hose-adapter-50mm-connector-galvanized-steel>: HTTP status code is not handled or not allowed 2026-01-25 06:23:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-level-48x24-high-capacity-rack-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 06:23:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/welded-workbench-butcher-block-top-drawer-fixed-height-30-x-72>: HTTP status code is not handled or not allowed 2026-01-25 06:23:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/j22201-iron-eagle-storm-fly-front-jacket-blue-hood-snaps-sm>: HTTP status code is not handled or not allowed 2026-01-25 06:23:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mb921b-neoprene-steel-toe-snugleg-boots-brick-redbrown-size-13>: HTTP status code is not handled or not allowed 2026-01-25 06:23:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3771-printed-message-carton-sealing-tape-2-x-110-yds-1-9-mil-clear-red-36-rolls-case>: HTTP status code is not handled or not allowed 2026-01-25 06:23:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eze-off-steel-delivery-hand-truck-8-mold-on-rubber-wheels-800-lb-capacity-54-1-2-height>: HTTP status code is not handled or not allowed 2026-01-25 06:23:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-wire-shelf-individual-basket-48w-x-18d>: HTTP status code is not handled or not allowed 2026-01-25 06:23:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-shelf-with-laminated-deck-72x36-gray-2>: HTTP status code is not handled or not allowed 2026-01-25 06:23:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/del-rectangular-brush-6-1-2-l-red>: HTTP status code is not handled or not allowed 2026-01-25 06:23:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o24129-icon-snap-fly-front-overall-fluorescent-orange-3xl>: HTTP status code is not handled or not allowed 2026-01-25 06:23:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jewelry-boxes-3-116l-x-2-18w-x-1h-white-100pack>: HTTP status code is not handled or not allowed 2026-01-25 06:23:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/del-galvanized-filter-non-sparking-9-1-3-l-x-9-1-3-w-x-1-9-10-h>: HTTP status code is not handled or not allowed 2026-01-25 06:23:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/del-food-grade-vacuum-hose-red-20-l>: HTTP status code is not handled or not allowed 2026-01-25 06:23:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-30-glass-filled-nylon-6-6-rod-2-5-dia>: HTTP status code is not handled or not allowed 2026-01-25 06:23:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-top-machine-table-24-x-36-x-42>: HTTP status code is not handled or not allowed 2026-01-25 06:23:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proform-f3-h013002-standard-harness-quick-connect-legs-m-l>: HTTP status code is not handled or not allowed 2026-01-25 06:23:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cubby-locker-welded-34-1-2w-x-13-1-2d-x-40h-9-compartments-medium-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:23:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-infinity-ventilated-steel-locker-six-tier-3-wide-12x15x12-unassembled-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:23:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reinforced-clear-vinyl-tubing-1-2-x-0-750-o-d-300-feet-coil>: HTTP status code is not handled or not allowed 2026-01-25 06:23:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/red-gpo-3-sheet-0-187-thick-x-48-w-x-48-l>: HTTP status code is not handled or not allowed 2026-01-25 06:23:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-top-machine-table-24-x-36-x-30-swivel-casters-w-brakes>: HTTP status code is not handled or not allowed 2026-01-25 06:23:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gray-fishpaper-sheet-0-015-thick-x-52-000-w-x-84-000-l>: HTTP status code is not handled or not allowed 2026-01-25 06:23:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-pencil-drawer-compact-85-w-x-14-l-x-1-1-4-h>: HTTP status code is not handled or not allowed 2026-01-25 06:23:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gloveworks-hd-diamond-textured-nitrile-industrial-gloves-m-green-100-per-box-10-box-per-cs>: HTTP status code is not handled or not allowed 2026-01-25 06:23:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-7-gauge-steel-workbench-fixed-height-30-x-48>: HTTP status code is not handled or not allowed 2026-01-25 06:23:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-flash-powdered-alkaline-concrete-cleaner-125-lb-drum>: HTTP status code is not handled or not allowed 2026-01-25 06:23:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/red-gpo-3-sheet-1-250-thick-x-48-w-x-96-l>: HTTP status code is not handled or not allowed 2026-01-25 06:23:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reverse-tuck-folding-cartons-4l-x-1-58w-x-4h-kraft-500pack>: HTTP status code is not handled or not allowed 2026-01-25 06:23:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-54-w-x-18-d-x-63-h>: HTTP status code is not handled or not allowed 2026-01-25 06:23:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-fair-conquer-green-link-deodorant-32-oz-spray-bottle-pleasant-scent-12-bottles-case>: HTTP status code is not handled or not allowed 2026-01-25 06:23:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-adjustable-3-shelf-unit-44-x-60-x-16-9346-3-03>: HTTP status code is not handled or not allowed 2026-01-25 06:23:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nomex-3-0-density-fiberglass-epoxy-honeycomb-0-125-thick-x-48-w-x-96-l-125-cell-ar2623>: HTTP status code is not handled or not allowed 2026-01-25 06:23:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pot-pan-high-foaming-liquid-hand-dishwashing-detergent-5-gallon-pail-1-pail>: HTTP status code is not handled or not allowed 2026-01-25 06:23:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/louvered-panel-36-x-19-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:23:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gray-pet-ertalyte-tx-rod-q-6-dia>: HTTP status code is not handled or not allowed 2026-01-25 06:23:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-vinyl-tubing-1-4-i-d-x-1-16-wall-x-100-feet-boxed>: HTTP status code is not handled or not allowed 2026-01-25 06:23:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foaming-shower-tub-tile-cleaner-quart-bottle-12-bottles-case>: HTTP status code is not handled or not allowed 2026-01-25 06:23:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-vinyl-tubing-5-8-i-d-x-1-8-wall-x-100-feet-boxed>: HTTP status code is not handled or not allowed 2026-01-25 06:23:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gray-pet-ertalyte-tx-rod-q-1-25-dia>: HTTP status code is not handled or not allowed 2026-01-25 06:23:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reverse-tuck-folding-cartons-5-14l-x-2-14w-x-5-14h-white-250pack>: HTTP status code is not handled or not allowed 2026-01-25 06:23:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/great-lakes-tin-erie-2-x-2-nail-up-tin-ceiling-tile-in-copper-t56-08>: HTTP status code is not handled or not allowed 2026-01-25 06:23:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/great-lakes-tin-erie-2-x-2-nail-up-tin-ceiling-tile-in-argento-t56-07>: HTTP status code is not handled or not allowed 2026-01-25 06:23:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/252-m16x1-5-a-steel-threaded-plug-m16-x-1-5-thread>: HTTP status code is not handled or not allowed 2026-01-25 06:23:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/twisttiemachinesandmaterials-17-2000twisttiematerial-brown-422943>: HTTP status code is not handled or not allowed 2026-01-25 06:23:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylon-membrane-filter-0-45-um-25-mm-diameter-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:23:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fm-080-1-1-4-total-water-main-shutoff-1-1-4-npt-full-port-lead-free-valve>: HTTP status code is not handled or not allowed 2026-01-25 06:23:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ptfe-syringe-filters-non-sterile-hydrophilic-0-22-um-13-mm-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:23:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grade-cfp1234-phase-separating-cellulose-filter-paper-15-0-cm-dia-11-um-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:23:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/libman-commercial-long-handle-grill-brush-black>: HTTP status code is not handled or not allowed 2026-01-25 06:23:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/libman-commercial-angle-broom>: HTTP status code is not handled or not allowed 2026-01-25 06:23:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gift-boxes-6l-x-4-12w-x-4-12h-kraft-100pack>: HTTP status code is not handled or not allowed 2026-01-25 06:23:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/great-lakes-tin-rochester-2-x-2-nail-up-tin-ceiling-tile-in-antique-white-t58-02>: HTTP status code is not handled or not allowed 2026-01-25 06:23:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnetic-accessory-smart-storage-box-black-sm010101>: HTTP status code is not handled or not allowed 2026-01-25 06:23:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grade-cfp5-qualitative-cellulose-filter-paper-0-195-mm-thick-9-0-cm-dia-2-5-um-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:23:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/481823087a-clip-s3-add-on-rack-unit-87-h-x-48-w-x-18-d-5xms230-shelves-galvanized>: HTTP status code is not handled or not allowed 2026-01-25 06:23:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anchor-cement-kit-each-1>: HTTP status code is not handled or not allowed 2026-01-25 06:23:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-round-cafe-table-gray-top-cafeworks-series>: HTTP status code is not handled or not allowed 2026-01-25 06:23:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2696-26-cordless-lithium-ion-6-tool-combo-kit>: HTTP status code is not handled or not allowed 2026-01-25 06:23:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chiavari-chairs-resin-gold>: HTTP status code is not handled or not allowed 2026-01-25 06:23:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-mobile-impulse-display-stand-square-baskets-3-tier-black>: HTTP status code is not handled or not allowed 2026-01-25 06:23:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/porta-king-8-x-8-guard-booth-gray-pre-assembled>: HTTP status code is not handled or not allowed 2026-01-25 06:23:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-with-36-shelf-bins-13-shelves-36x12x75-3>: HTTP status code is not handled or not allowed 2026-01-25 06:23:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coolant-pump-switch-5516855>: HTTP status code is not handled or not allowed 2026-01-25 06:23:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/libman-commercial-large-looped-end-wet-mop-head>: HTTP status code is not handled or not allowed 2026-01-25 06:23:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/great-lakes-tin-toronto-2-x-2-lay-in-tin-ceiling-tile-in-matte-white-y59-01>: HTTP status code is not handled or not allowed 2026-01-25 06:23:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-with-15-hopper-bins-36x18x75-6>: HTTP status code is not handled or not allowed 2026-01-25 06:23:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-32-4808-6-power-bit-phillips-3-bulk-10>: HTTP status code is not handled or not allowed 2026-01-25 06:23:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/libman-commercial-long-handle-utility-brush-white>: HTTP status code is not handled or not allowed 2026-01-25 06:23:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-perforated-basket-for-p360-series-ultrasonic-part-cleaners>: HTTP status code is not handled or not allowed 2026-01-25 06:23:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electronic-locking-storage-cabinet-48x24x72-charcoal>: HTTP status code is not handled or not allowed 2026-01-25 06:23:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/74h-nexel-chrome-wire-shelving-add-on-30wx21d>: HTTP status code is not handled or not allowed 2026-01-25 06:24:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/great-lakes-tin-jamestown-2-x-2-lay-in-tin-ceiling-tile-in-matte-white-y51-01>: HTTP status code is not handled or not allowed 2026-01-25 06:24:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/great-lakes-tin-jamestown-2-x-2-lay-in-tin-ceiling-tile-in-gloss-white-y51-00>: HTTP status code is not handled or not allowed 2026-01-25 06:24:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7060-spring-5512341>: HTTP status code is not handled or not allowed 2026-01-25 06:24:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-step-aluminum-rolling-ladder-24-w-ribbed-step-w-o-handrails>: HTTP status code is not handled or not allowed 2026-01-25 06:24:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bowl-clip-mango-orange-72-case>: HTTP status code is not handled or not allowed 2026-01-25 06:24:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-z-brite-wire-shelving-add-on-36wx14dx63h>: HTTP status code is not handled or not allowed 2026-01-25 06:24:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bright-air-max-odor-eliminator-air-freshener-meadow-breeze-8-oz-6-case>: HTTP status code is not handled or not allowed 2026-01-25 06:24:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-vinyl-bag-janitorial-cart>: HTTP status code is not handled or not allowed 2026-01-25 06:24:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fresh-products-wave-3d-urinal-deodorizer-screen-blue-ocean-mist-fragrance-60-screens-case>: HTTP status code is not handled or not allowed 2026-01-25 06:24:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/illinois-engineered-products-single-folding-gate-8w-to-9w-and-66h>: HTTP status code is not handled or not allowed 2026-01-25 06:24:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-2-wheeled-trash-container-35-gallon-blue-3954444f-bs8>: HTTP status code is not handled or not allowed 2026-01-25 06:24:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ledge-30-l-x-4-h-wire-shelves-black-epoxy>: HTTP status code is not handled or not allowed 2026-01-25 06:24:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/86h-nexel-chrome-wire-shelving-add-on-48wx21d>: HTTP status code is not handled or not allowed 2026-01-25 06:24:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/night-stick-mt-220-mini-tactical-pro-led-flashlight-2009045-lumens-2-aa-batteries>: HTTP status code is not handled or not allowed 2026-01-25 06:24:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/great-lakes-tin-hamilton-2-x-2-lay-in-tin-ceiling-tile-in-unfinished-y52-03>: HTTP status code is not handled or not allowed 2026-01-25 06:24:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 75038 pages (at 96 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:24:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/key-5-x-45-5518166>: HTTP status code is not handled or not allowed 2026-01-25 06:24:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/copper-block-1230r-5234681>: HTTP status code is not handled or not allowed 2026-01-25 06:24:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blend-prep-xl-tack-cloth-18-x-18-blue-10-box-020008b>: HTTP status code is not handled or not allowed 2026-01-25 06:24:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2221vs-v-belt-5510079>: HTTP status code is not handled or not allowed 2026-01-25 06:24:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-z-brite-wire-shelving-add-on-60wx14dx86h>: HTTP status code is not handled or not allowed 2026-01-25 06:24:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/butk2436-24-x-36-exposed-flange-knurled-knob>: HTTP status code is not handled or not allowed 2026-01-25 06:24:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/record-storage-rack-starter-60x36x96>: HTTP status code is not handled or not allowed 2026-01-25 06:24:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4500-wheel-adjustment-5512512>: HTTP status code is not handled or not allowed 2026-01-25 06:24:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aquaba-38-round-resin-pedestal-table-w-umbrella-hole-white-marble-on-white-frame>: HTTP status code is not handled or not allowed 2026-01-25 06:24:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylon-brush-w-quick-connect-for-1-id-tubes-blue-pack-of-15>: HTTP status code is not handled or not allowed 2026-01-25 06:24:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-natural-elements-he-liquid-detergent-linen-and-lilies-150-oz-bottle-4-case>: HTTP status code is not handled or not allowed 2026-01-25 06:24:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4210-belt-cover-5513479>: HTTP status code is not handled or not allowed 2026-01-25 06:24:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/probite-174-3-8-x-3-8-mnpt-lead-free-brass-straight-male-coupling>: HTTP status code is not handled or not allowed 2026-01-25 06:24:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/taper-brg--4t-32006x-1230r-5235161>: HTTP status code is not handled or not allowed 2026-01-25 06:24:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bntc2436-24-x-36-exposed-flange-cam-latch>: HTTP status code is not handled or not allowed 2026-01-25 06:24:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/panel-kit-72-w-workbench-with-18-w-louver-36-w-pegboard-mounting-rail-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:24:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4c-horizontal-mailbox-41h-double-column-20-mb1-doors-front-load-bronze-usps>: HTTP status code is not handled or not allowed 2026-01-25 06:24:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4c-horizontal-mailbox-23-1-2h-double-column-9-mb1-doors-front-load-aluminum-usps>: HTTP status code is not handled or not allowed 2026-01-25 06:24:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4c-horizontal-mailbox-23-1-2h-double-column-9-mb1-doors-front-load-bronze-usps>: HTTP status code is not handled or not allowed 2026-01-25 06:24:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7020-hose-clamp-ref-8a-5519487>: HTTP status code is not handled or not allowed 2026-01-25 06:24:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-2-shelf-tray-service-utility-cart-34-x-17-5-5-rubber-casters>: HTTP status code is not handled or not allowed 2026-01-25 06:24:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/we-253-adaptor>: HTTP status code is not handled or not allowed 2026-01-25 06:24:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2232ac-bearing-ball-5510148>: HTTP status code is not handled or not allowed 2026-01-25 06:24:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-wire-shelf-truck-36x24x80-1200-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:24:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4c-horizontal-mailbox-37-1-2h-single-column-2-pl5-doors-front-load-aluminum-usps>: HTTP status code is not handled or not allowed 2026-01-25 06:24:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4c-horizontal-mailbox-23-1-2h-single-parcel-box-1-pl6-front-load-aluminum-usps>: HTTP status code is not handled or not allowed 2026-01-25 06:24:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/prf122www-0-white-diamond-solid-door-refrigerator-freezer-combo-12-cu-ft>: HTTP status code is not handled or not allowed 2026-01-25 06:24:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-dunnage-rack-24-w-x-24-d-2>: HTTP status code is not handled or not allowed 2026-01-25 06:24:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disinfectant-toilet-bowl-cleaner-with-bleach-24-oz-bottle>: HTTP status code is not handled or not allowed 2026-01-25 06:24:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4b-horizontal-mailbox-30-a-doors-29-usable-front-loading-bronze-usps-access>: HTTP status code is not handled or not allowed 2026-01-25 06:24:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-antibacterial-dishwashing-liquid-orange-scent-28-oz-bottle-8-case>: HTTP status code is not handled or not allowed 2026-01-25 06:24:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnum-pro-250l-welding-gun-15l-cable>: HTTP status code is not handled or not allowed 2026-01-25 06:24:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pin-5x25-5517398>: HTTP status code is not handled or not allowed 2026-01-25 06:24:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cooper153-crx-10ial-air-cooled-welding-cobot-no-cart-package>: HTTP status code is not handled or not allowed 2026-01-25 06:24:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/embassy-air-vent-cap-assembly>: HTTP status code is not handled or not allowed 2026-01-25 06:24:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/centerfeed-hand-towel-2-ply-7-6-x-11-8-white-500-roll-6-rolls-case>: HTTP status code is not handled or not allowed 2026-01-25 06:24:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/embassy-hav-88-3-compact-hydronic-heater>: HTTP status code is not handled or not allowed 2026-01-25 06:24:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/embassy-vent-block-brass-end-block-kit>: HTTP status code is not handled or not allowed 2026-01-25 06:24:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/embossed-spray-bottle-24-oz-clear-24-case>: HTTP status code is not handled or not allowed 2026-01-25 06:24:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/embassy-hose-bib-adapter>: HTTP status code is not handled or not allowed 2026-01-25 06:24:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/keyguard-lab-coat-no-pockets-elastic-wrists-snap-front-single-collar-white-md-30case>: HTTP status code is not handled or not allowed 2026-01-25 06:24:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-feed-84-single-bench-wire-feeder-ui-heavy-duty-reel-usb-40-vdc>: HTTP status code is not handled or not allowed 2026-01-25 06:24:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/food-safe-motor-vfswdm3558-e-ip69-2hp-1750rpm-3ph-60hz-56c-3528m-tefc-f1-n>: HTTP status code is not handled or not allowed 2026-01-25 06:24:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-mil-polyethylene-apron-blue-28-x-46-100bag-10-bagscase>: HTTP status code is not handled or not allowed 2026-01-25 06:24:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/embassy-hav-88-3-2-speed-fan-switch>: HTTP status code is not handled or not allowed 2026-01-25 06:24:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/embassy-48-panel-track-heater-34-element>: HTTP status code is not handled or not allowed 2026-01-25 06:24:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swa-50-r-4860-scl-approach-ramp-for-scale-option-swa-50-only>: HTTP status code is not handled or not allowed 2026-01-25 06:24:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rack-protector-with-cutout-3-x-3-opening-18-h-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:24:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/laminated-polypropylene-boot-covers-white-large-100-pairscase>: HTTP status code is not handled or not allowed 2026-01-25 06:24:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metro-stackable-sidechair-cactus-green-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-25 06:24:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shss-m5x12mm-5640191>: HTTP status code is not handled or not allowed 2026-01-25 06:24:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fabric-softener-sheets-fresh-scent-120-sheets-box>: HTTP status code is not handled or not allowed 2026-01-25 06:24:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-lab-coat-no-pockets-elastic-wrists-snap-front-single-collar-blue-xl-30cs>: HTTP status code is not handled or not allowed 2026-01-25 06:24:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shine-up-153-mc-multi-surface-foaming-polish-lemon-scent-15-oz-aerosol-12-case>: HTTP status code is not handled or not allowed 2026-01-25 06:24:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-storage-drawer-cabinet-48x24x32-11>: HTTP status code is not handled or not allowed 2026-01-25 06:24:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-lab-coat-no-pockets-elastic-wrists-snap-front-single-collar-white-5xl-30cs>: HTTP status code is not handled or not allowed 2026-01-25 06:24:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-storage-drawer-cabinet-48x24x40-6>: HTTP status code is not handled or not allowed 2026-01-25 06:24:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7020-work-stop-rod-5712431>: HTTP status code is not handled or not allowed 2026-01-25 06:24:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bona-hardwood-floor-mop-15-microfiber-head-with-52-handle-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:24:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/474-ergomat153-anti-fatigue-mat-12-thick-2w-x-up-to-75l-black>: HTTP status code is not handled or not allowed 2026-01-25 06:24:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dispatcher-locker-triple-front-door-single-rear-door-24w-x-37-1-2d-x-79-1-2h>: HTTP status code is not handled or not allowed 2026-01-25 06:24:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-maverick-200200x-kubota-ready-pak-3-welder-335a-60-hz-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:24:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chill-its-6710-evaporative-cooling-triangle-hat-red-western-one-size>: HTTP status code is not handled or not allowed 2026-01-25 06:24:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eccentric-bolt---vsf-14-5519772>: HTTP status code is not handled or not allowed 2026-01-25 06:24:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-sticky-non-skid-shoe-covers-water-resistant-blue-lg-300case>: HTTP status code is not handled or not allowed 2026-01-25 06:24:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/viking153-2450-adv-welding-helmet-5-13-shade-black>: HTTP status code is not handled or not allowed 2026-01-25 06:24:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-x-2-drywall-screw-phillips-bugle-head-steel-black-phosphate-sharp-point-pkg-of-5-lbs>: HTTP status code is not handled or not allowed 2026-01-25 06:24:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-x-1-drywall-screw-phillips-bugle-head-black-phosphate-tek-self-drilling-point-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 06:24:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/office-packs-facial-tissue-2-ply-white-flat-box-100-sheets-box-30-boxes-case>: HTTP status code is not handled or not allowed 2026-01-25 06:24:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-x-1-1-4-drywall-screw-phillips-bugle-head-black-phosphate-tek-self-drilling-pt-100-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:24:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-x-3-drywall-screw-phillips-bugle-head-steel-black-phosphate-sharp-point-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 06:24:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-1100sf-standard-back-support-black-3-x-large>: HTTP status code is not handled or not allowed 2026-01-25 06:24:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-lab-coat-no-pockets-elastic-wrists-snap-front-single-collar-white-xl-30cs>: HTTP status code is not handled or not allowed 2026-01-25 06:24:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8201-spring-5783261>: HTTP status code is not handled or not allowed 2026-01-25 06:24:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scrubbers-spraybuffpad-red-10-case-261490>: HTTP status code is not handled or not allowed 2026-01-25 06:24:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/150-cfm-1-4-sones-energy-star-fan>: HTTP status code is not handled or not allowed 2026-01-25 06:24:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lightweight-polypropylene-bouffant-cap-100-latex-free-blue-19-100bag-10-bagscase>: HTTP status code is not handled or not allowed 2026-01-25 06:24:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8289-class-2-economy-t-shirt-lime-x-large>: HTTP status code is not handled or not allowed 2026-01-25 06:24:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-keyed-padlock-3-4-shackle-keyed-different>: HTTP status code is not handled or not allowed 2026-01-25 06:24:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-straight-leg-dunnage-rack-60-lx18-wx8-h>: HTTP status code is not handled or not allowed 2026-01-25 06:24:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/control-cable-extension-for-ln-15-flex-feeds-ln-25-pro-dual-power-25l>: HTTP status code is not handled or not allowed 2026-01-25 06:24:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-straight-leg-dunnage-rack-36-lx20-wx8-h>: HTTP status code is not handled or not allowed 2026-01-25 06:24:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rough-n-ready-plactic-stack-chair-platinum-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-25 06:24:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/frameless-safety-glasses-scratch-resistant-clear-lens>: HTTP status code is not handled or not allowed 2026-01-25 06:25:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergodyne-620-knee-sleeve-open-patella-spiral-stays-black-small>: HTTP status code is not handled or not allowed 2026-01-25 06:25:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compact-digital-counting-scale-15-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:25:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergodyne-620-knee-sleeve-open-patella-spiral-stays-black-medium>: HTTP status code is not handled or not allowed 2026-01-25 06:25:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-step-plastic-step-stand-20-w-x-28-1-2-d-x-33-1-2-h-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:25:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tall-kitchen-drawstring-trash-bags-13-gal-0-72-mil-24-x-27-38-gray-400-case>: HTTP status code is not handled or not allowed 2026-01-25 06:25:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7060-bracket-guide-lt-5512330>: HTTP status code is not handled or not allowed 2026-01-25 06:25:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stock-stop-5630131>: HTTP status code is not handled or not allowed 2026-01-25 06:25:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-scrubbing-sponge-yellow-green-20-sponges>: HTTP status code is not handled or not allowed 2026-01-25 06:25:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8215ba-class-2-econo-breakaway-vest-lime-4xl-5xl>: HTTP status code is not handled or not allowed 2026-01-25 06:25:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hd-sms-coverall-elastic-wrists-ankles-zipper-front-single-collar-white-lg-25case>: HTTP status code is not handled or not allowed 2026-01-25 06:25:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/32oz-natural-pp-jar-with-120-400-white-pp-unlined-cap-24pk>: HTTP status code is not handled or not allowed 2026-01-25 06:25:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/java-wicker-stacking-barstool-french-taupe-pack-of-8>: HTTP status code is not handled or not allowed 2026-01-25 06:25:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reversible-desk-return-36wx24d-driftwood>: HTTP status code is not handled or not allowed 2026-01-25 06:25:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bow-front-desk-shell-71wx41dx29-5h-driftwood>: HTTP status code is not handled or not allowed 2026-01-25 06:25:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-adjustable-t-bar-shelf-15-wx48-l>: HTTP status code is not handled or not allowed 2026-01-25 06:25:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 75137 pages (at 99 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:25:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7040-overload-440v-3ph-5712661>: HTTP status code is not handled or not allowed 2026-01-25 06:25:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-cantilever-solid-brute-shelf-24-wx42-l>: HTTP status code is not handled or not allowed 2026-01-25 06:25:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-l-deluxe-save-t-loc-strip-curtain-door-mounting-bracket-kit>: HTTP status code is not handled or not allowed 2026-01-25 06:25:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/500ml-natural-hdpe-wide-mouth-round-bottle-with-53-400-white-pp-cap-180-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:25:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2oz-natural-hdpe-oblong-bottle-with-33-400-white-pp-cap-48-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:25:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bar-storage-rack-add-on>: HTTP status code is not handled or not allowed 2026-01-25 06:25:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-adjustable-heavy-duty-shelf-15-wx72-l>: HTTP status code is not handled or not allowed 2026-01-25 06:25:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jet-jtm-949evs-mill-x-y-and-z-axis-powerfeeds-698158>: HTTP status code is not handled or not allowed 2026-01-25 06:25:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cannes-outdoor-stackable-dining-armchair-terra-cotta-pack-of-16>: HTTP status code is not handled or not allowed 2026-01-25 06:25:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-drawer-workbench-plastic>: HTTP status code is not handled or not allowed 2026-01-25 06:25:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/belt-7m600-6077100>: HTTP status code is not handled or not allowed 2026-01-25 06:25:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-drawer-workbench-shop-top>: HTTP status code is not handled or not allowed 2026-01-25 06:25:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jmd-40g-horse-powerf-geared-head-mill-drill-with-power-downfeed-1-1-2hp-115-230v-1ph>: HTTP status code is not handled or not allowed 2026-01-25 06:25:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jet-jtm-1050evs2-230-mill-x-axis-powerfeed-692191>: HTTP status code is not handled or not allowed 2026-01-25 06:25:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jet-jtm-4vs-1-mill-3-axis-acu-rite-200s-dro-knee-x-axis-powerfeed-698165>: HTTP status code is not handled or not allowed 2026-01-25 06:25:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/53-400-white-metal-cap-with-pulp-aluminum-foil-liner-packed-in-bags-of-12-288pk>: HTTP status code is not handled or not allowed 2026-01-25 06:25:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lcd-plasma-cart-beige-1>: HTTP status code is not handled or not allowed 2026-01-25 06:25:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spw-981lp-entry-level-pallet-stretch-wrapper-with-ramp-low-profile-3000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:25:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-adjustable-t-bar-shelf-20-wx72-l>: HTTP status code is not handled or not allowed 2026-01-25 06:25:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-adjustable-solid-brute-shelf-24-wx30-l>: HTTP status code is not handled or not allowed 2026-01-25 06:25:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-18w-x-36l-1-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 06:25:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-heavy-duty-4-shelf-rack-20-wx72-hx36-l>: HTTP status code is not handled or not allowed 2026-01-25 06:25:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-t-bar-4-shelf-rack-20-wx72-hx72-l>: HTTP status code is not handled or not allowed 2026-01-25 06:25:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-5w-x-18l-1-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 06:25:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triumph-style-bp7-hss-reduced-shank-drill-bright-38>: HTTP status code is not handled or not allowed 2026-01-25 06:25:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sentry-fire-safe-document-chest>: HTTP status code is not handled or not allowed 2026-01-25 06:25:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-adjustable-heavy-duty-shelf-24-wx54-l>: HTTP status code is not handled or not allowed 2026-01-25 06:25:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-heavy-duty-4-shelf-rack-24-wx72-hx36-l>: HTTP status code is not handled or not allowed 2026-01-25 06:25:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-8ml-fine-tip-ldpe-transfer-pipets-147mm-500-dispenser-box-10-boxes-unit-5000pk>: HTTP status code is not handled or not allowed 2026-01-25 06:25:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-step-steel-orange-tilt-and-roll-ladder-with-perforated-tread-tr-6-p-o>: HTTP status code is not handled or not allowed 2026-01-25 06:25:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/84oz-safety-coated-clear-jug-w-38-439-white-pp-f422-hdpe-foam-cap-6pk>: HTTP status code is not handled or not allowed 2026-01-25 06:25:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/27-5-x-140mm-15-dram-clear-borosilicate-vial-with-24-400-neck-finish-vial-only-72pk>: HTTP status code is not handled or not allowed 2026-01-25 06:25:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triumph-style-t1b-hss-jobbers-drill-black-oxide-364-12>: HTTP status code is not handled or not allowed 2026-01-25 06:25:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/short-sleeve-wicking-birdseye-t-shirt-w-pocket-hi-vis-yellow-5xl>: HTTP status code is not handled or not allowed 2026-01-25 06:25:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-t-bar-wallshelf-20-wx48-l>: HTTP status code is not handled or not allowed 2026-01-25 06:25:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/short-sleeve-wicking-birdseye-t-shirt-w-pocket-hi-vis-yellow-2xl>: HTTP status code is not handled or not allowed 2026-01-25 06:25:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/19-x-65mm-3-dram-vial-w-15-425-green-thermoset-f217-ptfe-lined-cap-cleaned-144pk>: HTTP status code is not handled or not allowed 2026-01-25 06:25:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-t-bar-wallshelf-24-wx60-l>: HTTP status code is not handled or not allowed 2026-01-25 06:25:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/food-safe-vfswdnm3555t-e-ip69-2hp-3480rpm-3ph-60hz-145tc-3546m-tenv-f1>: HTTP status code is not handled or not allowed 2026-01-25 06:25:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8295-class-2-polo-shirt-lime-2-x-large>: HTTP status code is not handled or not allowed 2026-01-25 06:25:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/long-sleevet-shirt-segmented-tape-hi-vis-yellow-medium>: HTTP status code is not handled or not allowed 2026-01-25 06:25:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-foot-5784692>: HTTP status code is not handled or not allowed 2026-01-25 06:25:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8925-class-e-thermal-pants-lime-small>: HTTP status code is not handled or not allowed 2026-01-25 06:25:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crank-hndl-5625111>: HTTP status code is not handled or not allowed 2026-01-25 06:25:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/angle-scale-5625171>: HTTP status code is not handled or not allowed 2026-01-25 06:25:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-5-swivel-casters-global-241632-241664-241666-hand-trucks>: HTTP status code is not handled or not allowed 2026-01-25 06:25:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crew-sweatshirt-hi-vis-orange-xl>: HTTP status code is not handled or not allowed 2026-01-25 06:25:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-4010-double-strap-wrist-support-tan-medium-left>: HTTP status code is not handled or not allowed 2026-01-25 06:25:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-8w-x-16l-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 06:25:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sea-safe-marine-horn-24-volt-135-24>: HTTP status code is not handled or not allowed 2026-01-25 06:25:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-18w-x-24l-1-mil-clear-100pack>: HTTP status code is not handled or not allowed 2026-01-25 06:25:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bearing-pin-5710551>: HTTP status code is not handled or not allowed 2026-01-25 06:25:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergodyne-420-wrist-wrap-w-thumb-loop-tan-s-m>: HTTP status code is not handled or not allowed 2026-01-25 06:25:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-mobile-round-table-mdf-with-protectedge-chrome-frame-banister-oak>: HTTP status code is not handled or not allowed 2026-01-25 06:25:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-x-35mm-0-50-dram-amber-vial-w-8-425-black-phenolic-pulp-vinyl-lined-cap-144pk>: HTTP status code is not handled or not allowed 2026-01-25 06:25:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xd-smooth-aluminum-underbody-with-barn-door-18x18x60-1705415>: HTTP status code is not handled or not allowed 2026-01-25 06:25:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8925-class-e-thermal-pants-orange-2-x-large>: HTTP status code is not handled or not allowed 2026-01-25 06:25:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-24w-x-32l-1-mil-clear-500pack>: HTTP status code is not handled or not allowed 2026-01-25 06:25:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-dummy-enclosure-only-30-series>: HTTP status code is not handled or not allowed 2026-01-25 06:25:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inventory-control-cone-black>: HTTP status code is not handled or not allowed 2026-01-25 06:25:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-filler-sleeve-30-series>: HTTP status code is not handled or not allowed 2026-01-25 06:25:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-hook-for-slatwall-satin-nickel>: HTTP status code is not handled or not allowed 2026-01-25 06:25:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sight-glass-9225-217>: HTTP status code is not handled or not allowed 2026-01-25 06:25:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-shelf-bracket-for-slatwall-satin-nickel>: HTTP status code is not handled or not allowed 2026-01-25 06:25:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2oz-natural-pp-hinged-vial-600pk>: HTTP status code is not handled or not allowed 2026-01-25 06:25:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spiral-costumer-rack-satin-nickel>: HTTP status code is not handled or not allowed 2026-01-25 06:25:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-14w-x-16l-1-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 06:25:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-economy-tab-dividers-8-tab-letter-white-24-sets-box>: HTTP status code is not handled or not allowed 2026-01-25 06:25:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-economy-scissors-8-length-straight-handle-stainless-steel-black>: HTTP status code is not handled or not allowed 2026-01-25 06:25:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-clear-front-report-cover-tang-fasteners-letter-size-green-25-box>: HTTP status code is not handled or not allowed 2026-01-25 06:25:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-chock-your-wheels-safety-warning-sign-14-x-10>: HTTP status code is not handled or not allowed 2026-01-25 06:25:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insul-sheet-roll-s2s-3-848100>: HTTP status code is not handled or not allowed 2026-01-25 06:25:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conntek-rul630pr-100-nema-l6-30-locking-all-weather-cord-set-100>: HTTP status code is not handled or not allowed 2026-01-25 06:25:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-message-stamp-completed-pre-inked-re-inkable-blue-ink>: HTTP status code is not handled or not allowed 2026-01-25 06:25:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-4w-x-52l-6-mil-clear-500pack>: HTTP status code is not handled or not allowed 2026-01-25 06:25:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-bariatric-bedside-commode-802-1203-0300>: HTTP status code is not handled or not allowed 2026-01-25 06:25:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quick-adjust-wire-shelf-truck-with-dolly-base-36x24x61-1600-pound-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:25:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-5-in-1-parka-hi-viz-yellow-4xl>: HTTP status code is not handled or not allowed 2026-01-25 06:25:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-dia-3-8-shank-2-oal-120-6-flute-single-end-countersink-cobalt-tin>: HTTP status code is not handled or not allowed 2026-01-25 06:25:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24x36-bib-typealuminized-rayon-ar36b>: HTTP status code is not handled or not allowed 2026-01-25 06:25:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/occumitts-my-way-beige-medium>: HTTP status code is not handled or not allowed 2026-01-25 06:26:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelf-bin-nestable-8-3-8-w-x-11-5-8-d-x-6-h-clear>: HTTP status code is not handled or not allowed 2026-01-25 06:26:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drive-shaft-washer-9180-102>: HTTP status code is not handled or not allowed 2026-01-25 06:26:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/food-safe-cfswdm41900t-ip69-15hp-1180rpm-3ph-60hz-284tc-4072m-tefc-f>: HTTP status code is not handled or not allowed 2026-01-25 06:26:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/small-coated-string-knitkevlar-plus-pvc-do-9366s>: HTTP status code is not handled or not allowed 2026-01-25 06:26:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hard-hat-shade-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:26:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brake-8-caster-top-locking-brake-with-side-lever>: HTTP status code is not handled or not allowed 2026-01-25 06:26:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vari-angle-worktable-aluminum-31-w-x-28-d-x-38-h>: HTTP status code is not handled or not allowed 2026-01-25 06:26:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-suction-disc-rubber-5-w-x-2-1-2-d-x-5-h>: HTTP status code is not handled or not allowed 2026-01-25 06:26:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caution-do-not-enter-7x10-rigid-plastic>: HTTP status code is not handled or not allowed 2026-01-25 06:26:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interoffice-press-seal-envelope-10-x-13-brown-100box>: HTTP status code is not handled or not allowed 2026-01-25 06:26:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plate-9180-308>: HTTP status code is not handled or not allowed 2026-01-25 06:26:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/belleville-sprng-w25mm-hbs-1321w-1321w-174>: HTTP status code is not handled or not allowed 2026-01-25 06:26:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caution-hearing-protection-required-7x10-pressure-sensitive-vinyl>: HTTP status code is not handled or not allowed 2026-01-25 06:26:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/distribution-container-with-hinged-lid-22x15-1-4x17-1-4-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:26:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/516705-med-disposable-nat-latex-100-glvs-bx-69-210-m>: HTTP status code is not handled or not allowed 2026-01-25 06:26:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 75233 pages (at 96 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:26:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silver-shield-unique-flexible-film-glove-len14-5-ssg-9>: HTTP status code is not handled or not allowed 2026-01-25 06:26:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pulling-device-to-be-used-with-the-15-5-separator>: HTTP status code is not handled or not allowed 2026-01-25 06:26:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ninja-flex-15-guage-red100-nylon-shell-gray-la-n9680m>: HTTP status code is not handled or not allowed 2026-01-25 06:26:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-12w-x-36l-3-mil-clear-250pack>: HTTP status code is not handled or not allowed 2026-01-25 06:26:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/latch-spring-1ss-3c-01703>: HTTP status code is not handled or not allowed 2026-01-25 06:26:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3kas6ylw-laminated-steel-safety-padlock-1-916w-x-34h-shackle-yellow-6set>: HTTP status code is not handled or not allowed 2026-01-25 06:26:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-selfstick-notes-3-x-3-4-pastel-colors-12-100sheet-padspack>: HTTP status code is not handled or not allowed 2026-01-25 06:26:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-l-x-36-w-production-bench-esd-square-edge-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:26:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/117075-9-sol-vex-unsupported-nitrile-line-37-145-9>: HTTP status code is not handled or not allowed 2026-01-25 06:26:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/id-label-ap5-19>: HTTP status code is not handled or not allowed 2026-01-25 06:26:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/narrow-2-arm-revers-quick-adj-set-75-ton-capacity-7-7-8-200-mm-open-11-7-8-300-mm-reach>: HTTP status code is not handled or not allowed 2026-01-25 06:26:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kraft-coin-envelope-5-light-brown-500box>: HTTP status code is not handled or not allowed 2026-01-25 06:26:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-l-x-36-w-production-bench-esd-square-edge-blue-1>: HTTP status code is not handled or not allowed 2026-01-25 06:26:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/narrow-3-arm-revers-quick-adj-55-ton-capacity-3-1-2-90-mm-open-3-7-8-100-mm-reach>: HTTP status code is not handled or not allowed 2026-01-25 06:26:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pivot-3130-5630491r>: HTTP status code is not handled or not allowed 2026-01-25 06:26:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classic-mid-length-fleece-gray-and-black-6-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:26:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classic-full-face-tube-liner-red-6-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:26:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/woodstock-decorative-queen-size-headboard-metal-black>: HTTP status code is not handled or not allowed 2026-01-25 06:26:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-size-stacking-bin-12-3-8-x-20-x-8-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:26:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-6w-x-24l-4-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 06:26:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-20w-x-42l-2-mil-clear-250pack>: HTTP status code is not handled or not allowed 2026-01-25 06:26:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/banquet-stacking-chair-fabric-2-1-2-seat-cushion-black>: HTTP status code is not handled or not allowed 2026-01-25 06:26:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/goin-green-plugstation-with-mounting-bracket-6646>: HTTP status code is not handled or not allowed 2026-01-25 06:26:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-jaw-self-centering-puller-55-ton-capacity-3-4-to-4-3-4-open-4-3-4-reach>: HTTP status code is not handled or not allowed 2026-01-25 06:26:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/goin-green-plugstation-with-mounting-bracket-6647>: HTTP status code is not handled or not allowed 2026-01-25 06:26:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/assemblys-51-1062-sou1177448>: HTTP status code is not handled or not allowed 2026-01-25 06:26:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paramount-16-person-ocker-12-x-18-x-12-assembled-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:26:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s1106blk-aluminum-safety-padlock-1-12w-x-1-12h-shackle-black>: HTTP status code is not handled or not allowed 2026-01-25 06:26:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/med-lg-n95-particulaterespirator-2300n95>: HTTP status code is not handled or not allowed 2026-01-25 06:26:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-exterior-36h-platform-2-rails-spiral-stair-kit-42dia-11-1-4h-vinyl-handrail>: HTTP status code is not handled or not allowed 2026-01-25 06:26:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-premium-locker-six-tier-12x15x12-18-door-assembled-parchment>: HTTP status code is not handled or not allowed 2026-01-25 06:26:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metaltech-aluminum-scaffold-platform-wood-deck-7-x-19>: HTTP status code is not handled or not allowed 2026-01-25 06:26:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pathway-hd-code-compliant-wheelchair-ramp-kit-16-x-48-w-guards-3-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:26:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/capacitor-holder-3t5t1ph-3et-1c-509>: HTTP status code is not handled or not allowed 2026-01-25 06:26:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metaltech-scaffold-base-plate>: HTTP status code is not handled or not allowed 2026-01-25 06:26:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-premium-locker-single-tier-12x12x72-3-door-ready-assemble-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:26:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-10w-x-22l-3-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 06:26:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metaltech-guard-rail-system-for-steel-maxi-scaffold-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:26:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-11w-x-20l-3-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 06:26:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-premium-locker-single-tier-15x15x72-1-door-ready-assemble-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:26:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/valves-56-1167-cle07176>: HTTP status code is not handled or not allowed 2026-01-25 06:26:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/atlantis-series-point-of-use-water-cooler-two-piece-hot-tank-hot-n-cold-8482-white>: HTTP status code is not handled or not allowed 2026-01-25 06:26:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simpson-61014-powershot-3500psi-196cc-2-5gpm-gas-pressure-washer-w-honda-gc200-engine-aaa-pump>: HTTP status code is not handled or not allowed 2026-01-25 06:26:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lamps-38-1003-blo4342>: HTTP status code is not handled or not allowed 2026-01-25 06:26:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-premium-locker-double-tier-12x12x36-2-door-parchment-2>: HTTP status code is not handled or not allowed 2026-01-25 06:26:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-manyard-single-legharhess-6-ft-216m-6ftgn>: HTTP status code is not handled or not allowed 2026-01-25 06:26:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-premium-locker-triple-tier-12x12x24-3-door-assembled-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:26:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nut-cs275-56>: HTTP status code is not handled or not allowed 2026-01-25 06:26:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-premium-locker-single-tier-12x18x72-3-door-ready-assemble-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:26:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11-l-x-4-3-4-w-pro-carbon-steel-narrow-london-masonry-trowel-wood-handle>: HTTP status code is not handled or not allowed 2026-01-25 06:26:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fiberglass-stacking-wash-box-24-x-14-3-8-x-7-3-4-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:26:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buckhorn-folding-bulk-shipping-container-48x45x34-2500-lbs-black>: HTTP status code is not handled or not allowed 2026-01-25 06:26:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylon-spacer-a-ck350-407>: HTTP status code is not handled or not allowed 2026-01-25 06:26:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thermostats-46-1485-hostb0031>: HTTP status code is not handled or not allowed 2026-01-25 06:26:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s1-safety-swivel-quick-coupling-3-8-high-flow-interchange-1-4-fnpt-connection>: HTTP status code is not handled or not allowed 2026-01-25 06:26:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fixed-plate-jdp-20evs-20evs-t67-2>: HTTP status code is not handled or not allowed 2026-01-25 06:26:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-valve-spring-jbj12-4>: HTTP status code is not handled or not allowed 2026-01-25 06:26:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jet-jwl-1640evs-16in-x-40in-wood-lathe>: HTTP status code is not handled or not allowed 2026-01-25 06:26:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-tempered-convex-jointer-3-4-x-5-8>: HTTP status code is not handled or not allowed 2026-01-25 06:26:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-unit-5-tier-poly-z-brite-24w-x-72l-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 06:26:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/o-ring-104x24-jbj8-12>: HTTP status code is not handled or not allowed 2026-01-25 06:26:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blowers-68-1105-imerphm20-2103>: HTTP status code is not handled or not allowed 2026-01-25 06:26:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-static-shielding-bags-6w-x-10l-28-mil-transparent-100pack>: HTTP status code is not handled or not allowed 2026-01-25 06:26:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-static-shielding-bags-6w-x-24l-28-mil-transparent-100pack>: HTTP status code is not handled or not allowed 2026-01-25 06:26:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/engineers-ball-peen-hammers-h-4fg>: HTTP status code is not handled or not allowed 2026-01-25 06:26:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bowls-26-3841-hob275690>: HTTP status code is not handled or not allowed 2026-01-25 06:26:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motors-68-1232-hospa0613>: HTTP status code is not handled or not allowed 2026-01-25 06:26:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paramount-clear-view-storage-cabinet-assembled-36x18x78-tan-3>: HTTP status code is not handled or not allowed 2026-01-25 06:26:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-unit-5-tier-poly-z-brite-21w-x-24l-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 06:26:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-step-steel-16in-w-tilt-roll-ladder-grip-strut-tread-kdtf106162>: HTTP status code is not handled or not allowed 2026-01-25 06:26:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-unit-5-tier-poly-z-brite-18w-x-36l-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 06:26:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gusseted-poly-bags-8w-x-4d-x-20h-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 06:26:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-sleeve-cos18-241>: HTTP status code is not handled or not allowed 2026-01-25 06:26:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gear-box-pulley-hbs1018w-97>: HTTP status code is not handled or not allowed 2026-01-25 06:26:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s1-composite-blow-gun-with-osha-nozzle-equipped-with-integrated-1-4-plug-iso-6150-profile>: HTTP status code is not handled or not allowed 2026-01-25 06:26:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/28-242-nylon-handle-flexible-putty-knife-2-inch-wide-blade>: HTTP status code is not handled or not allowed 2026-01-25 06:26:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s1-safety-quick-coupling-1-2-body-with-industrial-profile-with-3-4-fnpt-connection>: HTTP status code is not handled or not allowed 2026-01-25 06:26:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/27-step-steel-caged-walk-through-fixed-access-ladder-safety-yellow-wlfc1227-y>: HTTP status code is not handled or not allowed 2026-01-25 06:26:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-unit-5-tier-nexelon-18w-x-60l-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 06:26:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-unit-5-tier-poly-z-brite-18w-x-30l-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 06:26:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elements-34-1642-gro008852>: HTTP status code is not handled or not allowed 2026-01-25 06:26:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/back-plate-l-taper-for-8-inch-set-tru-bison-chucks-l0>: HTTP status code is not handled or not allowed 2026-01-25 06:27:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-8w-x-20l-6-mil-clear-500pack>: HTTP status code is not handled or not allowed 2026-01-25 06:27:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/back-plate-d-taper-for-set-tru-bison-chucks-12-inch-d1-8>: HTTP status code is not handled or not allowed 2026-01-25 06:27:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bearing-spacer-hbs814gh-800-42>: HTTP status code is not handled or not allowed 2026-01-25 06:27:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/28-593-window-scraper-with-1-blade>: HTTP status code is not handled or not allowed 2026-01-25 06:27:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-step-16in-w-10in-d-top-step-steel-rolling-ladder-grip-strut-tread-no-handrail-kdsr102162>: HTTP status code is not handled or not allowed 2026-01-25 06:27:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s1-safety-quick-coupling-3-8-automotive-profile-with-3-8-mnpt-connection>: HTTP status code is not handled or not allowed 2026-01-25 06:27:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-step-mobile-work-platform-40in-w-x-71in-l-36in-handrails-safety-yellow-wlwp153636sl-y>: HTTP status code is not handled or not allowed 2026-01-25 06:27:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-unit-5-tier-poly-z-brite-24w-x-24l-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 06:27:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/al55cwd-powerglide8482-defeater-handle-kit-med-handle-cw>: HTTP status code is not handled or not allowed 2026-01-25 06:27:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hard-master-jaws-for-bison-scroll-chuck-12-inch-3-jaw-3-piece-set>: HTTP status code is not handled or not allowed 2026-01-25 06:27:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gusseted-poly-bags-14w-x-14d-x-26h-3-mil-clear-250pack>: HTTP status code is not handled or not allowed 2026-01-25 06:27:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-retaining-ring-pliers-364>: HTTP status code is not handled or not allowed 2026-01-25 06:27:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cspb2420-concept8482-swing-out-panel-fits-2400x2000>: HTTP status code is not handled or not allowed 2026-01-25 06:27:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/868-00-knee-pad-gelite-hard-shell-textured-cover-gel-insert-dual-straps-button-closure>: HTTP status code is not handled or not allowed 2026-01-25 06:27:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-unit-5-tier-poly-z-brite-21w-x-60l-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 06:27:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 75330 pages (at 97 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:27:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/afk2410-floor-stand-kit-qty-2-2400x1006-steelgray>: HTTP status code is not handled or not allowed 2026-01-25 06:27:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-200mm-ip67-caliper-no-spc-output>: HTTP status code is not handled or not allowed 2026-01-25 06:27:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/atppy61-touch-up-paint-ansi-61-polyprim-12-oz-spray-can>: HTTP status code is not handled or not allowed 2026-01-25 06:27:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-inch-ready-set-staple-tool-rsc187-4>: HTTP status code is not handled or not allowed 2026-01-25 06:27:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-x12-wire-shelves-with-brackets-package-of-3>: HTTP status code is not handled or not allowed 2026-01-25 06:27:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-unit-5-tier-chrome-21w-x-30l-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 06:27:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-x-18-zipper-lock-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 06:27:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blue-easy-access-18-bushel-plastic-box-truck-with-attached-cargo-netting>: HTTP status code is not handled or not allowed 2026-01-25 06:27:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-add-on-unit-5-tier-nexelon-21w-x-54l-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 06:27:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gusseted-poly-bags-8w-x-4d-x-18h-3-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 06:27:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-inch-ready-set-staple-belt-lacing-galvanized-rs62j24>: HTTP status code is not handled or not allowed 2026-01-25 06:27:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-od-nylon-tubing-blue-color-100-roll-160-500-psi>: HTTP status code is not handled or not allowed 2026-01-25 06:27:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0-1-x-00005-0-001mm-ip65-digimatic-outside-micrometer-no-spc-output-ratchet-thimble>: HTTP status code is not handled or not allowed 2026-01-25 06:27:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-inch-ready-set-staple-belt-lacing-stainless-rs125sj36>: HTTP status code is not handled or not allowed 2026-01-25 06:27:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/middle-support-white-j-f225-127>: HTTP status code is not handled or not allowed 2026-01-25 06:27:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-edge-trimmer-jeb-150>: HTTP status code is not handled or not allowed 2026-01-25 06:27:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cwh5557-concept8482-hinged-window-kit-fits-550x570mm>: HTTP status code is not handled or not allowed 2026-01-25 06:27:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/three-phase-compact-starter-w-start-stop-reset-12-5-18-amps-230-vac>: HTTP status code is not handled or not allowed 2026-01-25 06:27:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-mm-od-nylon-tubing-natural-color-100-roll-160-500-psi>: HTTP status code is not handled or not allowed 2026-01-25 06:27:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spacer-zx-lathes-text-tl06121>: HTTP status code is not handled or not allowed 2026-01-25 06:27:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-ventilated-locker-six-tier-12x12x12-18-door-assembled-1>: HTTP status code is not handled or not allowed 2026-01-25 06:27:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-step-aluminum-rolling-ladder-24in-w-grip-strut-14in-d-top-step-no-handrails-wlar004245>: HTTP status code is not handled or not allowed 2026-01-25 06:27:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polyethylene-bags-20w-x-20l-6-mil-clear-250pack>: HTTP status code is not handled or not allowed 2026-01-25 06:27:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-piece-satin-combination-wrench-set-12-point>: HTTP status code is not handled or not allowed 2026-01-25 06:27:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proto-j1200-90asd-31-piece-satin-combination-asd-wrench-set-12-point>: HTTP status code is not handled or not allowed 2026-01-25 06:27:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oilers-72332>: HTTP status code is not handled or not allowed 2026-01-25 06:27:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ridgidmodel-no-418-hand-operated-oiler-whose-fittings54-hose>: HTTP status code is not handled or not allowed 2026-01-25 06:27:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-nexelon-24w-x-48l-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 06:27:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/propane-gas-infrared-heater-094570lps-straight-configuration-70-l-200000-btu>: HTTP status code is not handled or not allowed 2026-01-25 06:27:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-phase-disconnect-manual-motor-controller>: HTTP status code is not handled or not allowed 2026-01-25 06:27:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anti-static-flat-poly-bags-12w-x-24l-6-mil-pink-250pack>: HTTP status code is not handled or not allowed 2026-01-25 06:27:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shelving-36x24x72-with-5-shelves-wood-deck>: HTTP status code is not handled or not allowed 2026-01-25 06:27:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vivi-dressing-aid-pocketdresser>: HTTP status code is not handled or not allowed 2026-01-25 06:27:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cllr-column-safety-3096008>: HTTP status code is not handled or not allowed 2026-01-25 06:27:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hanging-chain-kit-1800css10-for-straight-configuration-infrared-heaters-10-l>: HTTP status code is not handled or not allowed 2026-01-25 06:27:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tube-cutter-wheels-74720>: HTTP status code is not handled or not allowed 2026-01-25 06:27:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anti-static-flat-poly-bags-30w-x-36l-4-mil-pink-100pack>: HTTP status code is not handled or not allowed 2026-01-25 06:27:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-shelving-48x12x72-with-4-shelves-no-deck>: HTTP status code is not handled or not allowed 2026-01-25 06:27:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-shelving-48x18x72-with-5-shelves-no-deck>: HTTP status code is not handled or not allowed 2026-01-25 06:27:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f22t136gvp-straight-section-type-1-250x250x3600-galvgray>: HTTP status code is not handled or not allowed 2026-01-25 06:27:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/propane-gas-infrared-heater-094540lpu-u-configuration-40-l-200000-btu>: HTTP status code is not handled or not allowed 2026-01-25 06:27:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-gas-infrared-heater-091520ngu-u-configuration-20-l-75000-btu>: HTTP status code is not handled or not allowed 2026-01-25 06:27:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/planters-hoe-h-110fg>: HTTP status code is not handled or not allowed 2026-01-25 06:27:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-chrome-18w-x-48l-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 06:27:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-shelving-36x18x72-with-5-shelves-wire-deck>: HTTP status code is not handled or not allowed 2026-01-25 06:27:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/angle-flange-5-inch-diameter-galvanized>: HTTP status code is not handled or not allowed 2026-01-25 06:27:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/f88t48hc-wire-trough-hinged-cover-800x800x4800-steelgray>: HTTP status code is not handled or not allowed 2026-01-25 06:27:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-chrome-21w-x-60l-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 06:27:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-electric-heater-10kw-240v>: HTTP status code is not handled or not allowed 2026-01-25 06:27:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/repeat-cycle-timer-0-60-sec-spdt-24-vac-vdc>: HTTP status code is not handled or not allowed 2026-01-25 06:27:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-electric-heater-15kw-240v-1>: HTTP status code is not handled or not allowed 2026-01-25 06:27:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/saddle-hanger-10-inch-diameter-galvanized>: HTTP status code is not handled or not allowed 2026-01-25 06:27:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aspb100125nm-hol-sealers8482-non-metallic-hole-seal-191-dia>: HTTP status code is not handled or not allowed 2026-01-25 06:27:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-covers-72w-x-42d-x-54h-3-mil-clear-50roll>: HTTP status code is not handled or not allowed 2026-01-25 06:27:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microporous-coverall-elastic-wrists-ankles-hood-white-x-large-25-case>: HTTP status code is not handled or not allowed 2026-01-25 06:27:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clamp-together-45-deg-elbow-15-clr-14-inch-diameter-glv-18-gage>: HTTP status code is not handled or not allowed 2026-01-25 06:27:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clamp-together-automatic-blast-gate-14-inch-diameter-galvanized>: HTTP status code is not handled or not allowed 2026-01-25 06:27:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esd-nesting-box-top-overall-dimensions-6-1-8l-x-4-7-8w-x-2-1-8h>: HTTP status code is not handled or not allowed 2026-01-25 06:27:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clamp-together-hose-adapter-16-inch-diameter-galvanized-20-gage>: HTTP status code is not handled or not allowed 2026-01-25 06:27:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-12w-x-15l-2-mil-green-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 06:27:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-bulk-box-20-bushel-truck-steel-chassis-base-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:27:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/panel-sheet-mover-truck-with-solid-steel-deck-2000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:27:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-16w-x-16l-8-mil-clear-250pack>: HTTP status code is not handled or not allowed 2026-01-25 06:27:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/screw-hd-raising-3691007>: HTTP status code is not handled or not allowed 2026-01-25 06:27:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/goliath-eyewear-clear-lens-with-black-frame>: HTTP status code is not handled or not allowed 2026-01-25 06:27:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-latch-assembly-2t-smh-016d>: HTTP status code is not handled or not allowed 2026-01-25 06:27:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-rack-add-on-w-steel-decking-84-w-x-36-d-x-96-h>: HTTP status code is not handled or not allowed 2026-01-25 06:27:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bulk-rack-starter-w-steel-decking-84-w-x-24-d-x-96-h>: HTTP status code is not handled or not allowed 2026-01-25 06:27:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-bulk-box-20-bushel-truck-steel-chassis-base-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:27:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/182-sjtw-6ft-indoor--outdoor-floodlight-holder-kit-gg-36001-green>: HTTP status code is not handled or not allowed 2026-01-25 06:27:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stop-pin-cap-zx-04507>: HTTP status code is not handled or not allowed 2026-01-25 06:27:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/venture-ii-eyewear-sun-block-bronze-lens-with-black-frame>: HTTP status code is not handled or not allowed 2026-01-25 06:27:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-inch-foot-rest-with-pan>: HTTP status code is not handled or not allowed 2026-01-25 06:27:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unprinted-open-end-static-shielding-bags-12w-x-18l-28-mil-clear-100pack>: HTTP status code is not handled or not allowed 2026-01-25 06:27:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-oz-cartridge-super-lube-silicone-high-dielectric-vacuum-grease>: HTTP status code is not handled or not allowed 2026-01-25 06:27:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cast-iron-grip-handle-weights-nist-class-f-with-traceable-certificate-25-kgs>: HTTP status code is not handled or not allowed 2026-01-25 06:27:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sign-disc-zx-lathes-zx-05301>: HTTP status code is not handled or not allowed 2026-01-25 06:27:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-qt-bottle-super-lube-synthetic-gear-oil-iso-220>: HTTP status code is not handled or not allowed 2026-01-25 06:27:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/46232-m10-x-1-0mm-metric-grease-fitting-90-5-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:27:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-cylindrical-weights-nist-class-f-4-kgs>: HTTP status code is not handled or not allowed 2026-01-25 06:27:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-x-24-x-74-chrome-wire-shelving-with-24-24-d-bins-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:27:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/el5002-lrg-tennis-elbow-support-ambidextrous-neoprene-wrap-pressure-pad-for-compression>: HTTP status code is not handled or not allowed 2026-01-25 06:27:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/import-precision-ground-dowel-pin-116-x-78-100pk>: HTTP status code is not handled or not allowed 2026-01-25 06:27:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-x-24-x-74-chrome-wire-shelving-with-20-24-d-bins-red>: HTTP status code is not handled or not allowed 2026-01-25 06:28:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silver-hard-hat-adaptor-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 06:28:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/44167-plastic-lever-action-barrel-pump-with-2-bung-adapter-and-extra-tri-sure-adapter>: HTTP status code is not handled or not allowed 2026-01-25 06:28:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thank-you-flat-handle-grocery-bags-16-bl-12w-x-7d-x-17h-kraft-300pack>: HTTP status code is not handled or not allowed 2026-01-25 06:28:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enriched-lotion-free-n-clean-soap-800ml-fg402363>: HTTP status code is not handled or not allowed 2026-01-25 06:28:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ball-bearing-ref-28--1230-5232751>: HTTP status code is not handled or not allowed 2026-01-25 06:28:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blue-velour-rope-59in-with-ends-for-portable-gold-post>: HTTP status code is not handled or not allowed 2026-01-25 06:28:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/55-ton-hydraulic-jack-deluxe-13081>: HTTP status code is not handled or not allowed 2026-01-25 06:28:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gusseted-poly-bags-24w-x-24d-x-48h-2-mil-clear-100pack>: HTTP status code is not handled or not allowed 2026-01-25 06:28:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-cabinets-aah-48rbmag2-b-storage-cabinet-assembled-48x18x78-black>: HTTP status code is not handled or not allowed 2026-01-25 06:28:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-shelving-60x24x39>: HTTP status code is not handled or not allowed 2026-01-25 06:28:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/end-piece-assy-new-style-500-2900960>: HTTP status code is not handled or not allowed 2026-01-25 06:28:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-aluminum-half-pallet-skid-bottom-48x24>: HTTP status code is not handled or not allowed 2026-01-25 06:28:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-cabinets-bca-367213-p-bookcase-assembled-36x13x72-putty>: HTTP status code is not handled or not allowed 2026-01-25 06:28:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 75428 pages (at 98 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:28:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-chain-1-link-25-l-hdpe-magenta>: HTTP status code is not handled or not allowed 2026-01-25 06:28:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v-belt--a-59-5051521>: HTTP status code is not handled or not allowed 2026-01-25 06:28:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-chain-1-link-25-l-hdpe-safety-pink>: HTTP status code is not handled or not allowed 2026-01-25 06:28:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-hanford-quad-post-with-support-brace-ornate-base-e1-economy-mailbox>: HTTP status code is not handled or not allowed 2026-01-25 06:28:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-outlet-remote-point-of-attachment-box-independent-regulator-hansen-fitting-poa-4r>: HTTP status code is not handled or not allowed 2026-01-25 06:28:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-chain-1-link-25-l-hdpe-red>: HTTP status code is not handled or not allowed 2026-01-25 06:28:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-foot-x-120-foot-blue-tarp>: HTTP status code is not handled or not allowed 2026-01-25 06:28:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/goop-hand-cleaner-with-pumice-4-5-lb-can>: HTTP status code is not handled or not allowed 2026-01-25 06:28:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lewiston-equine-post-only-w-support-bracket-horsehead-finial-decorative-fluted-base-white>: HTTP status code is not handled or not allowed 2026-01-25 06:28:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/molded-urethane-cable-hose-protection-bridges-48-x-24-x-4>: HTTP status code is not handled or not allowed 2026-01-25 06:28:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/molded-rubber-cable-hose-protection-bridges-24x24x4>: HTTP status code is not handled or not allowed 2026-01-25 06:28:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50-x-12-breathing-air-rated-hose-schrader-fittings-h-50-5s>: HTTP status code is not handled or not allowed 2026-01-25 06:28:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bushing-pin-ghdjmd-2450030>: HTTP status code is not handled or not allowed 2026-01-25 06:28:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-charcoal-3rd-stage-d-filter-element-bb100-150-series-bb100-d>: HTTP status code is not handled or not allowed 2026-01-25 06:28:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-drawer-vertical-file-huntington-office-furniture-medium-oak>: HTTP status code is not handled or not allowed 2026-01-25 06:28:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ipack-starter-kit-for-5-waterless-urinals>: HTTP status code is not handled or not allowed 2026-01-25 06:28:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-chain-3-4-link-25-l-hdpe-traffic-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:28:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/collet-nut-jsm-512512e-51225>: HTTP status code is not handled or not allowed 2026-01-25 06:28:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chain-breaker-for-3-4-to-11-4-pitch-chain>: HTTP status code is not handled or not allowed 2026-01-25 06:28:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dot-traffic-cones-28-h-14-x-14-base-7-lbs-pvc-traffic-orange-6-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:28:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/444-double-coated-film-tape-1-x-36-yds-4-mil-clear-36pack>: HTTP status code is not handled or not allowed 2026-01-25 06:28:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traffic-cones-28-h-14-x-14-base-7-lbs-pvc-sky-blue-6-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:28:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-medium-plastic-individual-bin-no-logo-9-3-8w-x-5-7-8d-x-4-7-8h-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:28:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1550-strapping-tape-34-x-60-yds-59-mil-clear-12pack>: HTTP status code is not handled or not allowed 2026-01-25 06:28:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/offices-to-go-return-desk-30wx24dx29-1-2h-walnut>: HTTP status code is not handled or not allowed 2026-01-25 06:28:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acroprint-model-150ar3-heavy-duty-time-recorder-prints-day-hour-minute>: HTTP status code is not handled or not allowed 2026-01-25 06:28:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acroprint-es900-electronic-time-clock-3-in-1-document-stamp>: HTTP status code is not handled or not allowed 2026-01-25 06:28:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uattend-rfid-ready-time-clock>: HTTP status code is not handled or not allowed 2026-01-25 06:28:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-precision-ansi-nickel-plated-roller-chain-50-1np-5-8-pitch-offset-link>: HTTP status code is not handled or not allowed 2026-01-25 06:28:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-precision-ansi-roller-chain-240-1r-3-pitch-10ft-box>: HTTP status code is not handled or not allowed 2026-01-25 06:28:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4008-double-sided-foam-tape-34-x-36-yds-125-mil-natural-12pack>: HTTP status code is not handled or not allowed 2026-01-25 06:28:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-foot-x-18-foot-blue-tarp>: HTTP status code is not handled or not allowed 2026-01-25 06:28:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flap-disc-769f-05909-t27-7-x-7-8-in-40-yf-weight-5-per-case>: HTTP status code is not handled or not allowed 2026-01-25 06:28:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flap-disc-769f-05911-t27-7-x-7-8-in-80-yf-weight-5-per-case>: HTTP status code is not handled or not allowed 2026-01-25 06:28:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sj4570-low-profile-fasteners-58w-x-150l-clear>: HTTP status code is not handled or not allowed 2026-01-25 06:28:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-precision-ansi-stainless-steel-roller-chain-25-1ss-1-4-pitch-connecting-link>: HTTP status code is not handled or not allowed 2026-01-25 06:28:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-precision-ansi-stainless-steel-roller-chain-41-1ss-1-2-pitch-10ft-box>: HTTP status code is not handled or not allowed 2026-01-25 06:28:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-foam-tape-1-x-36-yds-625-mil-black-12pack>: HTTP status code is not handled or not allowed 2026-01-25 06:28:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-foam-tape-34-x-36-yds-125-mil-white-16pack>: HTTP status code is not handled or not allowed 2026-01-25 06:28:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/057-476-s-p-bl-cat-5e-stp-1000-4-pair-bulk-blue-plenum-jacket-awg24-bare-copper>: HTTP status code is not handled or not allowed 2026-01-25 06:28:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/012-021-cat-6-rj45-shielded-modular-plugs-8p-8c-50-micro-inches-gold-plated>: HTTP status code is not handled or not allowed 2026-01-25 06:28:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cllr-assy-600-c2-c3-2904500>: HTTP status code is not handled or not allowed 2026-01-25 06:28:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-precision-ansi-stainless-steel-roller-chain-35-1ss-3-8-pitch-100ft-reel>: HTTP status code is not handled or not allowed 2026-01-25 06:28:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-precision-iso-metric-roller-chain-12b-1-3-4-pitch-50ft-reel>: HTTP status code is not handled or not allowed 2026-01-25 06:28:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/complete-mobile-electronic-packaging-workbench-plastic-square-edge-72-x-30>: HTTP status code is not handled or not allowed 2026-01-25 06:28:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-precision-iso-metric-roller-chain-05b-1-8mm-pitch-connecting-link>: HTTP status code is not handled or not allowed 2026-01-25 06:28:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/complete-mobile-electronic-packaging-workbench-plastic-safety-edge-72-x-30>: HTTP status code is not handled or not allowed 2026-01-25 06:28:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-suffocation-warning-poly-bags-10w-x-15l-2-mil-clear-100pack>: HTTP status code is not handled or not allowed 2026-01-25 06:28:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-foot-x-20-foot-camouflage-green-tarp>: HTTP status code is not handled or not allowed 2026-01-25 06:28:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-foot-x-24-foot-blue-tarp>: HTTP status code is not handled or not allowed 2026-01-25 06:28:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/airspade-ht108-2000-trench-rescue-kit>: HTTP status code is not handled or not allowed 2026-01-25 06:28:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/23-7518b-9l-cut-resistant-gloves-nitrile-micro-foam-coating-a3-c-size-9>: HTTP status code is not handled or not allowed 2026-01-25 06:28:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chain-breaker-for-3-8-to-3-4-pitch-chain>: HTTP status code is not handled or not allowed 2026-01-25 06:28:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-stacking-bin-8-1-4-x-13-5-8-x-6-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:28:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22-7913b-8m-cut-resistant-gloves-smooth-nitrile-coating-a2-b-size-8>: HTTP status code is not handled or not allowed 2026-01-25 06:28:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-precision-ansi-double-roller-chain-100-2r-1-1-4-pitch-connecting-link>: HTTP status code is not handled or not allowed 2026-01-25 06:28:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lock-handle-m10x20-text-jvm836-b86>: HTTP status code is not handled or not allowed 2026-01-25 06:28:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/airspade-ht156-105-scfm-nozzle>: HTTP status code is not handled or not allowed 2026-01-25 06:28:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tritan-precision-ansi-roller-chain-60-1r-3-4-pitch-connecting-link>: HTTP status code is not handled or not allowed 2026-01-25 06:28:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-plastic-garbage-can-with-lid-dolly-32-gal-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:28:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-foot-x-30-foot-blue-tarp>: HTTP status code is not handled or not allowed 2026-01-25 06:28:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-fan-motor-kit-for-761217-761218>: HTTP status code is not handled or not allowed 2026-01-25 06:28:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-piece-chocolate-faux-rattan-plastic-chair-set-w-matching-side-table>: HTTP status code is not handled or not allowed 2026-01-25 06:28:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-5w-x-16l-4-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 06:28:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-5w-x-24l-4-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 06:28:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/se-fm-floor-mounted-walk-in-fume-hood-with-vapor-proof-light-72w-x-36d-x-94h>: HTTP status code is not handled or not allowed 2026-01-25 06:28:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bearing-spacer-jmd18-035>: HTTP status code is not handled or not allowed 2026-01-25 06:28:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/screw-jtm-124vs-la-124>: HTTP status code is not handled or not allowed 2026-01-25 06:28:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spacer-jtm-1050-1050b-34>: HTTP status code is not handled or not allowed 2026-01-25 06:28:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-kit-for-electric-pallet-jack-gwk-8900-lw-xl-fits-raymond-model-8900>: HTTP status code is not handled or not allowed 2026-01-25 06:28:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40012-bottle-blaster-plastic-cast-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 06:28:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/complete-wheel-kit-for-manual-pallet-jack-gwk-tsp5500-ck-fits-total-source-model-tsp5500>: HTTP status code is not handled or not allowed 2026-01-25 06:28:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40411-top-and-side-open-blast-cabinet-w-dust-collector-steel>: HTTP status code is not handled or not allowed 2026-01-25 06:28:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/penco-recessed-handle-triple-tier-locker-12x12x24-assembled-3-wide-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:28:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poly-tubing-10w-x-725l-4-mil-black-1-roll>: HTTP status code is not handled or not allowed 2026-01-25 06:28:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-inch-deep-top-step-kit-grip-strut>: HTTP status code is not handled or not allowed 2026-01-25 06:28:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-mesh-decking-52-l-x-36-d-2620-lb-capacity-1>: HTTP status code is not handled or not allowed 2026-01-25 06:28:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-mesh-decking-46-l-x-42-d-2520-lb-capacity-1>: HTTP status code is not handled or not allowed 2026-01-25 06:28:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clip-bracket-for-mack-led-plus-40-20-degrees>: HTTP status code is not handled or not allowed 2026-01-25 06:28:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/minigrip-white-block-reclosable-bags-w-hang-hole-6w-x-9l-4-milclear1000pk>: HTTP status code is not handled or not allowed 2026-01-25 06:28:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-step-75-incline-ladder-24-inch-w-perforated>: HTTP status code is not handled or not allowed 2026-01-25 06:28:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-tpi-fan-head-non-oscillating-3>: HTTP status code is not handled or not allowed 2026-01-25 06:28:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/snap-on-magnifier-for-omnivue-add-x-4d-12d>: HTTP status code is not handled or not allowed 2026-01-25 06:29:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sleeve-nut-jtm-1055-jtm1055-v34>: HTTP status code is not handled or not allowed 2026-01-25 06:29:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-9w-x-6l-4-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 06:29:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/halux-universal-led-treatment-light-n50-3-p-fx-double-arm-clamp>: HTTP status code is not handled or not allowed 2026-01-25 06:29:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72x30-complete-pack-bench-standard-laminate>: HTTP status code is not handled or not allowed 2026-01-25 06:29:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/41200-front-open-dual-station-abrasive-blast-cabinet-steel>: HTTP status code is not handled or not allowed 2026-01-25 06:29:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/needle-bearing-jtm-1050-1050b-41>: HTTP status code is not handled or not allowed 2026-01-25 06:29:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-kit-for-electric-pallet-jack-gwk-npp60-lw-fits-cat-model-npp-60>: HTTP status code is not handled or not allowed 2026-01-25 06:29:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-kit-for-electric-pallet-jack-gwk-8300-lw-fits-raymond-model-8300>: HTTP status code is not handled or not allowed 2026-01-25 06:29:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-polypropylene-bags-30l-x-24w-15-mil-clear-500pack>: HTTP status code is not handled or not allowed 2026-01-25 06:29:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cantilever-rack-straight-arm-22>: HTTP status code is not handled or not allowed 2026-01-25 06:29:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cantilever-rack-straight-arm-with-2-lip-21>: HTTP status code is not handled or not allowed 2026-01-25 06:29:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triango-universal-led-fokus-treatment-light-100-1-w-dimming-wall-mount>: HTTP status code is not handled or not allowed 2026-01-25 06:29:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-shelf-bookcase-mahogany-executive-modular-furniture>: HTTP status code is not handled or not allowed 2026-01-25 06:29:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/horizontal-drum-positioner-650-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:29:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/block-table-adj-3055241>: HTTP status code is not handled or not allowed 2026-01-25 06:29:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 75525 pages (at 97 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:29:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-kit-for-electric-pallet-jack-gwk-b80zhd-lw-xl-fits-hyster-model-b80zhd>: HTTP status code is not handled or not allowed 2026-01-25 06:29:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40068-3-32-id-ceramic-nozzle-7-cfm-pressure>: HTTP status code is not handled or not allowed 2026-01-25 06:29:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alum-pie-plates-8-15-16-dia>: HTTP status code is not handled or not allowed 2026-01-25 06:29:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/snack-mix-original-flavor-5-5-oz>: HTTP status code is not handled or not allowed 2026-01-25 06:29:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bib11-bench-in-a-box-technical-workbench-60-w-x-30-d-plastic-laminate-top-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:29:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/minigrip174-reclosable-poly-bags-8w-x-10l-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 06:29:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speedpress-40-ultimate-steel-safety-ruler>: HTTP status code is not handled or not allowed 2026-01-25 06:29:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/80-fury-heavy-duty-degreaser-non-corrosive-low-foam-unscented-2l-bottles-4-cs-908682>: HTTP status code is not handled or not allowed 2026-01-25 06:29:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/70-tough-green-all-purpose-cleaner-degreaser-fruity-floral-2l-bottle-4-cs-908972>: HTTP status code is not handled or not allowed 2026-01-25 06:29:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40115-hose-rubber>: HTTP status code is not handled or not allowed 2026-01-25 06:29:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-hardware-kit-for-761224-outdoor-wall-mount-drinking-fountains>: HTTP status code is not handled or not allowed 2026-01-25 06:29:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/174-6292621-knife-set>: HTTP status code is not handled or not allowed 2026-01-25 06:29:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speakman-anystream-hotel-5-jet-4-1-4-dia-showerhead-polished-chrome-finish-2-5-gpm>: HTTP status code is not handled or not allowed 2026-01-25 06:29:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-qt-mixing-bowl-1075-dia>: HTTP status code is not handled or not allowed 2026-01-25 06:29:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-18w-x-30l-4-mil-clear-250pack>: HTTP status code is not handled or not allowed 2026-01-25 06:29:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sketch-pla-filament-orange>: HTTP status code is not handled or not allowed 2026-01-25 06:29:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/half-inch-locking-clip>: HTTP status code is not handled or not allowed 2026-01-25 06:29:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-polypropylene-bags-5l-x-3w-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 06:29:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-poly-bags-24w-x-36l-4-mil-clear-100pack>: HTTP status code is not handled or not allowed 2026-01-25 06:29:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wheel-a-bout-portable-home-access-ramp-wab1030-10l-x-30w-600-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:29:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-way-ceiling-sidewall-register-7>: HTTP status code is not handled or not allowed 2026-01-25 06:29:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rg-213-u-type-coaxial-cable-solid-pe-insulation-1-conductor-13-awg-bk>: HTTP status code is not handled or not allowed 2026-01-25 06:29:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-rubber-seal-strip-l-1300mm-261990-641250-641263-641265-641407-floor-scrubbers>: HTTP status code is not handled or not allowed 2026-01-25 06:29:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gland-jet-1t-t30201030>: HTTP status code is not handled or not allowed 2026-01-25 06:29:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/resealable-polypropylene-bags-24l-x-18w-15-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 06:29:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e4pbg-push-button-encl-4pb-x-225mm-44mm>: HTTP status code is not handled or not allowed 2026-01-25 06:29:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/back-panel-36-x-13>: HTTP status code is not handled or not allowed 2026-01-25 06:29:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clamp-nut-jsm5430-30>: HTTP status code is not handled or not allowed 2026-01-25 06:29:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gondolashelving-levelingwrench-796551>: HTTP status code is not handled or not allowed 2026-01-25 06:29:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-athletic-sneakers-size-14m-black>: HTTP status code is not handled or not allowed 2026-01-25 06:29:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/complete-electronic-packing-workbench-esd-safety-edge-60-x-36>: HTTP status code is not handled or not allowed 2026-01-25 06:29:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-removable-foam-tape-12-x-72-yds-3125-mil-white-2pack>: HTTP status code is not handled or not allowed 2026-01-25 06:29:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/diamond-polishing-17-green-3000-grit-polishing-daily-maintenance-pad-2-per-case>: HTTP status code is not handled or not allowed 2026-01-25 06:29:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjusting-rod-jjp8bt-91>: HTTP status code is not handled or not allowed 2026-01-25 06:29:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lifetime-in-ground-basketball-system-with-54in-steel-framed-backboard>: HTTP status code is not handled or not allowed 2026-01-25 06:29:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-mechanics-tool-trolley-stool-back-rest-450-lb-cap-black-vinyl>: HTTP status code is not handled or not allowed 2026-01-25 06:29:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lucky-dog-uptown-dog-welded-wire-kennel-with-cover-4ft-x4ft-x6ft-black>: HTTP status code is not handled or not allowed 2026-01-25 06:29:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/minigrip174-reclosable-poly-bags-3w-x-5l-2-mil-clear-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 06:29:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/moisture-barrier-bags-14w-x-30l-36-mil-silver-100pack>: HTTP status code is not handled or not allowed 2026-01-25 06:29:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tool-and-hardware-corral-for-lifetime-sheds>: HTTP status code is not handled or not allowed 2026-01-25 06:29:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flammable-cabinet-with-legs-manual-close-door-120-gallon>: HTTP status code is not handled or not allowed 2026-01-25 06:29:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stanco-economy-hrc-4-arc-clothing-kit-50-coat-15-leggings-hood-hard-hat-bag-ttek40-2xl>: HTTP status code is not handled or not allowed 2026-01-25 06:29:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lever-action-drum-pump-for-water-based-fluids>: HTTP status code is not handled or not allowed 2026-01-25 06:29:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ase30x30x6nk-pull-box-screw-cover-3000x3000x600-steelgray>: HTTP status code is not handled or not allowed 2026-01-25 06:29:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/strap-protector-2-x-2-x-3-225-thickness-package-qty-600>: HTTP status code is not handled or not allowed 2026-01-25 06:29:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-stainless-steel-squeegee-complete-16>: HTTP status code is not handled or not allowed 2026-01-25 06:29:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/linkup-portable-modular-dirt-aggregate-conveyor-with-4-belts-400-series-12l-to-30l-x-16w>: HTTP status code is not handled or not allowed 2026-01-25 06:29:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-steel-stack-on-desk-hutch-48w-black>: HTTP status code is not handled or not allowed 2026-01-25 06:29:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gray-flagged-polypropylene-floor-brush-36>: HTTP status code is not handled or not allowed 2026-01-25 06:29:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-blades-6>: HTTP status code is not handled or not allowed 2026-01-25 06:29:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proaluminum-acme-straight-handle>: HTTP status code is not handled or not allowed 2026-01-25 06:29:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durham-vertical-gas-cylinder-storage-cabinet-holds-4-20-lb-or-335-lb-lpg-cylinders>: HTTP status code is not handled or not allowed 2026-01-25 06:29:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/csd12126-concept-wall-mount-encl-1200x1200x600-steelgray>: HTTP status code is not handled or not allowed 2026-01-25 06:29:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-x-36-3-drawer-with-maple-top-workbench>: HTTP status code is not handled or not allowed 2026-01-25 06:29:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/valley-craft-2-wheel-aluminum-drum-truck-pneumatic-wheels>: HTTP status code is not handled or not allowed 2026-01-25 06:29:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-x-30-4-drawer-with-steel-top-workbench>: HTTP status code is not handled or not allowed 2026-01-25 06:29:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/csd302010-concept-wall-mount-encl-3000x2000x1000-steelgray>: HTTP status code is not handled or not allowed 2026-01-25 06:29:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-36-stainless-steel-unit-fixed-leg-work-table>: HTTP status code is not handled or not allowed 2026-01-25 06:29:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mapa-temp-ice-700-nitrile-3-4-coated-thermal-gloves-1-pair-size-9-700419>: HTTP status code is not handled or not allowed 2026-01-25 06:29:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/prosegur-2-alarm-mini-patriot-cable-super-lock-amrf-with-48-heavy-duty-lanyard-black-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-25 06:29:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/janitorial-shadow-board-blue-on-black-general-purpose-composite-sb101acp>: HTTP status code is not handled or not allowed 2026-01-25 06:29:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nilotron-metered-air-fresheners-fresh-clean-scent-7-oz-refill-12-case>: HTTP status code is not handled or not allowed 2026-01-25 06:29:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nilotron-metered-air-fresheners-cinnamon-spice-scent-7-oz-refill-12-case>: HTTP status code is not handled or not allowed 2026-01-25 06:29:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waterhog-fashion-diamond-mat-charcoal-2-x-3>: HTTP status code is not handled or not allowed 2026-01-25 06:29:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reverse-valve-case-jsm7572-65>: HTTP status code is not handled or not allowed 2026-01-25 06:29:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15ga-finish-nails-33-angled-2-1-2-length-316-stainless-steel-pkg-of-4000>: HTTP status code is not handled or not allowed 2026-01-25 06:29:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-reach-freezer-3-door-72-cu-80-w-x-32-2-d-x-82-5-h>: HTTP status code is not handled or not allowed 2026-01-25 06:29:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jbc-revolution-18-reflective-traffic-cone-yellow-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 06:29:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waterhog-fashion-diamond-mat-dark-brown-6-x-12>: HTTP status code is not handled or not allowed 2026-01-25 06:29:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ahe18x18x6-pull-box-hinged-cover-1800x1800x600-steelgray>: HTTP status code is not handled or not allowed 2026-01-25 06:29:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deodorizing-urinal-screen-cherry-scent-12-case>: HTTP status code is not handled or not allowed 2026-01-25 06:29:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jbc-revolution-18-reflective-traffic-cone-traffic-orange-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 06:29:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/temp-tec-nl56-14-insulated-neoprene-coated-gloves-heavy-weight-1-pair-size-10>: HTTP status code is not handled or not allowed 2026-01-25 06:29:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ingersoll-rand-125-air-needle-scaler>: HTTP status code is not handled or not allowed 2026-01-25 06:29:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iso-ss-316-cast-hex-head-plug-18-npt-male>: HTTP status code is not handled or not allowed 2026-01-25 06:29:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/worm-jbos-5-6286473>: HTTP status code is not handled or not allowed 2026-01-25 06:29:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ahe12x10x6-pull-box-hinged-cover-1200x1000x600-steelgray>: HTTP status code is not handled or not allowed 2026-01-25 06:29:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a1084dl-j-box-hinged-cover-contoured-type-12-1000x800x400-steelgray>: HTTP status code is not handled or not allowed 2026-01-25 06:29:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/waterhog-classic-diamond-mat-bluestone-3-x-5>: HTTP status code is not handled or not allowed 2026-01-25 06:29:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hndl-38-25a-6295374>: HTTP status code is not handled or not allowed 2026-01-25 06:29:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carpet-upholstery-shampoo-fresh-scent-gallon-bottle-4-case>: HTTP status code is not handled or not allowed 2026-01-25 06:29:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/handy-size-bound-memo-book-stiff-tan-cover-7-x-4-3-8-size-96-pages>: HTTP status code is not handled or not allowed 2026-01-25 06:29:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/certified-174-dri-way-jr-machine-120v-40-power-cord-50-60-hz>: HTTP status code is not handled or not allowed 2026-01-25 06:29:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oxy-force-dilution-system-with-key-lock-out-multiple-flow-rate>: HTTP status code is not handled or not allowed 2026-01-25 06:30:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/post-it-super-sticky-ultra-notes-3-x-3-assorted-colors-12-pads-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:30:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/filter-331022>: HTTP status code is not handled or not allowed 2026-01-25 06:30:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tubing-kit-1-id-x-100l>: HTTP status code is not handled or not allowed 2026-01-25 06:30:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a1412chftc-continuous-hinge-wflange-trough-collar-type-12-1400x1200x654-steelgray>: HTTP status code is not handled or not allowed 2026-01-25 06:30:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plate-25-a-6295369>: HTTP status code is not handled or not allowed 2026-01-25 06:30:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ingersoll-rand-2145qimax-3-4-impactool>: HTTP status code is not handled or not allowed 2026-01-25 06:30:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exq-pro-extreme-power-workstation-hutch-55w-x-30d-x-26-38h-black-with-red-handle>: HTTP status code is not handled or not allowed 2026-01-25 06:30:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fae-3100-refill-1-x-3-adhesive-bandages-plastic-40-per-box>: HTTP status code is not handled or not allowed 2026-01-25 06:30:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fae-3101-refill-1x-3-adhesive-bandage-fabric-40-per-box>: HTTP status code is not handled or not allowed 2026-01-25 06:30:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fae-3004-refill-3-4x-3-adhesive-bandages-plastic-25-per-box>: HTTP status code is not handled or not allowed 2026-01-25 06:30:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ex-professional-upgraded-spring-loaded-casters-5w-x-10d-x-9-12h-black-set-of-4>: HTTP status code is not handled or not allowed 2026-01-25 06:30:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speedi-boot-90-175-register-vent-boot-w-adj-hangers-for-hvac-duct-work-4-x-14-x-8>: HTTP status code is not handled or not allowed 2026-01-25 06:30:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/permanently-bound-composition-book-quadrille-rule-80-10-x-7-7-8-sheets>: HTTP status code is not handled or not allowed 2026-01-25 06:30:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rx-roller-cabinet-12-drawers-55w-x-25d-x-47h-black-w-chrome-drawer-pulls-trim>: HTTP status code is not handled or not allowed 2026-01-25 06:30:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-union-fitting>: HTTP status code is not handled or not allowed 2026-01-25 06:30:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 75624 pages (at 99 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:30:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-closing-right-exit-mirror>: HTTP status code is not handled or not allowed 2026-01-25 06:30:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-closing-left-entrance-satin>: HTTP status code is not handled or not allowed 2026-01-25 06:30:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a84xm7818ftc-heavy-duty-fs-disc-encl-wpanel-2-door-8412x7850x1812-paint>: HTTP status code is not handled or not allowed 2026-01-25 06:30:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a12n124-small-type-1-enclosure-1200x1200x400-steelgray>: HTTP status code is not handled or not allowed 2026-01-25 06:30:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3m-vinyl-duct-tape-3903-red-2-x-50-yd-6-3-mil-24-per-case>: HTTP status code is not handled or not allowed 2026-01-25 06:30:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manual-locking-left-exit-satin>: HTTP status code is not handled or not allowed 2026-01-25 06:30:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scotch-vinyl-electrical-color-coding-tape-35-gray-3-4-x-66-100-per-case>: HTTP status code is not handled or not allowed 2026-01-25 06:30:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90-176-ss-corner-guard-16-ga-counter-sunk-holes-rd-corners-48-lx2-wx2-h>: HTTP status code is not handled or not allowed 2026-01-25 06:30:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a726016ulpg-2-door-wfloor-stands-type-12-encl-7206x6006x1606-steelgray>: HTTP status code is not handled or not allowed 2026-01-25 06:30:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ase18x12x8nk-pull-box-screw-cover-1800x1200x800-steelgray>: HTTP status code is not handled or not allowed 2026-01-25 06:30:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speedi-grille-filter-air-vent-grille-vent-cover-12-x-12>: HTTP status code is not handled or not allowed 2026-01-25 06:30:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90-176-ss-corner-guard-16-ga-3-8-wings-sq-corners-96-lx3-1-2-wx3-1-2-h>: HTTP status code is not handled or not allowed 2026-01-25 06:30:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-solid-galvanized-steel-shelving-starter-unit-5-tier-60w-x-18d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 06:30:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nut-m4-jtm-1050-b-132>: HTTP status code is not handled or not allowed 2026-01-25 06:30:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scotch-vinyl-electrical-color-coding-tape-35-red-3-4-x-66-100-per-case>: HTTP status code is not handled or not allowed 2026-01-25 06:30:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-series-370-disc-horn-12volt-465hz>: HTTP status code is not handled or not allowed 2026-01-25 06:30:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/open-sided-wire-exchange-truck-two-wire-one-galvanized-shelf-1000-lb-cap-4>: HTTP status code is not handled or not allowed 2026-01-25 06:30:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-solid-galvanized-steel-shelving-starter-unit-5-tier-48w-x-24d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 06:30:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ball-retainer-jsm-802-80242>: HTTP status code is not handled or not allowed 2026-01-25 06:30:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30x12x72-solar-star-greenhouse-w-solid-polycarbonate-prop-heater>: HTTP status code is not handled or not allowed 2026-01-25 06:30:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ce5002-3-75hp-2-stage-60-gallon-vertical-air-compressor>: HTTP status code is not handled or not allowed 2026-01-25 06:30:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-green-wire-shelving-add-on-unit-5-tier-72w-x-21d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 06:30:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-green-wire-shelving-add-on-unit-5-tier-72w-x-21d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 06:30:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-green-wire-shelving-starter-unit-5-tier-60w-x-21d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 06:30:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chuck-key-jbm-5-23011076>: HTTP status code is not handled or not allowed 2026-01-25 06:30:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cutter-hd-guard-6296145>: HTTP status code is not handled or not allowed 2026-01-25 06:30:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milwaukee-m12-redlithium-xc6-battery>: HTTP status code is not handled or not allowed 2026-01-25 06:30:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelate-silver-epoxy-wire-shelving-starter-unit-5-tier-54w-x-21d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 06:30:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/champion-sprayon-galv-off>: HTTP status code is not handled or not allowed 2026-01-25 06:30:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30x12x48-solar-star-greenhouse-w-poly-top-and-ends-drop-down-sides>: HTTP status code is not handled or not allowed 2026-01-25 06:30:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bollard-safety-net-4-foot-l-by-32-foot-w-fits-8-3-4-inches-to-9-5-8-inches-dia-brsn-432-10>: HTTP status code is not handled or not allowed 2026-01-25 06:30:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gear--15-planer-6284713>: HTTP status code is not handled or not allowed 2026-01-25 06:30:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bollard-safety-net-4-foot-l-by-28-foot-w-fits-3-1-2-inches-to-4-1-2-inches-dia-brsn-428-5>: HTTP status code is not handled or not allowed 2026-01-25 06:30:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/armorultra-job-on-pallet-2-layer-epoxy-system-5000-sq-ft-white>: HTTP status code is not handled or not allowed 2026-01-25 06:30:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arbor-bolt-jms10scms-54>: HTTP status code is not handled or not allowed 2026-01-25 06:30:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bollard-safety-net-4-foot-l-by-24-foot-w-fits-3-5-8-inches-to-5-1-2-inches-dia-brsn-424-6>: HTTP status code is not handled or not allowed 2026-01-25 06:30:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-u-channel-12-l-x-1-4-w-clear>: HTTP status code is not handled or not allowed 2026-01-25 06:30:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/34x12x40-solar-star-greenhouse-w-poly-top-and-ends-drop-down-sides-prop-heater>: HTTP status code is not handled or not allowed 2026-01-25 06:30:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/screw-set-14-20x14-6295652>: HTTP status code is not handled or not allowed 2026-01-25 06:30:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-green-wire-shelving-starter-unit-5-tier-24w-x-21d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 06:30:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stor-n-pour-quart-containerwhite>: HTTP status code is not handled or not allowed 2026-01-25 06:30:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/majestic-greenhouse-20w-x-60l-w-top-side-polycarbonate>: HTTP status code is not handled or not allowed 2026-01-25 06:30:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/key-4-4-10--15-planer-6284712>: HTTP status code is not handled or not allowed 2026-01-25 06:30:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-green-wire-shelving-add-on-unit-5-tier-54w-x-24d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 06:30:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n999475-ecosol-security-bolts>: HTTP status code is not handled or not allowed 2026-01-25 06:30:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cast-acrylic-sheet-24-l-x-24-w-x-1-8-thick-blue-ps-cacc-22>: HTTP status code is not handled or not allowed 2026-01-25 06:30:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classical-cogged-ax-v-belt-ax38>: HTTP status code is not handled or not allowed 2026-01-25 06:30:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-vertical-gas-cylinder-cabinet-2-cylinder-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:30:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bollard-safety-net-4-foot-l-by-8-foot-w-fits-6-3-4-inches-to-7-5-8-inches-dia-brsn-48-8>: HTTP status code is not handled or not allowed 2026-01-25 06:30:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shaft-model-201planer-6012170>: HTTP status code is not handled or not allowed 2026-01-25 06:30:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-solid-galvanized-steel-shelving-starter-unit-5-tier-36w-x-24d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 06:30:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classical-b-v-belt-b150>: HTTP status code is not handled or not allowed 2026-01-25 06:30:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30x12x96-solar-star-greenhouse-w-poly-top-and-ends-roll-up-sides-prop-heater>: HTTP status code is not handled or not allowed 2026-01-25 06:30:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-green-wire-shelving-starter-unit-5-tier-42w-x-24d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 06:30:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8090253447-two-stage-horizontal-duplex-air-compressor-3-x-10hp-200-gallon>: HTTP status code is not handled or not allowed 2026-01-25 06:30:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/123324-safety-padlock-with-label-nylon-shackle-nylon-red>: HTTP status code is not handled or not allowed 2026-01-25 06:30:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/narrow-5vx-cogged-v-belt-5vx800>: HTTP status code is not handled or not allowed 2026-01-25 06:30:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-film-36-l-x-40-w-x-0-005-thick-clear>: HTTP status code is not handled or not allowed 2026-01-25 06:30:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50943-all-purpose-lockout-with-8-l-sheathed-metal-cable-fiberglass-reinforced-poly>: HTTP status code is not handled or not allowed 2026-01-25 06:30:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/66064-lockout-tag-danger-do-not-operate-2-sided-economy-polyester-25pack>: HTTP status code is not handled or not allowed 2026-01-25 06:30:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/auger-style-earth-anchor-5-8-x-48-x-6>: HTTP status code is not handled or not allowed 2026-01-25 06:30:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cast-acrylic-bar-12-l-x-4-w-x-3-16-thick-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:30:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26x12x48-solar-star-greenhouse-w-poly-top-and-ends-roll-up-sides-gas-heater>: HTTP status code is not handled or not allowed 2026-01-25 06:30:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/krylon-industrial-work-day-enamel-paint-rue-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:30:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fixed-plate-jsg-96-jsg96-204>: HTTP status code is not handled or not allowed 2026-01-25 06:30:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anchor-lock-down-jsg-0402-822027>: HTTP status code is not handled or not allowed 2026-01-25 06:30:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10x10-heavy-duty-commercial-canopy-12-5oz-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:30:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/red-rack-mobile-gear-storage-rack-locker-single-sided-three-20-sections-red>: HTTP status code is not handled or not allowed 2026-01-25 06:30:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mailing-tubes-with-caps-2dia-x-18l-0060-thick-red-50pack>: HTTP status code is not handled or not allowed 2026-01-25 06:30:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classical-b-v-belt-b53>: HTTP status code is not handled or not allowed 2026-01-25 06:30:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/113280-stop-sign-high-intensity-prismatic-reflective-sign-aluminum-18w-x-18h>: HTTP status code is not handled or not allowed 2026-01-25 06:30:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelate-silver-epoxy-wire-shelving-starter-unit-5-tier-72w-x-21d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 06:30:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/weathershield-commercial-canopy-18w-x-50l-white>: HTTP status code is not handled or not allowed 2026-01-25 06:30:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/narrow-5v-banded-v-belt-3-5v2120>: HTTP status code is not handled or not allowed 2026-01-25 06:30:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/25804-danger-construction-area-keep-out-sign-polystyrene-14w-x-10h>: HTTP status code is not handled or not allowed 2026-01-25 06:30:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/weathershield-commercial-canopy-18w-x-60l-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:30:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-green-wire-shelving-add-on-unit-5-tier-42w-x-24d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 06:30:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poly-mailers-with-tear-strip-7-12w-x-10-12l-25-mil-white-1000pack>: HTTP status code is not handled or not allowed 2026-01-25 06:30:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/env200-env-oil-only-pads15w-x-19llight-weightbale200>: HTTP status code is not handled or not allowed 2026-01-25 06:30:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-duty-steel-zinc-plated-wire-rope-thimble-1-4-diameter>: HTTP status code is not handled or not allowed 2026-01-25 06:30:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/33-w-stacking-bin-33-x-8-7-10-x-5-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:30:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/divider-33-w-stacking-bin-239863-pack-of-7>: HTTP status code is not handled or not allowed 2026-01-25 06:31:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grade-70-transport-binder-chain-3-8-dia-6600-lb-cap-20-box-bag>: HTTP status code is not handled or not allowed 2026-01-25 06:31:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paramount-wall-storage-cabinet-assembled-19-7-8x14-1-4x32-3-4-gray-269877gy>: HTTP status code is not handled or not allowed 2026-01-25 06:31:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-duty-steel-zinc-plated-wire-rope-thimble-1-2-diameter>: HTTP status code is not handled or not allowed 2026-01-25 06:31:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hndl-20-planer-mdl-208-6292782>: HTTP status code is not handled or not allowed 2026-01-25 06:31:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-duty-steel-zinc-plated-wire-rope-thimble-5-8-diameter>: HTTP status code is not handled or not allowed 2026-01-25 06:31:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelate-silver-epoxy-wire-shelving-starter-unit-5-tier-42w-x-18d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 06:31:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labels-with-invoice-enclosed-print-5l-x-3w-redwhite-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-25 06:31:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sparta-scratch-brush-11-38-l>: HTTP status code is not handled or not allowed 2026-01-25 06:31:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cast-acrylic-sheet-96-l-x-48-w-x-1-8-thick-orange>: HTTP status code is not handled or not allowed 2026-01-25 06:31:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-x-6-x-6-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-25 06:31:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-foam-buna-n-pvc-sheet-12-l-x-12-w-x-3-16-thick-black>: HTTP status code is not handled or not allowed 2026-01-25 06:31:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-x-8-x-8-doublewall-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-25 06:31:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wear-pad-mdl-jwss-10-200386>: HTTP status code is not handled or not allowed 2026-01-25 06:31:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/500-1-8-diameter-7x7-galvanized-aircraft-cable-2>: HTTP status code is not handled or not allowed 2026-01-25 06:31:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/opticlean-16-compartment-glass-rack-w-4-extenders-carlisle-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:31:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 75721 pages (at 97 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:31:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-x-8-x-8-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-25 06:31:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stayflats-plus-peel-seal-mailers-13w-x-18l-kraft-100pack>: HTTP status code is not handled or not allowed 2026-01-25 06:31:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bsc40-basic-universal-soc3dia-x-42lcase40>: HTTP status code is not handled or not allowed 2026-01-25 06:31:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-foam-buna-n-pvc-sheet-12-l-x-12-w-x-1-16-thick-black-acrylic-adhesive>: HTTP status code is not handled or not allowed 2026-01-25 06:31:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/weathershield-giant-commercial-canopy-24w-x-50l-white>: HTTP status code is not handled or not allowed 2026-01-25 06:31:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/76224-danger-do-not-operate-tag-2-sided-25pkg-polyester-3w-x-5-34h>: HTTP status code is not handled or not allowed 2026-01-25 06:31:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/65367-danger-do-not-operate-tag-2-sided-25pkg-polyester-3w-x-5-34h>: HTTP status code is not handled or not allowed 2026-01-25 06:31:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grade-70-transport-chain-1-4-dia-3150-lb-cap-10-box>: HTTP status code is not handled or not allowed 2026-01-25 06:31:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/13-x-13-x-15-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-25 06:31:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-1-2-x-12-1-2-x-12-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-25 06:31:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storplus-5-gallon-box-26-18-3-12white>: HTTP status code is not handled or not allowed 2026-01-25 06:31:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-champion-forged-5-14x7-14-rigid-10x2-12-duralast-ball-2500lb>: HTTP status code is not handled or not allowed 2026-01-25 06:31:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/94211-speed-limit-15-hip-reflective-speed-limit-sign-aluminum-24w-x-24h>: HTTP status code is not handled or not allowed 2026-01-25 06:31:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crank-lead-screw-15-planer-6284708>: HTTP status code is not handled or not allowed 2026-01-25 06:31:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vital-cluster-box-unit-w-vogue-traditional-accessories-4-mailboxes-2-parcel-lockers-dark-bronze>: HTTP status code is not handled or not allowed 2026-01-25 06:31:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storplus-35-gallon-box-18-12-6blue>: HTTP status code is not handled or not allowed 2026-01-25 06:31:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-pneumatic-wheel-9-x410350-4-58-ball>: HTTP status code is not handled or not allowed 2026-01-25 06:31:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700258gre-pegboard-floor-stand-w5-c-channel-sliding-16w-x-66h-gn-1-pc>: HTTP status code is not handled or not allowed 2026-01-25 06:31:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15-x-10-x-10-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-25 06:31:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelite-vented-plastic-shelving-starter-unit-5-tier-54w-x-24d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 06:31:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/225570-6-compartment-eye-shadow-tray-10w-x-1h-x-6d-clr-1-pc>: HTTP status code is not handled or not allowed 2026-01-25 06:31:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-x-14-x-19-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-25 06:31:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/key-5-5-22-5fk-g10-15-planer-6284770>: HTTP status code is not handled or not allowed 2026-01-25 06:31:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/washer-25mm-model-201planer-6012077>: HTTP status code is not handled or not allowed 2026-01-25 06:31:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelite-vented-plastic-shelving-starter-unit-5-tier-48w-x-24d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 06:31:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/76168-caution-accident-prevention-tag-2-sided-25pkg-polyester-3w-x-5-34h>: HTTP status code is not handled or not allowed 2026-01-25 06:31:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-flexonite-wheel-5x1-14-12-oilless>: HTTP status code is not handled or not allowed 2026-01-25 06:31:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/121186-ghs-3-safety-data-sheet-sds-binder-spanish-polyethylene-3-12w-x-11h>: HTTP status code is not handled or not allowed 2026-01-25 06:31:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/570-8-lrb-trailer-tire-wheel-assembly-bolt-circle-4-x-4>: HTTP status code is not handled or not allowed 2026-01-25 06:31:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stand-off-model-201-planer-6012288>: HTTP status code is not handled or not allowed 2026-01-25 06:31:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelving-starter-unit-5-tier-60w-x-21d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 06:31:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700228pur-multitower-pegboard-floor-display-20w-x-64h-x-20d-pl-1-pc>: HTTP status code is not handled or not allowed 2026-01-25 06:31:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shoulder-screw-model-201-plnr-6012284>: HTTP status code is not handled or not allowed 2026-01-25 06:31:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cast-acrylic-bar-12-l-x-1-w-x-3-16-thick-white>: HTTP status code is not handled or not allowed 2026-01-25 06:31:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-flexonite-wheel-3x1-14-38-oilless>: HTTP status code is not handled or not allowed 2026-01-25 06:31:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/66065-danger-do-not-operate-tag-bilingual-englighspanish-hd-polyester-25pack>: HTTP status code is not handled or not allowed 2026-01-25 06:31:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/redi-seal-envelopes-12w-x-15-12l-kraft-500pack>: HTTP status code is not handled or not allowed 2026-01-25 06:31:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/86615-emergency-shower-eye-wash-test-record-tag-2-sided-10pkg-polyester-4w-x-7h>: HTTP status code is not handled or not allowed 2026-01-25 06:31:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-hi-heat-plastex-wheel-5x1-14-12-roller>: HTTP status code is not handled or not allowed 2026-01-25 06:31:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/146123-cable-safety-padlock-with-label-4-316h-clearance-steel-cable-green>: HTTP status code is not handled or not allowed 2026-01-25 06:31:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bolt-m8-x-20-powerfeeders-6288749>: HTTP status code is not handled or not allowed 2026-01-25 06:31:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/funnel-king-1-2-3-quart-double-capped-funnel-dark-red>: HTTP status code is not handled or not allowed 2026-01-25 06:31:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprng-pin-4mm-x-25lg-model-201planer-6012059>: HTTP status code is not handled or not allowed 2026-01-25 06:31:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-175-watt-ruby-infra-red-lamp-with-mobile-base>: HTTP status code is not handled or not allowed 2026-01-25 06:31:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-z-brite-wire-shelving-starter-unit-1-dunnage-shelf-30w-x-24d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 06:31:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/regulated-labels-warning--keep-this-bag-away-3l-x-5w-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-25 06:31:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelite-vented-plastic-shelving-starter-unit-5-tier-36w-x-24d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 06:31:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hamilton-v-groove-wheel-8x3-1-tapered>: HTTP status code is not handled or not allowed 2026-01-25 06:31:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-z-brite-wire-shelving-starter-unit-1-dunnage-shelf-36w-x-24d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 06:31:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clasp-envelopes-10w-x-12l-kraft-500pack>: HTTP status code is not handled or not allowed 2026-01-25 06:31:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-z-brite-wire-shelving-starter-unit-1-dunnage-shelf-36w-x-24d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 06:31:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-screw-m12-x-175p-x-75lg-pfeeders-6288756>: HTTP status code is not handled or not allowed 2026-01-25 06:31:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-nexelon-wire-shelving-starter-unit-1-dunnage-shelf-24w-x-18d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 06:31:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simoniz-lemon-shine-ready-to-use-furniture-polish-32-oz>: HTTP status code is not handled or not allowed 2026-01-25 06:31:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-poly-z-brite-wire-shelving-starter-unit-1-dunnage-shelf-24w-x-18d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 06:31:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simonize-cool-blue-foaming-hand-soap-body-wash-gallon>: HTTP status code is not handled or not allowed 2026-01-25 06:31:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fellowes-galaxy-comb-bin-ding-machine>: HTTP status code is not handled or not allowed 2026-01-25 06:31:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-pvc-rectangular-tube-12-l-x-2-w-gray-bulk-rpt-pvc-14>: HTTP status code is not handled or not allowed 2026-01-25 06:31:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fellowes-thermal-bin-ding-machine-helios-30>: HTTP status code is not handled or not allowed 2026-01-25 06:31:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-plated-ultra-heavy-duty-spring-mount-for-non-lighted-flagstaffs-over-8-ft>: HTTP status code is not handled or not allowed 2026-01-25 06:31:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labels-with-fragile-handle-with-care-print-5l-x-3w-redwhite-roll-of-500-dl1160>: HTTP status code is not handled or not allowed 2026-01-25 06:31:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-square-plastic-trash-container-garbage-can-50-gallon-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:31:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700460wht-pinwheel-pegboard-display-wrb-135w-x-60h-wh-1-pc>: HTTP status code is not handled or not allowed 2026-01-25 06:31:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/colored-thermal-transfer-labels-4w-x-6l-3-core-yellow-4pack>: HTTP status code is not handled or not allowed 2026-01-25 06:31:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-specialty-labels-contains-cryogenic-liquid-3l-x-4-14w-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-25 06:31:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultem-pei-sheet-48-l-x-12-w-x-1-2-thick-amber>: HTTP status code is not handled or not allowed 2026-01-25 06:31:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10gce-digital-gravity-convection-lab-oven-0-7-cu-ft-600w>: HTTP status code is not handled or not allowed 2026-01-25 06:31:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultem-pei-sheet-48-l-x-24-w-x-1-4-thick-amber>: HTTP status code is not handled or not allowed 2026-01-25 06:31:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/700430clr-hshaped-pegboard-display-16w-x-60h-clr-1-pc>: HTTP status code is not handled or not allowed 2026-01-25 06:31:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-abs-tube-12-l-x-1-2-id-x-3-4-od-beige>: HTTP status code is not handled or not allowed 2026-01-25 06:31:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-x-20-x-12-regular-slotted-container>: HTTP status code is not handled or not allowed 2026-01-25 06:31:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cooler-backroom-shelving-tubular-bar-style-20-inch-48-inch-bar-3-shelves>: HTTP status code is not handled or not allowed 2026-01-25 06:31:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labels-with-next-day-air-print-3l-x-2w-fluorescent-red-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-25 06:31:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-flag-16-x16-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:31:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-pvc-rectangular-tube-36-l-x-2-w-gray-bulk-rpt-pvc-36>: HTTP status code is not handled or not allowed 2026-01-25 06:31:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-x-17-x-17-eo-container-350-double-wall-container-1>: HTTP status code is not handled or not allowed 2026-01-25 06:31:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-sheet-96-l-x-48-w-x-1-8-thick-white>: HTTP status code is not handled or not allowed 2026-01-25 06:31:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-wall-mounted-shelf-10-inch-60-inch-shelf>: HTTP status code is not handled or not allowed 2026-01-25 06:31:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-dia-round-number-paper-labels-with-14-print-fluorescent-yellow-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-25 06:31:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-wall-mounted-shelf-12-inch-36-inch-shelf>: HTTP status code is not handled or not allowed 2026-01-25 06:31:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-dia-round-paper-labels-w-quality-hold-print-fluorescent-orange-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-25 06:31:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-x-1-3-8-spring-pin-high-carbon-steel-zinc-clear-asme-b18-8-2-made-in-usa>: HTTP status code is not handled or not allowed 2026-01-25 06:31:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-epdm-strip-120-l-x-7-w-x-1-8-thick-black>: HTTP status code is not handled or not allowed 2026-01-25 06:31:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-buna-n-sheet-36-l-x-36-w-x-3-16-thick-blue-detectable-60a>: HTTP status code is not handled or not allowed 2026-01-25 06:32:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultem-pei-bar-24-l-x-1-w-x-1-thick-amber>: HTTP status code is not handled or not allowed 2026-01-25 06:32:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-wall-mounted-shelf-15-inch-48-inch-shelf-with-hooks>: HTTP status code is not handled or not allowed 2026-01-25 06:32:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-ground-wring-terminals-l510-3856318>: HTTP status code is not handled or not allowed 2026-01-25 06:32:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-heavy-duty-split-pole-warning-whip-w-o-light-12-x11-orange-rectangle-flag>: HTTP status code is not handled or not allowed 2026-01-25 06:32:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/332-x-1-cotter-pin-6622001>: HTTP status code is not handled or not allowed 2026-01-25 06:32:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/server-stainless-steel-pump-for-a-7-qt-vegetable-inset>: HTTP status code is not handled or not allowed 2026-01-25 06:32:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/belt-3vx220-goodyear-66-arbor-saw-6077225>: HTTP status code is not handled or not allowed 2026-01-25 06:32:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/split-ring-imp-s-s-2>: HTTP status code is not handled or not allowed 2026-01-25 06:32:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-1-2x14-self-seal-kra-bubble-mailers-with-opening-tear-strip-retail>: HTTP status code is not handled or not allowed 2026-01-25 06:32:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powerback-rubber-duct-5-ft-heavy-duty-single-1-ch>: HTTP status code is not handled or not allowed 2026-01-25 06:32:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-bar-24-l-x-2-w-x-1-8-thick-bronze>: HTTP status code is not handled or not allowed 2026-01-25 06:32:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/server-twin-warmer-w-pumps>: HTTP status code is not handled or not allowed 2026-01-25 06:32:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultem-pei-sheet-24-l-x-24-w-x-3-8-thick-amber>: HTTP status code is not handled or not allowed 2026-01-25 06:32:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/protective-mats-for-craft-tables-17-x-22-pack-of-two>: HTTP status code is not handled or not allowed 2026-01-25 06:32:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-16-x-2-cotter-pin-300-series-stainless-steel-asme-18-8-1-made-in-usa>: HTTP status code is not handled or not allowed 2026-01-25 06:32:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-80-2-50-4-narrow-flat-free-centipede-tread-3-centered-3-4-ball-bearings>: HTTP status code is not handled or not allowed 2026-01-25 06:32:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 75821 pages (at 100 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:32:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-foam-epdm-strip-120-l-x-1-w-x-3-8-thick-black>: HTTP status code is not handled or not allowed 2026-01-25 06:32:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-dia-round-number-paper-labels-with-0-print-dark-blue-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-25 06:32:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-x-3-x-2-mailers>: HTTP status code is not handled or not allowed 2026-01-25 06:32:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-x-4-x-4-mailers>: HTTP status code is not handled or not allowed 2026-01-25 06:32:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-x-4-x-2-mailers>: HTTP status code is not handled or not allowed 2026-01-25 06:32:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powerback-rubber-duct-10-ft-2-ch>: HTTP status code is not handled or not allowed 2026-01-25 06:32:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-socket-3-4-drive-6-point-1-3-16-short>: HTTP status code is not handled or not allowed 2026-01-25 06:32:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-x-4-x-3-mailers>: HTTP status code is not handled or not allowed 2026-01-25 06:32:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-socket-3-4-drive-6-point-1-1-2-short>: HTTP status code is not handled or not allowed 2026-01-25 06:32:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/03a2-sprocket-6288944>: HTTP status code is not handled or not allowed 2026-01-25 06:32:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-sheet-12-l-x-12-w-x-1-8-thick-black-antistatic>: HTTP status code is not handled or not allowed 2026-01-25 06:32:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classroom-adjustable-height-mobile-task-chair-polypropylene-with-padded-seat-black>: HTTP status code is not handled or not allowed 2026-01-25 06:32:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-48-l-x-3-4-w-x-1-2-thick-black-antistatic>: HTTP status code is not handled or not allowed 2026-01-25 06:32:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-24-l-x-2-1-2-w-x-1-4-thick-black-antistatic>: HTTP status code is not handled or not allowed 2026-01-25 06:32:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nhl-boston-bruins-hitch-cover-4-1-2-x-3-3-8-15143>: HTTP status code is not handled or not allowed 2026-01-25 06:32:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-steel-stability-ball-storage-rack-white>: HTTP status code is not handled or not allowed 2026-01-25 06:32:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fixed-height-pallet-stand-pdfs40-6ph2fllr-40-x-48-solid-deck-and-load-retainers>: HTTP status code is not handled or not allowed 2026-01-25 06:32:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nba-new-york-knicks-hitch-cover-4-1-2-x-3-3-8-15126>: HTTP status code is not handled or not allowed 2026-01-25 06:32:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-outdoor-in-ground-steel-flat-bench-perforated-metal-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:32:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-outdoor-in-ground-steel-flat-bench-perforated-metal-green>: HTTP status code is not handled or not allowed 2026-01-25 06:32:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/415c-strainer-2-inch-npt>: HTTP status code is not handled or not allowed 2026-01-25 06:32:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-purpose-ballistic-panel-6-6-h-x-4-l-designer-charcoal>: HTTP status code is not handled or not allowed 2026-01-25 06:32:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-socket-3-8-drive-12-point-20mm-short>: HTTP status code is not handled or not allowed 2026-01-25 06:32:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/impact-socket-3-8-drive-3-8-6-point-short>: HTTP status code is not handled or not allowed 2026-01-25 06:32:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/disposable-coverall-knit-cuffs-and-attached-hood-white-red-large-4565-blk-l>: HTTP status code is not handled or not allowed 2026-01-25 06:32:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-spill-plus-containment-pallet-p4-4-drum-with-no-drain>: HTTP status code is not handled or not allowed 2026-01-25 06:32:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/impact-socket-1-2-drive-15-16-6-point-short>: HTTP status code is not handled or not allowed 2026-01-25 06:32:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-24-l-x-3-4-w-x-1-8-thick-black-antistatic>: HTTP status code is not handled or not allowed 2026-01-25 06:32:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-10-1-2x16-self-seal-kra-padded-mailers>: HTTP status code is not handled or not allowed 2026-01-25 06:32:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-office-partition-panel-with-partial-window-60-1-4w-x-101-1-2h-gray-gl>: HTTP status code is not handled or not allowed 2026-01-25 06:32:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-composite-carbon-fiber-bar-12-l-x-1-w-x-1-8-thick-black-bulk-cs-cf-39>: HTTP status code is not handled or not allowed 2026-01-25 06:32:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nhl-edmonton-oilers-grill-mat-14235>: HTTP status code is not handled or not allowed 2026-01-25 06:32:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-office-partition-panel-with-pass-thru-cable-48-1-4w-x-101-1-2h-blue-gl>: HTTP status code is not handled or not allowed 2026-01-25 06:32:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-48-l-x-4-w-x-3-4-thick-black-antistatic>: HTTP status code is not handled or not allowed 2026-01-25 06:32:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-sheet-12-l-x-12-w-x-1-2-thick-black-antistatic>: HTTP status code is not handled or not allowed 2026-01-25 06:32:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cb12948-50-keys-lock-box-w-carrying-handle-13w-x-4d-x-1012h-grey-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 06:32:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/17-1-8-x-11-1-8-x-4-literature-mailers>: HTTP status code is not handled or not allowed 2026-01-25 06:32:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-12-l-x-1-1-2-w-x-1-thick-black-antistatic>: HTTP status code is not handled or not allowed 2026-01-25 06:32:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/moo-tel-calf-nursery-w-doors-26w-x-40l-white>: HTTP status code is not handled or not allowed 2026-01-25 06:32:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nhl-carolina-hurricanes-grill-mat-14229>: HTTP status code is not handled or not allowed 2026-01-25 06:32:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cb13054-register-style-cash-box-w-key-lock-1134w-x-914d-x-312h-black-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 06:32:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cb12956-100-keys-adjustable-lock-box-6w-x-2412d-x-1712h-grey-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 06:32:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pictogram-flame-labels-2l-x-2w-redwhiteblack-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-25 06:32:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ax13104-white-jewelry-safe-18w-x-14d-x-1712h-101-cubic-ft>: HTTP status code is not handled or not allowed 2026-01-25 06:32:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-14-1-2x20-self-seal-white-padded-mailers>: HTTP status code is not handled or not allowed 2026-01-25 06:32:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cb12960-1170-keys-adjustable-lock-box-55w-x-3212d-x-1512h-grey-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 06:32:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/circle-mailing-labels-1-14-dia-clear-roll-of-1000>: HTTP status code is not handled or not allowed 2026-01-25 06:32:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-door-enclosed-flannel-letter-board-w-oak-frame-60w-x-48h-burgundy>: HTTP status code is not handled or not allowed 2026-01-25 06:32:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oxygen-sensor-socket-3-8-drive-side-cutout-1-hex-bolster>: HTTP status code is not handled or not allowed 2026-01-25 06:32:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vertical-cylinder-storage-cabinet-egcvc4-sc-50-holds-14-50-lb-cylinders>: HTTP status code is not handled or not allowed 2026-01-25 06:32:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-door-enclosed-recycled-rubber-tackboard-96w-x48h-confetti-w-silver-frame>: HTTP status code is not handled or not allowed 2026-01-25 06:32:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/non-electric-office-partition-panel-with-raceway-36-1-4w-x-100h-tan-gl>: HTTP status code is not handled or not allowed 2026-01-25 06:32:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/non-electric-office-partition-panel-with-raceway-24-1-4w-x-100h-gray-gl>: HTTP status code is not handled or not allowed 2026-01-25 06:32:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/northstar-garage-18w-x-16h-x-52l-green>: HTTP status code is not handled or not allowed 2026-01-25 06:32:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-door-enclosed-recycled-rubber-tackboard-72w-x48h-black-w-silver-frame>: HTTP status code is not handled or not allowed 2026-01-25 06:32:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/daddy-long-legs-side-panel-50l-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:32:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-pegboard-shelf-cabinet-hdcp244878-pb32-4s95-12-gauge-4-w-24-1-16-d-78-h>: HTTP status code is not handled or not allowed 2026-01-25 06:32:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/madc804vp-notice-sign-employees-only-14w-x-10h-plastic>: HTTP status code is not handled or not allowed 2026-01-25 06:32:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-socket-3-8-drive-12-point-1-4-deep>: HTTP status code is not handled or not allowed 2026-01-25 06:32:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pictogram-exclamation-mark-labels-1l-x-1w-blackredwhite-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-25 06:32:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-1-2x14-self-seal-kra-padded-mailers-25-packs>: HTTP status code is not handled or not allowed 2026-01-25 06:32:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/daddy-long-legs-gable-end-14w-70percent-shade>: HTTP status code is not handled or not allowed 2026-01-25 06:32:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-outdoor-in-ground-steel-bench-with-backrest-perforated-metal-green>: HTTP status code is not handled or not allowed 2026-01-25 06:32:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kdd630-cinch-cable-lockout-plastic>: HTTP status code is not handled or not allowed 2026-01-25 06:32:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-48-l-x-2-1-2-w-x-3-4-thick-black-antistatic>: HTTP status code is not handled or not allowed 2026-01-25 06:32:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-bar-12-l-x-2-w-x-1-4-thick-black-antistatic>: HTTP status code is not handled or not allowed 2026-01-25 06:32:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aneroid-sphygmomanometer-large-adult-size-blue-01-100-016>: HTTP status code is not handled or not allowed 2026-01-25 06:32:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/colored-thermal-transfer-labels-4w-x-6l-3-core-dark-green-4pack>: HTTP status code is not handled or not allowed 2026-01-25 06:32:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-bench-workstation-steel-top-locking-door-4-drawers-shelf-54-1-16x18-1-4x36-3-8>: HTTP status code is not handled or not allowed 2026-01-25 06:32:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kdd162-120240-circuit-breaker-lockout-triple-pole-plastic>: HTTP status code is not handled or not allowed 2026-01-25 06:32:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rectangular-inventory-paper-labels-4l-x-4w-fluorescent-red-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-25 06:32:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-artic-blast-aluminum-snow-shovel>: HTTP status code is not handled or not allowed 2026-01-25 06:32:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-x-30-open-end-static-shielding-bags>: HTTP status code is not handled or not allowed 2026-01-25 06:32:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-x-18-open-end-static-shielding-bags>: HTTP status code is not handled or not allowed 2026-01-25 06:32:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14x16-2-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 06:32:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/07157-53447-esmart-14-pc-square-2-set-in-clamshell>: HTTP status code is not handled or not allowed 2026-01-25 06:32:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/msmk120va-danger-sign-no-smoking-or-open-flame-10w-x-7h-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 06:32:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-bit-3-8>: HTTP status code is not handled or not allowed 2026-01-25 06:32:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paper-labels-with-used-oil-print-3l-x-2w-fluorescent-yellow-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-25 06:32:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rectangular-inventory-paper-labels-4l-x-2w-red-roll-of-500>: HTTP status code is not handled or not allowed 2026-01-25 06:32:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24x42-4-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 06:32:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-uhmw-polyethylene-sheet-96-l-x-48-w-x-1-2-thick-black-antistatic>: HTTP status code is not handled or not allowed 2026-01-25 06:33:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kdd480-versatile-pneumatic-lockout-plastic>: HTTP status code is not handled or not allowed 2026-01-25 06:33:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/34x25-liter-under-the-counter-corrosive-cabinet-fully-lined-35w-x-22d-x-35-1-2h>: HTTP status code is not handled or not allowed 2026-01-25 06:33:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-x-1075-4-mil-anti-static-poly-tubing>: HTTP status code is not handled or not allowed 2026-01-25 06:33:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/impact-socket-3-4-drive-1-1-8-6-point-short>: HTTP status code is not handled or not allowed 2026-01-25 06:33:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-foam-epdm-sheet-24-l-x-12-w-x-1-thick-black>: HTTP status code is not handled or not allowed 2026-01-25 06:33:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelate-post-86in-h-leveler-installed-pack-of-four>: HTTP status code is not handled or not allowed 2026-01-25 06:33:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-barstool-24-h-silver-4-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:33:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-foam-epdm-roll-480-l-x-13-w-x-1-4-thick-black>: HTTP status code is not handled or not allowed 2026-01-25 06:33:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wrestling-marial-arts-roll-mats-24-x-6-x-2-black-121r-bk>: HTTP status code is not handled or not allowed 2026-01-25 06:33:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mlt405ltp-lockout-tag-danger-do-not-operate-hs-laminate-25pack>: HTTP status code is not handled or not allowed 2026-01-25 06:33:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-8-polypropylene-standard-series-clamp-cushion>: HTTP status code is not handled or not allowed 2026-01-25 06:33:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wrestling-marial-arts-roll-mats-36-x-6-x-1-5-8-royal-113r-ry>: HTTP status code is not handled or not allowed 2026-01-25 06:33:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/melc114vp-danger-sign-high-voltage-14w-x-10h-plastic>: HTTP status code is not handled or not allowed 2026-01-25 06:33:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-dc-3600c-cyclone-style-dust-collector-3600-cfm-5hp-3ph-220v>: HTTP status code is not handled or not allowed 2026-01-25 06:33:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-enclosed-wire-stockpicker-truck-dolly-base-36-x-24-x-69-1000-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 06:33:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 75918 pages (at 97 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:33:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-strapping-1-2-inches-w-x-0-point-015-inches-x-200-feet>: HTTP status code is not handled or not allowed 2026-01-25 06:33:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8flexo-pet-general-purpose-sleeving-dia-125-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:33:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6x8-6-mil-anti-static-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 06:33:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/msmk251vp-danger-sign-flammable-material-no-smoking-or-open-flames-10w-x-7h-plastic>: HTTP status code is not handled or not allowed 2026-01-25 06:33:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mcsp007vs-danger-sign-permit-required-confined-space-10w-x-7h-adhesive-vinyl>: HTTP status code is not handled or not allowed 2026-01-25 06:33:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-8-polypropylene-heavy-rubber-insert-grommet>: HTTP status code is not handled or not allowed 2026-01-25 06:33:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-wire-security-truck-36x24x70-with-dolly-base-1600-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:33:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-impact-socket-set-1-2-drive-6-point-deep-26-piece>: HTTP status code is not handled or not allowed 2026-01-25 06:33:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-bar-24-l-x-4-w-x-1-16-thick-clear-scratch-and-uv-resistant>: HTTP status code is not handled or not allowed 2026-01-25 06:33:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-rack-guard-back-panel-with-flush-mount-clip-120l-x-1-14w-x-48h>: HTTP status code is not handled or not allowed 2026-01-25 06:33:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20x30-4-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 06:33:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polyester-film-60-l-x-36-w-x-0-001-thick-clear>: HTTP status code is not handled or not allowed 2026-01-25 06:33:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-lumbar-back-support-cushion-14-x-13-navy-555-7300-2400hs>: HTTP status code is not handled or not allowed 2026-01-25 06:33:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16oz-ball-pein-hammer-w-fiberglass-handle>: HTTP status code is not handled or not allowed 2026-01-25 06:33:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4000-semi-outdoor-led-mass-notification-sign-full-color-24vdc-4080rgb>: HTTP status code is not handled or not allowed 2026-01-25 06:33:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-pvc-type-1-sheet-24-l-x-24-w-x-3-8-thick-clear>: HTTP status code is not handled or not allowed 2026-01-25 06:33:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-x3-x18-2-mil-gusseted-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 06:33:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4flexo-pet-general-purpose-sleeving-dia-1000-white>: HTTP status code is not handled or not allowed 2026-01-25 06:33:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/45-degree-single-socket-tee-3>: HTTP status code is not handled or not allowed 2026-01-25 06:33:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mdt246ctp-danger-out-of-service-tag-pf-cardstock-25pack>: HTTP status code is not handled or not allowed 2026-01-25 06:33:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mcsp140vs-danger-sign-confined-space-authorized-personnel-only-10w-x-7h-adhesive-vinyl>: HTTP status code is not handled or not allowed 2026-01-25 06:33:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kobi-tall-planter-20-l-x-20-w-x-38-h-square-graphite-grey>: HTTP status code is not handled or not allowed 2026-01-25 06:33:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9000i-rgb-indoor-led-message-sign-full-color-9660rgb>: HTTP status code is not handled or not allowed 2026-01-25 06:33:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gloveworks-powderfree-industrial-grade-latex-gloves-xl-100box-10-boxcs>: HTTP status code is not handled or not allowed 2026-01-25 06:33:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/desk-shell-48w-x-30d-x-29-half-h-artisan-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:33:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mdt623ptp-caution-tag-caution-pf-cardstock-25pack>: HTTP status code is not handled or not allowed 2026-01-25 06:33:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manual-stencil-machine-for-3-4-characters-model-s>: HTTP status code is not handled or not allowed 2026-01-25 06:33:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-bar-24-l-x-2-w-x-3-8-thick-clear-antistatic>: HTTP status code is not handled or not allowed 2026-01-25 06:33:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-2flexo-pet-general-purpose-sleeving-dia-40-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:33:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/malleable-plug-3>: HTTP status code is not handled or not allowed 2026-01-25 06:33:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/impact-socket-1-2-drive-1-6-point-deep>: HTTP status code is not handled or not allowed 2026-01-25 06:33:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6x6-1-5-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 06:33:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4x12-1-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 06:33:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-1-2-x-10-7-8-x-8-lewisbins-divider-box-light-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:33:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lewisbins-1-7-8-h-short-divider>: HTTP status code is not handled or not allowed 2026-01-25 06:33:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2x-premium-blacktop-crack-filler-1-gallon-jug>: HTTP status code is not handled or not allowed 2026-01-25 06:33:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-sheet-48-l-x-24-w-x-1-16-thick-clear-scratch-and-uv-resistant>: HTTP status code is not handled or not allowed 2026-01-25 06:33:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lewisbins-4-3-8-h-short-divider>: HTTP status code is not handled or not allowed 2026-01-25 06:33:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnetic-strips-2w-x-7-8h-25-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:33:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5x7-1-5-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 06:33:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5x8-1-5-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 06:33:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mecr620vp-caution-sign-low-headroom-10w-x-7h-plastic>: HTTP status code is not handled or not allowed 2026-01-25 06:33:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commercial-fire-safe-electronic-lock-25-7-16-x-23-7-16-x-39-13-16light-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:33:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g15l-175-ff-20hp-oil-injected-rotary-screw-132-gal-tank-mnt-175psi-3ph-208-230-460v>: HTTP status code is not handled or not allowed 2026-01-25 06:33:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g22-150-ff-30hp-oil-injected-rotary-screw-132-gal-tank-mnt-150psi-3ph-208-230-460v>: HTTP status code is not handled or not allowed 2026-01-25 06:33:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pj1696t-16-parallelogram-jointer-with-armorglide--helical-head-cutter-3ph-230v>: HTTP status code is not handled or not allowed 2026-01-25 06:33:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3x3-1-5-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 06:33:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8x18-1-5-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 06:33:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8x12-1-5-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 06:33:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-bar-12-l-x-1-w-x-1-16-thick-clear-scratch-and-uv-resistant>: HTTP status code is not handled or not allowed 2026-01-25 06:33:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bronco-dome-lid-for-32-gallon-black>: HTTP status code is not handled or not allowed 2026-01-25 06:33:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g22-175-ap-30hp-oil-injected-rotary-screw-132-gal-tank-mnt-175psi-3ph-208-230-460v>: HTTP status code is not handled or not allowed 2026-01-25 06:33:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flo-pac-grout-brush-with-nylon-bristle-8-1-8-black>: HTTP status code is not handled or not allowed 2026-01-25 06:33:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cpvc-bar-48-l-x-2-w-x-1-2-thick-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:33:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-led-pole-mount-radar-feedback-sign-solar-powered-26w-x-8-14d-x-26h-blackwhite>: HTTP status code is not handled or not allowed 2026-01-25 06:33:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9x24-1-5-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 06:33:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flo-pac-dust-mop-handle-60-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:33:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10x12-1-5-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 06:33:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trimline-waste-container-23-gal-beige>: HTTP status code is not handled or not allowed 2026-01-25 06:33:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machined-type-needle-roller-bearing-inch-double-sealed-2-bore-2-9-16-od>: HTTP status code is not handled or not allowed 2026-01-25 06:33:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mcsp116vp-danger-sign-confined-space-10w-x-7h-plastic>: HTTP status code is not handled or not allowed 2026-01-25 06:33:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-standard-railing-flange-1>: HTTP status code is not handled or not allowed 2026-01-25 06:33:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coxreels-tsh-n-350-hv-3-8inx50ft-300psi-supreme-duty-spring-retractable-low-pressure-safety-hose-reel>: HTTP status code is not handled or not allowed 2026-01-25 06:33:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sparta-hi-lo-polypropylene-scrub-brush-10-red>: HTTP status code is not handled or not allowed 2026-01-25 06:33:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-x-11-open-end-static-shielding-bags>: HTTP status code is not handled or not allowed 2026-01-25 06:33:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flo-pac-straight-red-gum-rubber-floor-squeegee-heavy-duty-steel-frame-36>: HTTP status code is not handled or not allowed 2026-01-25 06:33:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-freestanding-2-panel-corner-room-divider-with-partial-window-48-x-73-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:33:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12x12-1-5-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 06:33:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quat-stat153-5-disinfectant-lavender-scent-2-l-bottle-4case>: HTTP status code is not handled or not allowed 2026-01-25 06:33:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/af79-acid-free-bathroom-cleaner-concentrate-ocean-breeze-scent-2-l-bottle-4case>: HTTP status code is not handled or not allowed 2026-01-25 06:33:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sure-bet153-ii-foaming-disinfectant-citrus-scent-676-oz-bottle-4case>: HTTP status code is not handled or not allowed 2026-01-25 06:33:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coxreels-ez-sh-450-hv-1-2inx50ft-300psi-ez-coil-safety-heavy-duty-spring-retractable-low-pressure>: HTTP status code is not handled or not allowed 2026-01-25 06:33:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/freestanding-2-panel-corner-room-divider-36-x-72-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:33:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sparta-spectrum-bottle-brush-16-long-white>: HTTP status code is not handled or not allowed 2026-01-25 06:33:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-sheet-32-l-x-16-w-x-3-32-thick-clear>: HTTP status code is not handled or not allowed 2026-01-25 06:33:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/infusion-series-tea-and-coffee-brewers-tall-booster>: HTTP status code is not handled or not allowed 2026-01-25 06:33:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/push-all-purpose-cleaner--odor-eliminator-lemon--sage-scent-32-oz-bottle-12case>: HTTP status code is not handled or not allowed 2026-01-25 06:33:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4x6-1-5-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 06:33:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uvi-stretch-film-black-with-handles-1000l-x-20w-x-80-gauge-4-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:33:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/liquid-coffee-ambient-dispenser-344000026>: HTTP status code is not handled or not allowed 2026-01-25 06:33:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mfs2817-warning-forklift-traffic-floor-sign-17-diameter-adhesive-vinyl-1>: HTTP status code is not handled or not allowed 2026-01-25 06:33:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/easy-pour-coffee-ss-bottom-decanters-64-oz-regular-6-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:33:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acudor-12x12-aluminum-diamond-plate-floor-door-no-hinge>: HTTP status code is not handled or not allowed 2026-01-25 06:33:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20ml-amber-vials-in-a-box-rubber-lined-caps-case-of-72>: HTTP status code is not handled or not allowed 2026-01-25 06:34:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-x-3-open-end-static-shielding-bags>: HTTP status code is not handled or not allowed 2026-01-25 06:34:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/narrow-beverage-dispenser-35-gal>: HTTP status code is not handled or not allowed 2026-01-25 06:34:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/narrow-beverage-dispenser-4-gal>: HTTP status code is not handled or not allowed 2026-01-25 06:34:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cardboard-corrugated-boxes-11-x-11-x-6-200lb-testect32-25-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:34:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jat-596-17-piece-spray-gun-cleaning-kit>: HTTP status code is not handled or not allowed 2026-01-25 06:34:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/orthopedic-foam-bed-wedge-pillow-10-x-24-x-24-blue-802-8027-0100>: HTTP status code is not handled or not allowed 2026-01-25 06:34:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carbon-pile-battery-tester-6034>: HTTP status code is not handled or not allowed 2026-01-25 06:34:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-airpot-racks-univ-2-apr-35728>: HTTP status code is not handled or not allowed 2026-01-25 06:34:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/Sunex-16-speed-floor-drill-press>: HTTP status code is not handled or not allowed 2026-01-25 06:34:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pistol-grip-needle-scaler-sx246-1-4-npt>: HTTP status code is not handled or not allowed 2026-01-25 06:34:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/Sunex-12-ton-shop-press>: HTTP status code is not handled or not allowed 2026-01-25 06:34:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cpvc-bar-24-l-x-1-2-w-x-1-2-thick-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:34:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/Sunex-3-ton-service-jack>: HTTP status code is not handled or not allowed 2026-01-25 06:34:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oasis-pg8ebqsl-bi-level-drinking-fountain-with-quasar-electronic-bottle-filler-greystone>: HTTP status code is not handled or not allowed 2026-01-25 06:34:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 76016 pages (at 98 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:34:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hse-portable-electric-hot-water-pressure-washer-4000-psi-35-gpm-38-hose-230v>: HTTP status code is not handled or not allowed 2026-01-25 06:34:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/saf-t-stor-cylinder-cabinet-29-inch-w-x-29-inch-d-x-66-inch-h-capacity-8-horizontal-galvanized>: HTTP status code is not handled or not allowed 2026-01-25 06:34:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5400063-round-file-6-smooth-cut>: HTTP status code is not handled or not allowed 2026-01-25 06:34:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-polycarbonate-bar-48-l-x-4-w-x-1-8-thick-clear-antistatic>: HTTP status code is not handled or not allowed 2026-01-25 06:34:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rechargeable-cordless-stick-vacuum-8-cleaning-width>: HTTP status code is not handled or not allowed 2026-01-25 06:34:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pswl10-pipeline-10-l-wall-mount-faceout>: HTTP status code is not handled or not allowed 2026-01-25 06:34:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5x10-3-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 06:34:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-pump-kit-p7510>: HTTP status code is not handled or not allowed 2026-01-25 06:34:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/freestanding-3-panel-corner-room-divider-with-partial-window-48-x-60-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:34:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-emergency-light-dual-head-self-diagnostic>: HTTP status code is not handled or not allowed 2026-01-25 06:34:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speakman-se-911-pr-replacement-stay-open-ball-valve>: HTTP status code is not handled or not allowed 2026-01-25 06:34:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/35305-one-touch-stainless-steel-liquid-dispenser-square-bottle-natural-semi-opaque-white-4-oz>: HTTP status code is not handled or not allowed 2026-01-25 06:34:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/35510-pure-touch-stainless-steel-liquid-dispenser-square-natural-white-semi-opaque-8-oz>: HTTP status code is not handled or not allowed 2026-01-25 06:34:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/front-loading-deposit-safe-electronic-lock-14-w-x-14-d-x-20-h-black>: HTTP status code is not handled or not allowed 2026-01-25 06:34:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26x36-2-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 06:34:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30x30-2-mil-flat-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 06:34:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/saf-t-grip-cutting-boards-cbg182412wh>: HTTP status code is not handled or not allowed 2026-01-25 06:34:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ez-double-handle-strap-winder>: HTTP status code is not handled or not allowed 2026-01-25 06:34:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guardair-48-75xt-alum-ext-alloy-nozzle>: HTTP status code is not handled or not allowed 2026-01-25 06:34:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slide-check-racks-ck6512a>: HTTP status code is not handled or not allowed 2026-01-25 06:34:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acetal-plastic-spring-style-tubing-clamps-for-tubing-up-to-od-3pk>: HTTP status code is not handled or not allowed 2026-01-25 06:34:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guardair-ultra-whisper-jet-36-alum-ext-alum-nozzle-long-trigger>: HTTP status code is not handled or not allowed 2026-01-25 06:34:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guardair-air-miser-safety-air-gun>: HTTP status code is not handled or not allowed 2026-01-25 06:34:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-floor-scrubber-replacement-pads-10-3-4-x-5-1-4-green-1260>: HTTP status code is not handled or not allowed 2026-01-25 06:34:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-cpvc-bar-12-l-x-3-w-x-1-8-thick-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:34:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guardair-lazer-pistol-grip-36-alum-ext-alum-nozzle>: HTTP status code is not handled or not allowed 2026-01-25 06:34:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/saf-t-grip-board-mate-cbm1016>: HTTP status code is not handled or not allowed 2026-01-25 06:34:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-freestanding-2-panel-corner-room-divider-48-x-61-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:34:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s3636s-stainless-steel-wire-shelf-36w-x-36d>: HTTP status code is not handled or not allowed 2026-01-25 06:34:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-nylon-6-12-bar-24-l-x-1-w-x-1-8-thick-white>: HTTP status code is not handled or not allowed 2026-01-25 06:34:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-hdpe-sheet-96-l-x-54-w-x-1-2-thick-black-marine>: HTTP status code is not handled or not allowed 2026-01-25 06:34:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plus-astm-like-liquid-in-glass-thermometer-14f-wax-meltingpoint-100-to-180f>: HTTP status code is not handled or not allowed 2026-01-25 06:34:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyethylene-two-bottle-carrier-9-x-4-x-15>: HTTP status code is not handled or not allowed 2026-01-25 06:34:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-freestanding-3-panel-corner-room-divider-24-x-61-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:34:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classic-entrance-mat-waffle-pattern-3-8-thick-3-x-5-charcoal>: HTTP status code is not handled or not allowed 2026-01-25 06:34:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/79-91-degree-api-astm-9h-hydrometer>: HTTP status code is not handled or not allowed 2026-01-25 06:34:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/entrance-mat-3-8-thick-3-x-10-blue-black>: HTTP status code is not handled or not allowed 2026-01-25 06:34:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-x-3-x-37-square-mailing-tubes>: HTTP status code is not handled or not allowed 2026-01-25 06:34:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-nylon-6-6-bar-48-l-x-4-w-x-3-8-thick-black-mds-filled>: HTTP status code is not handled or not allowed 2026-01-25 06:34:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/prison-bench-6-foot-composite-lumber-seating-without-backrest-cedar>: HTTP status code is not handled or not allowed 2026-01-25 06:34:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40626-s-m-fishing-vest-mesh-tan-small-medium>: HTTP status code is not handled or not allowed 2026-01-25 06:34:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stepped-trays-w-indiv-notched-lids-b4702inl>: HTTP status code is not handled or not allowed 2026-01-25 06:34:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shell-shock-hand-cleaner-gallon-bottle-4-case-1041532>: HTTP status code is not handled or not allowed 2026-01-25 06:34:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-led-aluminum-workbench-light-kit>: HTTP status code is not handled or not allowed 2026-01-25 06:34:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-led-aluminum-workbench-light-kit>: HTTP status code is not handled or not allowed 2026-01-25 06:34:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-x-30-kra-tubes-with-caps-case>: HTTP status code is not handled or not allowed 2026-01-25 06:34:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1000-2000-specific-gravity-and-0-70-degree-baume-dual-scale-hydrometer>: HTTP status code is not handled or not allowed 2026-01-25 06:34:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bt-1220-all-purpose-tarp-12-x-20-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:34:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-rod-24-l-x-2-dia-white>: HTTP status code is not handled or not allowed 2026-01-25 06:34:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acrylic-leveling-table-8-x-8-x-3-8>: HTTP status code is not handled or not allowed 2026-01-25 06:34:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guardair-rubber-tip-safety-air-gun>: HTTP status code is not handled or not allowed 2026-01-25 06:34:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-partition-panel-57-3-4in-w-x-58in-h-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:34:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-x-36-white-mailing-tubes-with-caps-case>: HTTP status code is not handled or not allowed 2026-01-25 06:34:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0-70-degree-baume-hydrometer>: HTTP status code is not handled or not allowed 2026-01-25 06:34:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-anti-static-air-foam-rolls-18w-x-250l-x-14-thick-pink-4-rolls>: HTTP status code is not handled or not allowed 2026-01-25 06:34:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sn-20005-fire-resistant-safety-netting-4-ft-x-150-ft-orange-1-roll>: HTTP status code is not handled or not allowed 2026-01-25 06:34:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-hdpe-sheet-6-l-x-6-w-x-1-2-thick-black-marine>: HTTP status code is not handled or not allowed 2026-01-25 06:34:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roll-out-extra-shelving-add-on-3-shelf-48x48x62>: HTTP status code is not handled or not allowed 2026-01-25 06:34:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wet-zone-shadow-board-yellow-white-68-x-30-aluminum-sb121al>: HTTP status code is not handled or not allowed 2026-01-25 06:34:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-rod-24-l-x-2-1-4-dia-white>: HTTP status code is not handled or not allowed 2026-01-25 06:34:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dry-zone-shadow-board-green-blue-68-x-30-pro-series-acrylic-sb135fg>: HTTP status code is not handled or not allowed 2026-01-25 06:34:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1450-1500-specific-gravity-astm-134h-hydrometer-for-heavy-liquids>: HTTP status code is not handled or not allowed 2026-01-25 06:34:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vertical-bar-rack-all-welded>: HTTP status code is not handled or not allowed 2026-01-25 06:34:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/0650-0700-specific-gravity-astm-82h-hydrometer-for-light-liquids>: HTTP status code is not handled or not allowed 2026-01-25 06:34:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guardair-36-75xt-alum-ext-alloy-nozzle>: HTTP status code is not handled or not allowed 2026-01-25 06:34:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-nylon-6-12-bar-24-l-x-4-w-x-1-4-thick-off-white>: HTTP status code is not handled or not allowed 2026-01-25 06:34:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guardair-ultra-whisper-jet-36-alum-ext-alum-nozzle-short-trigger>: HTTP status code is not handled or not allowed 2026-01-25 06:34:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-ton-air-hydraulic-bottle-jack-14475>: HTTP status code is not handled or not allowed 2026-01-25 06:34:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milwaukee-12-pc-sawzall-blade-set>: HTTP status code is not handled or not allowed 2026-01-25 06:34:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-inch-5-tpi-the-ax-sawzall-blade-5-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:34:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/koch-filter-e-pak-950-6-pocket-paint-booth-exhaust-final-filter-24-in-x-24-in-x-15-in>: HTTP status code is not handled or not allowed 2026-01-25 06:34:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-cu-yd-orange-medium-duty-hopper-986661>: HTTP status code is not handled or not allowed 2026-01-25 06:34:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluke-289-fvf-true-rms-logging-multimeter-combo-kit-w-trendcapture>: HTTP status code is not handled or not allowed 2026-01-25 06:34:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-air-hydraulic-jack-25-10-ton-b25-2>: HTTP status code is not handled or not allowed 2026-01-25 06:34:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/air-hydraulic-jack-25-ton-min-height-14-2-max-height-24-1-25-1h>: HTTP status code is not handled or not allowed 2026-01-25 06:34:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-cu-yd-orange-medium-duty-hopper-986673>: HTTP status code is not handled or not allowed 2026-01-25 06:34:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-replacement-valve-white>: HTTP status code is not handled or not allowed 2026-01-25 06:34:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/strong-hold-heavy-duty-maintenance-storage-cabinet-60x24x78-269892>: HTTP status code is not handled or not allowed 2026-01-25 06:34:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pennsylvania-7600-series-hdhigh-resolution-counting-scale-10-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:34:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anti-static-foam-shippers-8w-x-14l-x-2-34d-pinkwhite-24pack>: HTTP status code is not handled or not allowed 2026-01-25 06:34:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1500-1999-wired-inventory-tag-2-part-carbon-style>: HTTP status code is not handled or not allowed 2026-01-25 06:34:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/calibrated-electronic-thermometer-with-waterproof-sensor-50-200c-58-392f>: HTTP status code is not handled or not allowed 2026-01-25 06:34:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coplin-staining-jar-10-slide-capacity-53cm-d-opening-plastic>: HTTP status code is not handled or not allowed 2026-01-25 06:34:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-hdpe-bar-48-l-x-3-w-x-1-4-thick-black-marine>: HTTP status code is not handled or not allowed 2026-01-25 06:35:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hon-10500-l-or-u-workstation-right-pedestal-desk-72w-x-36d-x-29-1-2h-mahogany>: HTTP status code is not handled or not allowed 2026-01-25 06:35:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silver-cut-off-wheel-4-1-2-x-094-x-7-8-t27-ceramic-grain>: HTTP status code is not handled or not allowed 2026-01-25 06:35:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pneumatic-fluid-extractor-2-3-gallon-mv7300>: HTTP status code is not handled or not allowed 2026-01-25 06:35:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clay-desiccant-bags-3w-x-4l-x-14d-white-1300-packs>: HTTP status code is not handled or not allowed 2026-01-25 06:35:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluke-c116-carrying-case-polyester-blk-yw>: HTTP status code is not handled or not allowed 2026-01-25 06:35:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/realiti-workcenter-stationary-includes-stainless-steel-top-monaco-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:35:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cleanware-microscope-optics-cleaning-kit>: HTTP status code is not handled or not allowed 2026-01-25 06:35:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-air-foam-rolls-36w-x-1250l-x-116-thick-white-2-rolls>: HTTP status code is not handled or not allowed 2026-01-25 06:35:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/science-lab-table-with-book-compartments-black-laminate-top-24wx48lx36h>: HTTP status code is not handled or not allowed 2026-01-25 06:35:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crinkle-paper-10-lbs-teal>: HTTP status code is not handled or not allowed 2026-01-25 06:35:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-1-4-x-3-1-8-8-gray-shipping-tag>: HTTP status code is not handled or not allowed 2026-01-25 06:35:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silhouette-banquet-stacking-chair-fabric-midnight-blue-9300-series>: HTTP status code is not handled or not allowed 2026-01-25 06:35:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honda-trx-rancher-350cc-atv-battery-2000-2006-12-amps-12v-b-terminals>: HTTP status code is not handled or not allowed 2026-01-25 06:35:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-edge-2w-10l-2-ply-endless-poly-flat-web-sling>: HTTP status code is not handled or not allowed 2026-01-25 06:35:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-3-4-x-2-7-8-7-light-green-shipping-tag>: HTTP status code is not handled or not allowed 2026-01-25 06:35:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-1-4-x-3-1-8-8-dark-blue-shipping-tag>: HTTP status code is not handled or not allowed 2026-01-25 06:35:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 76117 pages (at 101 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:35:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bluetooth-stainless-steel-stem-thermometer-50-300c-58-572f>: HTTP status code is not handled or not allowed 2026-01-25 06:35:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-ptfe-tube-36-l-x-3-4-id-x-1-1-2-od-white>: HTTP status code is not handled or not allowed 2026-01-25 06:35:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dispensing-drop-185ml-6oz-polyethylene-bottles-24mm-closure-12pk>: HTTP status code is not handled or not allowed 2026-01-25 06:35:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h-b-frio-temp-blood-bank-verification-thermometer-with-individual-calibration-report-5-to-20c>: HTTP status code is not handled or not allowed 2026-01-25 06:35:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-hdpe-bar-36-l-x-2-w-x-1-4-thick-white-marine>: HTTP status code is not handled or not allowed 2026-01-25 06:35:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-hdpe-bar-36-l-x-4-w-x-1-thick-white-marine>: HTTP status code is not handled or not allowed 2026-01-25 06:35:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48h-single-sided-floor-stand-with-64-drawer-interlocking-storage-cabinet>: HTTP status code is not handled or not allowed 2026-01-25 06:35:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pandemic-flu-kit-m-l>: HTTP status code is not handled or not allowed 2026-01-25 06:35:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-epoxy-wire-shelving-add-on-14425gz-42w-x-14d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 06:35:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-hdpe-bar-36-l-x-4-w-x-1-2-thick-white-marine>: HTTP status code is not handled or not allowed 2026-01-25 06:35:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-rod-24-l-x-2-3-4-dia-white>: HTTP status code is not handled or not allowed 2026-01-25 06:35:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-epoxy-wire-shelving-21487gz-48w-x-21d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 06:35:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/green-epoxy-wire-shelving-21547gz-54w-x-21d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 06:35:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-60-x-30-adj-height-workbench-w-drawer-black-shop-top-safety-edge-top>: HTTP status code is not handled or not allowed 2026-01-25 06:35:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluorescent-ruler-metric-english-scale-15cm-6>: HTTP status code is not handled or not allowed 2026-01-25 06:35:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/galileo-thermometer-18-to-30c-64-to-88f-7-spheres-17>: HTTP status code is not handled or not allowed 2026-01-25 06:35:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hoover-paper-vacuum-bag-type-y-uprights-windtunnel-tempo-type-z-uprights>: HTTP status code is not handled or not allowed 2026-01-25 06:35:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergopet-filters-12pk>: HTTP status code is not handled or not allowed 2026-01-25 06:35:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-rack-24x36x60-with-48-red-drawers>: HTTP status code is not handled or not allowed 2026-01-25 06:35:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-paper-vacuum-bag-backpack-6-qt>: HTTP status code is not handled or not allowed 2026-01-25 06:35:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-60-x-30-adj-height-workbench-w-drawer-riser-bk-plastic-laminate-safety-top>: HTTP status code is not handled or not allowed 2026-01-25 06:35:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-rod-60-l-x-1-3-8-dia-white>: HTTP status code is not handled or not allowed 2026-01-25 06:35:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-1-4-x-1-5-8-2-orange-fluorescent-wired-tag>: HTTP status code is not handled or not allowed 2026-01-25 06:35:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-3-4-x-1-7-8-3-pink-fluorescent-wired-tag>: HTTP status code is not handled or not allowed 2026-01-25 06:35:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-72-x-30-adj-height-workbench-w-drawer-riser-black-steel-square-edge-top>: HTTP status code is not handled or not allowed 2026-01-25 06:35:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-sided-rack-12x36x60-with-24-blue-drawers>: HTTP status code is not handled or not allowed 2026-01-25 06:35:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kendall-howard-advanced-classroom-training-table-24-x-60-folkstone>: HTTP status code is not handled or not allowed 2026-01-25 06:35:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-3-4-x-1-3-8-1-manila-pre-wired-tag-1>: HTTP status code is not handled or not allowed 2026-01-25 06:35:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interlocking-storage-cabinet-11-3-4w-11-3-8d-x-11h-14-ivory-drawers>: HTTP status code is not handled or not allowed 2026-01-25 06:35:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h-b-easy-read-general-purpose-liquid-in-glass-thermometer-10-to-110c-total-immersion>: HTTP status code is not handled or not allowed 2026-01-25 06:35:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-plastic-bin-16-1-4l-x-11-3-4w-x-22h-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:35:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/closed-ibc-transfer-system-8-gpm-pump-w-12-hose-automatic-nozzle>: HTTP status code is not handled or not allowed 2026-01-25 06:35:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/euro-drawer-5-9-16w-x-11-5-8d-x-2-5-8h-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:35:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-rack-24x36x60-with-144-yellow-drawers>: HTTP status code is not handled or not allowed 2026-01-25 06:35:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-rod-96-l-x-1-1-4-dia-black>: HTTP status code is not handled or not allowed 2026-01-25 06:35:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storage-container-with-lid-64-1-2-x-12-x-8-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:35:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storage-container-with-lid-57-x-18-x-7-3-4-red>: HTTP status code is not handled or not allowed 2026-01-25 06:35:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-hdpe-bar-48-l-x-1-w-x-1-4-thick-black-marine>: HTTP status code is not handled or not allowed 2026-01-25 06:35:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-hdpe-sheet-12-l-x-12-w-x-1-thick-white-marine>: HTTP status code is not handled or not allowed 2026-01-25 06:35:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/liquid-in-glass-aquarium-thermometer-10-to-40c-20-to-100f-organic-liquid-fill>: HTTP status code is not handled or not allowed 2026-01-25 06:35:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-hdpe-bar-12-l-x-6-w-x-1-thick-black-marine>: HTTP status code is not handled or not allowed 2026-01-25 06:35:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ghs-labeled-safety-vented-ethyl-acetate-wash-bottles-500ml-16oz-pe-w-green-pp-cap-4pk>: HTTP status code is not handled or not allowed 2026-01-25 06:35:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tempco-channel-strip-240v-t1-14l-500w>: HTTP status code is not handled or not allowed 2026-01-25 06:35:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lab-aire-ii-double-sided-non-electric-benchtop-glassware-dryer-2-tier-1475-x-10-x-154>: HTTP status code is not handled or not allowed 2026-01-25 06:35:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-1-2-x-7-1-2-clear-face-document-envelopes>: HTTP status code is not handled or not allowed 2026-01-25 06:35:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-round-bistro-table-2-barstool-set-designer-white-table-top-with-white-stools>: HTTP status code is not handled or not allowed 2026-01-25 06:35:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tempco-finned-strip-120v-t4-14l-750w>: HTTP status code is not handled or not allowed 2026-01-25 06:35:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-square-bistro-table-4-barstool-set-maple-table-top-with-black-stools>: HTTP status code is not handled or not allowed 2026-01-25 06:35:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buyers-stainless-steel-underbody-box-w-t-handle-black-24x24x60>: HTTP status code is not handled or not allowed 2026-01-25 06:35:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buyers-steel-underbody-box-w-ss-door-black-24x24x60>: HTTP status code is not handled or not allowed 2026-01-25 06:35:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-x-4-2-mil-reclosable-poly-bags-w-hang-hole>: HTTP status code is not handled or not allowed 2026-01-25 06:35:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plus-astm-like-liquid-in-glass-thermometer-16c-high-softening-point-30-to-200c>: HTTP status code is not handled or not allowed 2026-01-25 06:35:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tempco-brasscopper-immersion-1-14-9250d-2000w-120v-t-stat>: HTTP status code is not handled or not allowed 2026-01-25 06:35:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/armless-modular-loveseat-leather-white-hercules-alon-series>: HTTP status code is not handled or not allowed 2026-01-25 06:35:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-ptfe-bar-24-l-x-1-2-w-x-1-2-thick-off-white-glass-filled>: HTTP status code is not handled or not allowed 2026-01-25 06:35:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5x5-2-mil-white-block-reclosable-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 06:35:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/113177-79-pc-deluxe-roadside-emergency-kit-with-road-flares>: HTTP status code is not handled or not allowed 2026-01-25 06:35:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mro-industrial-enamel-20-oz-safety-green-6-cans-case-620-1452>: HTTP status code is not handled or not allowed 2026-01-25 06:35:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buyers-aluminum-underbody-box-w-barn-door-18x18x30>: HTTP status code is not handled or not allowed 2026-01-25 06:35:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buyers-stainless-steel-underbody-box-w-t-handle-polished-gray-18x18x24>: HTTP status code is not handled or not allowed 2026-01-25 06:35:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plain-spinbar-magnetic-stirring-bar-76-x-12mm>: HTTP status code is not handled or not allowed 2026-01-25 06:35:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/buyers-stainless-steel-underbody-box-w-t-handle-black-24x24x36>: HTTP status code is not handled or not allowed 2026-01-25 06:35:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-ptfe-sheet-48-l-x-48-w-x-1-8-thick-white>: HTTP status code is not handled or not allowed 2026-01-25 06:35:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-x-10-2-mil-reclosable-poly-bags>: HTTP status code is not handled or not allowed 2026-01-25 06:35:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/draft-direct-draw-dispensers-dd-59w-dd58c>: HTTP status code is not handled or not allowed 2026-01-25 06:35:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lab-wall-cabinet-35w-x-13d-x-30h-2-glass-steel-encased-doors-2-adj-shelves-champagne>: HTTP status code is not handled or not allowed 2026-01-25 06:35:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-security-shelving-unit-poly-z-brite-24-w-x-48-l-x-66-h>: HTTP status code is not handled or not allowed 2026-01-25 06:35:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-ptfe-bar-36-l-x-2-w-x-3-8-thick-white>: HTTP status code is not handled or not allowed 2026-01-25 06:35:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-48-l-x-1-w-x-3-8-thick-white>: HTTP status code is not handled or not allowed 2026-01-25 06:35:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-36-l-x-6-w-x-2-3-4-thick-black>: HTTP status code is not handled or not allowed 2026-01-25 06:35:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/optional-pneumatic-wheel-kit-steel-yard-ramps-forkli-dock-ramps>: HTTP status code is not handled or not allowed 2026-01-25 06:35:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-36-l-x-4-w-x-1-4-thick-white>: HTTP status code is not handled or not allowed 2026-01-25 06:35:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-48-l-x-2-w-x-1-1-2-thick-black>: HTTP status code is not handled or not allowed 2026-01-25 06:35:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plus-precision-liquid-in-glass-thermometer-1-to-51c-76mm-immersion-organic-liquid-fill>: HTTP status code is not handled or not allowed 2026-01-25 06:35:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-wire-shelves-utility-cart-chrome-18-w-x-42-l-x-39-h>: HTTP status code is not handled or not allowed 2026-01-25 06:35:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plus-astm-s66f-03-individually-calibrated-liquid-in-glass-thermometer-167-221f>: HTTP status code is not handled or not allowed 2026-01-25 06:35:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-wire-shelves-utility-cart-chrome-21-w-x-36-l-x-39-h>: HTTP status code is not handled or not allowed 2026-01-25 06:35:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-wall-mount-wire-shelving-24w-x-18d-x-54h-3-shelf-starter-poly-green>: HTTP status code is not handled or not allowed 2026-01-25 06:35:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-wire-shelf-platform-poly-z-brite-18-w-x-36-l-x-14-h>: HTTP status code is not handled or not allowed 2026-01-25 06:35:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plus-calibrated-liquid-in-glass-thermometer-20-to-110c-76mm-immersion>: HTTP status code is not handled or not allowed 2026-01-25 06:35:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reach-in-refrigerator-horizon-glass-half-glass-doors-78w-hr3-1g>: HTTP status code is not handled or not allowed 2026-01-25 06:35:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reach-in-refrigerator-horizon-glass-half-glass-doors-35w-hrp1w-1g>: HTTP status code is not handled or not allowed 2026-01-25 06:35:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-sheet-24-l-x-12-w-x-1-1-2-thick-white>: HTTP status code is not handled or not allowed 2026-01-25 06:35:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-x-30-green-tissue-paper>: HTTP status code is not handled or not allowed 2026-01-25 06:35:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rods-and-clips-3-rods-12-clips-for-solid-shelving-units-84-l>: HTTP status code is not handled or not allowed 2026-01-25 06:36:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reach-in-refrigerator-horizon-solid-half-solid-doors-78w-hrp3-1s>: HTTP status code is not handled or not allowed 2026-01-25 06:36:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reach-in-refrigerator-horizon-glass-half-glass-doors-52w-hr2-1g>: HTTP status code is not handled or not allowed 2026-01-25 06:36:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyethylene-coarse-porous-90-130-micron-filter-plate-for-18-id-buchner-funnels>: HTTP status code is not handled or not allowed 2026-01-25 06:36:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-wall-mount-wire-shelving-42w-x-24d-x-34h-2-shelf-starter-poly-green>: HTTP status code is not handled or not allowed 2026-01-25 06:36:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-12-l-x-2-w-x-1-thick-black>: HTTP status code is not handled or not allowed 2026-01-25 06:36:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-x-18in-us-handheld-stick-flag-with-gold-speartip-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-25 06:36:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-5-tier-72w-x-21d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 06:36:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aed-flat-wall-sign-8-x-10-metal>: HTTP status code is not handled or not allowed 2026-01-25 06:36:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cane-apron-lkapr2>: HTTP status code is not handled or not allowed 2026-01-25 06:36:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-48w-x-30d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 06:36:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-5-tier-60w-x-21d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 06:36:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-sheet-48-l-x-24-w-x-1-thick-white>: HTTP status code is not handled or not allowed 2026-01-25 06:36:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-washer-for-pipette-pump-green-pipettors>: HTTP status code is not handled or not allowed 2026-01-25 06:36:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 76214 pages (at 97 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:36:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-backless-swivel-bar-stool-with-chrome-frame-and-red-seat>: HTTP status code is not handled or not allowed 2026-01-25 06:36:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-5-tier-48w-x-36d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 06:36:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cracked-ice-yellow-retro-chrome-kitchen-chair-with-1-pulled-seat>: HTTP status code is not handled or not allowed 2026-01-25 06:36:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-check-valves-1-4-bore-1-2-to-3-8-taper-ldpe-12pk>: HTTP status code is not handled or not allowed 2026-01-25 06:36:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-48-l-x-4-w-x-5-8-thick-white>: HTTP status code is not handled or not allowed 2026-01-25 06:36:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ge-100es-spherical-plain-bearing-metric>: HTTP status code is not handled or not allowed 2026-01-25 06:36:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/file-dolly-15-x-32>: HTTP status code is not handled or not allowed 2026-01-25 06:36:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-ptfe-sheet-24-l-x-24-w-x-3-4-thick-off-white-glass-filled>: HTTP status code is not handled or not allowed 2026-01-25 06:36:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-24-l-x-5-w-x-1-2-thick-white>: HTTP status code is not handled or not allowed 2026-01-25 06:36:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/riteflow-aluminum-mounted-flowmeter-65mm-scale-size-4>: HTTP status code is not handled or not allowed 2026-01-25 06:36:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/riteflow-aluminum-mounted-flowmeter-65mm-scale-size-5>: HTTP status code is not handled or not allowed 2026-01-25 06:36:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-infrared-heatpad-200-watt-for-use-w-electric-insulated-waterers>: HTTP status code is not handled or not allowed 2026-01-25 06:36:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rotary-pipette-stand-94-places-polypropylene>: HTTP status code is not handled or not allowed 2026-01-25 06:36:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gem-50es-2rs-spherical-plain-bearing-metric-extended-inner-ring-sealed>: HTTP status code is not handled or not allowed 2026-01-25 06:36:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/05-516-powdered-eggs-bucket-144-servings>: HTTP status code is not handled or not allowed 2026-01-25 06:36:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-5-tier-72w-x-30d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 06:36:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-lamp-3-2kw-208v-90-symmetrical-infrared-heater-brown>: HTTP status code is not handled or not allowed 2026-01-25 06:36:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-24-l-x-3-w-x-1-4-thick-black>: HTTP status code is not handled or not allowed 2026-01-25 06:36:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-medium-duty-adjustable-grader-blade-tractor-implement-cat-1>: HTTP status code is not handled or not allowed 2026-01-25 06:36:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-36w-x-24d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 06:36:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gaz-200sa-spherical-plain-thrust-bearing-angular-contact-inch>: HTTP status code is not handled or not allowed 2026-01-25 06:36:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roto-flo-polystyrene-flow-indicator-for-5-16-to-7-16-id-tubing>: HTTP status code is not handled or not allowed 2026-01-25 06:36:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-48-l-x-4-w-x-1-1-2-thick-black>: HTTP status code is not handled or not allowed 2026-01-25 06:36:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leviton-5352-bu-20-amp-125v-heavy-duty-grade-duplex-receptacle-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:36:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commercial-electric-baseboard-convection-heater-72-l-120v>: HTTP status code is not handled or not allowed 2026-01-25 06:36:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poxygrid-100mm-petri-dish-carrying-rack-30-places>: HTTP status code is not handled or not allowed 2026-01-25 06:36:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-36w-x-30d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 06:36:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commercial-fan-forced-wall-heater-with-tamperproof-thermostat-4800w-277v>: HTTP status code is not handled or not allowed 2026-01-25 06:36:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gac-35f-spherical-plain-thrust-bearing-angular-contact-metric>: HTTP status code is not handled or not allowed 2026-01-25 06:36:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-voltage-wall-mounted-thermostats-ut8001>: HTTP status code is not handled or not allowed 2026-01-25 06:36:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-5-tier-42w-x-24d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 06:36:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-1600-1820-specific-gravity-combined-form-thermo-hydrometer>: HTTP status code is not handled or not allowed 2026-01-25 06:36:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/geh-15c-spherical-plain-bearing-metric-heavy-series-maintenance-free>: HTTP status code is not handled or not allowed 2026-01-25 06:36:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-5-tier-60w-x-21d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 06:36:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5120-2560-btu-fan-forced-wall-heaters-with-summer-fan-switch-120v-1>: HTTP status code is not handled or not allowed 2026-01-25 06:36:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/geh-100es-spherical-plain-bearing-metric-heavy-series>: HTTP status code is not handled or not allowed 2026-01-25 06:36:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16380-btu-fan-forced-wall-heater-without-summer-fan-switch-208v-4>: HTTP status code is not handled or not allowed 2026-01-25 06:36:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ge-20es-2rs-spherical-plain-bearing-metric-sealed>: HTTP status code is not handled or not allowed 2026-01-25 06:36:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-48-l-x-3-w-x-1-1-4-thick-white>: HTTP status code is not handled or not allowed 2026-01-25 06:36:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/delrin-acetal-bar-36-l-x-3-w-x-2-thick-white>: HTTP status code is not handled or not allowed 2026-01-25 06:36:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-add-on-unit-24w-x-14d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 06:36:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/solvent-based-paint-pen-markers-5-color-assortment-12pk>: HTTP status code is not handled or not allowed 2026-01-25 06:36:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fs24lvac-compact-vaccine-storage-all-freezer-20c-capable-1-4-cu-ft-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:36:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/180in-wide-complete-5-in-corner-compartments-satin-brushed-ic53660-ss>: HTTP status code is not handled or not allowed 2026-01-25 06:36:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/r-kive-letter-legal-box-12-3-4w-x-16-1-2d-x-10-3-8h-white-green>: HTTP status code is not handled or not allowed 2026-01-25 06:36:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/versatile-storage-unit-36h-x-18d>: HTTP status code is not handled or not allowed 2026-01-25 06:36:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spinbar-rare-earth-teflon-octagon-magnetic-stirring-bar-254-x-095cm-green>: HTTP status code is not handled or not allowed 2026-01-25 06:36:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/liberty-check-and-form-boxes-11w-x-24d-x-5h-white-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:36:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/geg-20es-spherical-plain-bearing-metric-extended-inner-ring>: HTTP status code is not handled or not allowed 2026-01-25 06:36:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gez-200es-2rs-spherical-plain-bearing-inch-sealed>: HTTP status code is not handled or not allowed 2026-01-25 06:36:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/geg-45es-spherical-plain-bearing-metric-extended-inner-ring>: HTTP status code is not handled or not allowed 2026-01-25 06:36:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-general-purpose-polyethylene-tray-with-faucet-21-1-2-x-25-1-2-x-4>: HTTP status code is not handled or not allowed 2026-01-25 06:36:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-general-purpose-polyethylene-tray-without-faucet-16-x-20-x-3>: HTTP status code is not handled or not allowed 2026-01-25 06:36:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/infection-prevention-station-wall-mountable-14-w-x-9-d-x-9-h-clear>: HTTP status code is not handled or not allowed 2026-01-25 06:36:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tray-and-shelf-folding-storage-with-25-assorted-trays>: HTTP status code is not handled or not allowed 2026-01-25 06:36:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acyrlic-refrigerator-medicine-lock-box-17-w-x-8-d-x-11-h-clear>: HTTP status code is not handled or not allowed 2026-01-25 06:36:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-ptfe-rod-12-l-x-4-dia-off-white-glass-filled>: HTTP status code is not handled or not allowed 2026-01-25 06:36:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-compact-chemical-dispenser-joining-piece>: HTTP status code is not handled or not allowed 2026-01-25 06:36:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gez-308es-2rs-spherical-plain-bearing-inch-sealed>: HTTP status code is not handled or not allowed 2026-01-25 06:36:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-5-tier-60w-x-14d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 06:36:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-hard-plastic-scraper-36-handle-5-x-6-blade-red>: HTTP status code is not handled or not allowed 2026-01-25 06:36:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oreck-lowboy-vacuum-kit>: HTTP status code is not handled or not allowed 2026-01-25 06:36:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/type-5000-medium-outdoor-waterproof-case-w-sponge-insert-foam-18-1-2-l-x-14-1-2-w-x7-1-2h-black>: HTTP status code is not handled or not allowed 2026-01-25 06:36:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spinbar-teflon-octagon-magnetic-stirring-bar-254-x-8mm-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:36:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-72w-x-14d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 06:36:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oreck-comvac-disposable-bags>: HTTP status code is not handled or not allowed 2026-01-25 06:36:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9539-16ex-16-inch-axial-explosion-proof-ex-industrial-plastic-blower>: HTTP status code is not handled or not allowed 2026-01-25 06:36:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roller-stand-with-23-to-38-1-2-height-range-1760-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:36:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spinbar-teflon-micro-flea-magnetic-stirring-bar-635-x-3mm-red>: HTTP status code is not handled or not allowed 2026-01-25 06:36:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9650-25-20-inch-diameter-ducting-25-length>: HTTP status code is not handled or not allowed 2026-01-25 06:36:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-sterileware-sampling-spoon-8ml-027oz-sterile-plastic-individually-wrapped-100pk>: HTTP status code is not handled or not allowed 2026-01-25 06:36:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-abs-bar-12-l-x-2-w-x-1-2-thick-black>: HTTP status code is not handled or not allowed 2026-01-25 06:36:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-face-out-w-disc-at-end-semi-gloss-black>: HTTP status code is not handled or not allowed 2026-01-25 06:36:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-way-heavy-duty-straight-arms-w-x-style-base-rectangular-tubing-chrome>: HTTP status code is not handled or not allowed 2026-01-25 06:36:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9600-15-16-inch-diameter-ducting-15-length>: HTTP status code is not handled or not allowed 2026-01-25 06:36:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-wye-y-tubing-connectors-for-1-4-tubing-polypropylene-12pk>: HTTP status code is not handled or not allowed 2026-01-25 06:36:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-square-grid-dump-bin-chrome>: HTTP status code is not handled or not allowed 2026-01-25 06:36:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-oval-flood-light-12-24vdc-6-leds-1492113>: HTTP status code is not handled or not allowed 2026-01-25 06:36:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cable-truck-step-perforated-step-12-inch-wx12-inch-h-black-5231512>: HTTP status code is not handled or not allowed 2026-01-25 06:36:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9548-25-12-inch-axial-explosion-proof-ex-plastic-blower-w-canister-25-ducting>: HTTP status code is not handled or not allowed 2026-01-25 06:36:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hytrol-stationary-floor-support-16w-max-leg-35-5-8h>: HTTP status code is not handled or not allowed 2026-01-25 06:36:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-304-stainless-steel-locker-12w-x-18d-x-78h-single-tier-1-wide-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 06:37:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/injection-molded-styrene-shoe-shelf-clear>: HTTP status code is not handled or not allowed 2026-01-25 06:37:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-304-stainless-steel-locker-18w-x-18d-x-78h-six-tier-3-wide-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 06:37:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/campbell-hausfeld-gr3200-combination-unit-30-gallon-compressor-5000-watt-generator-180a-welder>: HTTP status code is not handled or not allowed 2026-01-25 06:37:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gondola-base-for-grid-panels-black>: HTTP status code is not handled or not allowed 2026-01-25 06:37:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-30w-x-18d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 06:37:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-stainless-steel-wire-shelving-starter-72w-x-21d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 06:37:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-epoxy-wire-shelving-18545b-54w-x-18d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 06:37:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spinbar-teflon-octagon-magnetic-stirring-bar-762-x-127mm-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:37:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-premium-locker-single-tier-18x18x72-3-door-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:37:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-premium-locker-single-tier-18x21x72-3-door-assembled-parchment>: HTTP status code is not handled or not allowed 2026-01-25 06:37:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/water-cooler-rack-lt25>: HTTP status code is not handled or not allowed 2026-01-25 06:37:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-square-clear-flood-light-12-24vdc-4-leds-1492129>: HTTP status code is not handled or not allowed 2026-01-25 06:37:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-wide-vented-steel-locker-single-tier-24x24x74-1-door-blue-2>: HTTP status code is not handled or not allowed 2026-01-25 06:37:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 76309 pages (at 95 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:37:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spinbar-teflon-polygon-magnetic-stirring-bar-7-8-x-3-16-white-without-pivot-ring>: HTTP status code is not handled or not allowed 2026-01-25 06:37:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-underbed-truck-box-18x18x48-1725110>: HTTP status code is not handled or not allowed 2026-01-25 06:37:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-epoxy-wire-shelving-14368b-36w-x-14d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 06:37:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-plastic-ladle-250ml-9-handle>: HTTP status code is not handled or not allowed 2026-01-25 06:37:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-epoxy-wire-shelving-add-on-14608b-60w-x-14d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 06:37:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/100-truckpak-6-watt-wintergard-wet-heating-cable-120v>: HTTP status code is not handled or not allowed 2026-01-25 06:37:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lkav2031cr-avado-semiprofessional-kitchen-faucet-chrome-single-lever-handle>: HTTP status code is not handled or not allowed 2026-01-25 06:37:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/competitor-ventilated-three-tier-locker-2-wide-12w-x-12d-x-24h-assembled-blue-frost>: HTTP status code is not handled or not allowed 2026-01-25 06:37:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/140110-hf-actkp-km-klp-klm-hf-act-8l-x-4w-x-3h>: HTTP status code is not handled or not allowed 2026-01-25 06:37:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-two-way-stopcock-for-1-4-to-3-8-tubing-2mm-bore-ptfe>: HTTP status code is not handled or not allowed 2026-01-25 06:37:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/140020-tensioner-gripper-8l-x-6w-x-4h-galvanized>: HTTP status code is not handled or not allowed 2026-01-25 06:37:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-three-tier-locker-adder-1-wide-12w-x-18d-x-24h-unassembled-blue-frost>: HTTP status code is not handled or not allowed 2026-01-25 06:37:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-polyethylene-15ml-conical-centrifuge-tubes-with-rims-117cm-12pk>: HTTP status code is not handled or not allowed 2026-01-25 06:37:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/recyclockout-gate-valve-lockout-1-inch-to-2-5-inch-recycled-plastic-7103>: HTTP status code is not handled or not allowed 2026-01-25 06:37:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-epoxy-wire-shelving-21305b-30w-x-21d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 06:37:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22-x-28-bulletin-sign-holder-w-flat-base-chrome>: HTTP status code is not handled or not allowed 2026-01-25 06:37:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/201005-klp-guard-locking-switchstd-release-110v-12npt-pe>: HTTP status code is not handled or not allowed 2026-01-25 06:37:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/180017-seu-td-1-relay-std-screw-terminals-230v>: HTTP status code is not handled or not allowed 2026-01-25 06:37:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-1-2-in-d-paper-towel-holder-1-2-in-dia-zinc-plated-steel-pegboard-hook-for-locboard-1-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:37:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-x-36-plastic-square-edge-mobile-lab-bench>: HTTP status code is not handled or not allowed 2026-01-25 06:37:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60040dcadc-ultra-quiet-ultra-dry-oil-free-4-0-hp-60-0-gal-air-compressor>: HTTP status code is not handled or not allowed 2026-01-25 06:37:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-abs-bar-36-l-x-1-w-x-3-8-thick-black>: HTTP status code is not handled or not allowed 2026-01-25 06:37:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/competitor-ventilated-three-tier-locker-1-wide-12w-x-15d-x-24h-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:37:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spring-action-9-burette-clamp-opens-0-to-1>: HTTP status code is not handled or not allowed 2026-01-25 06:37:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-proculture-round-microcentrifuge-floating-bubble-rack-for-15ml-tubes-for-400ml-beakers>: HTTP status code is not handled or not allowed 2026-01-25 06:37:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-silicone-sheet-36-l-x-36-w-x-3-8-thick-red-food-70a>: HTTP status code is not handled or not allowed 2026-01-25 06:37:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sp-switch-grid-test-tube-rack-grids-for-20-25mm-tubes-fuchsia-4pk>: HTTP status code is not handled or not allowed 2026-01-25 06:37:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12l-x-11-3-4w-mdf-melamine-shelf-for-t502sc-hard-rock-maple>: HTTP status code is not handled or not allowed 2026-01-25 06:37:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/143057-glm-rope-pull-switch-we-stops-3nc-1no-12npt>: HTTP status code is not handled or not allowed 2026-01-25 06:37:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/seq-dual-speed-electric-chain-hoist-w-push-trolley-1-2-ton-15-lift-25-4-2-ft-min-115v>: HTTP status code is not handled or not allowed 2026-01-25 06:37:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-swing-jaw-tubing-clamps-for-tubing-up-to-3-8-od-3pk>: HTTP status code is not handled or not allowed 2026-01-25 06:37:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-single-tier-locker-1-wide-12w-x-12d-x-72h-assembled-blue-frost>: HTTP status code is not handled or not allowed 2026-01-25 06:37:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-animal-restrainer-for-150-300-gram-rats-and-hampsters-acrylic>: HTTP status code is not handled or not allowed 2026-01-25 06:37:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/seq-dual-speed-electric-chain-hoist-w-push-trolley-1-2-ton-10-lift-25-4-2-ft-min-115v>: HTTP status code is not handled or not allowed 2026-01-25 06:37:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cantilever-double-sided-add-on-unit-medium-duty-5>: HTTP status code is not handled or not allowed 2026-01-25 06:37:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cantilever-single-sided-starter-unit-medium-duty-3>: HTTP status code is not handled or not allowed 2026-01-25 06:37:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/max-attach-polymagnet-rare-earth-disc-1-00-dia-x-0-06-thick>: HTTP status code is not handled or not allowed 2026-01-25 06:37:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/max-attach-polymagnet-rare-earth-disc-0-75-dia-x-0-12-thick>: HTTP status code is not handled or not allowed 2026-01-25 06:37:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-double-tier-locker-1-wide-12w-x-12d-x-30h-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:37:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wide-mouth-2-l-1-2-gal-polypropylene-mason-jars-3pk>: HTTP status code is not handled or not allowed 2026-01-25 06:37:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-four-tier-locker-1-wide-12w-x-18d-x-18h-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:37:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cantilever-single-sided-starter-unit-extra-2>: HTTP status code is not handled or not allowed 2026-01-25 06:37:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trustat-b80-16314-vinyl-mat-with-ground-36-d-x-72-w-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:37:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-double-tier-locker-1-wide-12w-x-15d-x-36h-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:37:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cantilever-double-add-on-unit-extra-6>: HTTP status code is not handled or not allowed 2026-01-25 06:37:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-buna-n-sheet-12-l-x-12-w-x-3-8-thick-black-high-strength-90a-bulk-rs-h90-33>: HTTP status code is not handled or not allowed 2026-01-25 06:37:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-layer-rubber-mat-66220-with-ground-30-d-x-72-w-dark-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:37:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-buna-n-sheet-12-l-x-12-w-x-1-32-thick-black-high-strength-80a-bulk-rs-h80-9>: HTTP status code is not handled or not allowed 2026-01-25 06:37:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/statfree-t2-plus-153-66424-dissipative-dual-layer-rubber-mat-top-kit-24-d-x-36-w-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:37:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-center-shelf-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:37:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hard-carrying-case-with-customizable-foam-int-15-7-inch-x12-6-inch-x7-inch>: HTTP status code is not handled or not allowed 2026-01-25 06:37:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-1-8-shower-curtain-hook-ss-spring-stainless-steel-9536-000000>: HTTP status code is not handled or not allowed 2026-01-25 06:37:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/curved-glass-deli-case-with-rear-storage-77-12w-x-43-38d-x-47-14h>: HTTP status code is not handled or not allowed 2026-01-25 06:37:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-buna-n-strip-120-l-x-2-w-x-1-4-thick-black-high-strength-80a>: HTTP status code is not handled or not allowed 2026-01-25 06:37:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/13d-x-24l-duron-shelf-open-white>: HTTP status code is not handled or not allowed 2026-01-25 06:37:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/additional-level-72x24-steel-deck>: HTTP status code is not handled or not allowed 2026-01-25 06:37:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cm-1261-classic-tier-condiment-display-6-1-2-inch-w-x-12-inch-d-x-20-1-2-inch-h>: HTTP status code is not handled or not allowed 2026-01-25 06:37:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grab-n-go-low-profile-open-display-merchandiser-50-14w-x-31-14d-x-59-12h>: HTTP status code is not handled or not allowed 2026-01-25 06:37:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/grab-n-go-low-profile-open-display-merchandiser-41w-x-31-14d-x-59-12h>: HTTP status code is not handled or not allowed 2026-01-25 06:37:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cm-1611-55-squared-packet-organizer-9-1-4-inch-w-x-4-1-2-inch-d-x-2-inch-h-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 06:37:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u-shaped-round-tubing-hangrail-chrome>: HTTP status code is not handled or not allowed 2026-01-25 06:37:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-buna-n-roll-1440-l-x-36-w-x-3-32-thick-black-high-strength-90a>: HTTP status code is not handled or not allowed 2026-01-25 06:37:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/usa-buna-n-sheet-36-l-x-36-w-x-3-16-thick-black-high-strength-90a-acrylic-adhesive>: HTTP status code is not handled or not allowed 2026-01-25 06:37:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/panel-thumb-screw-3-8-16-1-thread-7-8-head-dia-11-16-head-h-stainless-pkg-of-2>: HTTP status code is not handled or not allowed 2026-01-25 06:37:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/patio-bar-height-stools-all-weather-textilene-sling-fabric-seat-back-black-2-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:37:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-8482-2-drum-spill-containment-platform>: HTTP status code is not handled or not allowed 2026-01-25 06:37:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-270ledmb24ad-smd-multi-mode-led-module-and-light-source-blue-24v-acdc>: HTTP status code is not handled or not allowed 2026-01-25 06:37:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/220-ac-dc-tig-welder-package-120-230v-5-220a-60hz-green>: HTTP status code is not handled or not allowed 2026-01-25 06:37:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-display-table-frame-only-satin-chrome>: HTTP status code is not handled or not allowed 2026-01-25 06:37:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-way-slatwall-merchandiser-white>: HTTP status code is not handled or not allowed 2026-01-25 06:37:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mount-anywhere-basket-24-6-6-w>: HTTP status code is not handled or not allowed 2026-01-25 06:37:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mount-anywhere-basket-24-12-6-b>: HTTP status code is not handled or not allowed 2026-01-25 06:37:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-58g-n5-100wh-rotating-beacon-100wh-green-120v-ac>: HTTP status code is not handled or not allowed 2026-01-25 06:37:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a0041-woodruff-key-1-pkg-qty>: HTTP status code is not handled or not allowed 2026-01-25 06:37:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/die-cast-adjustable-lever-m6-x-1-0-63mm-handle-63mm-stud-black-6n63a02k-gn300>: HTTP status code is not handled or not allowed 2026-01-25 06:37:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ecp44156t-4-150-hp-1190-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:37:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-125xbrzb24db-125xbr-xtra-brite-led-random-flash-pattern-blue-24-vdc>: HTTP status code is not handled or not allowed 2026-01-25 06:37:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-248mda24ad-48-mm-led-stacklight-sounder-module-24v-acdc-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:37:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vertical-plate-clamp-li-ing-attachment-1000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:37:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/174-150444-prop-65-regulatory-sign-wood-working-10-x-7-polyester-black-on-white>: HTTP status code is not handled or not allowed 2026-01-25 06:37:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/174-m-131-494-pk-b-494-color-polyester-labels-1-h-x-0-5-w-pink-white-180-roll>: HTTP status code is not handled or not allowed 2026-01-25 06:37:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/briggs-stratton-44s977-0033-g1-gas-engine-professional-v-twin-vertical-shaft-1-1-8-crank-shaft>: HTTP status code is not handled or not allowed 2026-01-25 06:37:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/55170-sgv-swivel-exhaust-fitting-128-mm-hose-1-pkg-qty>: HTTP status code is not handled or not allowed 2026-01-25 06:37:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c-general-purpose-three-phase-odp-motor-3-4-hp-3450-rpm-230-460v-odp>: HTTP status code is not handled or not allowed 2026-01-25 06:37:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/174-j20-179-7425j-153-polypropylene-label-1-h-x-1-w-1500-roll-white>: HTTP status code is not handled or not allowed 2026-01-25 06:38:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-599-class-2-transformer-120v-ac-primary-24v-ac-secondary>: HTTP status code is not handled or not allowed 2026-01-25 06:38:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-105xbrmg120a-dual-mode-led-signal-green-120v-ac>: HTTP status code is not handled or not allowed 2026-01-25 06:38:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/174-m-120-494-or-b-494-color-polyester-labels-0-5-h-x-1-w-orange-white-240-roll>: HTTP status code is not handled or not allowed 2026-01-25 06:38:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/die-li-ing-tongs-li-ing-attachment-3000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:38:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c0018-mini-pad-support-assembly-2-pkg-qty>: HTTP status code is not handled or not allowed 2026-01-25 06:38:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/belt-drive-motor-1-4-hp-1725-rpm-115v-tenv>: HTTP status code is not handled or not allowed 2026-01-25 06:38:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-869stra-n5-flush-mount-hornstrobe-120v-ac-amber>: HTTP status code is not handled or not allowed 2026-01-25 06:38:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30615-o-ring-8-mm-x-10-mm-1-pkg-qty>: HTTP status code is not handled or not allowed 2026-01-25 06:38:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blt-tapered-round-concrete-bollard-12-dia-x-30-h-type-b-mount-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:38:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cjl1301a-33-hp-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:38:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/centrifugal-blower-1550-rpm-115v-oao>: HTTP status code is not handled or not allowed 2026-01-25 06:38:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-rtv-silicone-sealant>: HTTP status code is not handled or not allowed 2026-01-25 06:38:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60x24-plastic-square-edge-production-bench-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:38:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cl3711t-10-hp-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:38:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 76408 pages (at 99 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:38:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-hinged-top-basket-truck-20-bu-green-vinyl-wood-base-all-swivel>: HTTP status code is not handled or not allowed 2026-01-25 06:38:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/06552-throttle-valve-seat-1-pkg-qty>: HTTP status code is not handled or not allowed 2026-01-25 06:38:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-oscillating-desk-fan-2>: HTTP status code is not handled or not allowed 2026-01-25 06:38:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cjm3112-3-4-hp-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:38:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/condenser-fans-motor-1-2-hp-1075-rpm-208-230v-teao-48y-frame>: HTTP status code is not handled or not allowed 2026-01-25 06:38:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-general-purpose-single-phase-odp-motor-1-3-hp-1725-rpm-115-230v-odp-56c-frame>: HTTP status code is not handled or not allowed 2026-01-25 06:38:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/174-j20-262-2595-153-vinyl-label-6-10-h-x-4-13-w-200-roll-white>: HTTP status code is not handled or not allowed 2026-01-25 06:38:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drain-valve-sk-329-529-110v-60hz-cube-ice-machines>: HTTP status code is not handled or not allowed 2026-01-25 06:38:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/water-level-temperature-sensor-sk-329-529-110v-60hz-cube-ice-machines>: HTTP status code is not handled or not allowed 2026-01-25 06:38:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-hinged-top-basket-truck-12-bu-brown-vinyl-wood-base-all-swivel>: HTTP status code is not handled or not allowed 2026-01-25 06:38:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-8-thick-3-w-corrugated-rubber-runner-full-roll-150-l>: HTTP status code is not handled or not allowed 2026-01-25 06:38:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-hinged-top-basket-truck-20-bu-maroon-vinyl-wood-base-all-swivel>: HTTP status code is not handled or not allowed 2026-01-25 06:38:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-test-plugs-3>: HTTP status code is not handled or not allowed 2026-01-25 06:38:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90-tooth-stubby-flex-head-teardrop-ratchet-with-3-8-drive-tang-5>: HTTP status code is not handled or not allowed 2026-01-25 06:38:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pool-pump-motor-3-4-hp-3450-rpm-208-230-460v-odp-48y-frame>: HTTP status code is not handled or not allowed 2026-01-25 06:38:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-electricians-data-shears>: HTTP status code is not handled or not allowed 2026-01-25 06:38:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adhesive-transfer-tape-969-1-2-x-18-yds-5-mil-clear>: HTTP status code is not handled or not allowed 2026-01-25 06:38:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-270ledg240v-led-bulb-green-240v>: HTTP status code is not handled or not allowed 2026-01-25 06:38:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/direct-drive-motor-1-6-hp-1075-rpm-115v-oao>: HTTP status code is not handled or not allowed 2026-01-25 06:38:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-crowd-control-yellow-15-magnetic-wall-mount-with-red-belt>: HTTP status code is not handled or not allowed 2026-01-25 06:38:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/removable-repositionable-tape-9415pc-1-x-72-yds-7-6-mil-translucent>: HTTP status code is not handled or not allowed 2026-01-25 06:38:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/belt-drive-motor-1-2-hp-1725-rpm-115-230v-oao>: HTTP status code is not handled or not allowed 2026-01-25 06:38:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edwards-signaling-270twm-270-threaded-wall-mount-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:38:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rug-ram-carpet-pole-carriage-mounted-120-l-2>: HTTP status code is not handled or not allowed 2026-01-25 06:38:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rug-ram-carpet-pole-fork-mounted-108-l-2>: HTTP status code is not handled or not allowed 2026-01-25 06:38:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1010497-bin-compartment-case-2-sided-12-locking-bins-26-l-x-12-w-x-3-h-black-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:38:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1010496-bin-compartment-case-2-sided-18-locking-bins-26-l-x-12-w-x-3-h-black-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:38:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/100-contractor-chalk-reel-blue-chalk>: HTTP status code is not handled or not allowed 2026-01-25 06:38:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-rivet-boltless-shelving-36-inch-w-x-30-inch-d-x-84-inch-h-8-levels-add-on-w-o-decking-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:38:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-rivet-boltless-shelving-48-inch-w-x-30-inch-d-x-84-inch-h-5-levels-starter-w-o-decking-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:38:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbl-rivet-boltless-shelving-96wx30dx84h-4-levels-add-on-no-decking-1400-lbs-shelf-cap-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:38:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/edge-2-x-4-clear-package-qty-12>: HTTP status code is not handled or not allowed 2026-01-25 06:38:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/room-air-conditioner-motor-1-6-hp-1075-rpm-277v-oao>: HTTP status code is not handled or not allowed 2026-01-25 06:38:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbl-rivet-boltless-shelving-72wx24dx84h-4-levels-add-on-no-decking-1000-lbs-shelf-cap-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:38:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rapidguard-2-lift-off-welded-wire-panel-5-w-x-8-h-panel>: HTTP status code is not handled or not allowed 2026-01-25 06:38:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tablet-cell-phone-locker-27w-x-12d-x-30-1-2h-15-door-3-wide-w-key-locks-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:38:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hydra-li-cart-1>: HTTP status code is not handled or not allowed 2026-01-25 06:38:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-checklist-for-electric-pallet-truck-checklist-caddy-986896>: HTTP status code is not handled or not allowed 2026-01-25 06:38:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-single-phase-tefc-motor-5-hp-1755-rpm-208-230v-tefc>: HTTP status code is not handled or not allowed 2026-01-25 06:38:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extruded-bumper-4-25-w-x-12-l-x-4-h>: HTTP status code is not handled or not allowed 2026-01-25 06:38:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cfm3156a-1-hp-1140-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:38:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/advanced-photo-paper-13in-x-19in-white-20-sheets-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:38:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-storage-cabinet-36-w-x-24-d-x-75-h-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:38:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdt-series-manual-trolley-11000-lbs-cap>: HTTP status code is not handled or not allowed 2026-01-25 06:38:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-photo-paper-11-3-4inx-16-1-2in-white-20-sheets-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:38:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-swivel-elbow-1-2-tube-x-1-2-swift-fit-universal-thread-1-1-4-l>: HTTP status code is not handled or not allowed 2026-01-25 06:38:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-swivel-run-tee-3-8-tube-x-3-8-ml-1-l>: HTTP status code is not handled or not allowed 2026-01-25 06:38:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/record-storage-shelving-69-inch-w-x-15-inch-d-x-84-inch-h-4-levels-starter-w-o-decking-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:38:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cecp4104t-4-30-hp-1770-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:38:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-straight-male-4mm-tube-x-m5-unf-bspp-5-6-l>: HTTP status code is not handled or not allowed 2026-01-25 06:38:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fan-and-blower-1-3-hp-1725-rpm-115v-odp>: HTTP status code is not handled or not allowed 2026-01-25 06:38:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbl-rivet-boltless-shelving-60wx18dx84h-4-levels-add-on-no-decking-2000-lbs-shelf-cap-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:38:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-single-phase-odp-motor-3-4-hp-3450-rpm-115-230v-odp>: HTTP status code is not handled or not allowed 2026-01-25 06:38:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ceuhm3611t-3-hp-1760-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:38:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/peakworks-srl505026le-selfretracting-lifeline-6l-dyneema-webbing-twin-leg-steel-carabiner>: HTTP status code is not handled or not allowed 2026-01-25 06:38:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-poly-trim-1-1-2-brush>: HTTP status code is not handled or not allowed 2026-01-25 06:38:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milwaukee-174-48-89-9221-step-drill-bit-set>: HTTP status code is not handled or not allowed 2026-01-25 06:38:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-union-8mm-tube-1-2-5-l>: HTTP status code is not handled or not allowed 2026-01-25 06:38:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-union-tee-8mm-tube-1-3-4-l>: HTTP status code is not handled or not allowed 2026-01-25 06:38:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-rivet-boltless-shelving-36-inch-w-x-36-inch-d-x-84-inch-h-8-levels-add-on-w-o-decking-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:38:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cdp3455-1-00-hp-1750-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:38:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oem-replacement-motor-1-50-hp-1400-rpm-120v-oao>: HTTP status code is not handled or not allowed 2026-01-25 06:38:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s38440-380-series-max-light-ratcheting-faceshield-dual-crown-clear-antifog>: HTTP status code is not handled or not allowed 2026-01-25 06:38:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/farm-duty-poultry-fan-motor-1-hp-1725-rpm-115-230v-teao-56-frame>: HTTP status code is not handled or not allowed 2026-01-25 06:38:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cem3769t-5-7-5-hp-3525-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:38:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-three-phase-odp-motor-40-hp-1780-rpm-230-460v-odp>: HTTP status code is not handled or not allowed 2026-01-25 06:38:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/men-indus-wrk-shirt-lng-slv-khaki-reg-l-sp14>: HTTP status code is not handled or not allowed 2026-01-25 06:38:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/farm-duty-single-phase-motor-5-hp-1755-rpm-230v-tefc-184t-frame>: HTTP status code is not handled or not allowed 2026-01-25 06:38:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-5-tier-starter-unit-with-solid-bottom-shelf-60-w-x-24-d-x-74-h>: HTTP status code is not handled or not allowed 2026-01-25 06:38:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-polymer-shelf-with-grid-mat-24-w-x-21-d>: HTTP status code is not handled or not allowed 2026-01-25 06:38:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-bnm3542-d-75-hp-1750-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:38:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-boltless-shelving-60-inch-w-x-18-inch-d-x-84-inch-h-3-levels-add-on-w-o-decking-black>: HTTP status code is not handled or not allowed 2026-01-25 06:38:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-swivel-elbow-5-16-tube-x-1-8-nptf-8-9-l>: HTTP status code is not handled or not allowed 2026-01-25 06:38:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/centrifugal-pump-motor-1-2-hp-3450-rpm-115-230v-odp-f56j-frame>: HTTP status code is not handled or not allowed 2026-01-25 06:38:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cdpwd3603-3-00-hp-1750-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:38:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oem-replacement-motor-1-4-hp-1100-rpm-208-230-220v-teao>: HTTP status code is not handled or not allowed 2026-01-25 06:38:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-cecp2334t-4-20-hp-1765-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:38:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cello-wrapped-wooden-toothpicks-1>: HTTP status code is not handled or not allowed 2026-01-25 06:38:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/royal-chef-hat-2>: HTTP status code is not handled or not allowed 2026-01-25 06:38:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-broom-cabinet-60-w-x-24-d-x-75-h-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:38:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-4-tier-add-on-unit-48-w-x-24-d-x-63-h>: HTTP status code is not handled or not allowed 2026-01-25 06:38:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clearview-industrial-cabinet-60-w-x-24-d-x-75-h-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:38:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/toolbox-z400-blue-shop-towel-3-pack-64-sheets-roll-8-3-pack-case-54483>: HTTP status code is not handled or not allowed 2026-01-25 06:38:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-4-tier-add-on-unit-42-w-x-21-d-x-63-h>: HTTP status code is not handled or not allowed 2026-01-25 06:38:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-3-8-basic-ind-plug-x-1-2-female-nptf-1-1-2-l>: HTTP status code is not handled or not allowed 2026-01-25 06:39:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mastercool-59161-brass-2-way-manifold-set-3-18-gauges-3-60-hoses-standard-14-fittings>: HTTP status code is not handled or not allowed 2026-01-25 06:39:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-5-tier-add-on-unit-36w-x-18-d-x-74-h>: HTTP status code is not handled or not allowed 2026-01-25 06:39:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oem-replacement-motor-1-hp-1125-rpm-208-230-460v-oao>: HTTP status code is not handled or not allowed 2026-01-25 06:39:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-l1206-50-33-hp-1425ip23-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:39:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/off-set-hardware-kit-add-on>: HTTP status code is not handled or not allowed 2026-01-25 06:39:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-uniform-cabinet-with-7-drawers-48-w-x-24-d-x-75-h-red>: HTTP status code is not handled or not allowed 2026-01-25 06:39:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-5-tier-add-on-unit-36-w-x-24-d-x-74-h>: HTTP status code is not handled or not allowed 2026-01-25 06:39:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-uniform-cabinet-with-full-width-rod-60-w-x-24-d-x-75-h-brown>: HTTP status code is not handled or not allowed 2026-01-25 06:39:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-mechanical-sweeper-brushless>: HTTP status code is not handled or not allowed 2026-01-25 06:39:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-jaws-handle>: HTTP status code is not handled or not allowed 2026-01-25 06:39:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-standard-duty-shelf-divider-18-w-x-8-h>: HTTP status code is not handled or not allowed 2026-01-25 06:39:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-standard-cut-end-wet-mop-heads-16-oz-1>: HTTP status code is not handled or not allowed 2026-01-25 06:39:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-straight-male-internal-hex-8mm-tube-x-1-4-swift-fit-universal-thread-5-9-l>: HTTP status code is not handled or not allowed 2026-01-25 06:39:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fabric-hopper-36-w-x-36-lx-45-h>: HTTP status code is not handled or not allowed 2026-01-25 06:39:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 76507 pages (at 99 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:39:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-reducer-6mm-stud-x-4mm-tube-1-1-6-l>: HTTP status code is not handled or not allowed 2026-01-25 06:39:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-boltless-shelving-72-inch-w-x-36-inch-d-x-84-inch-h-3-levels-add-on-w-o-decking-black>: HTTP status code is not handled or not allowed 2026-01-25 06:39:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-straight-male-4mm-tube-x-1-8-bspp-7-9-l>: HTTP status code is not handled or not allowed 2026-01-25 06:39:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-uniform-cabinet-with-7-drawers-60-w-x-24-d-x-75-h-white>: HTTP status code is not handled or not allowed 2026-01-25 06:39:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-boltless-shelving-72-inch-w-x-30-inch-d-1-level-w-wood-decking-black>: HTTP status code is not handled or not allowed 2026-01-25 06:39:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cabinet-with-electronic-lock-with-digital-screen-60-w-x-24-d-x-75-h-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:39:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-idvsm3661t-3-hp-6200-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:39:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-canvas-functional-cargo-short-charcoal-32x12-wp90ch3212>: HTTP status code is not handled or not allowed 2026-01-25 06:39:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-boltless-shelving-48-inch-w-x-18-inch-d-1-level-w-wire-decking-black>: HTTP status code is not handled or not allowed 2026-01-25 06:39:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-uniform-cabinet-with-4-shelves-60-w-x-24-d-x-75-h-brown>: HTTP status code is not handled or not allowed 2026-01-25 06:39:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-canvas-short-sleeve-work-shirt-light-blue-xl-ws20lbssxl>: HTTP status code is not handled or not allowed 2026-01-25 06:39:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-toilet-bowl-brush-white>: HTTP status code is not handled or not allowed 2026-01-25 06:39:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-l1511t-10-hp-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:39:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v-grip-pre-configured-wire-shelving-spill-containment-system-36-w-x-24-d-x-84-h-starter-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:39:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-boltless-shelving-48-inch-w-x-48-inch-d-1-level-w-wood-decking-black>: HTTP status code is not handled or not allowed 2026-01-25 06:39:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-flow-control-cylinder-flow-screw-adjustment-1-4-tube-x-10-32-universal-thread-0-l>: HTTP status code is not handled or not allowed 2026-01-25 06:39:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-rack-5-shelves-with-15-span-track-flow-units-48-w-x-48-d-x-84-h-textured-black>: HTTP status code is not handled or not allowed 2026-01-25 06:39:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/maximatic-9-sponge-mop-6-case-94222>: HTTP status code is not handled or not allowed 2026-01-25 06:39:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-l1313t-1-5-hp-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:39:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-30-mobile-security-cabinet-bench-maple-safety-edge-2>: HTTP status code is not handled or not allowed 2026-01-25 06:39:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/allergen-vacuum-bags-for-hushtone-ch54113-ch54115-ch54013-and-ch54015>: HTTP status code is not handled or not allowed 2026-01-25 06:39:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-straight-male-6mm-tube-x-1-8-swift-fit-universal-thread-7-9-l>: HTTP status code is not handled or not allowed 2026-01-25 06:39:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rivetwell-particle-board-decking-72-inch-w-x-36-inch-d-x-5-8-inch>: HTTP status code is not handled or not allowed 2026-01-25 06:39:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-maxiplus-window-squeegees-12-case-96158-s>: HTTP status code is not handled or not allowed 2026-01-25 06:39:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reinforced-safety-flag-true-red-24-x24>: HTTP status code is not handled or not allowed 2026-01-25 06:39:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-straight-male-1-4-tube-x-10-32-1-l>: HTTP status code is not handled or not allowed 2026-01-25 06:39:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-hamper-stainless-steel-18-dia-35-h-2-casters-foot-operated-w-pneumatic-top>: HTTP status code is not handled or not allowed 2026-01-25 06:39:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-x-36-reflective-roll-up-vinyl-sign-road-work-ahead>: HTTP status code is not handled or not allowed 2026-01-25 06:39:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-idm3661t-3te-hp-6000-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:39:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-l3504a-5-hp-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:39:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-x-48-non-reflective-roll-up-vinyl-sign-one-lane-road-ahead>: HTTP status code is not handled or not allowed 2026-01-25 06:39:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rivetwell-particle-board-decking-42-inch-w-x-24-inch-d-x-5-8-inch>: HTTP status code is not handled or not allowed 2026-01-25 06:39:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-x-48-non-reflective-roll-up-vinyl-sign-flagger-ahead-symbol>: HTTP status code is not handled or not allowed 2026-01-25 06:39:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deep-penetrating-febreze-fabric-refresher-odor-eliminator-2>: HTTP status code is not handled or not allowed 2026-01-25 06:39:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-straight-male-12mm-tube-x-1-4-swift-fit-universal-thread-1-1-4-l>: HTTP status code is not handled or not allowed 2026-01-25 06:39:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-tear-drop-pallet-rack-add-on-96-w-x-36-d-x-96-h-b2282517n>: HTTP status code is not handled or not allowed 2026-01-25 06:39:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-deck-for-rivetwell-48-inch-w-x-36-inch-d-x-1-4-inch-h-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:39:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-x-48-reflective-roll-up-vinyl-sign-road-work-ahead>: HTTP status code is not handled or not allowed 2026-01-25 06:39:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hardwood-deck-overlay-wire-shelving-36-w-x-14-d-x-1-thick>: HTTP status code is not handled or not allowed 2026-01-25 06:39:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hardwood-deck-overlay-wire-shelving-36-w-x-18-d-x-1-thick>: HTTP status code is not handled or not allowed 2026-01-25 06:39:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-station-windsor-scrub-sink-wall-mounted-with-foot-action-control-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 06:39:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hardwood-deck-overlay-wire-shelving-36-w-x-24-d-x-1-thick>: HTTP status code is not handled or not allowed 2026-01-25 06:39:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lid-for-st2500h>: HTTP status code is not handled or not allowed 2026-01-25 06:39:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kent-foot-stool-18-w-x-8-h-x-12-d-w-handrail-mri-safe>: HTTP status code is not handled or not allowed 2026-01-25 06:39:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-compartment-sink-18x18x12-deep-8-faucet-holes-18l-drainboard-galvanized-legs>: HTTP status code is not handled or not allowed 2026-01-25 06:39:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-floor-pads-red-15>: HTTP status code is not handled or not allowed 2026-01-25 06:39:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xtra-tt120h-heavy-duty-ultrasonic-cleaner-with-heater-timer-drain-3-5-gal>: HTTP status code is not handled or not allowed 2026-01-25 06:39:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/split-ring-1-100-o-d-x-932-i-d-x-084-thick-stainless-steel-plain>: HTTP status code is not handled or not allowed 2026-01-25 06:39:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-em2558t-4-150-hp-1785-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:39:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-tear-drop-pallet-rack-starter-48-w-x-48-d-x-96-h-796580n>: HTTP status code is not handled or not allowed 2026-01-25 06:39:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-station-windsor-scrub-sink-wall-mounted-w-infrared-water-control-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 06:39:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/panoramic-fold-out-poly-sheet-protector-center-loading-clear-11-x-8-1-2-25-bx>: HTTP status code is not handled or not allowed 2026-01-25 06:39:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-pocket-heavyweight-poly-portfolio-folder-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:39:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-easy-panel-48-w-x-12-h-white-4-pc>: HTTP status code is not handled or not allowed 2026-01-25 06:39:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-tear-drop-pallet-rack-starter-96-w-x-48-d-x-120-h-b2282538n>: HTTP status code is not handled or not allowed 2026-01-25 06:39:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-flow-control-valve-flow-knob-adjustment-3-8-tube-x-3-8-universal-thread-1-1-6-l>: HTTP status code is not handled or not allowed 2026-01-25 06:39:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-needle-valve-knob-adjustment-8mm-tube-x-3-8-swift-fit-universal-thread-1-l>: HTTP status code is not handled or not allowed 2026-01-25 06:39:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-polypropylene-sheet-protector-heavyweight-11-x-8-1-2-50-bx>: HTTP status code is not handled or not allowed 2026-01-25 06:39:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/101517-arc-flash-labels-warning-arc-flash-shock-hazzard-2-inches-by-4-inches-black-orange-white>: HTTP status code is not handled or not allowed 2026-01-25 06:39:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-m1024t-5-2-2-hp-1725-1140-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:39:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-tear-drop-pallet-rack-add-on-108-w-x-36-d-x-96-h-b2282511n>: HTTP status code is not handled or not allowed 2026-01-25 06:39:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/angular-instrument-table-60-l-x-15-w-x-34-h-stainless-steel>: HTTP status code is not handled or not allowed 2026-01-25 06:39:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-tear-drop-pallet-rack-add-on-96-w-x-42-d-x-144-h-b2282520n>: HTTP status code is not handled or not allowed 2026-01-25 06:39:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-flow-control-cylinder-flow-screw-10-32-fm-nptf-x-10-32-universal-thread-3-4-l>: HTTP status code is not handled or not allowed 2026-01-25 06:39:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/104374-toughstripe-floor-marking-tape-polyester-4-inches-wide-by-100-feet-long-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:39:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-motorsports-shirt-short-sleeve-m-silver-black-sp28>: HTTP status code is not handled or not allowed 2026-01-25 06:39:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/x-knee-boot-men-s-size-11-15-h-steel-toe-cleated-outsole-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:39:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-grade-dehumidifier-210-pints-day-240v>: HTTP status code is not handled or not allowed 2026-01-25 06:39:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/35mm-economy-rainsuits-yellow-35mm-pvc-on-polyester-4x>: HTTP status code is not handled or not allowed 2026-01-25 06:39:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-fdl3510m-1-hp-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:39:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blended-duck-zip-front-hooded-jacket-regular-l-brown-duck-jd20>: HTTP status code is not handled or not allowed 2026-01-25 06:39:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyurethane-scooter-stool-with-chrome-base>: HTTP status code is not handled or not allowed 2026-01-25 06:39:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/atrix-omega-needle-nozzle-black>: HTTP status code is not handled or not allowed 2026-01-25 06:39:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/atrix-hepa-filter-white-for-atibcv>: HTTP status code is not handled or not allowed 2026-01-25 06:39:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rack-guard-12-h-x-4-1-4-w-opening>: HTTP status code is not handled or not allowed 2026-01-25 06:39:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-em4117t-30-hp-1180-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:39:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yellow-lined-rain-hat-waterproof-35mm-pvc-on-polyester-large>: HTTP status code is not handled or not allowed 2026-01-25 06:39:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-baseball-hat-polyester-fluorescent-yellow-green>: HTTP status code is not handled or not allowed 2026-01-25 06:39:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-smock-loose-fit-short-sleeve-red-s-tp23>: HTTP status code is not handled or not allowed 2026-01-25 06:39:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-3-black-front-t-shirt-pullover-lime-polyester-5xl>: HTTP status code is not handled or not allowed 2026-01-25 06:40:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hanover-all-weather-commercial-grade-aluminum-30-in-square-slat-top-bistro-table>: HTTP status code is not handled or not allowed 2026-01-25 06:40:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-em3610t-3-hp-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:40:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/65693-large-steel-ball-valve-lockout-1-to-3-inches-valve-steel-7-lock-points>: HTTP status code is not handled or not allowed 2026-01-25 06:40:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-swivel-run-tee-1-2-tube-x-1-2-swift-fit-universal-thread-1-1-5-l>: HTTP status code is not handled or not allowed 2026-01-25 06:40:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/overshoe-w-gaiter-small-waterproof-black-with-red-soles>: HTTP status code is not handled or not allowed 2026-01-25 06:40:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anti-dust-chalk-white-chalk-12-box-501402>: HTTP status code is not handled or not allowed 2026-01-25 06:40:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xt-traction-overshoe-w-roll-a-way-gaiter-large-oil-resistant-black>: HTTP status code is not handled or not allowed 2026-01-25 06:40:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-precision-push-pins-100-box-clear-92707>: HTTP status code is not handled or not allowed 2026-01-25 06:40:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-em3661t-3-hp-1760-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:40:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-ball-valve-3-8-tube-2-3-8-l>: HTTP status code is not handled or not allowed 2026-01-25 06:40:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/b30-series-indoor-outdoor-industrial-vinyl-labels-4-w-white-b30c-4000-595-wt>: HTTP status code is not handled or not allowed 2026-01-25 06:40:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-flow-control-cylider-flow-screw-adjustment-8mm-tube-x-1-4-universal-thread-1-l>: HTTP status code is not handled or not allowed 2026-01-25 06:40:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g2-pvc-overshoe-size-extra-small-5-5-h-cleated-outsole-black-with-red-sole>: HTTP status code is not handled or not allowed 2026-01-25 06:40:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alpha-fittings-flow-control-cylinder-flow-screw-3-8-tube-x-1-4-universal-thread-1-1-6-l>: HTTP status code is not handled or not allowed 2026-01-25 06:40:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-yard-ramp-forkli-ramp-36-l-x-84-w-25-000-cap-lbs-with-ramp-clamps>: HTTP status code is not handled or not allowed 2026-01-25 06:40:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 76603 pages (at 96 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:40:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-oz-eyewash-eyepads-adhesive-strips-1-set-box>: HTTP status code is not handled or not allowed 2026-01-25 06:40:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-gas-heater-infrared-vacuum-straight-tube-50000-btu-20l>: HTTP status code is not handled or not allowed 2026-01-25 06:40:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-em3555-2-hp-3490-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:40:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultracorner-protectors>: HTTP status code is not handled or not allowed 2026-01-25 06:40:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/propane-heater-infrared-vacuum-u-tube-50000-btu-30l>: HTTP status code is not handled or not allowed 2026-01-25 06:40:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/first-aid-kit-200-piece-fabric-case>: HTTP status code is not handled or not allowed 2026-01-25 06:40:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultradrain-guard-oil-sediment-model-60-x-60-1-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:40:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kraft-corrugated-mailers-7-x-3-x-3-200lb-ect-32-b-50-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:40:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-lightweight-knee-boot-men-s-size-13-15-h-plain-toe-cleated-outsole-black>: HTTP status code is not handled or not allowed 2026-01-25 06:40:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/alcohol-wipes-70-isopropyl-100-box>: HTTP status code is not handled or not allowed 2026-01-25 06:40:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-3-short-sleeve-t-shirt-pullover-lime-polyester-3xl>: HTTP status code is not handled or not allowed 2026-01-25 06:40:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-gas-heater-infrared-positive-pressure-straight-tube-tough-guy-40000-btu-10l>: HTTP status code is not handled or not allowed 2026-01-25 06:40:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-gas-heater-infrared-vacuum-u-tube-75000-btu-30l>: HTTP status code is not handled or not allowed 2026-01-25 06:40:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/knee-boot-men-s-size-9-15-h-composite-safety-toe-cleated-outsole-brown>: HTTP status code is not handled or not allowed 2026-01-25 06:40:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-polyurethane-cleanroom-stool-with-casters-medium-bench-height-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:40:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ejmm3211t-3-hp-1760-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:40:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22-32in-clear-curved-combination-spot-flood-light-bar-with-40-led-1492172>: HTTP status code is not handled or not allowed 2026-01-25 06:40:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ecp83664t-5-2-hp-1165-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:40:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/big-d-fire-d-after-fire-one-shot-fogger>: HTTP status code is not handled or not allowed 2026-01-25 06:40:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jacket-size-men-s-2xl-storm-fly-front-attached-hood-green>: HTTP status code is not handled or not allowed 2026-01-25 06:40:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/northflex-oil-grip-nitrile-coated-gloves-nf35-7s-green-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 06:40:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/propane-heater-infrared-positive-pressure-straight-tube-175000-btu-50l>: HTTP status code is not handled or not allowed 2026-01-25 06:40:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60in-amber-modular-light-bar-with-12-led-modules-88930604>: HTTP status code is not handled or not allowed 2026-01-25 06:40:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metalin-zipper-bag-3-x-5-100-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:40:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anti-static-low-density-conductive-foam-24-x-36-x-14>: HTTP status code is not handled or not allowed 2026-01-25 06:40:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-em2332t-10-hp-1180-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:40:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ectm3770t-7-5-hp-1770-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:40:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coverall-zipper-front-elastic-wrist-and-ankle-stormflap-serged-seam-yellow-4x>: HTTP status code is not handled or not allowed 2026-01-25 06:40:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/propane-heater-infrared-vacuum-u-tube-150000-btu-40l>: HTTP status code is not handled or not allowed 2026-01-25 06:40:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-gas-heater-infrared-positive-pressure-straight-tube-tough-guy-75000-btu-20l>: HTTP status code is not handled or not allowed 2026-01-25 06:40:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-em2532t-25-hp-1180-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:40:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ecp84407tr-5-200-hp-1785-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:40:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/x-knee-boot-men-s-size-7-15-h-steel-toe-cleated-outsole-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:40:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oil-safe-utility-lid-orange>: HTTP status code is not handled or not allowed 2026-01-25 06:40:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubbermaid-angled-broom-with-aluminum-handle>: HTTP status code is not handled or not allowed 2026-01-25 06:40:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metalout-bag-3-mm-5-x-8-100-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:40:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-jl3504a-5-hp-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:40:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-9-x-4-8in-clear-rectangular-spot-light-with-15-led-1492290>: HTTP status code is not handled or not allowed 2026-01-25 06:40:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metalin-zipper-bag-10-x-12-100-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:40:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pressure-sensitive-pipe-marker-fire-protection-water-3>: HTTP status code is not handled or not allowed 2026-01-25 06:40:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/esd-tape-clear-2-x-72-yds-3-paper-core>: HTTP status code is not handled or not allowed 2026-01-25 06:40:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s19-2200p-thermostatic-valve-safety-54-gpm>: HTTP status code is not handled or not allowed 2026-01-25 06:40:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s19-2300p-thermostatic-valve-safety-82-gpm>: HTTP status code is not handled or not allowed 2026-01-25 06:40:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4in-amber-rectangular-surface-mount-ultra-thin-strobe-light-with-3-led-8892230>: HTTP status code is not handled or not allowed 2026-01-25 06:40:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/35mm-economy-rainsuits-yellow-35mm-pvc-on-polyester-xl>: HTTP status code is not handled or not allowed 2026-01-25 06:40:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/divider-kit-paramount-modular-drawer-cab-3-long-6-short-dividers-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:40:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reversible-insulated-zipper-vest-black-fl-lime-polyurethane-polyester-5xl>: HTTP status code is not handled or not allowed 2026-01-25 06:40:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/top-tray-w-vinyl-mat-paramount-modular-drawer-cabinet>: HTTP status code is not handled or not allowed 2026-01-25 06:40:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-efm3211t-3-hp-1760-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:40:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pressure-sensitive-pipe-marker-domestic-hot-water-3>: HTTP status code is not handled or not allowed 2026-01-25 06:40:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lexanpremium-molded-face-shield-fs-18l-10-x-185x-007-clear>: HTTP status code is not handled or not allowed 2026-01-25 06:40:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-27-barrel-nut-10-24-x-1-2-stainless-steel-100-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:40:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/automotive-test-and-bleeder-kit-mitmv8000>: HTTP status code is not handled or not allowed 2026-01-25 06:40:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/amber-40-led-beacon-light-5-125in-diameter-x-3-75in-tall-sl501a>: HTTP status code is not handled or not allowed 2026-01-25 06:40:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-metal-wall-cabinet-w-z-logo>: HTTP status code is not handled or not allowed 2026-01-25 06:40:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/intruder-defense-system-mounting-cleat-dga-1-use-with-dso-1>: HTTP status code is not handled or not allowed 2026-01-25 06:40:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/400d-coverall-zipper-front-hood-elastic-wrist-ankle-stormflap-white-blue-2x>: HTTP status code is not handled or not allowed 2026-01-25 06:40:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/type-r-class-2-hi-visibility-safety-vest-5-point-breakaway-polyester-mesh-xxs-xs>: HTTP status code is not handled or not allowed 2026-01-25 06:40:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stackable-air-mover-w-telescopic-handle-wheels-3-speeds-3-4-hp-p-800h>: HTTP status code is not handled or not allowed 2026-01-25 06:40:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-el3501-33-hp-1740-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:40:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ehfm2523t-15-hp-1765-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:40:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/400d-coverall-zipper-front-elastic-wrist-ankle-stormflap-white-blue-2x>: HTTP status code is not handled or not allowed 2026-01-25 06:40:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-steel-platform-loading-dock-12-x-8-20-000-lb-cap-986956>: HTTP status code is not handled or not allowed 2026-01-25 06:40:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/products-20-ton-air-manual-bottle-jack-ast5302a>: HTTP status code is not handled or not allowed 2026-01-25 06:40:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-2-premium-pullover-hi-visibility-t-shirt-lime-polyester-4xl>: HTTP status code is not handled or not allowed 2026-01-25 06:40:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-degree-papertape-framing-nailer>: HTTP status code is not handled or not allowed 2026-01-25 06:40:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-ecp84308t-5-40-hp-1190-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:40:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-165-psi-pancake-compressor>: HTTP status code is not handled or not allowed 2026-01-25 06:40:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pants-black-polyurethane-on-75-denier-ripstop-polyester-sm>: HTTP status code is not handled or not allowed 2026-01-25 06:40:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-mounting-bracket-for-3-375in-thin-mount-horizontal-strobe-light-8892325>: HTTP status code is not handled or not allowed 2026-01-25 06:40:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-5in-amber-round-marker-clearance-light-with-2-led-5622522>: HTTP status code is not handled or not allowed 2026-01-25 06:40:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-leg-18-gauge-7-32-narrow-crown-finish-staples-5000-qty>: HTTP status code is not handled or not allowed 2026-01-25 06:40:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-mag-bit-holder-1pc-stainless-steel-w-c-ring-x-12>: HTTP status code is not handled or not allowed 2026-01-25 06:40:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-4in-amber-green-surface-mount-ultra-thin-strobe-light-4-led-8892250>: HTTP status code is not handled or not allowed 2026-01-25 06:40:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-3-8-in-crown-stapler>: HTTP status code is not handled or not allowed 2026-01-25 06:40:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-to-1-1-2-15-degree-coil-roofing-nailer>: HTTP status code is not handled or not allowed 2026-01-25 06:40:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-16-mag-nutsetter-x-10-s2-modified-steel-gunmetal-grey>: HTTP status code is not handled or not allowed 2026-01-25 06:40:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/parabolic-reflector-extension-30-000-to-40-000-btu-ceramic-heaters>: HTTP status code is not handled or not allowed 2026-01-25 06:40:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fellowes-standard-monitor-riser>: HTTP status code is not handled or not allowed 2026-01-25 06:40:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-3-8-x-113-ring-shank-21-degree-plastic-collated-stick-framing-nails-5000-qty>: HTTP status code is not handled or not allowed 2026-01-25 06:40:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roll-set-leveling-caster-built-in-ratchet-300-lb-cap-swivel-plate-mount-60f>: HTTP status code is not handled or not allowed 2026-01-25 06:40:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/z-green-5-gallons-184839>: HTTP status code is not handled or not allowed 2026-01-25 06:40:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-vem3774t-10-hp-1760-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:41:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iron-horse-2300lb-riveted-shelving-with-wire-decking-4-shelf-60-h-x-36-w-x-18-d-black>: HTTP status code is not handled or not allowed 2026-01-25 06:41:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-vebm3546t-1-hp-1760-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:41:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-x-120-wire-bright-steel-round-head-framing-nails-2000-qty>: HTTP status code is not handled or not allowed 2026-01-25 06:41:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camtray-18x26-rec-horizon-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:41:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manual-label-dispenser-for-up-to-6w-labels>: HTTP status code is not handled or not allowed 2026-01-25 06:41:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lab-design-island-bench-8-5-w-x-5-d-x-3-3-4-h-6-door-model-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:41:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30h-backless-barstool-metal-square-red>: HTTP status code is not handled or not allowed 2026-01-25 06:41:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-vinyl-sign-caution-chock-wheels-before-loading-or-unloading>: HTTP status code is not handled or not allowed 2026-01-25 06:41:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-vebm3615t-5-hp-1750-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:41:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-vl1408t-3-hp-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:41:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-plastic-sign-danger-chemical-storage>: HTTP status code is not handled or not allowed 2026-01-25 06:41:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-plastic-sign-danger-chemical-storage-area-authorized-personnel-only-1>: HTTP status code is not handled or not allowed 2026-01-25 06:41:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-partition-4-way-90-degree-connector-kit-for-38-h-base-panel>: HTTP status code is not handled or not allowed 2026-01-25 06:41:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-vinyl-sign-danger-do-not-start-out-of-order-1>: HTTP status code is not handled or not allowed 2026-01-25 06:41:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-vecp84106t-4-20-hp-3510-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:41:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 76700 pages (at 97 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:41:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/torx-174-55-insert-bit-x-1-1-4-5-16-hex-shank-bull-nose-s2-modified-steel-gunmetal-grey>: HTTP status code is not handled or not allowed 2026-01-25 06:41:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-plastic-sign-caution-this-door-must-be-kept-closed-2>: HTTP status code is not handled or not allowed 2026-01-25 06:41:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-vejmm3154t-1-5-hp-1755-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:41:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/head-harness-for-face-shield-black>: HTTP status code is not handled or not allowed 2026-01-25 06:41:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/face-seal-for-sr-570-black>: HTTP status code is not handled or not allowed 2026-01-25 06:41:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mortorq-1-insert-bit-x-1-s2-modified-steel-gunmetal-grey>: HTTP status code is not handled or not allowed 2026-01-25 06:41:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-vm3535-33-hp-1140-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:41:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carbon-steel-hand-winch-with-brake-1000-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:41:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/phillips-3-power-bit-x-2-3-4-extra-hard-s2-modified-steel-black-manganese-phosphate>: HTTP status code is not handled or not allowed 2026-01-25 06:41:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/square-2-power-bit-x-2-3-4-s2-modified-steel-gunmetal-grey>: HTTP status code is not handled or not allowed 2026-01-25 06:41:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ac2201wlh-power-grip-magnetic-pickup-tool-92-5-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:41:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slotted-5-6-power-bit-x-3-1-2-s2-modified-steel-gunmetal-grey>: HTTP status code is not handled or not allowed 2026-01-25 06:41:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-plastic-sign-danger-hard-hat-area-2>: HTTP status code is not handled or not allowed 2026-01-25 06:41:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4100-w-vertical-panel-channelizer-barricade-w-oversized-handle-white>: HTTP status code is not handled or not allowed 2026-01-25 06:41:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-vm3153t-75-hp-1140-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:41:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-zdvsm4406t-4-150-hp-3600-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:41:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-motor-zdnm3767t-5te-hp-6000-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:41:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-2-tier-wire-shelving-starter-unit-30w-x-24d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 06:41:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/csp-sg-4kit-eg-y-all-plastic-manhole-guard-nonconductive-engineer-grade-sheeting-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:41:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/little-giant-wstc2-3072-all-welded-cabinet-workbench-steel-top-center-shelf>: HTTP status code is not handled or not allowed 2026-01-25 06:41:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-16-18-t-slot-nut-3-8-table-slot-5-8-base-width-13-32-height-carbon-steel-black-oxide-tn-0>: HTTP status code is not handled or not allowed 2026-01-25 06:41:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-16-14-flange-nut-3-4-hex-1-flange-dia-9-16-height-stainless-steel-cn-15ss>: HTTP status code is not handled or not allowed 2026-01-25 06:41:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-2-tier-wire-shelving-starter-unit-48w-x-21d-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 06:41:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lynx-distributor-16-w-x-7-h-blue-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 06:41:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-vinyl-sign-men>: HTTP status code is not handled or not allowed 2026-01-25 06:41:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-jumper-cables-2-awg-20-ft-black-red>: HTTP status code is not handled or not allowed 2026-01-25 06:41:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mm3-2d-2448-fl-48-w-mobile-bench-cabinets-center-shelf-2-locking-doors>: HTTP status code is not handled or not allowed 2026-01-25 06:41:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-4-tier-esd-18w-x-36l-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 06:41:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g-2-1-2cg-2-1-2-w-g-connector-for-3-4-pipe>: HTTP status code is not handled or not allowed 2026-01-25 06:41:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-vinyl-sign-notice-keep-this-door-closed-2>: HTTP status code is not handled or not allowed 2026-01-25 06:41:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/watts-pleated-cartridges-ff-4-12odx20l-1-micron>: HTTP status code is not handled or not allowed 2026-01-25 06:41:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-4-tier-esd-18w-x-24l-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 06:41:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-4-tier-esd-14w-x-54l-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 06:41:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cam-follower-heavy-sealed-hex-head-2-1-2-l-stud-175-w-roller-3-roller-dia>: HTTP status code is not handled or not allowed 2026-01-25 06:41:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-battery-cable-with-5-16-ring-terminals-4-awg-4-ft-black>: HTTP status code is not handled or not allowed 2026-01-25 06:41:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-5-tier-esd-14w-x-48l-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 06:41:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/battery-cable-set-with-5-16-ring-terminals-4-awg-3-ft-black-red>: HTTP status code is not handled or not allowed 2026-01-25 06:41:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sk3000-structural-channel-pallet-rack-4-inch-x-96-inch-channel-beam>: HTTP status code is not handled or not allowed 2026-01-25 06:41:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/r-bg-boring-gimlet-for-5-8-r-cables-steel>: HTTP status code is not handled or not allowed 2026-01-25 06:41:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spanish-vinyl-sign-peligro-blank>: HTTP status code is not handled or not allowed 2026-01-25 06:41:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-2-tier-esd-30w-x-60l-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 06:41:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extreme-duty-aluminum-correctional-cart-54-x-22-2000-lb-cap-986982>: HTTP status code is not handled or not allowed 2026-01-25 06:41:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machinist-jaw-round-channel-vise-with-swivel-base-6>: HTTP status code is not handled or not allowed 2026-01-25 06:41:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/illinois-engineered-products-peco865-double-eco-gate-8482-6-w-to-8-w-6-h>: HTTP status code is not handled or not allowed 2026-01-25 06:41:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/light-duty-floor-crane-4l0014-500-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:41:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-battery-cable-with-3-8-ring-terminals-1-0-awg-4-ft-red>: HTTP status code is not handled or not allowed 2026-01-25 06:41:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dura-kap-industrial-pant-brown-34x34-pt20>: HTTP status code is not handled or not allowed 2026-01-25 06:41:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-step-heavy-duty-mobile-work-platform-24-x-48-platform>: HTTP status code is not handled or not allowed 2026-01-25 06:41:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-174-reach-freezer-1-door-29-wx32-2-dx82-5-h-23-cu>: HTTP status code is not handled or not allowed 2026-01-25 06:41:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ngd268-combustible-gas-leak-detector-for-extreme-environments>: HTTP status code is not handled or not allowed 2026-01-25 06:41:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-shelf-cart-esd-3-shelf-18w-x-30l-polyurethane-4-swivel-2-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 06:41:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-infinity-ventilated-steel-locker-single-tier-3-wide-12x12x72-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:41:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leviton-80414-w-quickport-decora-multimedia-blank-insert>: HTTP status code is not handled or not allowed 2026-01-25 06:41:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caution-sign-10x7-vinyl-do-not-operate-without-guards>: HTTP status code is not handled or not allowed 2026-01-25 06:41:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelf-truck-esd-18w-x-48l-x-80h-polyurethane-4-swivel-2-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 06:41:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leviton-80609-2-gang-decora-gfci-device-decora-brown>: HTTP status code is not handled or not allowed 2026-01-25 06:41:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-step-heavy-duty-mobile-work-platform-handrails-36-x-48-platform>: HTTP status code is not handled or not allowed 2026-01-25 06:41:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precision-battery-monitor-bmv-702-bluetooth-black-abs-plastic-6-5-95-vdc>: HTTP status code is not handled or not allowed 2026-01-25 06:41:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelf-truck-esd-18w-x-30l-x-60h-polyurethane-4-swivel-2-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 06:41:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelf-truck-esd-18w-x-24l-x-60h-polyurethane-4-swivel-2-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 06:41:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mg-2-express-lavatory-system-mg-series-w-sprayhead-w-infrared-control>: HTTP status code is not handled or not allowed 2026-01-25 06:41:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flexbollard-52h-x-7-14w-w99h-signpost-adhesive-blueblack-xl-b528-d-bluewhite>: HTTP status code is not handled or not allowed 2026-01-25 06:41:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/band-saw-blade-compact-48-39-0529>: HTTP status code is not handled or not allowed 2026-01-25 06:41:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-battery-cable-with-3-8-ring-terminals-2-0-awg-20-ft-black>: HTTP status code is not handled or not allowed 2026-01-25 06:41:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-boxed-protective-wear-dispenser-23-3-8-w-x-5-3-8-d-x-36-1-2-h-quartz>: HTTP status code is not handled or not allowed 2026-01-25 06:41:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-battery-cable-with-5-16-ring-terminals-1-0-awg-20-ft-black>: HTTP status code is not handled or not allowed 2026-01-25 06:41:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ch-270067-6-test-ball-plug-13-psi-30-ft>: HTTP status code is not handled or not allowed 2026-01-25 06:41:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/danger-sign-14x10-aluminum-keep-hands-clear-of-moving-machinery>: HTTP status code is not handled or not allowed 2026-01-25 06:41:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-603-mini-lever-hoist-1100-lbs-cap-10ft-lift>: HTTP status code is not handled or not allowed 2026-01-25 06:41:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-4-tier-esd-18w-x-24l-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 06:41:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blow-model-commercial-duty-adjustable-height-folding-table-30in-x72in-gray-granite>: HTTP status code is not handled or not allowed 2026-01-25 06:41:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ch-262137-15-16-muni-ball-plug-3-bypass-13-psi-30-ft>: HTTP status code is not handled or not allowed 2026-01-25 06:41:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/danger-sign-10x7-vinyl-no-smoking-matches-or-open-flames>: HTTP status code is not handled or not allowed 2026-01-25 06:41:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honeywell-24-v-fan-relay-spdt-switching>: HTTP status code is not handled or not allowed 2026-01-25 06:41:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/danger-sign-14x10-aluminum-no-smoking-beyond-this-point>: HTTP status code is not handled or not allowed 2026-01-25 06:41:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42811-satin-finishing-star-eyelet-3-aluminum-oxide-very-fine>: HTTP status code is not handled or not allowed 2026-01-25 06:41:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/notice-sign-14x10-rigid-plastic-report-all-injuries-at-once>: HTTP status code is not handled or not allowed 2026-01-25 06:41:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10555-conditioning-qc-disc-type-r-2-aluminum-oxide-very-fine>: HTTP status code is not handled or not allowed 2026-01-25 06:41:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/locking-respiratory-hygiene-station-17-1-16-w-x-4-1-4-d-x-10-1-8-h-beige>: HTTP status code is not handled or not allowed 2026-01-25 06:41:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vital-cluster-box-unit-w-vogue-classic-accessories-12-unit-1-parcel-locker-dark-bronze>: HTTP status code is not handled or not allowed 2026-01-25 06:41:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-federal-straight-arrow-stencil-1-8-thick-polytough-plastic-white>: HTTP status code is not handled or not allowed 2026-01-25 06:41:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oty-30796-purple-primercleaner-16-oz>: HTTP status code is not handled or not allowed 2026-01-25 06:41:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vital-cluster-box-unit-w-vogue-classic-accessories-16-unit-2-parcel-lockers-black>: HTTP status code is not handled or not allowed 2026-01-25 06:42:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-shelf-truck-esd-2-shelf-24w-x-42l-x-40h-polyurethane-4-swivel-2-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 06:42:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-shelf-truck-esd-3-shelf-21w-x-42l-x-40h-polyurethane-4-swivel-2-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 06:42:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-shelf-truck-esd-3-shelf-21w-x-30l-x-40h-polyurethane-4-swivel-2-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 06:42:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6x24x4-1-2-open-top-bin-boxes>: HTTP status code is not handled or not allowed 2026-01-25 06:42:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honeywell-high-low-aquastat-100-240-operating-temperature>: HTTP status code is not handled or not allowed 2026-01-25 06:42:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honeywell-pilot-burner-nator-lp-gas-batwing-tip-style-b-style-mounting-primary>: HTTP status code is not handled or not allowed 2026-01-25 06:42:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ch-270270-8-end-of-pipe-gripper-plug-17-psi-40ft>: HTTP status code is not handled or not allowed 2026-01-25 06:42:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caution-sign-14x10-vinyl-eye-protection-required>: HTTP status code is not handled or not allowed 2026-01-25 06:42:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-2-tier-esd-14w-x-72l-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 06:42:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caution-sign-14x10-rigid-plastic-eye-protection-required>: HTTP status code is not handled or not allowed 2026-01-25 06:42:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ch-270059-5-test-ball-plug-13-psi-30-ft>: HTTP status code is not handled or not allowed 2026-01-25 06:42:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelf-truck-esd-30w-x-36l-x-69h-polyurethane-4-swivel-2-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 06:42:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hc-31403-yellow-gas-line-thread-seal-tape-with-ptfe-12-x-260-dispenser-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:42:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42in-round-table-economy-stack-chair-set-gray-nebula-laminate-table-blue-vinyl-chair>: HTTP status code is not handled or not allowed 2026-01-25 06:42:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ch-275058-6-8-test-ball-plug-17-psi-40-ft>: HTTP status code is not handled or not allowed 2026-01-25 06:42:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 76799 pages (at 99 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:42:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-sided-stock-curtain-package-w-hardware-24w-12w-24w-x-9h-white-clear-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:42:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-freestanding-office-partition-panel-24-14-w-x-61-12-h-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:42:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-panel-display-tower-5-ft-9-inch-fabric-color-navy>: HTTP status code is not handled or not allowed 2026-01-25 06:42:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zurn-z88000-deck-mounted-single-hole-lab-turret>: HTTP status code is not handled or not allowed 2026-01-25 06:42:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1000v-insulated-long-arm-cable-cutter-26-inch>: HTTP status code is not handled or not allowed 2026-01-25 06:42:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-3-tier-esd-24w-x-72l-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 06:42:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/freestanding-office-partition-panel-60-14-w-x-42-h-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:42:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/goffs-welding-screen-6w-x-6h-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:42:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vestil-forkli-or-hoist-bulk-bag-li-er-bbl-4-4000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:42:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/speakman-s-1554-tub-spout>: HTTP status code is not handled or not allowed 2026-01-25 06:42:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/individual-character-stencil-12-number-8>: HTTP status code is not handled or not allowed 2026-01-25 06:42:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silicone-foam-with-high-temp-adhesive-18-thick-x-12-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 06:42:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-roll-vertical-48-inch>: HTTP status code is not handled or not allowed 2026-01-25 06:42:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mw-pg-484-012-adj-adjustable-pallet-guard-48w-x-40d-x-12h-black>: HTTP status code is not handled or not allowed 2026-01-25 06:42:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-office-partition-panel-with-partial-window-60-14-w-x-96-h-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:42:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/85238153-50l-hose-with-couplers>: HTTP status code is not handled or not allowed 2026-01-25 06:42:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/biztalk-174-belt-clip-for-mb400-black>: HTTP status code is not handled or not allowed 2026-01-25 06:42:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rust-oleum-sierra-perfor-metlmx-0-voc-dtm-acryl-enam-smi-gls-safe-rd>: HTTP status code is not handled or not allowed 2026-01-25 06:42:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rust-oleum-sierra-perfor-metlmx-0-voc-dtm-acryl-enam-semi-gls-safe-bl>: HTTP status code is not handled or not allowed 2026-01-25 06:42:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-3-tier-esd-18w-x-24l-x-34h>: HTTP status code is not handled or not allowed 2026-01-25 06:42:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rust-oleum-companion-aerosol-economy-spray-grip>: HTTP status code is not handled or not allowed 2026-01-25 06:42:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-freestanding-office-partition-panel-24-14-w-x-73-12-h-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:42:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flexible-stainless-steel-faucet-connector-20-l-1-2-x-1-2-fip-x-fip>: HTTP status code is not handled or not allowed 2026-01-25 06:42:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multipurpose-pastel-colored-paper-3r11050-8-12-x-11-blue-500-sheetsream>: HTTP status code is not handled or not allowed 2026-01-25 06:42:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electric-office-partition-panel-60-14-w-x-46-h-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:42:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-40x1-4-military-pan-head-machine-screws-MS51957-13B>: HTTP status code is not handled or not allowed 2026-01-25 06:42:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-shot-7500w-portable-generator-electric-start-black>: HTTP status code is not handled or not allowed 2026-01-25 06:42:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/non-electric-office-partition-panel-with-raceway-60-14-w-x-46-h-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:42:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/individual-character-stencil-36-letter-o>: HTTP status code is not handled or not allowed 2026-01-25 06:42:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/business-4200-copyprint-paper-3r02047rm-20-lbs-8-12-x-11-white-500-sheetsream>: HTTP status code is not handled or not allowed 2026-01-25 06:42:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t200-head-harness-tychem-2000-hood-hx5-backpack-papr>: HTTP status code is not handled or not allowed 2026-01-25 06:42:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steno-book-wassorted-colored-cvr-80221-6-x-9-green-tint-80-sheetspad-4-padpack>: HTTP status code is not handled or not allowed 2026-01-25 06:42:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-link-t-link-hard-hat-assembly-tychem-4000-hood-hx5-papr>: HTTP status code is not handled or not allowed 2026-01-25 06:42:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-office-partition-panel-wpartial-window-pass-thru-cable-60-14wx47-12h-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:42:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-office-partition-panel-with-pass-thru-cable-60-14-w-x-47-12-h-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:42:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-16-18x2-1-2-pan-head-floor-board-screws-type-f-3140FTPB>: HTTP status code is not handled or not allowed 2026-01-25 06:42:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/neenah-paper-astrobrights-colored-card-stock-22791-8-12-x-11-sunburst-yellow-250pack>: HTTP status code is not handled or not allowed 2026-01-25 06:42:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wire-shelving-starter-unit-4-tier-esd-14w-x-60l-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 06:42:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/office-partition-panel-with-partial-window-pass-thru-cable-48-14-w-x-76-h-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:42:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/highlandself-stick-pads-6539yw-1-12-x-2-yellow-100-sheets-12pack>: HTTP status code is not handled or not allowed 2026-01-25 06:42:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-18-gauge-430-stainless-steel-workbench-galvanized-legs-undershelf-48-x30>: HTTP status code is not handled or not allowed 2026-01-25 06:42:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-18-ga-430-ss-workbench-2-backsplash-galvanized-legs-undershelf-30-x24>: HTTP status code is not handled or not allowed 2026-01-25 06:42:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-16-18x1-1-4-self-clinching-stud-12-rib-full-thread-300-series-stainless-steel-3120SCN300>: HTTP status code is not handled or not allowed 2026-01-25 06:42:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-safety-sign-fire-extinguisher-inside-vinyl-3>: HTTP status code is not handled or not allowed 2026-01-25 06:42:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/digital-index-color-card-stock-25100-8-12-x-11-blue-250pack>: HTTP status code is not handled or not allowed 2026-01-25 06:42:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/asb1084-lawn-and-garden-tire-assembly-size-16-x-650-8>: HTTP status code is not handled or not allowed 2026-01-25 06:42:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rust-oleum-high-performance-v2100-rust-prevent-enamel-aero-safety-red>: HTTP status code is not handled or not allowed 2026-01-25 06:42:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8x2-1-4-modified-truss-head-self-drilling-screws-0836KPM>: HTTP status code is not handled or not allowed 2026-01-25 06:42:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8x1-1-2-indented-hex-washer-self-tapping-screws-type-a-b-thread-0824ABPW>: HTTP status code is not handled or not allowed 2026-01-25 06:42:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oblong-punch-die-set-with-storage-case-12-piece>: HTTP status code is not handled or not allowed 2026-01-25 06:42:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brighton-twin-size-tufted-upholstered-platform-bed-in-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:42:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/melbourne-metal-upholstered-king-size-headboard-in-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:42:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-lockout-tags-do-not-operate-equipment-tag-out>: HTTP status code is not handled or not allowed 2026-01-25 06:42:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/punching-productivity-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:42:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rust-oleum-2500-alkyd-accelerator>: HTTP status code is not handled or not allowed 2026-01-25 06:42:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-pole-circuit-breaker-lockout-1>: HTTP status code is not handled or not allowed 2026-01-25 06:42:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/413-1-1-1-4-chrome-tubeless-tire-valve>: HTTP status code is not handled or not allowed 2026-01-25 06:42:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/btguwm-thermostat-guard-metal-ring-base-675-hx45-wx3563-d>: HTTP status code is not handled or not allowed 2026-01-25 06:42:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electrical-lockout-kit-bilingual>: HTTP status code is not handled or not allowed 2026-01-25 06:42:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-cable-lockout-with-6-foot-cable>: HTTP status code is not handled or not allowed 2026-01-25 06:42:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-rubber-overboots-mens-black-size-1516-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 06:42:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lockout-tags-electrical-panel-locked-out>: HTTP status code is not handled or not allowed 2026-01-25 06:42:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silicone-o-ring-2mm-wide-3mm-id-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-25 06:42:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tribeca-twin-size-tufted-upholstered-platform-bed-in-black>: HTTP status code is not handled or not allowed 2026-01-25 06:42:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chemical-resistant-high-temperature-teflon-ptfe-tubing-116id-x-18od-x-10-ft>: HTTP status code is not handled or not allowed 2026-01-25 06:42:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hanger-hook-301766-magliner-174-gemini-bulk-container-edition-hand-truck>: HTTP status code is not handled or not allowed 2026-01-25 06:42:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-16x1-1-4-concrete-screw-anchors-1020CNPF>: HTTP status code is not handled or not allowed 2026-01-25 06:42:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ingersoll-rand-activated-carbon-filter-64-cfm>: HTTP status code is not handled or not allowed 2026-01-25 06:42:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-16x1-indented-hex-head-machine-screws-3716MH>: HTTP status code is not handled or not allowed 2026-01-25 06:42:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silicone-o-ring-1-5mm-wide-7mm-id-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-25 06:42:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/full-face-viton-flange-gasket-for-1-pipe-116-thick-class-150>: HTTP status code is not handled or not allowed 2026-01-25 06:42:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ingersoll-rand-air-compressor-2475n5-value-460v-3ph-60hz>: HTTP status code is not handled or not allowed 2026-01-25 06:42:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/printed-barricade-tape-caution-caution-2>: HTTP status code is not handled or not allowed 2026-01-25 06:42:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ingersoll-rand-non-cycling-air-dryer-15-cfm>: HTTP status code is not handled or not allowed 2026-01-25 06:42:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/riverdale-full-tufted-upholstered-platform-bed-light-gray-pocket-spring-mattress>: HTTP status code is not handled or not allowed 2026-01-25 06:42:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/viton-o-ring-dash-014-pack-of-25>: HTTP status code is not handled or not allowed 2026-01-25 06:42:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-25-11110-160-psi-14-lm-plastic>: HTTP status code is not handled or not allowed 2026-01-25 06:42:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2x1-4-round-head-type-u-drive-screws-0204U188>: HTTP status code is not handled or not allowed 2026-01-25 06:42:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-fuel-ventless-gas-fireplace-system-10000-btu-t-stat-control-apple-spice-fs100t-3as>: HTTP status code is not handled or not allowed 2026-01-25 06:42:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shutter-stamp-with-microban-red-blue-file-1-5-8-x-1-2>: HTTP status code is not handled or not allowed 2026-01-25 06:42:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adhesive-peel-stick-butyl-pad-8-x-8-black>: HTTP status code is not handled or not allowed 2026-01-25 06:42:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dp200-36-round-traffic-channelizer-post-orange>: HTTP status code is not handled or not allowed 2026-01-25 06:42:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-25-11110-160-psikpa-14-lm-plastic>: HTTP status code is not handled or not allowed 2026-01-25 06:43:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premier-stainless-steel-glass-door-laboratory-refrigerator-72-cu-ft-abt-hc-ssp-72g>: HTTP status code is not handled or not allowed 2026-01-25 06:43:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-4-11110-100-psikpa-14-lm-plastic>: HTTP status code is not handled or not allowed 2026-01-25 06:43:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-2-11111-200-psi-14-lm-polished-brass>: HTTP status code is not handled or not allowed 2026-01-25 06:43:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-ink-pad-for-printer-p20-and-dual-pad-printer-p20-red>: HTTP status code is not handled or not allowed 2026-01-25 06:43:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traditional-date-stamp-six-years-1-3-8-x-3-16>: HTTP status code is not handled or not allowed 2026-01-25 06:43:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lux-lspp2b-yxl-ls-wick-b-eye-polo-yel-xl>: HTTP status code is not handled or not allowed 2026-01-25 06:43:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/viton-o-ring-dash-124-pack-of-5>: HTTP status code is not handled or not allowed 2026-01-25 06:43:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/record-storage-rack-add-on-96x36x96>: HTTP status code is not handled or not allowed 2026-01-25 06:43:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10qt-quietpro-cn-runningvac-hepa-intercept-microfilter-bag-10pk-106995>: HTTP status code is not handled or not allowed 2026-01-25 06:43:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/viton-o-ring-dash-115-pack-of-5>: HTTP status code is not handled or not allowed 2026-01-25 06:43:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/l-shaped-adjustable-height-workbench-maple-square-edge-72-in-w-x-78-in-d>: HTTP status code is not handled or not allowed 2026-01-25 06:43:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-32x3-16-socket-set-screws-half-dog-point-1103SSD>: HTTP status code is not handled or not allowed 2026-01-25 06:43:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lux-lspp2b-y5x-ls-wick-b-eye-polo-yel-5x>: HTTP status code is not handled or not allowed 2026-01-25 06:43:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lifetime-48-in-premium-snow-sled-2-pack-red>: HTTP status code is not handled or not allowed 2026-01-25 06:43:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 76895 pages (at 96 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:43:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/modular-drawer-cabinet-7-drawers-w-lock-w-o-dividers-30x27x29-1-2-black>: HTTP status code is not handled or not allowed 2026-01-25 06:43:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proguard-intercept-micro-filter-bag-10-gallon-3-pack-107181>: HTTP status code is not handled or not allowed 2026-01-25 06:43:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lux-lspp3b-yxl-ls-wick-b-eye-polo-yel-xl>: HTTP status code is not handled or not allowed 2026-01-25 06:43:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lux-lspp3b-ym-ls-wick-b-eye-polo-yel-l>: HTTP status code is not handled or not allowed 2026-01-25 06:43:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bakery-boxes-9in-x-9in-x-4in-white-200-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:43:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kold-locker-indoor-freezer-minus-10-degrees-f-with-floor-1-1-2hp-lh-door-96-inches-w-x-96-inches-d-x-91-inches-h>: HTTP status code is not handled or not allowed 2026-01-25 06:43:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-32x3-8-knurled-thumb-screws-0806TK188>: HTTP status code is not handled or not allowed 2026-01-25 06:43:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-8x3-toggle-anchors-0648TBCM>: HTTP status code is not handled or not allowed 2026-01-25 06:43:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gel-ocity-quick-dry-retractable-gel-pen-0-7mm-assorted-ink-barrel-12-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:43:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thermoformed-plastic-parts-tray-12-1-2-x-8-1-2-x-3-5-compartments-black-269955>: HTTP status code is not handled or not allowed 2026-01-25 06:43:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-24-cap-nuts-10NC>: HTTP status code is not handled or not allowed 2026-01-25 06:43:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pallet-rack-solid-steel-deck-52-w-x-42-d-796897>: HTTP status code is not handled or not allowed 2026-01-25 06:43:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-silicone-foam-with-high-temp-adhesive-12-thick-x-2w-x-6l>: HTTP status code is not handled or not allowed 2026-01-25 06:43:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/commercial-led-backlit-panel-50w-5500-lumens-4000k-100-277v>: HTTP status code is not handled or not allowed 2026-01-25 06:43:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silicone-foam-tube-34-id-x-1-14-od-x-5l>: HTTP status code is not handled or not allowed 2026-01-25 06:43:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hirsh-industries-10000-series-lateral-file-36in-wide-5-drawer-charcoal>: HTTP status code is not handled or not allowed 2026-01-25 06:43:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4x8-1-2-wedge-anchor-icc-compliant-icbo-75136AWG>: HTTP status code is not handled or not allowed 2026-01-25 06:43:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glow-sign-vinyl-le-arrow>: HTTP status code is not handled or not allowed 2026-01-25 06:43:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-16x4-hex-tap-bolts-3764BHT>: HTTP status code is not handled or not allowed 2026-01-25 06:43:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/non-illuminated-mushroom-head-pb-metal-bezel-22mm-black-p9m-em6nn>: HTTP status code is not handled or not allowed 2026-01-25 06:43:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sharp-mechanical-pencil-0-7-mm-hb-2-5-assorted-barrel-colors-z-ms-mk1-3-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:43:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glow-sign-rigid-plastic-10x14-no-exit>: HTTP status code is not handled or not allowed 2026-01-25 06:43:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-lockers-four-tier-12-x-18-x-18-4-door-assembled-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:43:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/easy-peel-laser-mailing-labels-3-1-3-x-4-clear-300-box-5664>: HTTP status code is not handled or not allowed 2026-01-25 06:43:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blue-marvel-car-truck-wash-5-gallon-1-pail>: HTTP status code is not handled or not allowed 2026-01-25 06:43:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glow-sign-rigid-plastic-emergency-telephone>: HTTP status code is not handled or not allowed 2026-01-25 06:43:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sarasa-dry-gel-x1-retractable-gel-pen-medium-0-7mm-blue-ink-barrel-dozen>: HTTP status code is not handled or not allowed 2026-01-25 06:43:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-dividers-for-qtb304-model-252190-pack-of-4-269966>: HTTP status code is not handled or not allowed 2026-01-25 06:43:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glow-sign-rigid-plastic-push-bar-to-open>: HTTP status code is not handled or not allowed 2026-01-25 06:43:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/permanent-self-adhesive-laser-inkjet-file-folder-labels-white-1500-box-5366>: HTTP status code is not handled or not allowed 2026-01-25 06:43:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6209zz-double-shielded-45mm-bore-85mm-od>: HTTP status code is not handled or not allowed 2026-01-25 06:43:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shipping-labels-with-paper-receipt-5-1-16-x-7-5-8-white-50-pack-5127>: HTTP status code is not handled or not allowed 2026-01-25 06:43:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-easy-peel-labels-2-1-2-dia-brown-kraft-225-pk-22808>: HTTP status code is not handled or not allowed 2026-01-25 06:43:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-fixed-spreader-beam-yellow-80000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:43:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-four-drawer-procedure-cart-with-pushbutton-e-lock-forest-green>: HTTP status code is not handled or not allowed 2026-01-25 06:43:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-epoxy-wire-shelving-add-on-a18488b-48w-x-18d-x-86h>: HTTP status code is not handled or not allowed 2026-01-25 06:43:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bathroom-partition-wrap-around-strike-keeper-zamak-baked-enamel-hdwt-t318>: HTTP status code is not handled or not allowed 2026-01-25 06:43:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-six-drawer-narrow-procedure-cart-tall-height-mini-width-key-lock-white>: HTTP status code is not handled or not allowed 2026-01-25 06:43:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6307-open-35mm-bore-80mm-od>: HTTP status code is not handled or not allowed 2026-01-25 06:43:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-four-drawer-procedure-cart-with-standard-key-lock-light-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:43:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-16-x-9-working-length-j-hook-style-b>: HTTP status code is not handled or not allowed 2026-01-25 06:43:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/orbis-clear-container-with-hinged-lid-21-4-5x15-1-5x9-4-5>: HTTP status code is not handled or not allowed 2026-01-25 06:43:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mini-m-series-five-drawer-anesthesia-cart-w-standard-key-lock-teal>: HTTP status code is not handled or not allowed 2026-01-25 06:43:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jv1-jones-valve-condensate-drain-blow-out-vavle>: HTTP status code is not handled or not allowed 2026-01-25 06:43:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-8-11-drop-in-anchor-62ADR>: HTTP status code is not handled or not allowed 2026-01-25 06:43:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6308-open-40mm-bore-90mm-od>: HTTP status code is not handled or not allowed 2026-01-25 06:43:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rack-large-plastic-compartment-boxes>: HTTP status code is not handled or not allowed 2026-01-25 06:43:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/removable-inkjet-laser-id-labels-3-1-3-x-4-white-150-pack-6464>: HTTP status code is not handled or not allowed 2026-01-25 06:43:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/column-sentry-fit-column-protector-8x8-square-opening-43-inches-h-medium-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:43:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ecofriendly-labels-2-3-x-3-7-16-white-750-pack-48266>: HTTP status code is not handled or not allowed 2026-01-25 06:43:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-all-purpose-labels-2-x-4-2500-box-ml1000b>: HTTP status code is not handled or not allowed 2026-01-25 06:43:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-four-drawer-procedure-cart-with-pushbutton-e-lock-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:43:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/threaded-rod-rack-2>: HTTP status code is not handled or not allowed 2026-01-25 06:43:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-8-toggle-anchors-284891>: HTTP status code is not handled or not allowed 2026-01-25 06:43:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4x1-1-4-fender-washers-1420WF>: HTTP status code is not handled or not allowed 2026-01-25 06:43:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6206zz-double-shielded-30mm-bore-62mm-od>: HTTP status code is not handled or not allowed 2026-01-25 06:43:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/countertop-compact-all-fridge-1-6-cu-ft-dar016b1bm>: HTTP status code is not handled or not allowed 2026-01-25 06:43:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-lb-bulk-grinder-g3-hd-red>: HTTP status code is not handled or not allowed 2026-01-25 06:43:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/axiom-12-cup-auto-coffee-brewer-axiom-15-31l2u-pf>: HTTP status code is not handled or not allowed 2026-01-25 06:43:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-soft-heat-brewer-docking-system-15g57l-black-no-fl>: HTTP status code is not handled or not allowed 2026-01-25 06:43:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portion-control-grinder-2-hoppers-g9-2t-dbc>: HTTP status code is not handled or not allowed 2026-01-25 06:43:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-dielectric-union>: HTTP status code is not handled or not allowed 2026-01-25 06:43:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dino-lite-2nd-pole-for-ms35b-36b>: HTTP status code is not handled or not allowed 2026-01-25 06:43:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/airpot-coffee-brewer-cw15-aps-pf>: HTTP status code is not handled or not allowed 2026-01-25 06:43:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-tall-emergency-crash-cart-6-drawers-36-3-4-w-x-26-l-x-43-h-brushed-silver>: HTTP status code is not handled or not allowed 2026-01-25 06:43:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hunter-trak-8482-modern-commercial-ceiling-fan-84-24887-cfm-matte-black>: HTTP status code is not handled or not allowed 2026-01-25 06:43:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/casters-soft-wheels>: HTTP status code is not handled or not allowed 2026-01-25 06:43:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multifunction-mesh-task-chair-with-seat-slider-fabric-black-apollo-series>: HTTP status code is not handled or not allowed 2026-01-25 06:43:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/luxor-174-ec112hd-b-black-2-tub-1-flat-shelf-cart-35-1-4-x-18-500-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 06:43:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6019-open-95mm-bore-145mm-od>: HTTP status code is not handled or not allowed 2026-01-25 06:43:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-x-5-working-length-j-hook-style-9>: HTTP status code is not handled or not allowed 2026-01-25 06:43:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/62388-s252c-class-2-mesh-solid-surveyor-vest-hi-vis-lime-2xl>: HTTP status code is not handled or not allowed 2026-01-25 06:43:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/19266-americana-full-brim-hard-hat-w-accessory-slots-mega-ratchet-4-pt-suspension-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:43:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dot-placard-hot-1>: HTTP status code is not handled or not allowed 2026-01-25 06:43:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dot-placard-four-digit-1090-2>: HTTP status code is not handled or not allowed 2026-01-25 06:43:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s163-ansi-class-3-long-raincoat-hi-vis-lime-large-lg-62029>: HTTP status code is not handled or not allowed 2026-01-25 06:43:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-8-x-7-working-length-j-hook-style-b>: HTTP status code is not handled or not allowed 2026-01-25 06:43:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paddle-sign-stop-slow-paddle>: HTTP status code is not handled or not allowed 2026-01-25 06:43:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/110-125-volt-start-capacitor-378-455-mfd-378b>: HTTP status code is not handled or not allowed 2026-01-25 06:43:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oty-11957-1-1-2-inch-3-inch-galvanized-all-flash-18-inchl-x-18-inchw-base>: HTTP status code is not handled or not allowed 2026-01-25 06:43:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/triple-column-storage-cabinet-electronic-keypad-lock-60-w-x-28-d-x-81-h-purple>: HTTP status code is not handled or not allowed 2026-01-25 06:43:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/saf-t-flag-plastic-diagonal>: HTTP status code is not handled or not allowed 2026-01-25 06:43:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oty-11832-1-2-inch-1-inch-aluminum-no-calk-roof-flashing-solar-12-1-2-inchl-x-9-inchw-base>: HTTP status code is not handled or not allowed 2026-01-25 06:43:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-x-14-working-length-j-hook-style-9>: HTTP status code is not handled or not allowed 2026-01-25 06:44:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-spread-basket-lifting-beam-yellow-1000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:44:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arsenal-5938-polyester-hoist-bucket-top-14938>: HTTP status code is not handled or not allowed 2026-01-25 06:44:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dot-placard-fuel-oil-1>: HTTP status code is not handled or not allowed 2026-01-25 06:44:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-socks-4-x-3-12-gallon-capacity-12pack>: HTTP status code is not handled or not allowed 2026-01-25 06:44:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dot-placard-poison-inhalation-hazard-6-2>: HTTP status code is not handled or not allowed 2026-01-25 06:44:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-tall-isolation-cart-4-drawers-key-lock-18-w-x-18-l-x-40-3-4-h-hammertone-red>: HTTP status code is not handled or not allowed 2026-01-25 06:44:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dot-placard-3082-9>: HTTP status code is not handled or not allowed 2026-01-25 06:44:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6917zz-double-shielded-85mm-bore-120mm-od>: HTTP status code is not handled or not allowed 2026-01-25 06:44:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-816-thermal-flip-top-gloves-black-xl-17345>: HTTP status code is not handled or not allowed 2026-01-25 06:44:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-910-half-finger-impact-gloves-wrist-support-black-l-17714>: HTTP status code is not handled or not allowed 2026-01-25 06:44:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8377-type-r-class-3-quilted-bomber-jacket-lime-m-25623>: HTTP status code is not handled or not allowed 2026-01-25 06:44:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-8-x-7-working-length-j-hook-style-9>: HTTP status code is not handled or not allowed 2026-01-25 06:44:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blood-pressure-monitors-calibrated-cuff-10-yr-adult-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:44:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/schoolmate-personal-monostereo-hdph>: HTTP status code is not handled or not allowed 2026-01-25 06:44:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-diamond-series-built-in-undercounter-auto-defrost-freezer-right-hinged-door>: HTTP status code is not handled or not allowed 2026-01-25 06:44:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 76995 pages (at 100 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:44:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-diamond-series-undercounter-manual-defrost-freezer-40-c-4-cu-ft>: HTTP status code is not handled or not allowed 2026-01-25 06:44:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6915zz-double-shielded-75mm-bore-105mm-od>: HTTP status code is not handled or not allowed 2026-01-25 06:44:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classic-series-classroom-chair-gray-vented-back-18-5-8-w-x-30-5-8-h-1>: HTTP status code is not handled or not allowed 2026-01-25 06:44:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-outdoor-wall-mounted-fan>: HTTP status code is not handled or not allowed 2026-01-25 06:44:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poly-z-brite-heavy-duty-wire-shelving-ds24247z-24w-x-24d-x-74h>: HTTP status code is not handled or not allowed 2026-01-25 06:44:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-96-economy-h-beam-adjustable-length-yellow-102-l-x-40-w-x-36-h-540lbs-4000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:44:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/miscellaneous-dangerous-goods-label-paper-500-roll>: HTTP status code is not handled or not allowed 2026-01-25 06:44:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-spread-basket-lifting-beam-yellow-4000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:44:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/markerboard-activity-table-24in-x-36in-rectangle-ada-compliant-adjustable-height>: HTTP status code is not handled or not allowed 2026-01-25 06:44:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pfhkbl-handle-flush-keylocking-fits-doors-cast-zinc>: HTTP status code is not handled or not allowed 2026-01-25 06:44:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-radial-ball-bearing-6818-open-90mm-bore-115mm-od>: HTTP status code is not handled or not allowed 2026-01-25 06:44:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/condensate-drain-pan-38-oz-capacity-120v-8oz-per-hr>: HTTP status code is not handled or not allowed 2026-01-25 06:44:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/red-arrows-up-air-label-paper-2-15-16-inch-x-4-1-8-inch-500-roll>: HTTP status code is not handled or not allowed 2026-01-25 06:44:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexelon-3-shelf-container-rack-bk18246n-24w-x-18d-x-63h>: HTTP status code is not handled or not allowed 2026-01-25 06:44:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/miscellaneous-dangerous-goods-placard-wordless-permanent-vinyl-25-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:44:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/p1224cn-cage-nut-pkg-20-1224-steelzinc>: HTTP status code is not handled or not allowed 2026-01-25 06:44:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/poly-z-brite-3-shelf-container-rack-with-2-solid-shelves-bk18485sz-48w-x-18d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 06:44:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/strobe-120vac-clear-lp1-120c>: HTTP status code is not handled or not allowed 2026-01-25 06:44:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hazard-class-9-lithium-battery-label-pvc-free-500-roll>: HTTP status code is not handled or not allowed 2026-01-25 06:44:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electronic-potential-relay-ac-hard-start-kit-one-half-to-10-hp>: HTTP status code is not handled or not allowed 2026-01-25 06:44:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/500mm-drylin-n-40mm-miniature-guide-rail>: HTTP status code is not handled or not allowed 2026-01-25 06:44:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drylin-with-10-80-double-guide-rail>: HTTP status code is not handled or not allowed 2026-01-25 06:44:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/as1032b-screw-pkg-rack-p-qty-20-1032x75-steelblack>: HTTP status code is not handled or not allowed 2026-01-25 06:44:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-well-non-treated-plate-with-lid-individual-sterile-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:44:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vynckier-vj1210w-vj-12-inch-x-10-inch-non-metallic-enclosure-4-corner-screws>: HTTP status code is not handled or not allowed 2026-01-25 06:44:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vynckier-vj1412hwpl2-vj-14-inch-x-12-inch-non-metallic-enclosure-hinge-2-padlockable-latches>: HTTP status code is not handled or not allowed 2026-01-25 06:44:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96-deep-well-storage-plate-1-1ml-pp-v-bottom-non-sterile-25-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:44:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-narrow-procedure-cart-6-drawers-key-lock-18-w-x-18-l-x-40-3-4-h-burgundy>: HTTP status code is not handled or not allowed 2026-01-25 06:44:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/retail-hose-with-static-wire-frh10012>: HTTP status code is not handled or not allowed 2026-01-25 06:44:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m-series-short-emergency-crash-cart-md30-emg1-std-width-4-drawers-breakaway-lock-light-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:44:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/185mm-dia-slewing-ring-bearing-12364-lbs-max-axial-static>: HTTP status code is not handled or not allowed 2026-01-25 06:44:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/q3pbpcd-qline-pb-encl-3pbx305mm-polycarb>: HTTP status code is not handled or not allowed 2026-01-25 06:44:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cell-strainer-100-181-m-yellow-bulk-packed-sterile-50-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:44:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/96-well-non-treated-plate-without-lid-individual-sterile-100-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:44:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/labor-law-poster-iowa-spanish>: HTTP status code is not handled or not allowed 2026-01-25 06:44:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/boss-reception-desk-71w-x-30d-x-42h-cherry>: HTTP status code is not handled or not allowed 2026-01-25 06:44:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stay-home-when-you-are-sick-poster-synthetic-paper>: HTTP status code is not handled or not allowed 2026-01-25 06:44:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/syringe-filter-pes-0-10-181-m-30mm-sterile-green-polypropylene-30-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:44:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fabric-task-chair-high-back-black>: HTTP status code is not handled or not allowed 2026-01-25 06:44:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tall-isolation-cart-3-drawers-standard-key-lock-37-1-2-wx22-lx42-63-100-h-sand>: HTTP status code is not handled or not allowed 2026-01-25 06:44:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/walk-on-floor-sign-emergency-shower>: HTTP status code is not handled or not allowed 2026-01-25 06:44:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/850cm-178-roller-bottle-tissue-culture-treated-vented-cap-sterile-12pk>: HTTP status code is not handled or not allowed 2026-01-25 06:44:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-pallet-40-x-48-four-way-entry-3000-fork-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:44:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyjohn-pjn3-portable-restroom-evergreen>: HTTP status code is not handled or not allowed 2026-01-25 06:44:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mattress-pallet-twin-xl-size-80-x-39-two-way-entry-1000-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:44:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polyjohn-pjn3-portable-restroom-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:44:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-v-max-1-4-inch-screwdriver-kit>: HTTP status code is not handled or not allowed 2026-01-25 06:44:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/facility-visi-sign-women-2>: HTTP status code is not handled or not allowed 2026-01-25 06:44:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1161422-comfort-vest-style-harness-tongue-buckle-quick-connect-xl>: HTTP status code is not handled or not allowed 2026-01-25 06:44:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extra-shelf-for-boltless-heavy-duty-die-rack-36-w-x-18-d-medium-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:44:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1161402-comfort-vest-style-positioning-harness-quick-connect-xl>: HTTP status code is not handled or not allowed 2026-01-25 06:44:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tilt-back-drum-truck-polyurethane-wheels-1000-lb-capacity-987054>: HTTP status code is not handled or not allowed 2026-01-25 06:44:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shippable-rugged-transit-case-929-carry-case-with-wheels-26l-x-20w-x-13h-black>: HTTP status code is not handled or not allowed 2026-01-25 06:44:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/engraved-occupancy-sign-occupied-vacant-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:44:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s75x6-female-buttress-x-2-male-npt-pipe-thread-ibc-valve>: HTTP status code is not handled or not allowed 2026-01-25 06:44:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/boltless-heavy-duty-die-rack-36-w-x-24-d-x-72-h-5-shelves-medium-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:44:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/511-rotunda-graphics-case-trade-show-case-14l-x-14w-x-38h-black>: HTTP status code is not handled or not allowed 2026-01-25 06:44:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1161568-vest-style-positioning-harness-quick-connect-xl>: HTTP status code is not handled or not allowed 2026-01-25 06:44:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-female-camlock-dust-cap>: HTTP status code is not handled or not allowed 2026-01-25 06:44:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-female-npt-pipe-thread-x-2-part-a-camlock-stubbyadapter>: HTTP status code is not handled or not allowed 2026-01-25 06:44:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/suredry-10-scope-cabinet-w-dri-scope-aid-e-lock-30-w-x-24-d-x-93-h-mauve>: HTTP status code is not handled or not allowed 2026-01-25 06:44:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1161224-comfort-construction-style-climbing-harness-tongue-buckle-quick-connect-m-l>: HTTP status code is not handled or not allowed 2026-01-25 06:44:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s100x8-female-buttress-x-3-part-a-camlock-adapter>: HTTP status code is not handled or not allowed 2026-01-25 06:44:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1161573-vest-style-harness-pass-through-2xl>: HTTP status code is not handled or not allowed 2026-01-25 06:44:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/em3458-condenser-fan-1-3-1-6-hp-1075-rpm-208-230v-totally-enclosed>: HTTP status code is not handled or not allowed 2026-01-25 06:44:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/syringe-filter-pvdf-0-10-181-m-13mm-sterile-light-blue-polypropylene-75-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:44:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-32-insert-for-soft-wood-flanged-901032-20>: HTTP status code is not handled or not allowed 2026-01-25 06:44:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/65-gallon-poly-overpack>: HTTP status code is not handled or not allowed 2026-01-25 06:44:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/15ml-centrifuge-tube-sure-cap-bag-sterile-polypropylene-500-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:44:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-16-24-thin-wall-insert-for-metal-319-524>: HTTP status code is not handled or not allowed 2026-01-25 06:44:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-20-thin-wall-insert-for-metal-319-820>: HTTP status code is not handled or not allowed 2026-01-25 06:44:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/161438-comfort-vest-style-positioning-climbing-harness-pass-through-xl>: HTTP status code is not handled or not allowed 2026-01-25 06:44:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/engraved-occupancy-sign-alarm-on-off-red>: HTTP status code is not handled or not allowed 2026-01-25 06:44:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powerstud-wedge-expansion-anchor-sd1-12-x-512>: HTTP status code is not handled or not allowed 2026-01-25 06:44:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-column-medical-storage-cabinet-key-lock-glass-doors-42-wx29-3-5-dx76-h-beige>: HTTP status code is not handled or not allowed 2026-01-25 06:44:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wedgebit-drill-bit-sdsplus-size-38-usable-6-l-overall-8-l>: HTTP status code is not handled or not allowed 2026-01-25 06:44:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/275-omni-telescoping-case-lined-with-1-2-foam-trade-show-case-44l-x-22w-x-12h-black>: HTTP status code is not handled or not allowed 2026-01-25 06:44:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powerstud-wedge-expansion-anchor-sd1-34-x-512>: HTTP status code is not handled or not allowed 2026-01-25 06:44:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hdcm-17mkg-dax-carbon-fiber-6-point-suspension-cap-matte-black>: HTTP status code is not handled or not allowed 2026-01-25 06:44:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m5-08-tapered-ultrasonic-insert-double-vane-dv-m50-th>: HTTP status code is not handled or not allowed 2026-01-25 06:44:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-blower-hadp9-34-hp-3-ph-tefc-ccw-bottom-horizontal>: HTTP status code is not handled or not allowed 2026-01-25 06:44:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-blower-hadp15-5-hp-3-ph-tefc-cw-bottom-horizontal>: HTTP status code is not handled or not allowed 2026-01-25 06:44:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aluminum-blower-hadp10-34-hp-1-ph-tefc-cw-top-horizontal>: HTTP status code is not handled or not allowed 2026-01-25 06:44:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-ga-triple-tier-locker-w-keyless-entry-lock-9-compartments-72-wx24-dx75-h-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:45:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8-16-flush-press-insert-stainless-240-6-cr>: HTTP status code is not handled or not allowed 2026-01-25 06:45:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polly-products-econo-mizer-4-ft-backed-bench-cedar-bench-black-frame>: HTTP status code is not handled or not allowed 2026-01-25 06:45:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-32-flanged-press-insert-stainless-260-332-cr>: HTTP status code is not handled or not allowed 2026-01-25 06:45:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polly-products-econo-mizer-6-ft-backed-bench-with-arms-cedar-bench-black-frame>: HTTP status code is not handled or not allowed 2026-01-25 06:45:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2595kl-digital-chair-scale-600-lb-x-0-2-lb>: HTTP status code is not handled or not allowed 2026-01-25 06:45:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a36h2410sslp-continuous-hinge-wclamps-type-4x-3600x2400x1000in-ss-type-304>: HTTP status code is not handled or not allowed 2026-01-25 06:45:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-ga-double-tier-ventilated-locker-4-compartments-48-w-x-24-d-x-75-h-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:45:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/thermoplastic-tape-1-width-x-9-ft-length-blue-1-roll>: HTTP status code is not handled or not allowed 2026-01-25 06:45:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machine-labels-danger-starts-automatically>: HTTP status code is not handled or not allowed 2026-01-25 06:45:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a20h2008ss6lp-continuous-hinge-wclamps-type-4x-2000x2000x800in-ss-type-316l>: HTTP status code is not handled or not allowed 2026-01-25 06:45:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zinc-die-cast-adjustable-lever-steel-38-16-126-stud-248l>: HTTP status code is not handled or not allowed 2026-01-25 06:45:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/14-ga-4-tier-locker-w-keyless-entry-lock-4-compartments-24-w-x-24-d-x-75-h-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:45:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cp304012-concepthmi-encl-wblack-trim-type-4-300x400x120mm>: HTTP status code is not handled or not allowed 2026-01-25 06:45:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-small-space-lp-gas-grill-cover>: HTTP status code is not handled or not allowed 2026-01-25 06:45:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-lug-style-butterfly-valve-w-epdm-sealsincludes-10-position-handle>: HTTP status code is not handled or not allowed 2026-01-25 06:45:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 77094 pages (at 99 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:45:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-lug-style-butterfly-valve-w-buna-seals-and-dbl-acting-pneum-actuator>: HTTP status code is not handled or not allowed 2026-01-25 06:45:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/siesta-marina-resin-dining-arm-chair-teak-brown>: HTTP status code is not handled or not allowed 2026-01-25 06:45:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a24h2412sslp3pt-continuous-hinge-w3point-ltch-type-4x-2400x2400x1200in-ss-type-304>: HTTP status code is not handled or not allowed 2026-01-25 06:45:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a48h3612ss6lp3pt-continuous-hinge-w3point-ltch-type-4x-4800x3600x1200inss-type-316l>: HTTP status code is not handled or not allowed 2026-01-25 06:45:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-54-watertight-limit-switch-36-1-gear-ratio-w-4-contact-blocks>: HTTP status code is not handled or not allowed 2026-01-25 06:45:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-cantilever-arm-w-lip>: HTTP status code is not handled or not allowed 2026-01-25 06:45:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vrtcl-concrete-pressure-tong-w-grip-teeth-1100-lbs-cap-1-3-16-6-5-16-grip-range>: HTTP status code is not handled or not allowed 2026-01-25 06:45:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vrtcl-concrete-pressure-tong-w-urethane-pads-1100-lbs-cap-3-8-9-7-16-grip-range>: HTTP status code is not handled or not allowed 2026-01-25 06:45:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-front-grille-for-nexel-models-243007-243009>: HTTP status code is not handled or not allowed 2026-01-25 06:45:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-55-limit-switch-111-1-gear-ratio-w-2-contact-blocks>: HTTP status code is not handled or not allowed 2026-01-25 06:45:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lithonia-ibe-22lm-mvolt-50k-led-linear-high-bay-166w-21712-lumens-5000k-dimmable-dlc-premium>: HTTP status code is not handled or not allowed 2026-01-25 06:45:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vinyl-surface-mounted-corner-guard-90-degrees-corner-1-1-2-inch-wings-8-ft-height-taupe-undrilled>: HTTP status code is not handled or not allowed 2026-01-25 06:45:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bilingual-machine-labels-danger-electrical-hazard-authorized-personnel-only>: HTTP status code is not handled or not allowed 2026-01-25 06:45:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/horizontal-twin-crossarm-with-4-pad-vacuum-lifter-680-lbs-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:45:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-falcon-lapel-microphone-soft-ear-hook-ep303>: HTTP status code is not handled or not allowed 2026-01-25 06:45:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vertical-lifting-90-degree-turning-plate-clamp-orange-steel-2000-lbs-cap-3-4-opening>: HTTP status code is not handled or not allowed 2026-01-25 06:45:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/surface-mount-bottle-filling-station-refrigerated-non-filtered-stainless-steel-115v>: HTTP status code is not handled or not allowed 2026-01-25 06:45:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powerpoint-hoist-ring-360-pivot-3300-lb-load-capacity-625-11-unc-bolt>: HTTP status code is not handled or not allowed 2026-01-25 06:45:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/utility-bag-14in-x-10in-0-75-mil-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:45:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pur-fill-1g-professional-gun-foam-750ml-gf01>: HTTP status code is not handled or not allowed 2026-01-25 06:45:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beam-seating-2-cool-gray-seats>: HTTP status code is not handled or not allowed 2026-01-25 06:45:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/secure-top-heavy-duty-badge-holders-horizontal-4-x-3-clear-25-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:45:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-pc-ss-npt-ball-valve-w-da-pneumatic>: HTTP status code is not handled or not allowed 2026-01-25 06:45:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/abs-plastic-bar-18-thick-x-4-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 06:45:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pictorial-osha-sign-plastic-danger-keep-all-cylinders-chained>: HTTP status code is not handled or not allowed 2026-01-25 06:45:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/id-badge-holder-w-clip-vertical-3-x-4-clear-50-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:45:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-342-dn-7-4-nipple-w-1-4-female-npt>: HTTP status code is not handled or not allowed 2026-01-25 06:45:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/resealable-id-badge-holder-vertical-2-5-8-x-3-3-4-clear-50-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:45:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pictorial-osha-sign-plastic-notice-this-is-a-smoke-free-workplace>: HTTP status code is not handled or not allowed 2026-01-25 06:45:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/frosted-rigid-badge-holder-2-1-8-x-3-3-8-clear-horizontal-25-bx>: HTTP status code is not handled or not allowed 2026-01-25 06:45:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/american-lock-solid-aluminum-rectangular-padlock-red-no-a1307red>: HTTP status code is not handled or not allowed 2026-01-25 06:45:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-3-pc-ss-npt-ball-valve-w-nema-4-115vac>: HTTP status code is not handled or not allowed 2026-01-25 06:45:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-4-3-way-t-port-brass-npt-ball-valve-w-sr-pneumatic>: HTTP status code is not handled or not allowed 2026-01-25 06:45:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/restricted-area-plastic-no-trespassing>: HTTP status code is not handled or not allowed 2026-01-25 06:45:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/restricted-area-plastic-no-unauthorized-personnel-beyond-this-point>: HTTP status code is not handled or not allowed 2026-01-25 06:45:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/restricted-area-aluminum-only-authorized-persons-to-enter-this-area>: HTTP status code is not handled or not allowed 2026-01-25 06:45:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/restricted-area-aluminum-private-no-trespassing>: HTTP status code is not handled or not allowed 2026-01-25 06:45:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-18-thick-x-6-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 06:45:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nickel-plated-brass-non-drip-nipple-1-2-body-size-blue-color-ring-nbr-seals>: HTTP status code is not handled or not allowed 2026-01-25 06:45:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-677-dn14-coupling-w-valve-2-21-25-l>: HTTP status code is not handled or not allowed 2026-01-25 06:45:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8mm-pc-92-ft-pl-zinc>: HTTP status code is not handled or not allowed 2026-01-25 06:45:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-lab-coat-white-polycombed-cotton-regular-38>: HTTP status code is not handled or not allowed 2026-01-25 06:45:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-1-thick-x-5-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 06:45:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-spreader-beam-30-000-lbs-capacity-72-chain-top-rigging-yellow-steel>: HTTP status code is not handled or not allowed 2026-01-25 06:45:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/american-lock-solid-aluminum-rectangular-padlock-orange-no-a1106orj>: HTTP status code is not handled or not allowed 2026-01-25 06:45:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nickel-plated-brass-non-drip-coupling-3-8-body-size-blue-color-ring-nbr-seals>: HTTP status code is not handled or not allowed 2026-01-25 06:45:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unistrut-1-58-conduit-emt-clamp-p1426eg-electro-galvanized-12>: HTTP status code is not handled or not allowed 2026-01-25 06:45:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-ton-combination-hitch-w-mounting-kit-2in-hitch-ball-10052>: HTTP status code is not handled or not allowed 2026-01-25 06:45:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vertical-reel-lifter-500-lbs-capacity-yellow-steel>: HTTP status code is not handled or not allowed 2026-01-25 06:45:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-position-adjustable-eye-assembly-w-3in-heavy-duty-channel-b20145>: HTTP status code is not handled or not allowed 2026-01-25 06:45:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unistrut-1-58-4-hole-90-fitting-p1359eg-electro-galvanized>: HTTP status code is not handled or not allowed 2026-01-25 06:45:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/womens-gripper-front-lab-coat-white-polycombed-cotton-m>: HTTP status code is not handled or not allowed 2026-01-25 06:45:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unistrut-1-58-90-gusseted-fitting-wwelded-corner-p2484weg-7-hole-electro-galvanized>: HTTP status code is not handled or not allowed 2026-01-25 06:45:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-light-13-flush-mount-frosted-melon-glass-textured-white>: HTTP status code is not handled or not allowed 2026-01-25 06:45:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-lab-coat-navy-polycombed-cotton-regular-50>: HTTP status code is not handled or not allowed 2026-01-25 06:45:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-light-12-flush-mount-large-alabaster-mushroom-textured-white>: HTTP status code is not handled or not allowed 2026-01-25 06:45:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-fold-paper-towels-white-250-sheets-pack-16-packs-case>: HTTP status code is not handled or not allowed 2026-01-25 06:45:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unistrut-1-58-90-fitting-p1458eg-3-hole-electro-galvanized>: HTTP status code is not handled or not allowed 2026-01-25 06:45:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unistrut-1-58-rigid-steel-conduit-clamp-p1121eg-electro-galvanized-4>: HTTP status code is not handled or not allowed 2026-01-25 06:45:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unistrut-1-58-wing-shape-fitting-p2348eg-electro-galvanized>: HTTP status code is not handled or not allowed 2026-01-25 06:45:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-x-19-louver-panel-tan-3>: HTTP status code is not handled or not allowed 2026-01-25 06:45:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xx-garolite-bar-14-thick-x-2-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 06:45:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caster-wheel-assembly-for-pw23-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 06:45:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-mounting-kit-tan-2>: HTTP status code is not handled or not allowed 2026-01-25 06:45:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/empire-1-light-7-mini-pendant-w-hang-straight-canopy-old-bronze>: HTTP status code is not handled or not allowed 2026-01-25 06:45:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-bar-1-thick-x-1-14-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 06:45:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-plastic-bar-w-lse-acrylic-adhesive-12-thick-x-1-12-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 06:45:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/odp-w-rigid-base-150-hp-1800-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:45:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-assembly-for-b60zhd-pallet-trucks-hy-4616703-xl-a-sg>: HTTP status code is not handled or not allowed 2026-01-25 06:45:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-way-die-cast-zinc-trailer-connector-trailer-side-tc2002>: HTTP status code is not handled or not allowed 2026-01-25 06:45:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/odp-w-rigid-base-100-hp-3600-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:45:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-industrial-long-sleeve-work-shirt-gray-polycotton-regular-3xl>: HTTP status code is not handled or not allowed 2026-01-25 06:45:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drive-wheel-hub-for-mp-mpb-040-ac-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 06:45:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lab-coat-royal-blue-nomexaramid-xs>: HTTP status code is not handled or not allowed 2026-01-25 06:45:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-bar-1-14-thick-x-4-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 06:45:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-sheet-2-thick-x-48-wide-x-96-long>: HTTP status code is not handled or not allowed 2026-01-25 06:45:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-for-wp-2300-pallet-trucks-cr-813015>: HTTP status code is not handled or not allowed 2026-01-25 06:45:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x36x74-chrome-wire-shelving-with-32-6-h-shelf-bins-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:45:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/protective-rail-barrier-4-ft-rail-436724>: HTTP status code is not handled or not allowed 2026-01-25 06:45:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lodestar-electric-chain-hoist-1-4-ton-6-ft-lift-32-fpm-230v-1-2-hp-2-step-controller>: HTTP status code is not handled or not allowed 2026-01-25 06:45:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lodestar-electric-chain-hoist-1-8-ton-6-ft-lift-60-fpm-460v-1-2-hp-3-step-controller>: HTTP status code is not handled or not allowed 2026-01-25 06:45:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18-protective-rail-barrier-post-for-single-rail-436731>: HTTP status code is not handled or not allowed 2026-01-25 06:45:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-protective-rail-barrier-post-for-double-rail-436732>: HTTP status code is not handled or not allowed 2026-01-25 06:45:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hw405xl-heatworx-450-heat-resistant-gloves-1-pair-black-x-large>: HTTP status code is not handled or not allowed 2026-01-25 06:46:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-built-in-combination-lock-wrap-around-latch-technology-purple-dial>: HTTP status code is not handled or not allowed 2026-01-25 06:46:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-absorbent-tarp-5-x-5>: HTTP status code is not handled or not allowed 2026-01-25 06:46:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-pop-up-pool-plus-150-gallon>: HTTP status code is not handled or not allowed 2026-01-25 06:46:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-sheet-12-thick-x-32-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 06:46:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-assembly-for-rmx-50-65-pallet-trucks-pr-27626-01-a>: HTTP status code is not handled or not allowed 2026-01-25 06:46:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-glue-easy-squeeze-gel-14-oz-super-glue-liquid>: HTTP status code is not handled or not allowed 2026-01-25 06:46:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-sheet-2-thick-x-16-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 06:46:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-drum-rack-containment-systems-2-drum-system-no-drain>: HTTP status code is not handled or not allowed 2026-01-25 06:46:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-security-combination-padlock-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:46:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/axle-assembly-for-8300-8400-8500-pallet-trucks-ra-671-015-16>: HTTP status code is not handled or not allowed 2026-01-25 06:46:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/liquid-super-glue-clear-014oz-1-ea>: HTTP status code is not handled or not allowed 2026-01-25 06:46:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ez-squeeze-153-one-hole-punch-10-sheet-capacity-20-case>: HTTP status code is not handled or not allowed 2026-01-25 06:46:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-spill-pallet-p2-with-drain>: HTTP status code is not handled or not allowed 2026-01-25 06:46:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-sheet-2-12-thick-x-32-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 06:46:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/office-konnect-153-desk-organizer-power-base-with-phone-stand-6-case>: HTTP status code is not handled or not allowed 2026-01-25 06:46:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-assembly-for-wrt-60-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 06:46:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 77194 pages (at 100 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:46:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x36x75-steel-shelving-with-21-giant-stacking-bins-ivory>: HTTP status code is not handled or not allowed 2026-01-25 06:46:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-sheet-3-thick-x-24-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 06:46:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x36x75-steel-shelving-with-15-giant-stacking-bins-green>: HTTP status code is not handled or not allowed 2026-01-25 06:46:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-containment-berm-agriculture-economy-model-10-x-10-x-13>: HTTP status code is not handled or not allowed 2026-01-25 06:46:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drive-tire-hub-for-w-40z-b218-45z-c215-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 06:46:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/emergency-disconnect-switch-for-w-45z-hd-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 06:46:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/head-harness-for-7hbw23-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 06:46:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-duty-mixer-600-qts-cap-5-hp-direct-drive-4-5-impeller-dia>: HTTP status code is not handled or not allowed 2026-01-25 06:46:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/work-height-platform-truck-with-6-deep-deck-30-x-48-adj-height-polyurethane-retaining-lip>: HTTP status code is not handled or not allowed 2026-01-25 06:46:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/haws-axion-advantage-ax14-eye-face-wash-shower-system-abs-plastic>: HTTP status code is not handled or not allowed 2026-01-25 06:46:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-window-display-sign-15-x-12-58-clear>: HTTP status code is not handled or not allowed 2026-01-25 06:46:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tumbler-nt-9oz-clear>: HTTP status code is not handled or not allowed 2026-01-25 06:46:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tumbler-dm-12oz-ruby-red>: HTTP status code is not handled or not allowed 2026-01-25 06:46:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-assembly-for-wp-2300-pallet-trucks-cr-813014-xl>: HTTP status code is not handled or not allowed 2026-01-25 06:46:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-ner-electric-chain-hoist-hook-suspension-2-15-14>: HTTP status code is not handled or not allowed 2026-01-25 06:46:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/correll-desk-height-computer-table-30x72-gray-granite>: HTTP status code is not handled or not allowed 2026-01-25 06:46:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-sheet-116-thick-x-8-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 06:46:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/desk-organizer-with-9-compartments-and-3-inch-x-5-inch-photo-frame-clear>: HTTP status code is not handled or not allowed 2026-01-25 06:46:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camcover-10-13-16-amber>: HTTP status code is not handled or not allowed 2026-01-25 06:46:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/koch-filter-merv-8-high-capacity-xl8-pleated-extended-surface-24-inch-w-x-24-inch-h-x-1-inch-d>: HTTP status code is not handled or not allowed 2026-01-25 06:46:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/desktop-organizer-with-6-compartments-black>: HTTP status code is not handled or not allowed 2026-01-25 06:46:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-mesh-desktop-organizer-with-5-drawers-black>: HTTP status code is not handled or not allowed 2026-01-25 06:46:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camcover-12-clear>: HTTP status code is not handled or not allowed 2026-01-25 06:46:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/koch-filter-merv-11-standard-capacity-xl11-pleated-panel-ext-surface-20-inch-w-x-20-inch-h-x-2-inch-d>: HTTP status code is not handled or not allowed 2026-01-25 06:46:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-lb-lever-hoist-1-12-20-foot-lift>: HTTP status code is not handled or not allowed 2026-01-25 06:46:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-assembly-for-pe-4000-pallet-trucks-cr-100611-040>: HTTP status code is not handled or not allowed 2026-01-25 06:46:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/koch-filter-merv-8-high-capacity-xl8-pleated-extended-surface-16-inch-w-x-20-inch-h-x-2-inch-d>: HTTP status code is not handled or not allowed 2026-01-25 06:46:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-w-lse-acrylic-adhesive-132-thick-x-1-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 06:46:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gt-beam-trolly-22000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:46:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cup-mug-beige>: HTTP status code is not handled or not allowed 2026-01-25 06:46:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-plastic-bar-w-lse-acrylic-adhesive-18-thick-x-12-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 06:46:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storage-tote-orange-t142-19l-x-12-1-2w-x-4-3-8h>: HTTP status code is not handled or not allowed 2026-01-25 06:46:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pt-beam-trolly-2200-lb-capacity-3-to-8-flange-width>: HTTP status code is not handled or not allowed 2026-01-25 06:46:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bowl-nappie-12-5oz-slate-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:46:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-ner-electric-chain-hoist-motor-trolley-1-15-28>: HTTP status code is not handled or not allowed 2026-01-25 06:46:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x42x75-steel-shelving-with-27-magnum-giant-hopper-bins-red>: HTTP status code is not handled or not allowed 2026-01-25 06:46:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-cf-hand-chain-hoist-2-10-lift>: HTTP status code is not handled or not allowed 2026-01-25 06:46:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tandem-load-wheel-for-sp-3000-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 06:46:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-bank-top-chest-w-12-drawers-42-w-x-22-3-8-d-x-27-2-3-h-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:46:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-for-8210-pallet-trucks-ra-312636-000-xl>: HTTP status code is not handled or not allowed 2026-01-25 06:46:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-ner-dual-speed-elec-hoist-motor-trolley-14-20-539>: HTTP status code is not handled or not allowed 2026-01-25 06:46:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-sheet-4-thick-x-8-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 06:46:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-sner-electric-chain-hoist-push-trolley-1-10-7>: HTTP status code is not handled or not allowed 2026-01-25 06:46:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polycarbonate-plastic-sheet-18-thick-x-48-wide-x-60-long>: HTTP status code is not handled or not allowed 2026-01-25 06:46:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-bar-3-thick-x-4-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 06:46:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leak-alert>: HTTP status code is not handled or not allowed 2026-01-25 06:46:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-tcr-air-hoist-pendant-control-2-10-11>: HTTP status code is not handled or not allowed 2026-01-25 06:46:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/solar-powered-integrated-daylight-ai-smart-3200-lumens-standard-3000k>: HTTP status code is not handled or not allowed 2026-01-25 06:46:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wood-storage-galvanized-steel-shed-53661-8-15-16-w-x-3-9-16-d-x-5-1-4-h>: HTTP status code is not handled or not allowed 2026-01-25 06:46:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gripper-front-short-butcher-coat-wpockets-white-polyestercotton-s>: HTTP status code is not handled or not allowed 2026-01-25 06:46:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/solar-powered-mini-ufo-led-flood-light-pure-white>: HTTP status code is not handled or not allowed 2026-01-25 06:46:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-ner-dual-speed-elec-hoist-motor-trolley-1-10-1425>: HTTP status code is not handled or not allowed 2026-01-25 06:46:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zurn-z80000-pr1-low-consumption-pre-rinse-spray-valve-1-24-gpm>: HTTP status code is not handled or not allowed 2026-01-25 06:46:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gripper-front-butcher-coat-winterior-top-pocket-spun-polyester-white-s>: HTTP status code is not handled or not allowed 2026-01-25 06:46:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/non-insulated-hot-food-holding-cabinet-vhfa18>: HTTP status code is not handled or not allowed 2026-01-25 06:46:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/collarless-butcher-wrap-wo-pockets-light-blue-polyestercombed-cotton-m>: HTTP status code is not handled or not allowed 2026-01-25 06:46:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/harrington-ner-dual-speed-elec-hoist-motor-trolley-14-15-366>: HTTP status code is not handled or not allowed 2026-01-25 06:46:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-purpose-container-190-oz-5700ml-polystyrene-snap-lid-white-20-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:46:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pre-filled-container-20ml-0-67-oz-click-close-lid-attached-hazard-label-96-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:46:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-5211zz-double-row-angular-contact-bearing-double-shielded-55mm-bore-x-100mm-od-x-33-3mm-w>: HTTP status code is not handled or not allowed 2026-01-25 06:46:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/graduated-histology-container-1000ml-32-oz-polypropylene-yellow-screwcap-100-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:46:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-mulitfan-hood-shutter-wire-guard-120v>: HTTP status code is not handled or not allowed 2026-01-25 06:46:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/maxtrac-snow-blower-garden-tractor-tire-chains-4-link-spacing-pair-1062955>: HTTP status code is not handled or not allowed 2026-01-25 06:46:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/control-head-for-w-40z-b218-45z-c215-pallet-trucks-hy-1467778>: HTTP status code is not handled or not allowed 2026-01-25 06:46:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nachi-5312-2ns-double-row-angular-contact-bearing-double-sealed-60mm-bore-x-130mm-od-x-54mm-w>: HTTP status code is not handled or not allowed 2026-01-25 06:46:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/axle-for-pe-3000-pallet-trucks-cr-116802>: HTTP status code is not handled or not allowed 2026-01-25 06:46:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kwikool-portable-air-conditioner-kib1411-1-point-1-ton-13850-btu-with-two-12-in-condenser-flanges>: HTTP status code is not handled or not allowed 2026-01-25 06:46:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/antistatic-weighing-boat-with-pour-spout-146-x-89-x-25mm-140ml-polystyrene-case-of-250>: HTTP status code is not handled or not allowed 2026-01-25 06:46:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-bar-2-12-thick-x-2-12-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 06:46:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crc-food-grade-chain-lubes-16-oz-aerosol-can>: HTTP status code is not handled or not allowed 2026-01-25 06:46:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-14-thick-x-6-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 06:46:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/graduated-tite-rite-container-40ml-1-34-oz-polypropylene-48mm-opening-white-screwcap-600pk>: HTTP status code is not handled or not allowed 2026-01-25 06:46:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gripper-front-butcher-coat-wknit-cuffs-pocket-less-spun-polyester-white-3xl>: HTTP status code is not handled or not allowed 2026-01-25 06:46:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/18x36x74-chrome-wire-shelving-with-55-4-h-shelf-bins-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:46:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hinged-lid-industrial-tote-14619006-black-racer-red-12-gallon-45-liter-21-3-4-x-15-3-8-x-12-1-2>: HTTP status code is not handled or not allowed 2026-01-25 06:46:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24x36x74-chrome-wire-shelving-with-77-4-h-shelf-bins-green>: HTTP status code is not handled or not allowed 2026-01-25 06:46:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-pedestal-post-global-outdoor-fans-292448-292449>: HTTP status code is not handled or not allowed 2026-01-25 06:46:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-purpose-container-64-oz-1920ml-polyethylene-snap-lid-white-25-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:46:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ballymore-10-step-perforated-cantilever-ladder-42-overhang>: HTTP status code is not handled or not allowed 2026-01-25 06:46:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gripper-front-butcher-frock-wtop-2-lower-outside-pockets-white-polycotton-twill-m>: HTTP status code is not handled or not allowed 2026-01-25 06:46:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-assembly-for-mpw060e-a897-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 06:46:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mid-back-leathersoft-gaming-chair-adjustable-foot-ring-chrome-base-white>: HTTP status code is not handled or not allowed 2026-01-25 06:46:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-x-20-canvas-dropcloth>: HTTP status code is not handled or not allowed 2026-01-25 06:46:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bushing-for-pth-50-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 06:46:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-for-mpw-060len-pallet-trucks-yl-524140950>: HTTP status code is not handled or not allowed 2026-01-25 06:46:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quantum-univeral-mount-led-green-exit-w-3-face-panels>: HTTP status code is not handled or not allowed 2026-01-25 06:47:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-153-100-cotton-blue-huck-towels-50-lb-box>: HTTP status code is not handled or not allowed 2026-01-25 06:47:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-square-bar-height-breakroom-table-black>: HTTP status code is not handled or not allowed 2026-01-25 06:47:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-epoxy-wire-shelving-14545b-54w-x-14d-x-54h>: HTTP status code is not handled or not allowed 2026-01-25 06:47:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kd-bike-rack-6-bike-version>: HTTP status code is not handled or not allowed 2026-01-25 06:47:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spun-polyester-long-cook-shirt-black-spun-polyester-3xl>: HTTP status code is not handled or not allowed 2026-01-25 06:47:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/butcher-wrap-white-polyestercombed-cotton-5xl>: HTTP status code is not handled or not allowed 2026-01-25 06:47:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-men-s-10-viking-yellow-overshoe-w-4-way-cleated-outsole-pvc-size-medium>: HTTP status code is not handled or not allowed 2026-01-25 06:47:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flanged-bushing-for-w-series-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 06:47:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stratus-1-4-perforated-plastic-panel-ceiling-t-grid-pk5>: HTTP status code is not handled or not allowed 2026-01-25 06:47:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-button-front-chef-coat-pearl-buttons-white-polyestercotton-3xl>: HTTP status code is not handled or not allowed 2026-01-25 06:47:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/neoprene-rubber-strip-no-adhesive-50a-18-thick-x-3-wide-x-10-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 06:47:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-alligator-clips-w-10-ft-7x19-stranded-flex-steel-cable>: HTTP status code is not handled or not allowed 2026-01-25 06:47:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ice-storage-bin-30w-32-12d-46h-ss-finish>: HTTP status code is not handled or not allowed 2026-01-25 06:47:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/af-914ct-cold-temperature-aluminum-foil-tape-72-mm-x-150-ft>: HTTP status code is not handled or not allowed 2026-01-25 06:47:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 77293 pages (at 99 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:47:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-men-s-boot-16-flex-3-black-steel-toe-pvc-size-7>: HTTP status code is not handled or not allowed 2026-01-25 06:47:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inkjoy-50st-ballpoint-pens-1-mm-blue-ink-60-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:47:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-men-s-56-6-chest-wader-black-steel-toe-w-cleated-outsole-pvc-size-6>: HTTP status code is not handled or not allowed 2026-01-25 06:47:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-men-s-35-chest-wader-black-steel-toe-steel-midsole-pvc-size-7>: HTTP status code is not handled or not allowed 2026-01-25 06:47:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/housing-for-pe-4000-pallet-trucks-an-940>: HTTP status code is not handled or not allowed 2026-01-25 06:47:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22-inch-light-bar-for-f-pickup-truck-drill-free-cab-mount>: HTTP status code is not handled or not allowed 2026-01-25 06:47:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-assembly-for-mp-mpb-040-ac-pallet-trucks-yl-065269600-xl-a>: HTTP status code is not handled or not allowed 2026-01-25 06:47:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-grinders-buffer-712e-20c-2p-grndr-utility>: HTTP status code is not handled or not allowed 2026-01-25 06:47:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldor-grinders-buffer-7307d-20c-2p-grndr-deluxe>: HTTP status code is not handled or not allowed 2026-01-25 06:47:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drive-tire-for-st-3000-pallet-trucks-cr-127251>: HTTP status code is not handled or not allowed 2026-01-25 06:47:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-kit-for-electric-pallet-jack-truck-gwk-b80z-lw-xl-fits-hyster-model-b80z>: HTTP status code is not handled or not allowed 2026-01-25 06:47:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/21x24x69-chrome-wire-cart-with-11-3-h-grid-containers-red-1>: HTTP status code is not handled or not allowed 2026-01-25 06:47:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-kit-for-electric-pallet-jack-truck-gwk-pr3000-lw-fits-crown-model-pr-3000>: HTTP status code is not handled or not allowed 2026-01-25 06:47:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-button-front-chef-coat-pearl-buttons-white-polyestercotton-xs>: HTTP status code is not handled or not allowed 2026-01-25 06:47:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-webtex-yellow-jacket-w-attached-hood-pvc-size-2x>: HTTP status code is not handled or not allowed 2026-01-25 06:47:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/load-wheel-axle-for-wpt60-pallet-trucks>: HTTP status code is not handled or not allowed 2026-01-25 06:47:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/solenoid-for-102xm-walkie-pallet-truck>: HTTP status code is not handled or not allowed 2026-01-25 06:47:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-bar-34-thick-x-2-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 06:47:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/xx-garolite-rod-1-38-diameter-x-2-ft-long>: HTTP status code is not handled or not allowed 2026-01-25 06:47:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/female-headless-half-mannequin-wo-arms-with-base-upper-white>: HTTP status code is not handled or not allowed 2026-01-25 06:47:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/top-register-stand-wadj-rear-storage-72w-x-22d-x-42h-maple>: HTTP status code is not handled or not allowed 2026-01-25 06:47:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cap-plug-12-13mm-santoprene-for-vacuum-and-test-tubes-white-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:47:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/diamond-culture-tube-cap-for-25mm-glass-culture-tubes-polypropylene-yellow-500-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:47:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-15-yellow-steel-toe-boot-polyurethane-size-9>: HTTP status code is not handled or not allowed 2026-01-25 06:47:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wash-bottle-acetone-500ml-ldpe-multi-lingual-vented-red-ghs-labeled-5-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:47:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microscope-slides-diamond-glass-25-x-75mm-charged-90-ground-edges-aqua-frosted-72-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:47:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/female-mannequin-left-hand-on-hip-right-leg-sideways-gloss-finish-white>: HTTP status code is not handled or not allowed 2026-01-25 06:47:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ridgeline-full-brim-hard-hat-mate-copper-pattern-4-point-ratchet-suspension>: HTTP status code is not handled or not allowed 2026-01-25 06:47:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microscope-slides-diamond-glass-25-x-75mm-charged-90-ground-edges-blue-frosted-72-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:47:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ridgeline-full-brim-hard-hat-matte-white-graphite-pattern-4-point-ratchet-suspension>: HTTP status code is not handled or not allowed 2026-01-25 06:47:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/g1-full-facepiece-scba-respirator-medium-10161813>: HTTP status code is not handled or not allowed 2026-01-25 06:47:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milwaukee-chuck-key-48-66-3280>: HTTP status code is not handled or not allowed 2026-01-25 06:47:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/four-shelf-mesh-truck-24-x-60>: HTTP status code is not handled or not allowed 2026-01-25 06:47:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4xr-pump-sampling-line-25-ft-10153104>: HTTP status code is not handled or not allowed 2026-01-25 06:47:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/milwaukee-sds-bit-12-22-24-48-20-7479>: HTTP status code is not handled or not allowed 2026-01-25 06:47:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-18-thick-x-2-12-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 06:47:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/diamond-culture-tube-cap-for-25mm-glass-culture-tubes-polypropylene-green-500-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:47:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-116-thick-x-6-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 06:47:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/advantage-420-half-mask-respirator-small-10102182>: HTTP status code is not handled or not allowed 2026-01-25 06:47:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cap-flange-plug-13mm-lavender-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:47:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-men-s-boot-16-goliath-black-steel-toe-w-power-lug-outsole-pvc-size-14>: HTTP status code is not handled or not allowed 2026-01-25 06:47:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-men-s-boot-16-economy-black-steel-toe-steel-mid-sole-pvc-size-7>: HTTP status code is not handled or not allowed 2026-01-25 06:47:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fiberglass-fabric-reinforced-silicone-rubber-roll-no-adhesive-70a-116-thick-x-36w-x-10-ftl>: HTTP status code is not handled or not allowed 2026-01-25 06:47:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-aa-alkaline-batteries-24-per-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:47:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anchor-d-ring-with-stud-nut-washer>: HTTP status code is not handled or not allowed 2026-01-25 06:47:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4828-b-nantucket-decorative-corbels-black-polyethylene-8-x-7-x-4-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:47:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ramp-cleat-non-slip-outdoor-rubber-mat-18-thick-3-x-10-black>: HTTP status code is not handled or not allowed 2026-01-25 06:47:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slide-box-for-100-slides-cork-lined-orange>: HTTP status code is not handled or not allowed 2026-01-25 06:47:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4830-w-nantucket-window-box-white-polyethylene-25-gallon-capacity-36-x-10-x-115>: HTTP status code is not handled or not allowed 2026-01-25 06:47:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/door-scraper-non-slip-commercial-entrance-mat-58-thick-3-x-6-black>: HTTP status code is not handled or not allowed 2026-01-25 06:47:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-head-cap-screw-m8-x-1-0-x-70mm-steel-zinc-clear-class-8-8-din-960-pkg-of-50>: HTTP status code is not handled or not allowed 2026-01-25 06:47:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onguard-men-s-35-hip-wader-black-plain-toe-w-cleated-outsole-pvc-size-8>: HTTP status code is not handled or not allowed 2026-01-25 06:47:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-centrifugal-fan-metal-housing-766-cfm>: HTTP status code is not handled or not allowed 2026-01-25 06:47:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rotor-adapter-for-use-with-gcm-series-for-05ml-mct-tubes-8-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:47:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-back-draft-damper>: HTTP status code is not handled or not allowed 2026-01-25 06:47:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-profile-span-track-42-d-x-9-w-for-2-1-2-w-step-beams-796931>: HTTP status code is not handled or not allowed 2026-01-25 06:47:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-deluxe-office-partition-panel-48-1-4w-x-73-1-2h-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:47:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sample-tube-4ml-polypropylene-external-threads-ss-pg-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:47:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-sloped-shelf-portable-bin-cart-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:47:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-bar-1-14-thick-x-5-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 06:47:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gumwand-brass-brushes-10-pack-gw3>: HTTP status code is not handled or not allowed 2026-01-25 06:47:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2230-radar-climaplus-ceiling-panels-mineral-fiber-white-24-x-24>: HTTP status code is not handled or not allowed 2026-01-25 06:47:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2410-radar-climaplus-ceiling-panels-mineral-fiber-white-48-x-24>: HTTP status code is not handled or not allowed 2026-01-25 06:47:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hex-nut-m5-x-0-8-steel-zinc-clear-class-8-8-din-934-pkg-of-100>: HTTP status code is not handled or not allowed 2026-01-25 06:47:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cap-flange-plug-13mm-black-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:47:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-uhmw-polyethylene-plastic-sheet-34-thick-x-16-wide-x-32-long>: HTTP status code is not handled or not allowed 2026-01-25 06:47:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-uhmw-polyethylene-plastic-sheet-34-thick-x-16-wide-x-32-long>: HTTP status code is not handled or not allowed 2026-01-25 06:47:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paramount-janitorial-cabinet-cabinet-assembled-36x18x72-tan-269903tn>: HTTP status code is not handled or not allowed 2026-01-25 06:47:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-sheet-2-12-thick-x-12-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 06:47:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-office-partition-panel-with-window-60-1-4w-x-60h-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:47:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-lockout-hasp-steel-1-12-jaw-dia-x-2-38w-x-5l>: HTTP status code is not handled or not allowed 2026-01-25 06:47:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transfer-pipet-50ml-blood-bank-graduated-to-2ml-155mm-5000-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:47:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/freezing-box-3-cardboard-100-place-10x10-format-fits-3ml-4ml-and-5ml-vials-white-48-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:47:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/twin-rack-u-frame-48-w-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:47:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-suction-hose-d-38x700-261990-641250-641263>: HTTP status code is not handled or not allowed 2026-01-25 06:47:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-plastic-sheet-2-thick-x-24-wide-x-24-long>: HTTP status code is not handled or not allowed 2026-01-25 06:47:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/control-key-for-1925-series-padlocks>: HTTP status code is not handled or not allowed 2026-01-25 06:47:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tach-it-nh-8-120-light-heavy-weight-ul-recognized-cable-tie>: HTTP status code is not handled or not allowed 2026-01-25 06:47:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-sheet-3-thick-x-12-wide-x-48-long>: HTTP status code is not handled or not allowed 2026-01-25 06:47:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microscope-slides-diamond-white-glass-25-x-75mm-90-ground-edges-orange-frosted1440-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:47:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-safety-padlock-zenex-1-12w-x-1-34h-orange>: HTTP status code is not handled or not allowed 2026-01-25 06:47:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microscope-cover-glass-20mm-x-20mm--2-thickness-1-oz-vacuum-pack-10-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:47:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-vertical-hanger-bar-portable-bin-cart-black>: HTTP status code is not handled or not allowed 2026-01-25 06:47:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-safety-padlock-zenex-1-12w-x-1-34h-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:47:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-safety-padlock-zenex-1-12w-x-3h-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:48:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8-vertical-hanger-bar-portable-bin-cart-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:48:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/master-lock-lockout-station-16-hanger-clips-23-12w-x-4-12d-x-27h-deluxe>: HTTP status code is not handled or not allowed 2026-01-25 06:48:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-acetal-plastic-bar-w-lse-acrylic-adhesive-116-thick-x-1-12-wide-x-12-long>: HTTP status code is not handled or not allowed 2026-01-25 06:48:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reflective-strip-1-x-4-silver-order-by-the-sheet-16-strips>: HTTP status code is not handled or not allowed 2026-01-25 06:48:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/americana-vented-full-brim-with-4-point-slide-lock-suspension-white>: HTTP status code is not handled or not allowed 2026-01-25 06:48:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-tone-pullover-hooded-sweatshirt-ansi-class-3-lime-black-m-lbpuhsw-c3-m>: HTTP status code is not handled or not allowed 2026-01-25 06:48:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microscope-slides-diamond-glass-25-x-75mm-45-beveled-edges-clipped-corners1440-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:48:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-drawer-full-height-36-w-cabinet-red-2>: HTTP status code is not handled or not allowed 2026-01-25 06:48:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anvil-14x1-12-std-blk-steel-cw-nipple>: HTTP status code is not handled or not allowed 2026-01-25 06:48:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-drawer-bench-high-24-w-cabinet-red-6>: HTTP status code is not handled or not allowed 2026-01-25 06:48:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8160-clear-petg-shield-040>: HTTP status code is not handled or not allowed 2026-01-25 06:48:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storage-bin-cabinet-14-blue-hook-on-bins-2-adj-shelves-36wx18dx72h>: HTTP status code is not handled or not allowed 2026-01-25 06:48:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cap-plug-12mm-green-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:48:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/250k-btu-ng-radiant-heater-246649>: HTTP status code is not handled or not allowed 2026-01-25 06:48:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/small-parts-storage-cabinet-w-pegboard-96-blue-bins-2-shelves>: HTTP status code is not handled or not allowed 2026-01-25 06:48:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 77393 pages (at 100 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:48:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transfer-pipet-230ml-extra-long-300mm-12-inches-long-sterile-polypropylenes-100-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:48:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transfer-pipet-70ml-general-purpose-standard-155mm-500-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:48:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/200g-deicing-sprayer-55hp-ge660-pump-75-12-hose-manual-reel-7-pro-boom>: HTTP status code is not handled or not allowed 2026-01-25 06:48:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transfer-pipet-exact-volume-75ul-0075ml-75mm-long-5000-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:48:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transfer-pipet-40ml-general-purpose-blood-bank-130mm-5000-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:48:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-1-2-inch-dial-1-4-inch-npt-bottom-0-200-psi>: HTTP status code is not handled or not allowed 2026-01-25 06:48:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/300g-skid-sprayer-8hp-k55-pump-300-12-hose-electric-reel>: HTTP status code is not handled or not allowed 2026-01-25 06:48:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/storage-bin-cabinet-138-blue-hook-on-bins-3-adj-shelves-48wx24dx72h>: HTTP status code is not handled or not allowed 2026-01-25 06:48:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-keypad-see-thru-door-cabinet-46-ld-244-kp-48w-x-24d-x-78h>: HTTP status code is not handled or not allowed 2026-01-25 06:48:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-inch-dial-liquid-filled-1-4-inch-bottom-0-160psi>: HTTP status code is not handled or not allowed 2026-01-25 06:48:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e-1236vs-with-acu-rite-303-dro-with-taper-attachment>: HTTP status code is not handled or not allowed 2026-01-25 06:48:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-inch-dial-liquid-filled-1-4-inch-bottom-0-200psi>: HTTP status code is not handled or not allowed 2026-01-25 06:48:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-inch-dial-liquid-filled-1-4-inch-bottom-0-300psi>: HTTP status code is not handled or not allowed 2026-01-25 06:48:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e13-plastic-visor-carrier-for-omega-ii-americana-and-liberty-caps-black>: HTTP status code is not handled or not allowed 2026-01-25 06:48:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/63-high-post-chrome-leveler-post-2-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:48:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-x15-1-2-kraft-clasp-envelopes>: HTTP status code is not handled or not allowed 2026-01-25 06:48:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e-1236vs-with-taper-attachment>: HTTP status code is not handled or not allowed 2026-01-25 06:48:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transfer-pipet-58ml-fine-tip-157mm-sterile-polypropylenes-400-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:48:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jtm-1254rvs-with-newall-dp700-dro-x-powerfeed-air-power-drawbar>: HTTP status code is not handled or not allowed 2026-01-25 06:48:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-heavy-duty-tripod>: HTTP status code is not handled or not allowed 2026-01-25 06:48:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/149629-toughstripe-max-floor-marking-tape-vinyl-2-inch-wide-100-feet-long-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:48:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nps-signature-science-lab-table-black-24-x-60-hpl-top>: HTTP status code is not handled or not allowed 2026-01-25 06:48:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-drawer-bench-high-24-w-cabinet-light-gray-5>: HTTP status code is not handled or not allowed 2026-01-25 06:48:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wp3412-bk-2-port-decorator-outlet-strap-black>: HTTP status code is not handled or not allowed 2026-01-25 06:48:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e-1440vs-with-acu-rite-303-dro>: HTTP status code is not handled or not allowed 2026-01-25 06:48:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/accent-rail-rounded-w-textured-finish-aluminum-retainer-2-h-x-12-l-black>: HTTP status code is not handled or not allowed 2026-01-25 06:48:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/monthly-refillable-desk-pad-22-x-17-white-2016>: HTTP status code is not handled or not allowed 2026-01-25 06:48:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/38-deluxe-triple-brush-push-power-sweeper-132-gal-cap-bg-497>: HTTP status code is not handled or not allowed 2026-01-25 06:48:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vertical-air-curtain-merch-30-14l-56-38h-tac-14gs>: HTTP status code is not handled or not allowed 2026-01-25 06:48:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/siesta-maya-55-rectangle-dining-table-with-laminated-top-white>: HTTP status code is not handled or not allowed 2026-01-25 06:48:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-drawer-bench-high-30-w-cabinet-beige-3>: HTTP status code is not handled or not allowed 2026-01-25 06:48:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/siesta-ocean-square-resin-side-table-black>: HTTP status code is not handled or not allowed 2026-01-25 06:48:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72in-yellow-plus-ii-quarter-in-hose-with-sealright-22072>: HTTP status code is not handled or not allowed 2026-01-25 06:48:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tru-ray-construction-paper-12-x-18-holiday-green25-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:48:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/durham-steel-compartment-box-rack-bearing-20-x-15-3-4-x-15-with-4-of-24-compartment-boxes>: HTTP status code is not handled or not allowed 2026-01-25 06:48:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flux-cored-welding-wire-e71t-gs-035-x-10lb-spool>: HTTP status code is not handled or not allowed 2026-01-25 06:48:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wireless-tube-rack-16mm-full-size-72-place-nylon-orange>: HTTP status code is not handled or not allowed 2026-01-25 06:48:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/welders-essentials-kit-student-kit>: HTTP status code is not handled or not allowed 2026-01-25 06:48:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-swivel-ring-dsr-u-075>: HTTP status code is not handled or not allowed 2026-01-25 06:48:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fall-protection-anchorage-point-m-12>: HTTP status code is not handled or not allowed 2026-01-25 06:48:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e-1340vs-with-acu-rite-303-css-dro-with-taper-attachment-and-collet-closer>: HTTP status code is not handled or not allowed 2026-01-25 06:48:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50570-small-2-hole-bottle-holder-polyethylene>: HTTP status code is not handled or not allowed 2026-01-25 06:48:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-boxes-15-x-15-x-7->: HTTP status code is not handled or not allowed 2026-01-25 06:48:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microscope-slides-glass-25-x-75mm-45-beveled-edges-frosted-1-end-1-side1440-pk>: HTTP status code is not handled or not allowed 2026-01-25 06:48:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tong-6-scallop-white>: HTTP status code is not handled or not allowed 2026-01-25 06:48:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/test-tube-16-x-100mm-12ml-polypropylene-no-rim-2000-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:48:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/centrifuge-tube-10ml-oak-ridge-style-round-bottom-attached-screw-cap-polypropylene-12-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:48:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-corrugated-mailer-7-x-7-x-4->: HTTP status code is not handled or not allowed 2026-01-25 06:48:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/microscope-slides-glass-25-x-75mm-90-ground-edges-safety-corners-lilac-frosted1440-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:48:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-welded-gear-locker-with-door-foot-locker-24x24x72-red>: HTTP status code is not handled or not allowed 2026-01-25 06:48:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-8-27-npt-to-3-16-barbed-adapter-glass-filled-black-nylon>: HTTP status code is not handled or not allowed 2026-01-25 06:48:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/all-welded-gear-locker-with-door-foot-locker-legs-24x24x72-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:48:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kraft-corrugated-sheets-44-x-44>: HTTP status code is not handled or not allowed 2026-01-25 06:48:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pipette-diamond-fixed-volume-40ul-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:48:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pipette-diamond-pro-fixed-volume-500ul-orange>: HTTP status code is not handled or not allowed 2026-01-25 06:48:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mailing-tubes-with-caps-4-x-20>: HTTP status code is not handled or not allowed 2026-01-25 06:48:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evs-949-mill-with-3-axis-acu-rite-203-knee-dro-and-x-axis-jet-powerfeed>: HTTP status code is not handled or not allowed 2026-01-25 06:48:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72-x-36-maple-square-edge-mobile-work-bench-adjustable-height-1-3-4-top>: HTTP status code is not handled or not allowed 2026-01-25 06:48:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/budget-science-table-chemical-resistant-top-24x48>: HTTP status code is not handled or not allowed 2026-01-25 06:48:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-x-30-mobile-plastic-square-edge-work-bench-adjustable-height-1-5-8-top>: HTTP status code is not handled or not allowed 2026-01-25 06:48:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-36-gal-concrete-receptacle-dome-lid-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:48:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/advantage-full-brim-hard-hat-non-vented-4-pt-ratchet-suspension-white>: HTTP status code is not handled or not allowed 2026-01-25 06:48:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/crinkle-latex-coated-gloves-red-black-medium-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 06:48:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/centrifuge-tube-30ml-oak-ridge-round-bottom-style-attached-screw-cap-polypropylene-12-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:48:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/etm-949-mill-2-axis-newall-dp700-dro-x-axis-jet-powerfeed-usa-powered-draw-bar>: HTTP status code is not handled or not allowed 2026-01-25 06:48:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/blockhead-safety-hard-hat-8-pt-ratchet-suspension-full-brim-style-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:48:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/showfest-octagonal-10x15-black>: HTTP status code is not handled or not allowed 2026-01-25 06:48:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/etm-949-mill-with-x-y-axis-jet-powerfeeds-and-usa-air-powered-draw-bar>: HTTP status code is not handled or not allowed 2026-01-25 06:48:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-corrugated-mailer-7-x-6-x-3->: HTTP status code is not handled or not allowed 2026-01-25 06:48:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tube-stand-reinforced-polypropylene-13mm-31-place-red>: HTTP status code is not handled or not allowed 2026-01-25 06:48:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stackbin-4000-series-electric-lift-hardboard-over-stainless-96-w-x-36-d-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:48:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/market-tray-12x24-white>: HTTP status code is not handled or not allowed 2026-01-25 06:48:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/market-tray-12x24-green>: HTTP status code is not handled or not allowed 2026-01-25 06:48:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/etm-949-mill-with-2-axis-acu-rite-303-dro-and-x-y-z-axis-jet-powerfeeds>: HTTP status code is not handled or not allowed 2026-01-25 06:48:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/market-tray-8x30-green>: HTTP status code is not handled or not allowed 2026-01-25 06:48:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-x-12-x-06-lf-hopper-front-container-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:48:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/40-8691-1-1-2-inch-trans-flo-gold-aodd-pump-aluminum-with-ptfe-neoprene-diaphragms-fnpt-ports>: HTTP status code is not handled or not allowed 2026-01-25 06:48:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-economy-tissue-paper-18-x-24>: HTTP status code is not handled or not allowed 2026-01-25 06:48:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1849-sg-43-1-8-x-1-8-standard-cut-pointed-tree-bur>: HTTP status code is not handled or not allowed 2026-01-25 06:48:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/side-loading-boxes-44-x-6-x-35>: HTTP status code is not handled or not allowed 2026-01-25 06:48:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camrack-base-rack-7-1-2-inch-inside-stack-height-brown>: HTTP status code is not handled or not allowed 2026-01-25 06:48:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kraft-literature-mailer-9-x-7-1-2-x-3>: HTTP status code is not handled or not allowed 2026-01-25 06:48:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4251-95-3x2-inch-cast-iron-heavy-duty-straight-centrifugal-pump-buna-n-seal-15hp-215jm-3-phase>: HTTP status code is not handled or not allowed 2026-01-25 06:48:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tube-stand-reinforced-polypropylene-25mm-18-place-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:49:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1853-sd-3-3-8-x-1-4-aluminum-cut-ball-shaped-bur>: HTTP status code is not handled or not allowed 2026-01-25 06:49:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-industrial-duty-exhaust-fan-1-phase-1-3-hp>: HTTP status code is not handled or not allowed 2026-01-25 06:49:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-square-mailing-tubes-3-x-3-x-12->: HTTP status code is not handled or not allowed 2026-01-25 06:49:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bdb-919-belt-drive-bench-lathe-3-4-hp-115v-single-phase-with-stand>: HTTP status code is not handled or not allowed 2026-01-25 06:49:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/justrite-smokers-cease-fire-cigarette-butt-receptacle-4-gallon-beige>: HTTP status code is not handled or not allowed 2026-01-25 06:49:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-cantilever-192-inch-upright-48-inch-brace-kit-adder>: HTTP status code is not handled or not allowed 2026-01-25 06:49:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrugated-boxes-20-x-20-x-11->: HTTP status code is not handled or not allowed 2026-01-25 06:49:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1845-sa-42-3-32-x-1-8-standard-cut-cylindrical-bur-w-o-end-cut>: HTTP status code is not handled or not allowed 2026-01-25 06:49:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jumbo-mailing-tubes-8-x-60>: HTTP status code is not handled or not allowed 2026-01-25 06:49:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-corrugated-mailer-8-x-5-x-5->: HTTP status code is not handled or not allowed 2026-01-25 06:49:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/manual-aluminum-drum-truck-with-brake-12-wheels-short-chime-hook>: HTTP status code is not handled or not allowed 2026-01-25 06:49:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rv-pette-pro-dispenser-tip-starter-pack-for-repeat-volume-pipettors>: HTTP status code is not handled or not allowed 2026-01-25 06:49:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1852-sl-3l6-3-8-x-1-4-standard-cut-included-angle-bur>: HTTP status code is not handled or not allowed 2026-01-25 06:49:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securall-65g-sliding-door-vertical-flammable-drum-cabinet-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:49:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1850-sm-41-1-8-x-1-8-double-cut-pointed-cone-bur>: HTTP status code is not handled or not allowed 2026-01-25 06:49:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 77492 pages (at 99 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:49:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/open-front-smoking-shelter-half-side-panels-13-ft-9-inch-w-x-7-ft-d-x-7-ft-11-inch-h-clear-roof>: HTTP status code is not handled or not allowed 2026-01-25 06:49:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carson-magnivisor-deluxe-head-visor-magnifier>: HTTP status code is not handled or not allowed 2026-01-25 06:49:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securall-45g-self-close-acid-corrosive-cabinet-md-green>: HTTP status code is not handled or not allowed 2026-01-25 06:49:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paper-grocery-bags-25-8-1-4-w-x-5-1-4-d-x-18-h-kraft-500-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:49:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1845-sa-3l6-3-8-x-1-4-double-cut-cylindrical-bur-w-o-end-cut>: HTTP status code is not handled or not allowed 2026-01-25 06:49:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lt30701-1-hole-wall-mount-lavatory-cotton-white>: HTTP status code is not handled or not allowed 2026-01-25 06:49:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1847-sc-41-1-8-x-3-32-standard-cut-cylindrical-ball-nose-bur>: HTTP status code is not handled or not allowed 2026-01-25 06:49:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-1-2-molded-ring-binder-side-open-burgundy>: HTTP status code is not handled or not allowed 2026-01-25 06:49:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-deluxe-literature-mailer-16-x-12-x-4->: HTTP status code is not handled or not allowed 2026-01-25 06:49:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pvc-500-food-processor>: HTTP status code is not handled or not allowed 2026-01-25 06:49:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-industrial-8482-power-scissor-li-table-hand-foot-control-48-x-48-3300-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:49:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kraft-deluxe-literature-mailer-11-3-4-x-10-3-4-x-2-1-4>: HTTP status code is not handled or not allowed 2026-01-25 06:49:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vacuum-bottle-narrow-mouth-heavy-duty-hdpe-bottle-white-83mm-screw-cap-4-liters-10-gallons->: HTTP status code is not handled or not allowed 2026-01-25 06:49:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fantech-centrifugal-4-duct-fan-molded-housing-167>: HTTP status code is not handled or not allowed 2026-01-25 06:49:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/72xl037-standard-timing-belt-xl-3-8-x-7-3-16-t36-trapezoidal>: HTTP status code is not handled or not allowed 2026-01-25 06:49:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fantech-6-centrifugal-duct-fan-metal-housing-303>: HTTP status code is not handled or not allowed 2026-01-25 06:49:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1200-pound-single-door-magnetic-lock>: HTTP status code is not handled or not allowed 2026-01-25 06:49:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bottle-wide-mouth-polypropylene-attached-polypropylene-screw-cap-125ml-12-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:49:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-step-aluminum-rolling-ladder-16w-grip-tread-21d-top-step>: HTTP status code is not handled or not allowed 2026-01-25 06:49:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1856-sj-6-15-88mm-x-0mm-cle-sj-60-deg-standard-cut-included-angle-bur>: HTTP status code is not handled or not allowed 2026-01-25 06:49:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-x-40-roll-ultimat-ii-dark-blue-ur-3640xdb>: HTTP status code is not handled or not allowed 2026-01-25 06:49:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-step-aluminum-wheel-barrow-style-rolling-ladder-24w-plat-ribbed-tread-wlartr106244c>: HTTP status code is not handled or not allowed 2026-01-25 06:49:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/omnicart-chart-carousel-double-tier-counter-top-stand-60-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:49:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11-step-aluminum-rolling-ladder-24w-ribbed-tread-21d-top-step>: HTTP status code is not handled or not allowed 2026-01-25 06:49:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/170xl037-standard-timing-belt-xl-3-8-x-17-t85-trapezoidal>: HTTP status code is not handled or not allowed 2026-01-25 06:49:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-black-stealthnet-heavy-duty-bird-netting-25-x-25-n1x-b110>: HTTP status code is not handled or not allowed 2026-01-25 06:49:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cle-line-1846-sb-51-1-4-x-1-8-double-cut-cylindrical-bur-w-end-cut>: HTTP status code is not handled or not allowed 2026-01-25 06:49:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unex-gravity-flow-roller-rack-with-span-track-add-on-48w-x-96d-x-84h-with-4-levels>: HTTP status code is not handled or not allowed 2026-01-25 06:49:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unex-gravity-flow-roller-rack-with-span-track-add-on-96w-x-96d-x-84h-with-4-levels>: HTTP status code is not handled or not allowed 2026-01-25 06:49:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-step-cross-over-ladder-54-1-2-l>: HTTP status code is not handled or not allowed 2026-01-25 06:49:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unex-gravity-flow-roller-rack-with-knuckled-span-track-starter-96w-x-96d-x-84h-with-4-levels>: HTTP status code is not handled or not allowed 2026-01-25 06:49:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/banana-jack-alligator-clip>: HTTP status code is not handled or not allowed 2026-01-25 06:49:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yellow-jacket-case-quarts-vacuum-pump-oil>: HTTP status code is not handled or not allowed 2026-01-25 06:49:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/140mxl025-standard-timing-belt-mxl-1-4-x-14-t175-trapezoidal>: HTTP status code is not handled or not allowed 2026-01-25 06:49:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60-1-4-x-73-1-2-deluxe-office-partition-panel-blue-277533bl>: HTTP status code is not handled or not allowed 2026-01-25 06:49:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/catheter-procedure-cart-24l-x-48w-x-68h>: HTTP status code is not handled or not allowed 2026-01-25 06:49:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-door-protection-kit-6-x-10-5-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:49:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bottle-wide-mouth-ldpe-attached-polypropylene-screw-cap-500ml-12-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:49:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-1-2-topwind-square-jack-drop-leg-91310-26-travel>: HTTP status code is not handled or not allowed 2026-01-25 06:49:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-stair-climbing-hand-truck-300-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:49:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/48-1-4-x-47-1-2-h-deluxe-office-partition-panel-with-electric-gray-277552gy>: HTTP status code is not handled or not allowed 2026-01-25 06:49:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/supco-bullet-piercing-valve-38>: HTTP status code is not handled or not allowed 2026-01-25 06:49:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tpb36bkhkit-pegboard-back-panel-twilight-black-48-inch-x-24-inch>: HTTP status code is not handled or not allowed 2026-01-25 06:49:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/trustat-b80-vinyl-roll-blue-0080in-x-24in-x-50ft>: HTTP status code is not handled or not allowed 2026-01-25 06:49:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dissipative-dual-layer-rubber-roll-dark-blue-0060in-x-30in-x-50ft>: HTTP status code is not handled or not allowed 2026-01-25 06:49:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-storage-3210mcws-538l-x-418h-x-3w-multicolor-26-pc>: HTTP status code is not handled or not allowed 2026-01-25 06:49:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/statfree-uc2-dissipative-dual-layer-rubber-tray-liner-sky-blue-0080in-x-16in-x-24in>: HTTP status code is not handled or not allowed 2026-01-25 06:49:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/630xh200-standard-timing-belt-xh-2-x-63-t72-trapezoidal>: HTTP status code is not handled or not allowed 2026-01-25 06:49:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gg244-18-18-squeegee-trowel>: HTTP status code is not handled or not allowed 2026-01-25 06:49:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/255l050-standard-timing-belt-l-1-2-x-25-1-2-t68-trapezoidal>: HTTP status code is not handled or not allowed 2026-01-25 06:49:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/285l075-standard-timing-belt-l-3-4-x-28-1-2-t76-trapezoidal>: HTTP status code is not handled or not allowed 2026-01-25 06:49:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-station-transportable-jib-crane-8-h-x-6-span>: HTTP status code is not handled or not allowed 2026-01-25 06:49:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/supco-robert-shaw-41-402-replacementhot-surface-igniter>: HTTP status code is not handled or not allowed 2026-01-25 06:49:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-1-4-x-47-1-2-h-deluxe-office-partition-panel-non-electric-w-raceway-blue-277546bl>: HTTP status code is not handled or not allowed 2026-01-25 06:49:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/snap-n-rack-tube-rack-for-30mm-tubes-24-place-polypropylene-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:49:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rug-ram-carpet-pole-fork-mounted-120-l-1>: HTTP status code is not handled or not allowed 2026-01-25 06:49:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/375l100-standard-timing-belt-l-1-x-37-1-2-t100-trapezoidal>: HTTP status code is not handled or not allowed 2026-01-25 06:49:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/floating-router-plate>: HTTP status code is not handled or not allowed 2026-01-25 06:49:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-x-24-inch-heavy-duty-adjustable-height-steel-work-platform-gray-5-h-to-8-h>: HTTP status code is not handled or not allowed 2026-01-25 06:49:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pipette-tip-01-10ul-certified-universal-natural-31mm-pipetman-style-960-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:49:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magsnap-360-dual-wire-metal-wristband-large>: HTTP status code is not handled or not allowed 2026-01-25 06:49:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/penco-vanguard-executive-locker-24x24x72-no-legs-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:49:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/statshield-metal-in-bag-14in-x-18in-100-bagspack>: HTTP status code is not handled or not allowed 2026-01-25 06:49:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/floor-mount-round-safety-bollard-with-plastic-cap-and-base-yellow-60-height>: HTTP status code is not handled or not allowed 2026-01-25 06:49:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-mailbox-mailbox-w-9-w-x-21-d-x-7-h-white>: HTTP status code is not handled or not allowed 2026-01-25 06:49:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/retractable-ratchet-strap-for-motorized-hand-trucks>: HTTP status code is not handled or not allowed 2026-01-25 06:49:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/global-174-24-oz-cotton-cut-end-mop-head-4ply-wide-band>: HTTP status code is not handled or not allowed 2026-01-25 06:49:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pharmacy-4-6-cu-ft-solid-door-refrigerator1-7-cu-ft-solid-door-freezer-auto-defrost>: HTTP status code is not handled or not allowed 2026-01-25 06:49:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arc-welding-bench-w-see-thru-sides-center-drawer-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:49:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-drawer-bench-w-steel-casters-1-cabinet-2-drawers-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:49:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chemical-cartridges-acid-gas-2-pack-815356>: HTTP status code is not handled or not allowed 2026-01-25 06:49:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/laboratory-solid-door-5-2-cu-ft-refrigeratorsolid-door-4-cu-ft-capacity-freezer-40>: HTTP status code is not handled or not allowed 2026-01-25 06:49:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machine-w-2-shelves-60-w-x-18-d-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:49:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compact-hand-towel-roll-dispenser-smoke>: HTTP status code is not handled or not allowed 2026-01-25 06:49:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bottle-with-screwcap-wide-mouth-round-graduated-ldpe-cap-polypropylene-1000ml-20-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:49:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/schedule-40-pvc-3-way-l-fitting-2dia-white-36-u>: HTTP status code is not handled or not allowed 2026-01-25 06:49:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/structural-plastic-adjustable-vented-shelf-66-x-24>: HTTP status code is not handled or not allowed 2026-01-25 06:49:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/retractable-weld-screen-5-5-x-20-semi-orange-weld-shade>: HTTP status code is not handled or not allowed 2026-01-25 06:49:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/255h100-standard-timing-belt-h-1-x-25-1-2-t51-trapezoidal>: HTTP status code is not handled or not allowed 2026-01-25 06:49:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/first-aid-kit-25-person-ansi-compliant-plastic-436739>: HTTP status code is not handled or not allowed 2026-01-25 06:49:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-duct-tape-2-inch-x-60-yards-silver>: HTTP status code is not handled or not allowed 2026-01-25 06:49:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ul181b-fx-flexible-duct-closure-tape-3-inch-x-120-yards-black>: HTTP status code is not handled or not allowed 2026-01-25 06:49:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beaker-pmp-tpx-molded-graduations-100ml>: HTTP status code is not handled or not allowed 2026-01-25 06:49:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kflex-4-6-kflex-tubing>: HTTP status code is not handled or not allowed 2026-01-25 06:50:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wet-zone-store-board-accu-shield-fb-x-blue-on-black>: HTTP status code is not handled or not allowed 2026-01-25 06:50:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ex310nist-multimeter-wncv-redblack-nist-certified>: HTTP status code is not handled or not allowed 2026-01-25 06:50:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/satin-hat-and-coat-hook-silver>: HTTP status code is not handled or not allowed 2026-01-25 06:50:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spare-rc-blade>: HTTP status code is not handled or not allowed 2026-01-25 06:50:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-4-cu-yd-self-dumping-steel-hopper-57-with-bump-release-4000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:50:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ep-725-6-poly-tubing-dispenser-with-slide-cutter-6>: HTTP status code is not handled or not allowed 2026-01-25 06:50:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-duty-single-rivet-beam-36-w-x-2-3-d-x-1-1-4-h-14-gauge>: HTTP status code is not handled or not allowed 2026-01-25 06:50:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camkiosk-hand-sink-cart-granite-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:50:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/goal-post-dock-door-guard-steel-hdpe-plastic-yellow-120-x-120>: HTTP status code is not handled or not allowed 2026-01-25 06:50:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/225-22-bl-gn-225-cabinet-u-handles-cast-iron-with-tapped-holes>: HTTP status code is not handled or not allowed 2026-01-25 06:50:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kac-lc0-we-nw-hd-v-4xl-white>: HTTP status code is not handled or not allowed 2026-01-25 06:50:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kac-111nwi-10-21-green>: HTTP status code is not handled or not allowed 2026-01-25 06:50:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-size-partially-equipped-soda-service-granite-sand>: HTTP status code is not handled or not allowed 2026-01-25 06:50:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-rivet-angle-beam-60-w-x-48-d-x-84-h-4-level-starter-no-deck-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:50:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-twoline-guardrail-steel-hdpe-plastic-yellow-72-x-42>: HTTP status code is not handled or not allowed 2026-01-25 06:50:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 77591 pages (at 99 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:50:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smooth-bollard-post-sleeve-3-hdpe-dome-top-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:50:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pipette-tip-1-200ul-natural-for-use-with-mla-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:50:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tornado-dual-air-jumbo-series-wet-dry-industrial-vacuum-black>: HTTP status code is not handled or not allowed 2026-01-25 06:50:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-100bs13hx2-1-14-pitch-2-finished-bore-13-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:50:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-oneline-guardrail-steel-hdpe-plastic-yellow-144-x-36>: HTTP status code is not handled or not allowed 2026-01-25 06:50:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tuffblok-xl-tube-rack-4-way-polypropylene-green-5-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:50:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/king-size-pillow-cases-46-inch-x-42-inch-white-12-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:50:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tuffblok-xl-tube-rack-4-way-polypropylene-natural-5-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:50:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/funnel-buchner-polypropylene-80mm>: HTTP status code is not handled or not allowed 2026-01-25 06:50:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cap-insert-for-new-cryoclear-vials-green-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:50:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/carbon-dioxide-gas-cylinder-1013-5-bal-n2-17l-p>: HTTP status code is not handled or not allowed 2026-01-25 06:50:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securall-sump-liner-b8000>: HTTP status code is not handled or not allowed 2026-01-25 06:50:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securall-additional-explosion-proof-light-interior>: HTTP status code is not handled or not allowed 2026-01-25 06:50:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/queen-size-fitted-bed-sheets-80-inch-x-60-inch-x-12-inch-white-12-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:50:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/industrial-tank-hdpe-10-liter-24-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:50:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/optional-accordion-skirting-presto-4000-lb-pneumatic-scissor-li-s>: HTTP status code is not handled or not allowed 2026-01-25 06:50:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/beaker-with-handle-polypropylene-molded-graduations-1000ml-48-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:50:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/psnu1-8m-powerswap-nucleus-mini-lithium-iron-phosphate-battery>: HTTP status code is not handled or not allowed 2026-01-25 06:50:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-o-smith-direct-drive-blower-motor-1075-rpm-115-volts-2>: HTTP status code is not handled or not allowed 2026-01-25 06:50:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/siesta-dream-folding-outdoor-chair-black>: HTTP status code is not handled or not allowed 2026-01-25 06:50:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/biopsy-pad-for-cassettes-foam-blue-302mm-x-254mm-x-2mm-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:50:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-holder-for-72-long-station-heavy-gauge-formed-steel-pre-drilled-for-es12>: HTTP status code is not handled or not allowed 2026-01-25 06:50:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6355-krew-d-153-spf-50-sunscreen-lotion-32oz>: HTTP status code is not handled or not allowed 2026-01-25 06:50:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/securall-84x108x100-agri-chemical-building>: HTTP status code is not handled or not allowed 2026-01-25 06:50:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-o-smith-5-5-8-direct-drive-blower-motor-208-230-volts-1625-rpm-2>: HTTP status code is not handled or not allowed 2026-01-25 06:50:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kenwood-nx-p1300auk-5-watt-two-way-uhf-analog-portable-radio-451-470-mhz>: HTTP status code is not handled or not allowed 2026-01-25 06:50:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6355-krew-d-174-spf-50-sunscreen-lotion-32oz-4-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:50:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-12bs25hx25-metric-34-pitch-25mm-finished-bore-25-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:50:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8in-mold-on-rubber-rigid-caster-for-little-giant-carts>: HTTP status code is not handled or not allowed 2026-01-25 06:50:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/prolube-43552-90-hydraulic-coupler-1000-psi-1-8-npt>: HTTP status code is not handled or not allowed 2026-01-25 06:50:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-o-smith-5-5-8-masterfit-8482-indoor-blower-motor-115v-1>: HTTP status code is not handled or not allowed 2026-01-25 06:50:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-12bs25hx35-metric-34-pitch-35mm-finished-bore-25-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:50:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-o-smith-nesbitt-replacement-motor-850-rpm-115-volts-1-12hp>: HTTP status code is not handled or not allowed 2026-01-25 06:50:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-pallet-covers-50-x-46-x-86-3-mil-50-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:50:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50bs35hx34-58-pitch-34-finished-bore-35-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:50:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bag-taper-for-3-8-w-tape>: HTTP status code is not handled or not allowed 2026-01-25 06:50:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leviton-16262-w-15a-125v-decora-plus-duplex-receptacle-industrial-grade-white>: HTTP status code is not handled or not allowed 2026-01-25 06:50:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50bs34hx11516-58-pitch-1-1516-finished-bore-34-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:50:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-pallet-covers-51-x-49-x-85-2-mil-50-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:50:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50bs32hx1316-58-pitch-1-316-finished-bore-32-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:50:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chipboard-pads-12-x-18-x-0022-420-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:50:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/insulated-box-liners-8-x-8-x-8-25-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:50:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-twist-ties-6-x-5-32-yellow-2000-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:50:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bubble-roll-with-dispenser-24-x-50-x-1-2>: HTTP status code is not handled or not allowed 2026-01-25 06:50:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/50-10-6-2-amp-battery-charger>: HTTP status code is not handled or not allowed 2026-01-25 06:50:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compact-boutique-folding-crib-with-oversized-casters-foam-mattress-dapper-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:50:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/packard-c-frame-nutone-replacement-motor-120-volts-3000-rpm-8>: HTTP status code is not handled or not allowed 2026-01-25 06:50:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-crib-with-4-inch-oversized-casters-slatted-end-panel>: HTTP status code is not handled or not allowed 2026-01-25 06:50:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-bags-15-x-18-2-mil-red-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:50:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60bs25hx114-34-pitch-1-14-finished-bore-25-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:50:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-pallet-covers-46-x-36-x-72-3-mil-50-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:50:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hot-cold-water-therapy-bag-small-1-pack-11-1950>: HTTP status code is not handled or not allowed 2026-01-25 06:50:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-100bs11hx1316-1-14-pitch-1-316-finished-bore-11-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:50:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/100-amp-battery-load-tester>: HTTP status code is not handled or not allowed 2026-01-25 06:50:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/transport-tube-5ml-polypropylene-ss-attach-white-screwcap-mg-str-500-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:50:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-pallet-covers-48-x-40-x-48-3-mil-50-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:50:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bright-yellow-discs-3-4-dia>: HTTP status code is not handled or not allowed 2026-01-25 06:50:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-o-smith-5-shaded-pole-motor-208-230-volts-1050-rpm-1>: HTTP status code is not handled or not allowed 2026-01-25 06:50:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cold-bricks-9-x-4-x-1-1-2-6-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:50:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rubber-bulb-battery-filler>: HTTP status code is not handled or not allowed 2026-01-25 06:50:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/number-3-hydraulic-adjustable-height-packing-workbench-laminate-safety-edge-60-x-30-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:50:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-3-3-shaded-pole-open-motor-115-volts-3000-rpm-5>: HTTP status code is not handled or not allowed 2026-01-25 06:50:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-discs-3-4-dia>: HTTP status code is not handled or not allowed 2026-01-25 06:50:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50bs36x11516-58-pitch-1-1516-finished-bore-36-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:50:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shrink-bags-6-x-9-x-80-gauge-500-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:50:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/battery-charger-with-engine-starter-100-30-6-2-amp>: HTTP status code is not handled or not allowed 2026-01-25 06:50:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60bs26hx1716-34-pitch-1-716-finished-bore-26-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:50:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/number-1-adjustable-height-complete-packing-workbench-butcher-block-top-square-edge-72x36-black>: HTTP status code is not handled or not allowed 2026-01-25 06:50:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/battery-jumper-clip>: HTTP status code is not handled or not allowed 2026-01-25 06:50:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/a-o-smith-5-split-capacitor-split-capacitor-fan-coil-motor-1500-rpm-115v-1>: HTTP status code is not handled or not allowed 2026-01-25 06:50:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-2-4>: HTTP status code is not handled or not allowed 2026-01-25 06:50:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50bs28hx118-58-pitch-1-18-finished-bore-28-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:50:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/24-w-plastic-curved-push-broom-w-standard-socket-wood-steel-sectional-handle>: HTTP status code is not handled or not allowed 2026-01-25 06:50:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/desktop-charging-station-system-heavy-use-bundle-kbep-9b6c9>: HTTP status code is not handled or not allowed 2026-01-25 06:50:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/desktop-charging-station-system-medium-use-bundle-kbep-6b3c3>: HTTP status code is not handled or not allowed 2026-01-25 06:50:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-16bs14hx38-metric-1-pitch-38mm-finished-bore-14-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:50:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50bs24hx34-58-pitch-34-finished-bore-24-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:50:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bomber-ii-jacket-black-with-fluorescent-yellow-green-tape-5xl>: HTTP status code is not handled or not allowed 2026-01-25 06:50:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-3-3-shaded-pole-totally-enclosed-motor-115-volts-1500-rpm-12>: HTTP status code is not handled or not allowed 2026-01-25 06:50:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/top-load-only-3-5-fluorescent-red-black>: HTTP status code is not handled or not allowed 2026-01-25 06:50:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-10bs23hx24-metric-58-pitch-24mm-finished-bore-23-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:50:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/to-go-rack-silver-ep-24w-x-42l-x-63h-4-wire-shelves-dividers-ledges>: HTTP status code is not handled or not allowed 2026-01-25 06:51:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fluorescent-green-discs-1-2-dia>: HTTP status code is not handled or not allowed 2026-01-25 06:51:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50bs24hx114-58-pitch-1-14-finished-bore-24-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:51:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-3-3-double-sha-motor-115-volts-1500-rpm-4>: HTTP status code is not handled or not allowed 2026-01-25 06:51:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/six-tier-6-door-premium-steel-locker-12-w-x-18-d-x-12-h-black-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 06:51:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/six-tier-6-door-premium-steel-locker-12-w-x-15-d-x-12-h-black-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 06:51:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/radiant-leak-finder-container-8-oz>: HTTP status code is not handled or not allowed 2026-01-25 06:51:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/to-go-rack-chrome-24w-x-36l-x-34h-3-wire-shelves-dividers-ledges>: HTTP status code is not handled or not allowed 2026-01-25 06:51:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/to-go-rack-chrome-24w-x-30l-x-34h-3-wire-shelves-dividers-ledges>: HTTP status code is not handled or not allowed 2026-01-25 06:51:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mr-chain-2-medium-duty-stanchion-40-h-safety-orange-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 06:51:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/curbside-truck-chrome-21w-x-54l-x-69h-4-wire-shelves-polyurethane-2-rigid-casters>: HTTP status code is not handled or not allowed 2026-01-25 06:51:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/general-purpose-grade-colored-masking-tape-blue-48mm-x-55m-pack-of-24>: HTTP status code is not handled or not allowed 2026-01-25 06:51:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-41bs40x1-12-pitch-1-finished-bore-40-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:51:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-aluminum-camlock-fitting-female-coupler-x-mpt-thread>: HTTP status code is not handled or not allowed 2026-01-25 06:51:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/to-go-rack-silver-ep-18w-x-36l-x-34h-3-wire-shelves-dividers-ledges>: HTTP status code is not handled or not allowed 2026-01-25 06:51:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/non-flammable-gas-blank-4-4-3-4-green-white>: HTTP status code is not handled or not allowed 2026-01-25 06:51:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-120a10hx138-a-plate-1-12-pitch-1-38-bore-10-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:51:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 77689 pages (at 98 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:51:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-multi-color-vest-red-s-m>: HTTP status code is not handled or not allowed 2026-01-25 06:51:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hazard-class-7-fissile-4-4-white-black>: HTTP status code is not handled or not allowed 2026-01-25 06:51:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-3-3-shaded-pole-dra-inducer-blower-115-230-volts-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:51:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-10bs25hx19-metric-58-pitch-19mm-finished-bore-25-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:51:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sanipack-macerating-pump-1-2-hp-110v>: HTTP status code is not handled or not allowed 2026-01-25 06:51:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dust-mop-treatment-rtu-1-quart-bottles-12-case>: HTTP status code is not handled or not allowed 2026-01-25 06:51:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/calcium-lime-remover-1-gallon-bottles-4-case>: HTTP status code is not handled or not allowed 2026-01-25 06:51:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/liquid-laundry-detergent-55-gallon-drum>: HTTP status code is not handled or not allowed 2026-01-25 06:51:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inspected-4-4-green-white>: HTTP status code is not handled or not allowed 2026-01-25 06:51:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/inventory-date-1-3-8-2-white-black>: HTTP status code is not handled or not allowed 2026-01-25 06:51:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-voltage-thermostat-non-programmable-1h-1c-single-stage>: HTTP status code is not handled or not allowed 2026-01-25 06:51:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/low-voltage-thermostat-7-day-5-1-1-or-non-programmable-2h-2c>: HTTP status code is not handled or not allowed 2026-01-25 06:51:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/curbside-cart-chrome-24w-x-36l-x-43h-3-wire-shelves-pneumatic-2-rigid-casters>: HTTP status code is not handled or not allowed 2026-01-25 06:51:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-class-2-five-point-breakaway-vest-lime-5xl>: HTTP status code is not handled or not allowed 2026-01-25 06:51:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/non-ansi-teflon-protection-heavy-weight-sweatshirt-w-segment-tape-md>: HTTP status code is not handled or not allowed 2026-01-25 06:51:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/no-4696t-t-s-a-accepted-luggage-combination-padlock-1-5-16-inch-w-syo-2-pack-black>: HTTP status code is not handled or not allowed 2026-01-25 06:51:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-slotted-pallet-rack-beam-96-inches-l-x-5-1-2-inches-h-9-674-lbs-cap-per-pair>: HTTP status code is not handled or not allowed 2026-01-25 06:51:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stationary-tip-out-bin-unit-double-sided-24-h-beige>: HTTP status code is not handled or not allowed 2026-01-25 06:51:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-3-night-glow-sherpa-line-heavy-weight-sierra-jacket-lg>: HTTP status code is not handled or not allowed 2026-01-25 06:51:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/class-3-premium-two-tone-hooded-rain-coat-black-bottom-lime-4-5xl>: HTTP status code is not handled or not allowed 2026-01-25 06:51:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80bs24x21516-1-pitch-2-1516-finished-bore-24-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:51:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/battery-powered-hepa-wet-dry-vacuum-2-6-gallon-cap>: HTTP status code is not handled or not allowed 2026-01-25 06:51:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-drawer-mobile-cabinet-48-wx24-dx37-1-2-h-avalanche-blue-3>: HTTP status code is not handled or not allowed 2026-01-25 06:51:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/must-ship-today-3-5-fluorescent-red-black>: HTTP status code is not handled or not allowed 2026-01-25 06:51:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90-tooth-and-12-point-flex-head-metric-combination-ratcheting-wrench-set-of-12>: HTTP status code is not handled or not allowed 2026-01-25 06:51:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-drawer-double-mobile-cabinet-60-wx27-dx45-1-2-h-black>: HTTP status code is not handled or not allowed 2026-01-25 06:51:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/corrosive-liquid-acidic-inorganic-nos-un3264-4-4-3-4-white-black>: HTTP status code is not handled or not allowed 2026-01-25 06:51:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/646-1w-3-8-mnpt-x-1-4-mnpt-hex-nipple-hose-fitting>: HTTP status code is not handled or not allowed 2026-01-25 06:51:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60btl40-taper-bushed-34-pitch-40-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:51:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caution-this-side-up-3-4-red-white>: HTTP status code is not handled or not allowed 2026-01-25 06:51:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6-drawer-mobile-cabinet-36-wx24-dx37-1-2-h-light-gray-1>: HTTP status code is not handled or not allowed 2026-01-25 06:51:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fragile-magnetic-tape-4-4-3-4-white-red>: HTTP status code is not handled or not allowed 2026-01-25 06:51:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-drawer-mobile-cabinet-36-wx24-dx35-1-2-h-avalanche-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:51:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/646w-1-4-mnpt-hex-nipple>: HTTP status code is not handled or not allowed 2026-01-25 06:51:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sunburst-discreet-fly-light-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:51:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-eu0752b-3-4-hp-1-phase-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:51:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tootsie-child-s-play-candy-variety-bag-4-75-lb-gr>: HTTP status code is not handled or not allowed 2026-01-25 06:51:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-drawer-mobile-cabinet-24-wx21-dx35-1-4-h-red-1>: HTTP status code is not handled or not allowed 2026-01-25 06:51:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/slide-hammer-set-10-box-slugger-81100>: HTTP status code is not handled or not allowed 2026-01-25 06:51:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exq-4-drawer-2-shelf-pro-side-cabinet-24-w-x-30-d-x-h-63-3-8-blue-w-black-handles>: HTTP status code is not handled or not allowed 2026-01-25 06:51:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vacuum-pump-6cfm-electric-15600>: HTTP status code is not handled or not allowed 2026-01-25 06:51:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/can-test-box-3415>: HTTP status code is not handled or not allowed 2026-01-25 06:51:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pro-fuel-system-tester-mv5545>: HTTP status code is not handled or not allowed 2026-01-25 06:51:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-js075upr-3-4-hp-1-phase-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:51:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-drawer-mobile-multi-drawer-cabinet-48-wx27-dx37-1-2-h-beige>: HTTP status code is not handled or not allowed 2026-01-25 06:51:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-1004-1-4-hp-1-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:51:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/26106-companion-labels-8-3-4-x-5-3-4-black-red-white-200-roll>: HTTP status code is not handled or not allowed 2026-01-25 06:51:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ductless-mini-split-air-conditioner-w-heat-pump-wifi-enabled-12000-btu-19-seer-230v-white>: HTTP status code is not handled or not allowed 2026-01-25 06:51:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heinz-ketchup-squeeze-bottle-20-oz-3-pack-gr>: HTTP status code is not handled or not allowed 2026-01-25 06:51:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-step-18-w-aluminum-wheelbarrow-ladder-serrated-grating>: HTTP status code is not handled or not allowed 2026-01-25 06:51:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-ec13b-2-hp-1-phase-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:51:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-6127-1-3-hp-1-phase-1625-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:51:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-step-16-wx60-d-stainless-steel-rolling-safety-ladder-perforated-tread>: HTTP status code is not handled or not allowed 2026-01-25 06:51:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lindor-assorted-chocolate-truffles-5-1-oz-3-pack-gr>: HTTP status code is not handled or not allowed 2026-01-25 06:51:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-t12c2zcr-1-2-hp-1-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:51:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-step-24-wx76-d-stainless-steel-rolling-safety-ladder-serrated-grating>: HTTP status code is not handled or not allowed 2026-01-25 06:51:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wide-collapsible-hamper-steel-black-vinyl-bag>: HTTP status code is not handled or not allowed 2026-01-25 06:51:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-step-24-wx90-d-stainless-steel-rolling-safety-ladder-perforated-tread>: HTTP status code is not handled or not allowed 2026-01-25 06:51:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/11-step-24-wx82-d-stainless-steel-rolling-safety-ladder-serrated-grating>: HTTP status code is not handled or not allowed 2026-01-25 06:51:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/big-party-snack-box-gr>: HTTP status code is not handled or not allowed 2026-01-25 06:51:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nissin-cup-noodles-with-shrimp-2-25-oz-24-count-gr>: HTTP status code is not handled or not allowed 2026-01-25 06:51:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kraft-mac-cheese-easy-mac-cups-12-count-gr>: HTTP status code is not handled or not allowed 2026-01-25 06:51:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-t2c1j14cr-2-hp-1-phase-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:51:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drywall-screw-gun-kit-with-collated-drywall-screwgun-attachment-dcf620cm2>: HTTP status code is not handled or not allowed 2026-01-25 06:51:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-step-24-wx58-d-stainless-steel-rolling-safety-ladder-perforated-tread>: HTTP status code is not handled or not allowed 2026-01-25 06:51:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-step-18-w-aluminum-tilt-roll-ladder-serrated-grating>: HTTP status code is not handled or not allowed 2026-01-25 06:51:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-step-16-w-stainless-steel-tilt-roll-ladder-serrated-grating>: HTTP status code is not handled or not allowed 2026-01-25 06:51:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/20-deep-mobile-pedestal-slim-ped-box-file-with-hidden-drawer-front-black>: HTTP status code is not handled or not allowed 2026-01-25 06:51:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/42-wide-2-drawer-lateral-file-cabinet-arctic-silver>: HTTP status code is not handled or not allowed 2026-01-25 06:51:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ax-series-18-shutter-mounted-fan-1-3-hp-1700-rpm-230-460v>: HTTP status code is not handled or not allowed 2026-01-25 06:51:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brio-plus-commerical-steam-cleaner-kit>: HTTP status code is not handled or not allowed 2026-01-25 06:51:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-ada-cooler-stainless-steel-2-statio-wall-hung-115v-60hz-5-amps>: HTTP status code is not handled or not allowed 2026-01-25 06:51:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-floor-cooler-light-gray-granite-floor-115v-60hz-5-amps>: HTTP status code is not handled or not allowed 2026-01-25 06:51:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-wd1c2jcr-1-hp-1-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:51:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pipeline-gloss-white-adjustable-2-way-garment-rack>: HTTP status code is not handled or not allowed 2026-01-25 06:51:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/linea-54-in-w-x-66-in-h-statuary-bronze-garment-rack>: HTTP status code is not handled or not allowed 2026-01-25 06:51:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-eb978-2-1-4-hp-1-phase-3450-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:51:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steel-shelving-48-wx24-dx75-h-closed-4-shelf-7-drawer-red>: HTTP status code is not handled or not allowed 2026-01-25 06:51:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-agl15fl1h2-special-hp-1-phase-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:51:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-eh492-3-4-hp-3-phase-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:51:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compact-pro-industrial-led-high-bay-12-000-lumens-120v-277v-5000k-white>: HTTP status code is not handled or not allowed 2026-01-25 06:51:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/contractor-select-8-watt-emergency-led-driver-ca-title-20-white>: HTTP status code is not handled or not allowed 2026-01-25 06:52:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/csvt-low-bay-vapor-tight-led-strip-light-48-inch-3000-5000-lumens-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:52:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/height-adjustable-table-60-x-30-cherry-black-base>: HTTP status code is not handled or not allowed 2026-01-25 06:52:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wesco-self-propelled-battery-powered-platform-truck-30x60-1100-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 06:52:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/closed-shelving-drawer-unit-36x18x39-45-drawers-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:52:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/closed-shelving-drawer-unit-36x12x75-72-drawers-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:52:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/closed-shelving-drawer-unit-36x12x75-72-drawers-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:52:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/closed-shelving-drawer-unit-36x18x75-90-drawers-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:52:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-d32c2j149-1-1-2-hp-1-phase-1725-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:52:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/telescopic-transmission-jack-0-5-ton-178-20-lbs>: HTTP status code is not handled or not allowed 2026-01-25 06:52:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lista-5-drawer-18-w-mobile-cabinet-classic-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:52:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wwe-khn47-10-1-h-182-4tc-cast-iron-helical-bevel-speed-reducer-182-4tc-input-flange-10-1-foot-mt>: HTTP status code is not handled or not allowed 2026-01-25 06:52:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flask-erlenmeyer-globe-glass-narrow-mouth-dual-graduations-astm-e1404-250ml-12-box>: HTTP status code is not handled or not allowed 2026-01-25 06:52:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wwe-khn97-10-1-h-284-6tc-cast-iron-helical-bevel-speed-reducer-284-6tc-input-flange-10-1-foot-mt>: HTTP status code is not handled or not allowed 2026-01-25 06:52:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unslotted-steel-pallet-rack-beam-120inl-x-5-1-2inh-5630-lb-cap-set-of-2>: HTTP status code is not handled or not allowed 2026-01-25 06:52:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-ee508-3-4-hp-3-phase-3450-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:52:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 77787 pages (at 98 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:52:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-ready-built-ii-locker-six-tier-3-wide-12x18x78-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:52:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/superior-portable-gate-11-to-15-openings>: HTTP status code is not handled or not allowed 2026-01-25 06:52:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-h7e3g-7-5-hp-3-phase-1185-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:52:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-ready-built-ii-locker-six-tier-1-wide-12x15x78-parchment>: HTTP status code is not handled or not allowed 2026-01-25 06:52:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wwe-khn87-30-1-h-254-6tc-cast-iron-helical-bevel-speed-reducer-254-6tc-input-flange-30-1-foot-mt>: HTTP status code is not handled or not allowed 2026-01-25 06:52:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-knock-down-corrosion-resistant-locker-single-tier-3-wide-12x18x78>: HTTP status code is not handled or not allowed 2026-01-25 06:52:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-ready-built-ii-locker-double-tier-1-wide-12x15x78-parchment>: HTTP status code is not handled or not allowed 2026-01-25 06:52:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-d7p2h-7-5-hp-3-phase-1765-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:52:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/182004-rfx-lp600-low-profile-wireless-wheel-load-scale-20000-x-20-lb>: HTTP status code is not handled or not allowed 2026-01-25 06:52:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-ready-built-ii-locker-single-tier-3-wide-12x12x78-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:52:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compact-respiratory-hygiene-dispenser-station-pt-dhs>: HTTP status code is not handled or not allowed 2026-01-25 06:52:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-ready-built-ii-locker-double-tier-3-wide-12x12x78-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:52:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-s7p2d-7-5-hp-3-phase-1765-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:52:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-welded-double-point-ventilated-locker-double-tier-1-wide-18x18x72>: HTTP status code is not handled or not allowed 2026-01-25 06:52:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hallowell-assembled-corrosion-resistant-locker-double-tier-3-wide-12x18x78>: HTTP status code is not handled or not allowed 2026-01-25 06:52:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fully-powered-double-mast-stacker-with-118-adj-and-straddle-legs-2200-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 06:52:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/braeside-displays-hp-2432-hg-hanging-sneeze-guard-acrylic-23-1-2-w-x-31-1-2-h>: HTTP status code is not handled or not allowed 2026-01-25 06:52:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-s3p3d-3-hp-3-phase-1180-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:52:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-rivet-steel-shelving-with-wood-deck-96-w-x-24-d-x-96-h-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:52:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lau-3-blade-condenser-propeller-10-diameter-cw-rotation>: HTTP status code is not handled or not allowed 2026-01-25 06:52:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-inch-heavy-duty-plastic-chain-25-feet-red>: HTTP status code is not handled or not allowed 2026-01-25 06:52:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-g642-0-75-hp-dc-phase-1750-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:52:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-galvanized-steel-blower-wheel-3-13-16-diameter-5-16-bore-3>: HTTP status code is not handled or not allowed 2026-01-25 06:52:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-chain-2-inch-links-25-feet-trade-size-8-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:52:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-master-link-2-inch-link-black-10-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:52:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lau-4-blade-condenser-propeller-24-diameter-ccw-rotation-1>: HTTP status code is not handled or not allowed 2026-01-25 06:52:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-galvanized-steel-blower-wheel-3-13-16-diameter-5-16-bore-1>: HTTP status code is not handled or not allowed 2026-01-25 06:52:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/traffic-cone-and-chain-kit-traffic-orange>: HTTP status code is not handled or not allowed 2026-01-25 06:52:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-chain-1-1-2-inch-links-in-a-pail-300-feet-trade-size-6-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:52:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fasco-double-inlet-blower-wheel-6-3-16-diameter-1-2-bore-2>: HTTP status code is not handled or not allowed 2026-01-25 06:52:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/light-duty-36-inch-ground-pole-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:52:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-c200p2g-200-hp-3-phase-1785-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:52:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-bags-9x12-6-mil-500-ctn>: HTTP status code is not handled or not allowed 2026-01-25 06:52:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-poly-bags-6x14-1-5-mil-1000-ctn>: HTTP status code is not handled or not allowed 2026-01-25 06:52:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-c150p3cb-150-hp-3-phase-1188-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:52:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-d2p3d-2-hp-3-phase-1175-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:52:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-poly-bags-13x20-1-mil-1000-ctn>: HTTP status code is not handled or not allowed 2026-01-25 06:52:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cotter-pin-assortment-stainless-steel-plain-1200-piece>: HTTP status code is not handled or not allowed 2026-01-25 06:52:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-3360-1-5-hp-1-phase-1040-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:52:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-rod-6-30-degree-bend>: HTTP status code is not handled or not allowed 2026-01-25 06:52:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wwe-kan87-50-1-h-184tc-cast-iron-helical-bevel-speed-reducer-184tc-input-flange-50-1>: HTTP status code is not handled or not allowed 2026-01-25 06:52:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wwe-kan47-7-5-1-h-143-5tc-cast-iron-helical-bevel-speed-reducer-143-5tc-input-flange-7-5-1>: HTTP status code is not handled or not allowed 2026-01-25 06:52:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wwe-kan77-80-1-h-182tc-cast-iron-helical-bevel-speed-reducer-182tc-input-flange-80-1>: HTTP status code is not handled or not allowed 2026-01-25 06:52:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wwe-kan67-110-1-h-143tc-cast-iron-helical-bevel-speed-reducer-143tc-input-flange-110-1>: HTTP status code is not handled or not allowed 2026-01-25 06:52:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/us-motor-5841-1-2-hp-1-phase-1075-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:52:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bin-clip>: HTTP status code is not handled or not allowed 2026-01-25 06:52:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-pegboard-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-25 06:52:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wwe-kan77-30-1-h-184tc-cast-iron-helical-bevel-speed-reducer-184tc-input-flange-30-1>: HTTP status code is not handled or not allowed 2026-01-25 06:52:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wwe-kan67-15-1-h-143-5tc-cast-iron-helical-bevel-speed-reducer-143-5tc-input-flange-15-1>: HTTP status code is not handled or not allowed 2026-01-25 06:52:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kan-series-helical-bevel-speed-reducer-b5-output-flg-kit-size-77-b5-output-flg-kit-hardw-incl>: HTTP status code is not handled or not allowed 2026-01-25 06:52:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/interion-174-mobile-office-partition-panel-48-1-4-w-x-96-h-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:52:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-drawer-steel-flat-file-30-x-42-documents-black>: HTTP status code is not handled or not allowed 2026-01-25 06:52:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/10-drawer-steel-flat-file-30-x-42-documents-sand>: HTTP status code is not handled or not allowed 2026-01-25 06:52:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-bar-675-w-cold-plate-chicago>: HTTP status code is not handled or not allowed 2026-01-25 06:52:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arg15pv-pharma-vac-performance-series-upright-vaccine-refrigerator-glass-door-15-cu-ft>: HTTP status code is not handled or not allowed 2026-01-25 06:52:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arg12pv-pharma-vac-performance-series-upright-vaccine-refrigerator-glass-door-12-cu-ft>: HTTP status code is not handled or not allowed 2026-01-25 06:52:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/versa-food-bars-serving-buffet-cold-food-72-x-29-low-black>: HTTP status code is not handled or not allowed 2026-01-25 06:52:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ars15pv-pharma-vac-performance-series-upright-vaccine-refrigerator-solid-door-15-cu-ft>: HTTP status code is not handled or not allowed 2026-01-25 06:52:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ergonomic-8-h-adjustable-footrest-qty-4>: HTTP status code is not handled or not allowed 2026-01-25 06:52:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/versa-food-bars-serving-buffet-cold-food-60-x-29-low-6-heavy-duty-swivel-casters-with-brakes-navy-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:52:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/letter-size-steel-wall-rack>: HTTP status code is not handled or not allowed 2026-01-25 06:52:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-24-pamphlet-display>: HTTP status code is not handled or not allowed 2026-01-25 06:52:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/locker-base-kit-for-3-lockers-36-x-15-black>: HTTP status code is not handled or not allowed 2026-01-25 06:52:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobile-beverage-cart>: HTTP status code is not handled or not allowed 2026-01-25 06:52:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/locker-base-12-x-12-black>: HTTP status code is not handled or not allowed 2026-01-25 06:52:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camwear-colander-12-3-4-x-20-7-8-x-5-deep-fits-1-1-size-food-pans-clear-polycarbonate-nsf>: HTTP status code is not handled or not allowed 2026-01-25 06:52:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-locker-double-tier-1-wide-15-x-18-x-36-deep-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:52:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/end-panel-for-slope-top-locker-18-x-60-beige>: HTTP status code is not handled or not allowed 2026-01-25 06:52:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hot-cold-cart-tall-granite-gray-fahrenheit>: HTTP status code is not handled or not allowed 2026-01-25 06:52:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camchiller-cold-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:52:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gray-counter-blue-panel-l-shaped-reception-station-2>: HTTP status code is not handled or not allowed 2026-01-25 06:52:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camshelving-elements-mobile-starter-unit-18-w-x-48-l-x-78-h-4-shelf-4-premium-casters-brushed-graphite>: HTTP status code is not handled or not allowed 2026-01-25 06:52:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-mixed-recyclables-waste-30-gallon-western-breeze-27-15-30>: HTTP status code is not handled or not allowed 2026-01-25 06:52:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/versa-food-bars-work-table-cold-food-60-x-36-standard-6-swivel-casters-with-brakes-black>: HTTP status code is not handled or not allowed 2026-01-25 06:52:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dock-door-seal-61-heavy-duty-40-oz-w-wear-pleats-8w-x-10h-20-projection-black>: HTTP status code is not handled or not allowed 2026-01-25 06:52:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ultra-camtainer-beverage-carrier-insulated-plastic-10-1-2-gallon-capacity-slate-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:52:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dock-door-seal-51-heavy-duty-40-oz-8w-x-8h-20-projection-black>: HTTP status code is not handled or not allowed 2026-01-25 06:52:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s0300-safety-spectacle-adjustable-temple-side-shields-anti-scratch-black-clear>: HTTP status code is not handled or not allowed 2026-01-25 06:52:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/adjustable-spreader-li-ing-beam-4000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:52:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-flat-top-cabinet-sliding-doors-30-x-60>: HTTP status code is not handled or not allowed 2026-01-25 06:52:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-channel-frame-mirror-48-wx36-h>: HTTP status code is not handled or not allowed 2026-01-25 06:53:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/round-storage-container-18-qt-covers-sold-separately-clear-nsf>: HTTP status code is not handled or not allowed 2026-01-25 06:53:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/camshelving-elements-shelf-plate-kit-21-w-x-72-l-venteded-brushed-graphite>: HTTP status code is not handled or not allowed 2026-01-25 06:53:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reducing-manifold-5-16-tube-x-5-32-tube-x-1-8-swift-fit>: HTTP status code is not handled or not allowed 2026-01-25 06:53:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-angle-frame-mirror-with-shelf-24-wx30-h>: HTTP status code is not handled or not allowed 2026-01-25 06:53:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caster-base-set-96>: HTTP status code is not handled or not allowed 2026-01-25 06:53:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plug-1-2-tube-black>: HTTP status code is not handled or not allowed 2026-01-25 06:53:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/meal-delivery-cart-tall-profile-1-door-28x36-1-2x58-1-8-granite-green-with-cream-color-door>: HTTP status code is not handled or not allowed 2026-01-25 06:53:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/economy-workbench-2-3-4-backsplash-legs-crossbracing-72-w-x-30-d>: HTTP status code is not handled or not allowed 2026-01-25 06:53:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/swivel-male-elbow-release-collet-4mm-tube-x-m5-bspp>: HTTP status code is not handled or not allowed 2026-01-25 06:53:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mini-bulkhead-connector-6mm-tube>: HTTP status code is not handled or not allowed 2026-01-25 06:53:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mini-swivel-elbow-6mm-tube-x-m5>: HTTP status code is not handled or not allowed 2026-01-25 06:53:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/male-adapter-10mm-straight-x-3-8-bspp>: HTTP status code is not handled or not allowed 2026-01-25 06:53:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cast-iron-jaw-coupling-hub-style-l100-1-3-8-inch-bore-diameter-5-16-x-5-32-keyway>: HTTP status code is not handled or not allowed 2026-01-25 06:53:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cast-iron-jaw-coupling-hub-style-l050-1-4-inch-bore-diameter-no-keyway>: HTTP status code is not handled or not allowed 2026-01-25 06:53:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cast-iron-jaw-coupling-hub-style-l070-1-2-inch-bore-diameter-no-keyway>: HTTP status code is not handled or not allowed 2026-01-25 06:53:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9-64-balldriver-screwdriver-long-10708>: HTTP status code is not handled or not allowed 2026-01-25 06:53:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 77883 pages (at 96 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:53:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12mm-balldriver-l-wrench-10980>: HTTP status code is not handled or not allowed 2026-01-25 06:53:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-control-metal-release-collet-5-32-tube-x-10-32-unf-flow-in-screw-adjustment>: HTTP status code is not handled or not allowed 2026-01-25 06:53:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-npt-breather-vent-brass-body-100-micron-14-to-176-f>: HTTP status code is not handled or not allowed 2026-01-25 06:53:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-8npt-breather-vent-brass-body-100-micron-14-to-176-f>: HTTP status code is not handled or not allowed 2026-01-25 06:53:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flow-control-8mm-tube-x-1-8-metal-release-collet-flow-out-screw-adjustment>: HTTP status code is not handled or not allowed 2026-01-25 06:53:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/needle-valve-12mm-tube-x-1-2>: HTTP status code is not handled or not allowed 2026-01-25 06:53:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/natural-convection-incubator-256-liters-115-volt>: HTTP status code is not handled or not allowed 2026-01-25 06:53:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forced-air-circulation-incubator-32-liters-115-volt>: HTTP status code is not handled or not allowed 2026-01-25 06:53:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fillet-weld-gage-4-13-16-inch-l>: HTTP status code is not handled or not allowed 2026-01-25 06:53:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/combination-square-13-3-8-inch-l>: HTTP status code is not handled or not allowed 2026-01-25 06:53:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/machinist-square-w-wide-base-15-3-4-inch-l-x-9-13-16-inch-w>: HTTP status code is not handled or not allowed 2026-01-25 06:53:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-electronic-height-gages-w-driving-wheel-0-12-inch-0-300mm-range>: HTTP status code is not handled or not allowed 2026-01-25 06:53:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/magnifier-w-illumination-and-4x-magnification>: HTTP status code is not handled or not allowed 2026-01-25 06:53:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-sidedshield-36-inch-reach-w-interlock-led-3-arms-14-inch-l-x-12-inch-w-x-12-inch-h>: HTTP status code is not handled or not allowed 2026-01-25 06:53:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shield-octagonal-24-inch-reach-w-interlock-led-24-inch-l-x-10-inch-w-29-lb>: HTTP status code is not handled or not allowed 2026-01-25 06:53:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shield-octagonal-36-inch-reach-w-interlock-led-28-inch-l-x-10-inch-w-35-lb>: HTTP status code is not handled or not allowed 2026-01-25 06:53:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shield-octagonal-direct-mount-w-o-interlock-9-inch-l-x-8-inch-w-18-inch-offset>: HTTP status code is not handled or not allowed 2026-01-25 06:53:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-ply-looped-end-mop-w-1-green-band-medium-red-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-25 06:53:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/looped-end-mop-w-scrubber-red-band-large-blue-pack-of-12>: HTTP status code is not handled or not allowed 2026-01-25 06:53:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ecp-single-face-foam-dispenser-24-inch-l>: HTTP status code is not handled or not allowed 2026-01-25 06:53:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ecp-battery-powered-cord-strap-tensioner-w-cutter>: HTTP status code is not handled or not allowed 2026-01-25 06:53:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ecp-economy-kit-for-5-8-inch-polyester-strap>: HTTP status code is not handled or not allowed 2026-01-25 06:53:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ecp-kit-for-3-4-inch-strap-open-seals>: HTTP status code is not handled or not allowed 2026-01-25 06:53:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ecp-bi-support-roll-stand-24-inch-l>: HTTP status code is not handled or not allowed 2026-01-25 06:53:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/7-tower-guard-zip-ties-pack-of-30>: HTTP status code is not handled or not allowed 2026-01-25 06:53:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bird-shock-small-solar-charger-12v-black>: HTTP status code is not handled or not allowed 2026-01-25 06:53:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bird-shock-flex-track-lead-out-wire-250l-roll-beige>: HTTP status code is not handled or not allowed 2026-01-25 06:53:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bird-shock-flex-track-50l-roll-black>: HTTP status code is not handled or not allowed 2026-01-25 06:53:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bird-shock-flex-track-lead-out-wire-250l-roll-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:53:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bird-flite-narrow-bird-spike-10l>: HTTP status code is not handled or not allowed 2026-01-25 06:53:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bird-flite-narrow-bird-spike-50l>: HTTP status code is not handled or not allowed 2026-01-25 06:53:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-anchor-pack-of-100>: HTTP status code is not handled or not allowed 2026-01-25 06:53:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/synthetic-winch-line-w-integrated-shackle-516-dia-x-125l>: HTTP status code is not handled or not allowed 2026-01-25 06:53:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/faosc-food-service-cabinet-w-o-meds-50-persons-large-plastic-case>: HTTP status code is not handled or not allowed 2026-01-25 06:53:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sj-15-inch-snow-blower-kit-w-hand-rotation--dual-charger-48v-4-amp-hours>: HTTP status code is not handled or not allowed 2026-01-25 06:53:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbw-vinyl-bumper-truck-all-swivel-casters-18-bushel-capacity-navy>: HTTP status code is not handled or not allowed 2026-01-25 06:53:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbw-antimicrobial-vinyl-basket-truck-all-swivel-20-bushel-capacity-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:53:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sensor-assembly-for-zems-hardwired-sensor-water-closet>: HTTP status code is not handled or not allowed 2026-01-25 06:53:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aquasense-zer-tm-series-ez-gear-driven-1-0-gpf-exposed-sensor-diaphragm-urinal-flush-valve>: HTTP status code is not handled or not allowed 2026-01-25 06:53:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/371-carton-sealing-tape-2-in-x-110-yds-1-9-mil-clear>: HTTP status code is not handled or not allowed 2026-01-25 06:53:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exeter-6-garden-bench-weathered-acorn>: HTTP status code is not handled or not allowed 2026-01-25 06:53:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glennville-lounge-chair-whitewash>: HTTP status code is not handled or not allowed 2026-01-25 06:53:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/muskoka-adirondack-5-piece-bistro-dining-set-30h-table-whitewash>: HTTP status code is not handled or not allowed 2026-01-25 06:53:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/flat-jamb-whiskey-oak>: HTTP status code is not handled or not allowed 2026-01-25 06:53:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/key-west-silver-lines-slab-door-wood-36w-x-80h-whiskey-oak>: HTTP status code is not handled or not allowed 2026-01-25 06:53:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tampa-glazed-light-slab-door-wood--glass-36w-x-80h-ice-maple>: HTTP status code is not handled or not allowed 2026-01-25 06:53:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/orlando-silver-lines-slab-door-wood-24w-x-80h-whiskey-oak>: HTTP status code is not handled or not allowed 2026-01-25 06:53:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/orlando-silver-lines-slab-door-wood-24w-x-80h-white>: HTTP status code is not handled or not allowed 2026-01-25 06:53:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/key-west-night-lines-slab-door-wood-36w-x-80h-ice-maple>: HTTP status code is not handled or not allowed 2026-01-25 06:53:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arcadia-glazed-light-slab-door-wood--glass-24w-x-80h-white>: HTTP status code is not handled or not allowed 2026-01-25 06:53:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/orlando-gold-lines-slab-door-wood-24w-x-80h-white>: HTTP status code is not handled or not allowed 2026-01-25 06:53:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/key-largo-night-lines-slab-door-wood-28w-x-80h-whiskey-oak>: HTTP status code is not handled or not allowed 2026-01-25 06:53:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/key-west-silver-lines-slab-door-wood-28w-x-80h-white>: HTTP status code is not handled or not allowed 2026-01-25 06:53:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/key-west-night-lines-slab-door-wood-32w-x-80h-white>: HTTP status code is not handled or not allowed 2026-01-25 06:53:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/key-west-gold-lines-slab-door-wood-30w-x-80h-ice-maple>: HTTP status code is not handled or not allowed 2026-01-25 06:53:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/l-type-casing-white>: HTTP status code is not handled or not allowed 2026-01-25 06:53:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/privacy-pocket-lock-gold>: HTTP status code is not handled or not allowed 2026-01-25 06:53:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/oia-5-piece-indoor-outdoor-steel-patio-table-set-w-round-chairs-gold>: HTTP status code is not handled or not allowed 2026-01-25 06:53:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-reclining-shower-chair-footrest-335-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:53:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cushion-max-anti-fatigue-mat-58-thick-25l-x-4w-charcoal>: HTTP status code is not handled or not allowed 2026-01-25 06:53:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hog-heavenanti-fatigue-mat-58-thick-20l-x-4w-black>: HTTP status code is not handled or not allowed 2026-01-25 06:53:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hog-heavenanti-fatigue-mat-58-thick-8l-x-3w-black-striped-border>: HTTP status code is not handled or not allowed 2026-01-25 06:53:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hog-heavenanti-fatigue-mat-58-thick-15l-x-5w-black-striped-border>: HTTP status code is not handled or not allowed 2026-01-25 06:53:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polywoven-dunnage-air-bags-2-ply-48l-x-46-12w-pack-of-810>: HTTP status code is not handled or not allowed 2026-01-25 06:53:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/espresso-maker-6-cups-stainless-steel-silver>: HTTP status code is not handled or not allowed 2026-01-25 06:53:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pour-over-coffee-set-1200ml>: HTTP status code is not handled or not allowed 2026-01-25 06:53:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/telero-digital-kitchen-scale-red>: HTTP status code is not handled or not allowed 2026-01-25 06:53:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/infinity-3-tier-3-door-locker-15-in-w-x-21-in-d-x-24-in-h-gray-assembled>: HTTP status code is not handled or not allowed 2026-01-25 06:53:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/infinity-4-tier-4-door-locker-18-in-w-x-18-in-d-x-18-in-h-gray-assembled>: HTTP status code is not handled or not allowed 2026-01-25 06:53:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-hivis-3-in-1-rainwear-jacket-2xl-black-lime>: HTTP status code is not handled or not allowed 2026-01-25 06:53:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chillshield-mens-insulated-pants-small-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:53:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-hivis-softshell-insulated-pants-large-black-lime>: HTTP status code is not handled or not allowed 2026-01-25 06:53:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-hivis-softshell-insulated-pants-xl-black-lime>: HTTP status code is not handled or not allowed 2026-01-25 06:53:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chillshield-mens-insulated-jacket-4xl-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:53:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classicware-plastic-plates-6-dia-clear-pack-of-180>: HTTP status code is not handled or not allowed 2026-01-25 06:53:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dinnerware-plate-9-dia-white-pack-of-500>: HTTP status code is not handled or not allowed 2026-01-25 06:53:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-aluminum-foil-roll-500-l-x-12-w-silver>: HTTP status code is not handled or not allowed 2026-01-25 06:53:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roast-1-lb-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 06:53:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powdered-creamer-sugar-free-chocolate-creme-10-2-oz-canister-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 06:53:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/earthchoice-recycled-pet-container-7-3-8-l-x-5-7-8-w-x-2-3-8-h-pack-of-280>: HTTP status code is not handled or not allowed 2026-01-25 06:53:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/smartlock-sandwich-container-5-3-4-l-x-5-3-4-w-x-3-1-4-h-pack-of-504>: HTTP status code is not handled or not allowed 2026-01-25 06:53:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bagasse-food-container-9-l-x-6-w-x-3-3-16-h-pack-of-250>: HTTP status code is not handled or not allowed 2026-01-25 06:54:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/earthchoice-container-10-1-2-l-x-9-1-2-w-x-3-h-pack-of-132>: HTTP status code is not handled or not allowed 2026-01-25 06:54:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-roll-table-cover-300-l-x-40-w-white>: HTTP status code is not handled or not allowed 2026-01-25 06:54:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gen-freezer-paper-1-000-l-x-15-w-white>: HTTP status code is not handled or not allowed 2026-01-25 06:54:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/easy-grip-disposable-plastic-party-cups-16-oz-assorted-colors-pack-of-400>: HTTP status code is not handled or not allowed 2026-01-25 06:54:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foam-school-tray-w-5-compartment-white-pack-of-500>: HTTP status code is not handled or not allowed 2026-01-25 06:54:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/one-piece-paper-bakery-boxes-6-1-4-l-x-3-3-4-w-x-2-1-8-h-white-pack-of-250>: HTTP status code is not handled or not allowed 2026-01-25 06:54:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paper-hot-drink-cups-lids-combo-10-oz-pack-300>: HTTP status code is not handled or not allowed 2026-01-25 06:54:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paper-hot-coffee-cups-20-oz-pack-of-500>: HTTP status code is not handled or not allowed 2026-01-25 06:54:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polypropylene-cocktail-straws-8-l-black-pack-of-5000>: HTTP status code is not handled or not allowed 2026-01-25 06:54:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quiet-classic-bowl-12-oz-white-pack-of-125>: HTTP status code is not handled or not allowed 2026-01-25 06:54:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/digital-compact-bench-scale-440-lb-x-0-1-lb-12-in-x-12-in-platform>: HTTP status code is not handled or not allowed 2026-01-25 06:54:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/copan-flexible-minitip-size-nylon-flocked-swab-w-100-mm-breakpoint-white>: HTTP status code is not handled or not allowed 2026-01-25 06:54:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cordless-one-hand-reciprocating-saw-bare-tool-24v-0-3000-spm>: HTTP status code is not handled or not allowed 2026-01-25 06:54:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-combi-glass-hepa--carbon-filter>: HTTP status code is not handled or not allowed 2026-01-25 06:54:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 77979 pages (at 96 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:54:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/colormaxx-paint-primer-12-oz-flat-black-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 06:54:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/colormaxx-paint-primer-11-oz-flat-crystal-clear-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 06:54:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-strength-cylinder-spray-adhesive-288-lb-capacity-transparent>: HTTP status code is not handled or not allowed 2026-01-25 06:54:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-visibility-windbreaker-water-resistant-jacket-type-r-class-3-orange-5xl>: HTTP status code is not handled or not allowed 2026-01-25 06:54:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-cotton-clavicle-strap-large>: HTTP status code is not handled or not allowed 2026-01-25 06:54:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cervical-collar-soft-3-34l-x-19h-medium-x-long>: HTTP status code is not handled or not allowed 2026-01-25 06:54:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/serpentine-style-cervical-collar-firm-21l-x-4h-medium>: HTTP status code is not handled or not allowed 2026-01-25 06:54:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/standard-instant-cold-packs-8l-x-11w-pack-of-24>: HTTP status code is not handled or not allowed 2026-01-25 06:54:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/back-wheelchair-cushion-16w-black>: HTTP status code is not handled or not allowed 2026-01-25 06:54:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/terry-bib-w-hook-loop-strap-adult-18l-x-30w-blue-12-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:54:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bariatric-crutches-tall-adult>: HTTP status code is not handled or not allowed 2026-01-25 06:54:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/equagel-adjustable-cushion-18l-x-16w-x-2-12h-green>: HTTP status code is not handled or not allowed 2026-01-25 06:54:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bariatric-quad-cane-small-base-500-lb-capacity-black>: HTTP status code is not handled or not allowed 2026-01-25 06:54:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guardian-k3-wheelchair-w-height-adj-full-length-arms-swing-away-footrests-20w-seat>: HTTP status code is not handled or not allowed 2026-01-25 06:54:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-tread-patient-slippers-size-l-red-48-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:54:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pediatric-gowns-tissue-poly-tissue-21-x-36-blue-50-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:54:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/protection-plus-mesh-incontinence-underpants-size-xl-for-waist-size-32-48-300-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:54:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/povidone-iodine-pvp-swabstick-50-per-box10-boxes-per-case>: HTTP status code is not handled or not allowed 2026-01-25 06:54:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/comprecares-reusable-compression-garment-kit-xx-large-tall-black>: HTTP status code is not handled or not allowed 2026-01-25 06:54:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-aneroid-sphygmomanometer-w-adult-cuff>: HTTP status code is not handled or not allowed 2026-01-25 06:54:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pathway-straight-wheelchair-ramp-kit-aluminum-5-top-platform-22l-x-36w>: HTTP status code is not handled or not allowed 2026-01-25 06:54:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vision-type-n-protection-safety-eyewear-smoky-lens-silver-frame>: HTTP status code is not handled or not allowed 2026-01-25 06:54:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vision-type-e-protection-safety-eyewear-blue-mirror-lens-black-frame>: HTTP status code is not handled or not allowed 2026-01-25 06:54:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-sided-mesh-truck-steel-3600-lb-capacity-54-1-2l-x-24-1-2w-x-46-1-4h>: HTTP status code is not handled or not allowed 2026-01-25 06:54:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/garden-cart-steel-1200-lb-capacity-36l-x-24-1-4w-x-35h>: HTTP status code is not handled or not allowed 2026-01-25 06:54:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/garden-truck-w-pneumatic-casters-1000-lb-cap-51-9-16l-x-24-5-16w-x-35-1-8h>: HTTP status code is not handled or not allowed 2026-01-25 06:54:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spindle-attachment-for-adjust-a-tray-truck-steel-16-3-8l-x-1w-x-3-1-4h>: HTTP status code is not handled or not allowed 2026-01-25 06:54:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mesh-truck-w-3-fixed-shelves-steel-2000-lb-cap-66-1-2l-x-32w-x-56-7-16h>: HTTP status code is not handled or not allowed 2026-01-25 06:54:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-pressure-balance-cartridge-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-25 06:54:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rbk4110-rebuild-kit-for-american-standard-shower-valves>: HTTP status code is not handled or not allowed 2026-01-25 06:54:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rear-connect-diverter-tub-spout-slip-on-12-x-34-rear-lift>: HTTP status code is not handled or not allowed 2026-01-25 06:54:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deck-mount-single-handle-pantry-faucet-w-8-gooseneck-spout>: HTTP status code is not handled or not allowed 2026-01-25 06:54:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-mount-two-handle-faucet-w-6-gooseneck-spout>: HTTP status code is not handled or not allowed 2026-01-25 06:54:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/acrylic-handle-for-moen-single-handle-faucets-plastic-pack-of-6>: HTTP status code is not handled or not allowed 2026-01-25 06:54:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aquaplus-axp-255ah-agm-carpet-extractor-100-psi-20-gallon-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:54:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/apex-vault-19-12l-x-23-12w-x-25-12h-black>: HTTP status code is not handled or not allowed 2026-01-25 06:54:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/versa-cuff-light-resistance-12l-green>: HTTP status code is not handled or not allowed 2026-01-25 06:54:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/versa-bar-prime-18-lb-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:54:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reebok-pump-set-black--red>: HTTP status code is not handled or not allowed 2026-01-25 06:54:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wall-ball-20-lb>: HTTP status code is not handled or not allowed 2026-01-25 06:54:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/business-furniture-hybrid-l-shaped-table-desk-71w-x-71-5-16d-x-29-15-16h-white>: HTTP status code is not handled or not allowed 2026-01-25 06:54:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/business-furniture-hybrid-l-shaped-table-desk-71w-x-77-3-8d-x-29-15-16h-storm-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:54:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/business-furniture-hybrid-bookcase-w-2-shelves-36w-x-16d-x-30h-storm-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:54:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/business-furniture-hybrid-lower-piler-filer-30w-x-17d-x-26h-white>: HTTP status code is not handled or not allowed 2026-01-25 06:54:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bottom-mount-refrigeratorfreezer-combo-unit-10-cuft-capacity-silver>: HTTP status code is not handled or not allowed 2026-01-25 06:54:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/countertop-ice-maker-portable-makes-25-lb-per-day-red>: HTTP status code is not handled or not allowed 2026-01-25 06:54:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/healthcare-hydrogen-peroxide-disinfectant-cleaner-1-gal-cap-bottle-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-25 06:54:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/optional-side-panel-for-wood-lockers-cherry>: HTTP status code is not handled or not allowed 2026-01-25 06:54:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-tier-6-door-digital-wood-locker-36-w-x-15-d-x-72-h-cherry-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 06:54:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-tier-4-door-wood-locker-12-w-x-15-d-x-72-h-cherry-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 06:54:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cabinet-w-fixed-shelves-assembled-36w-x-24d-x-72h-denim-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:54:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fs-48-c-cabinet-w-fixed-shelves-assembled-48w-x-18d-x-72h-charcoal>: HTTP status code is not handled or not allowed 2026-01-25 06:54:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/organizer-cabinet-w-fixed-shelves-assembled-48w-x-18d-x-78h-pastel-green>: HTTP status code is not handled or not allowed 2026-01-25 06:54:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cabinet-w-fixed-shelves-assembled-36w-x-24d-x-78h-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:54:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cabinet-w-fixed-shelves-assembled-36w-x-24d-x-72h-pastel-green>: HTTP status code is not handled or not allowed 2026-01-25 06:54:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cabinet-w-drawer--shelves-assembled-30w-x-18d-x-72h-walnut>: HTTP status code is not handled or not allowed 2026-01-25 06:54:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cabinet-w-drawer--shelves-assembled-36w-x-18d-x-72h-denim-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:54:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/specimentransport-vial-polystyrene-non-sterile-17-x-50-mm-7-ml-cap-pk-of-700>: HTTP status code is not handled or not allowed 2026-01-25 06:54:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cap-inserts-for-cryogenic-vials-yellow-pack-of-500>: HTTP status code is not handled or not allowed 2026-01-25 06:54:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/teardrop-pallet-rack-starter-96-in-w-x-24-in-d-x-120-in-h>: HTTP status code is not handled or not allowed 2026-01-25 06:54:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/onyx-standard-safety-vest-w-lime-contrasting-trim-2xl-black>: HTTP status code is not handled or not allowed 2026-01-25 06:54:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-waterproof-parka-jacket-w-fleece-lining-nylon-navy-x-large>: HTTP status code is not handled or not allowed 2026-01-25 06:54:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fr-waterproof-flame-resistant-insulated-bib-pant-sm-lime>: HTTP status code is not handled or not allowed 2026-01-25 06:54:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fr-waterproof-flame-resistant-insulated-bib-pant-sm-black>: HTTP status code is not handled or not allowed 2026-01-25 06:54:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quartz-duck-sherpa-lined-heavy-weight-jacket-brown-2xl>: HTTP status code is not handled or not allowed 2026-01-25 06:54:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quartz-job-shirt-w-14-zipper-cotton-navy-5xl>: HTTP status code is not handled or not allowed 2026-01-25 06:54:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/enhanced-visibility-premium-heavy-duty-vest-w-multi-pockets-lxl-black>: HTTP status code is not handled or not allowed 2026-01-25 06:54:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/friedrich--commercial-windowwall-air-conditioner-cool-only-1000-watts-230-v-12k-btu>: HTTP status code is not handled or not allowed 2026-01-25 06:54:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forkliftable-truck-w-1-deck-3600-lb-cap-53-12l-x-30w-x-58-12h-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:54:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/forkliftable-truck-w-2-shelves-3600-lb-cap-53-12l-x-24w-x-57h-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:54:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-sided-shelf-truck-w-mesh-sides-3600-lb-cap-53-12l-x-30w-x-57h-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:54:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8384-hi-vis-quilted-parka-winter-jacket-small-lime>: HTTP status code is not handled or not allowed 2026-01-25 06:54:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8384-hi-vis-quilted-parka-winter-jacket-2xl-lime>: HTTP status code is not handled or not allowed 2026-01-25 06:54:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8386-hi-vis-shell-jacket-xl-orange>: HTTP status code is not handled or not allowed 2026-01-25 06:54:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8384-hi-vis-quilted-parka-winter-jacket-4xl-orange>: HTTP status code is not handled or not allowed 2026-01-25 06:54:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8287-hi-vis-winter-jacket--vest-w-detachable-sleeves-small-orange>: HTTP status code is not handled or not allowed 2026-01-25 06:54:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8910bk-hi-vis-pants-w-black-bottom-class-e-mesh-4xl5xl-lime>: HTTP status code is not handled or not allowed 2026-01-25 06:54:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8915bk-hi-vis-rain-pants-w-black-bottom-class-e-polyester-large-lime>: HTTP status code is not handled or not allowed 2026-01-25 06:54:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-7031-cut-resistant-gloves-nitrile-coated-ansi-a3-l-gray-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 06:54:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-7031-cut-resistant-gloves-nitrile-coated-ansi-a3-s-gray-144-pairs>: HTTP status code is not handled or not allowed 2026-01-25 06:54:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8253hdz-heavy-duty-mesh-surveyors-vest-class-2-4xl5xl-orange>: HTTP status code is not handled or not allowed 2026-01-25 06:54:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-7141-dir-cut-resistant-gloves-nitrile-coated-ansi-a4-2xl-lime-72-pairs>: HTTP status code is not handled or not allowed 2026-01-25 06:54:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-7070-cut-resistant-gloves-nitrile-coated-ansi-a7-2xl-green-12-pairs>: HTTP status code is not handled or not allowed 2026-01-25 06:54:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-7024-cut-resistant-gloves-polyurethane-coated-ansi-a2-xl-gray-12-pairs>: HTTP status code is not handled or not allowed 2026-01-25 06:54:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8928-high-visibility-insulated-bib-pants-ansi-class-e-300d-polyester-xl-orange>: HTTP status code is not handled or not allowed 2026-01-25 06:54:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/proflex-7024-cut-resistant-gloves-polyurethane-coated-ansi-a2-l-blue-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 06:55:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/skullerz-8966-lightweight-cap-style-hard-hat-vented-class-c-lime>: HTTP status code is not handled or not allowed 2026-01-25 06:55:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dagr174-anti-fog-safety-glasses-inoutdoor-lens-black-frame>: HTTP status code is not handled or not allowed 2026-01-25 06:55:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baldr174-gasket-insert-fgi-foam-black-frame>: HTTP status code is not handled or not allowed 2026-01-25 06:55:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chill-its-6695-sun-protection-arm-sleeves-ml-white-custom-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 06:55:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n-ferno-6466-thermal-jacket-500d-nylon-shell-4xl-black>: HTTP status code is not handled or not allowed 2026-01-25 06:55:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chill-its-6695-sun-protection-arm-sleeves-xl2xl-blue-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 06:55:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/34-air-impact-wrench-1100-ft-lbs-torque-d-handle>: HTTP status code is not handled or not allowed 2026-01-25 06:55:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-air-impact-wrench-quiet-1700-ft-lbs-torque-industrial-duty-pistol-grip>: HTTP status code is not handled or not allowed 2026-01-25 06:55:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/horizontal-air-die-grinder-14-collet-burr-20000-rpm-04-hp-rear-exhaust>: HTTP status code is not handled or not allowed 2026-01-25 06:55:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pneumatic-backfill-tamper-575-round-malleable-butt-4-stroke-2-bore>: HTTP status code is not handled or not allowed 2026-01-25 06:55:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spike-driver-575-stroke-1250-bpm-206-bore>: HTTP status code is not handled or not allowed 2026-01-25 06:55:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pavement-breaker-1-18-x-6-hex-shank-575-stroke-length-262-bore>: HTTP status code is not handled or not allowed 2026-01-25 06:55:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/light-duty-spring-balancer-66-lb-capacity-52-lift-black>: HTTP status code is not handled or not allowed 2026-01-25 06:55:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chipping-hammer-68-round-shank-2-stroke-2200-bpm>: HTTP status code is not handled or not allowed 2026-01-25 06:55:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electric-powered-tugger-3000-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:55:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5000-arbor-series-w-4-ft-barrel-60-cfm>: HTTP status code is not handled or not allowed 2026-01-25 06:55:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 78081 pages (at 102 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:55:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5000-arbor-series-w-4-ft-barrel-225-cfm>: HTTP status code is not handled or not allowed 2026-01-25 06:55:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rotoplat-708-semi-automatic-turntable-stretch-wrapper-86h-x-65-dia-4400-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:55:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chest-freezer-35176c-2666-cu-ft-capacity-white>: HTTP status code is not handled or not allowed 2026-01-25 06:55:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/upright-vaccine-refrigerator-w-factory-installed-data-logger-18-cuft-solid-door>: HTTP status code is not handled or not allowed 2026-01-25 06:55:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-freezer-w-ac-adapter-088-cu-ft-capacity-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:55:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wheelchair-20-in-w-seat-flip-back-arms-swing-away-footrests-black>: HTTP status code is not handled or not allowed 2026-01-25 06:55:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/msa-safety--full-brim-hard-hat-non-vented-fas-trac-iii-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:55:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/msa-safety--full-brim-hard-hat-vented-fas-trac-iii-orange>: HTTP status code is not handled or not allowed 2026-01-25 06:55:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/s15-sensor-vacuum-15-cleaning-width>: HTTP status code is not handled or not allowed 2026-01-25 06:55:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/procedure-core-bench-w-2-bay-centre-divider-36l-x-27w-x-78-34h-g2g-pkg>: HTTP status code is not handled or not allowed 2026-01-25 06:55:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drive-belt-for-hl-m1120ld-frm-1120c-frm-1010-frm-1120ld>: HTTP status code is not handled or not allowed 2026-01-25 06:55:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-kit-for-w-300a>: HTTP status code is not handled or not allowed 2026-01-25 06:55:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silicone-rubber-pad-for-fs-800h>: HTTP status code is not handled or not allowed 2026-01-25 06:55:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silicone-rubber-pad-for-kf-300f-kf-305f>: HTTP status code is not handled or not allowed 2026-01-25 06:55:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-kit-for-w-455a>: HTTP status code is not handled or not allowed 2026-01-25 06:55:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/silicone-rubber-pad-for-kf-300hc-kf-305hc>: HTTP status code is not handled or not allowed 2026-01-25 06:55:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-kit-for-fkr-400a>: HTTP status code is not handled or not allowed 2026-01-25 06:55:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/90deg-high-raised-curved-guide-for-roller-conveyors-18in-oaw>: HTTP status code is not handled or not allowed 2026-01-25 06:55:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/roundtangular-convex-mirror-plexi-glass-outdoor-20l-x-30w>: HTTP status code is not handled or not allowed 2026-01-25 06:55:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ratchet-wrench-38-drive-160-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:55:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tough-coat-adv-spray-paint-w-rust-barrier-technology-20-oz-cap-gloss-safety-red>: HTTP status code is not handled or not allowed 2026-01-25 06:55:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ball-shaped-bur-single-cut-1-12l-x-18-shank-dia-18-len-of-cut>: HTTP status code is not handled or not allowed 2026-01-25 06:55:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/radius-tree-shaped-bur-aluma-cut-2-12l-x-14-shank-dia>: HTTP status code is not handled or not allowed 2026-01-25 06:55:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cylindrical-w-endcut-bur-double-cut-6-34l-x-14-shank-dia>: HTTP status code is not handled or not allowed 2026-01-25 06:55:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pointed-tree-shaped-bur-single-cut-2-12l-x-14-shank-dia>: HTTP status code is not handled or not allowed 2026-01-25 06:55:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/radius-cylindrical-bur-double-cut-2l-x-14-shank-dia-58-len-of-cut>: HTTP status code is not handled or not allowed 2026-01-25 06:55:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ez-lock153-sanding-polishing-rotary-accessories-kit-pack-of-8>: HTTP status code is not handled or not allowed 2026-01-25 06:55:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pu-air-hose-38-inner-dia-x-25l>: HTTP status code is not handled or not allowed 2026-01-25 06:55:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/radius-cylindrical-bur-for-stainless-steel-2-34l-x-14-shank-dia>: HTTP status code is not handled or not allowed 2026-01-25 06:55:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/worksite-table-saw-w-gravity-rise-wheeled-stand-10-blade-size>: HTTP status code is not handled or not allowed 2026-01-25 06:55:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t5-led-tube-light-bulb-miniature-bi-pin-base-13w-1700-lumens-5000k-super-white>: HTTP status code is not handled or not allowed 2026-01-25 06:55:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tuberoller-w-10-rollers-115v-5060-hz-2-80-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:55:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nextpette-starter-kit-w-carousel-for-p7700-10-20-200-1000-stand>: HTTP status code is not handled or not allowed 2026-01-25 06:55:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/led-half-cylindrical-wall-sconce-12w-840-lumens-80-cri-13-size-white>: HTTP status code is not handled or not allowed 2026-01-25 06:55:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tube-rack-for-96-well-adapter-20x20ml-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-25 06:55:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/m1-grade-calibration-weight-20g-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:55:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hotplate-plate-size-7-18w-x-7-18d-115v>: HTTP status code is not handled or not allowed 2026-01-25 06:55:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/twin-tank-air-compressor-7-hp-10-gallon-capacity-wheelbarrow-112-cfm-150-psi>: HTTP status code is not handled or not allowed 2026-01-25 06:55:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nylon-pouch-w-snap-for-bur-sets-no-logo-4-x-6-size>: HTTP status code is not handled or not allowed 2026-01-25 06:55:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mega-mover-dolly-for-20-bushel-tub-44-12l-x-28-12w-1500-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:55:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbi-sala-nano-lok-personal-twin-leg-self-retracting-lifeline-w-steel-web-snap-hook-72l>: HTTP status code is not handled or not allowed 2026-01-25 06:55:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/long-ball-end-mill-2-flute-4l-x-12-dia-2-cut-length>: HTTP status code is not handled or not allowed 2026-01-25 06:55:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cubitron-ii-cloth-disc-900dz-220-grit-5-dia-x-nh-pack-of-12500>: HTTP status code is not handled or not allowed 2026-01-25 06:55:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/excelon-plus-oil-fog-lubricator-metal-bowl-w-guard-14-ptf>: HTTP status code is not handled or not allowed 2026-01-25 06:55:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-starter-48l-x-24w-x-48h-antique-white>: HTTP status code is not handled or not allowed 2026-01-25 06:55:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-sided-starter-36l-x-12w-x-72h-black>: HTTP status code is not handled or not allowed 2026-01-25 06:55:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-sided-starter-36l-x-12w-x-84h-black>: HTTP status code is not handled or not allowed 2026-01-25 06:55:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-starter-36l-x-28w-x-60h-black>: HTTP status code is not handled or not allowed 2026-01-25 06:55:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-add-on-48l-x-24w-x-54h-black>: HTTP status code is not handled or not allowed 2026-01-25 06:55:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-add-on-48l-x-24w-x-60h-black>: HTTP status code is not handled or not allowed 2026-01-25 06:55:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-sided-starter-36l-x-14w-x-48h-black>: HTTP status code is not handled or not allowed 2026-01-25 06:55:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/double-sided-add-on-48l-x-36w-x-48h-antique-white>: HTTP status code is not handled or not allowed 2026-01-25 06:55:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/end-unit-w-t-joiner-support-clip-36l-x-18w-x-96h-white>: HTTP status code is not handled or not allowed 2026-01-25 06:55:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/end-unit-w-t-joiner-support-clip-36l-x-14w-x-48h-antique-white>: HTTP status code is not handled or not allowed 2026-01-25 06:55:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/single-sided-starter-36l-x-20w-x-84h-black>: HTTP status code is not handled or not allowed 2026-01-25 06:55:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/end-unit-w-t-joiner-support-clip-36l-x-12w-x-84h-black>: HTTP status code is not handled or not allowed 2026-01-25 06:55:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/end-unit-w-t-joiner-support-clip-36l-x-16w-x-60h-antique-white>: HTTP status code is not handled or not allowed 2026-01-25 06:55:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portable-air-conditioner-7500-btu-753w-115v-black>: HTTP status code is not handled or not allowed 2026-01-25 06:55:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/optimus-power-scissor-lift-table-hand-operated-3000-cap-3ph-230v-48l-x-28w>: HTTP status code is not handled or not allowed 2026-01-25 06:55:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/optimus-power-scissor-lift-table-foot-operated-3000-cap-3ph-480v-48l-x-36w>: HTTP status code is not handled or not allowed 2026-01-25 06:55:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tlaltakal36cl-ao-v-tec-mini-personal-fall-limiter-twin-leg-6l>: HTTP status code is not handled or not allowed 2026-01-25 06:55:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gondola-wire-shelf-divider-fits-13l-x-3h-shelf-chrome-finish>: HTTP status code is not handled or not allowed 2026-01-25 06:55:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uv-air-purifier-w-hepa-filter-58w-110v-white>: HTTP status code is not handled or not allowed 2026-01-25 06:55:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/krestol-heritage-hand-cleaner-1-gal-bottle-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-25 06:55:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/v-gard-h1pro-safety-helmet-non-vented-fas-trac-iii-pivot-ratchet-suspension-black>: HTTP status code is not handled or not allowed 2026-01-25 06:55:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/marque-plexi-double-unit-reception-desk-workstation-124-14w-x-49d-x-45-12h-mahogany>: HTTP status code is not handled or not allowed 2026-01-25 06:55:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/glowear-8210z-bk-hi-vis-safety-vest-ansi-class-2-lxl-orange>: HTTP status code is not handled or not allowed 2026-01-25 06:55:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bottle-media-globe-glass-gl45-screw-cap-dual-graduations-250ml-10-box>: HTTP status code is not handled or not allowed 2026-01-25 06:55:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/array153-ac2-ceiling-mount-hepa-air-purifier-715-cfm-120v-ac-black>: HTTP status code is not handled or not allowed 2026-01-25 06:55:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drum-pump-60-15-36981>: HTTP status code is not handled or not allowed 2026-01-25 06:55:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/skullerz-aegir-mi-half-frame-safety-glasses-blue-mirror-lens-lime-frame>: HTTP status code is not handled or not allowed 2026-01-25 06:55:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/spout-cap-blue-small-34402>: HTTP status code is not handled or not allowed 2026-01-25 06:55:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/identification-tag-4-inch-x-2-inch-dark-green-37081>: HTTP status code is not handled or not allowed 2026-01-25 06:55:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/container-vent-gray-24017>: HTTP status code is not handled or not allowed 2026-01-25 06:55:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobilecare-sharps-container-holder-w-side-mount-accessory-rail-10l-x-9w-x-9h-red>: HTTP status code is not handled or not allowed 2026-01-25 06:55:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mobilecare-keyed-lock-cart-27w-x-27d-x-39h-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:55:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/array153-lookout-stand-display-desktop-120v-ac-white>: HTTP status code is not handled or not allowed 2026-01-25 06:55:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/475-roll-stand-for-535-spot-check-vital-signs-monitor>: HTTP status code is not handled or not allowed 2026-01-25 06:55:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/digital-bariatric-scale-600-lb-cap-18l-x-18w-platform>: HTTP status code is not handled or not allowed 2026-01-25 06:55:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h1-3-bar-aluminum-ladder-roof-rack-for-ford-transit-connect-2014-on-white>: HTTP status code is not handled or not allowed 2026-01-25 06:55:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electronic-bench-scale-stainless-steel-150-lb-cap-16l-x-14w-platform>: HTTP status code is not handled or not allowed 2026-01-25 06:55:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h1-3-bar-steel-ladder-rack-for-ram-promaster-cargo-van-2013-on-white>: HTTP status code is not handled or not allowed 2026-01-25 06:55:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h1-2-bar-steel-ladder-rack-system-for-dodge-ram-van-1981-on-black>: HTTP status code is not handled or not allowed 2026-01-25 06:56:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h1-2-bar-steel-ladder-rack-system-for-dodge-ram-van-1981-on-white>: HTTP status code is not handled or not allowed 2026-01-25 06:56:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/h1-3-bar-steel-ladder-rack-system-for-dodge-ram-van-1981-on-black>: HTTP status code is not handled or not allowed 2026-01-25 06:56:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/wg-composite-bulkhead-ford-transit-low-roof>: HTTP status code is not handled or not allowed 2026-01-25 06:56:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-mega-swivel-anchor>: HTTP status code is not handled or not allowed 2026-01-25 06:56:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/knee-pad-for-cpr-training-16l-x-12w-navy>: HTTP status code is not handled or not allowed 2026-01-25 06:56:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/secure-click-particulate-filter-p95-d7p71-for-secure-click-cartridges-d8000-series>: HTTP status code is not handled or not allowed 2026-01-25 06:56:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honeywell-safety-dc365-surgical-n95-respirator-10-box>: HTTP status code is not handled or not allowed 2026-01-25 06:56:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/guard-air-cooled-standby-generator-26kw-120-240v-1-phase-ng-lp-wifi-enabled>: HTTP status code is not handled or not allowed 2026-01-25 06:56:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80bs35x1716-1-pitch-1-716-finished-bore-35-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:56:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60bs36x158-34-pitch-1-58-finished-bore-36-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:56:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/battery-heater-kit-for-air-cooled-standby-generator-9-24kw>: HTTP status code is not handled or not allowed 2026-01-25 06:56:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hazmat-storage-building-locker-non-combustible-120w-x-96d-x-108h-white>: HTTP status code is not handled or not allowed 2026-01-25 06:56:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60bs36x1716-34-pitch-1-716-finished-bore-36-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:56:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/compact-vaccine-refrigerator-1-cuft-capacity-glass-door>: HTTP status code is not handled or not allowed 2026-01-25 06:56:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ada-height-vaccine-refrigerator-6-cuft-capacity-119-lb-weight-glass-door>: HTTP status code is not handled or not allowed 2026-01-25 06:56:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 78179 pages (at 98 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:56:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-batteries-for-aed-plus-multi-colored-pack-of-10>: HTTP status code is not handled or not allowed 2026-01-25 06:56:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gssid-pre-decon-kit-youth>: HTTP status code is not handled or not allowed 2026-01-25 06:56:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-safety-crafted-waterproof-western-work-boots-steel-toe-size-9m-10-inch-h-tan>: HTTP status code is not handled or not allowed 2026-01-25 06:56:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/masslinn-dusting-tool-23w-x-5d-orange-6-carton>: HTTP status code is not handled or not allowed 2026-01-25 06:56:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cs6-hand-sanitizer-floor-stand-with-dispenser-1200-ml-graphite>: HTTP status code is not handled or not allowed 2026-01-25 06:56:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-12a40-metric-a-plate-34-pitch-24mm-bore-40-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:56:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-20bs16hx38-metric-1-14-pitch-38mm-finished-bore-16-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:56:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/foodservice-surface-sanitizer-fragrance-free-1-gal-bottle>: HTTP status code is not handled or not allowed 2026-01-25 06:56:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-16bs23x25-metric-1-pitch-25mm-finished-bore-23-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:56:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-safety-crafted-waterproof-logger-work-boots-steel-toe-size-8-5m-8-inch-h-dark-brown>: HTTP status code is not handled or not allowed 2026-01-25 06:56:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/es4-soap-push-style-dispenser-1200-ml-white>: HTTP status code is not handled or not allowed 2026-01-25 06:56:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-safety-crafted-waterproof-work-boots-steel-toe-size-6w-6-inch-h-brown>: HTTP status code is not handled or not allowed 2026-01-25 06:56:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/select-a-size-kitchen-roll-paper-towels-6-x-11-wht-98-sheets-roll-12-rolls-ctn>: HTTP status code is not handled or not allowed 2026-01-25 06:56:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/conditioner-clean-scent-30-ml-288-carton>: HTTP status code is not handled or not allowed 2026-01-25 06:56:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/select-kitchen-roll-towels-2-ply-8-x-11-white-450-roll-12-carton>: HTTP status code is not handled or not allowed 2026-01-25 06:56:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/loading-dock-fan-multi-directional-output-1000-cfm-115v>: HTTP status code is not handled or not allowed 2026-01-25 06:56:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60sf48-qd-bushed-34-pitch-48-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:56:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-100bs23hx21516-1-14-pitch-2-1516-finished-bore-23-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:56:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80bs30x134-1-pitch-1-34-finished-bore-30-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:56:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/the-safety-crafted-waterproof-work-boots-steel-toe-size-12m-6-inch-h-brown>: HTTP status code is not handled or not allowed 2026-01-25 06:56:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/steri-drape-2-incise-pouch-2057-34-inch-x-29-inch-incise-area-12-inch-x-12-inch-10-ctn-4-ctn-cs>: HTTP status code is not handled or not allowed 2026-01-25 06:56:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-100bs22hx1716-1-14-pitch-1-716-finished-bore-22-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:56:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/groundbreaker-work-boots-w-internal-met-guard-steel-toe-size-11m-6-inch-h-brown>: HTTP status code is not handled or not allowed 2026-01-25 06:56:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/groundbreaker-work-boots-w-external-met-guard-steel-toe-size-9-5w-8-inch-h-brown>: HTTP status code is not handled or not allowed 2026-01-25 06:56:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-100bs21hx2716-1-14-pitch-2-716-finished-bore-21-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:56:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bair-hugger-warming-blanket-585-underbody-74-inch-x-36-inch-with-drape-24-inch-x-24-inch-10-cs>: HTTP status code is not handled or not allowed 2026-01-25 06:56:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/attest-biological-indicator-incubator-127>: HTTP status code is not handled or not allowed 2026-01-25 06:56:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bair-hugger-plus-warming-gown-81102-small-44-inch-l-20-case>: HTTP status code is not handled or not allowed 2026-01-25 06:56:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80bs33x2316-1-pitch-2-316-finished-bore-33-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:56:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-clearseal-microcentrifuge-tubes-with-o-ring-loop-caps-2-0-ml-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:56:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vitals-skate-inspired-work-shoes-composite-toe-size-10-5w-wintermoss-whitecap-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:56:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vitals-skate-inspired-work-shoes-composite-toe-size-9-5w-wintermoss-whitecap-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:56:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-nuvaclean-uv-pipette-carousel-with-germicidal-uv-lamp-240v>: HTTP status code is not handled or not allowed 2026-01-25 06:56:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vitals-skate-inspired-work-shoes-composite-toe-size-9m-rinsed-black-tower-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:56:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zig-elusion-heritage-work-men-s-low-cut-sneaker-composite-toe-size-13m-black>: HTTP status code is not handled or not allowed 2026-01-25 06:56:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zig-elusion-heritage-work-men-s-low-cut-sneaker-composite-toe-size-10w-black>: HTTP status code is not handled or not allowed 2026-01-25 06:56:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-24bs20hx42-metric-1-12-pitch-42mm-finished-bore-20-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:56:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50b17hx58mpb-58-pitch-58-plain-bore-17-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:56:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-06bs25hx19-metric-38-pitch-19mm-finished-bore-25-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:56:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80sh12h-1-pitch-qd-bushed-12-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:56:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zig-elusion-heritage-work-men-s-low-cut-sneaker-composite-toe-size-6-5w-black>: HTTP status code is not handled or not allowed 2026-01-25 06:56:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50sds22h-qd-bushed-58-pitch-22-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:56:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-40bs20hx114-12-pitch-1-14-finished-bore-20-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:56:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/true-skate-inspired-work-shoes-composite-toe-size-9w-black-gum>: HTTP status code is not handled or not allowed 2026-01-25 06:56:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/true-skate-inspired-work-shoes-composite-toe-size-14m-navy>: HTTP status code is not handled or not allowed 2026-01-25 06:56:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/true-skate-inspired-work-shoes-composite-toe-size-8-5w-black>: HTTP status code is not handled or not allowed 2026-01-25 06:56:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-large-bulb-transfer-pipette-sterile-7-ml-500-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:56:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50bs15hx78-58-pitch-78-finished-bore-15-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:56:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50bs15hx114-58-pitch-1-14-finished-bore-15-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:56:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35bs23hx34-38-pitch-34-finished-bore-23-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:56:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bio-pureamp-8-strips-qpcr-tube-with-separated-strip-cap-0-2-ml-white-120-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:56:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/true-skate-inspired-work-shoes-composite-toe-size-5-5w-black>: HTTP status code is not handled or not allowed 2026-01-25 06:56:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-legend-380f-377cc-snowmobile-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 06:56:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/true-skate-inspired-work-shoes-composite-toe-size-8-5w-navy>: HTTP status code is not handled or not allowed 2026-01-25 06:56:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60bs13hx118-34-pitch-1-18-finished-bore-13-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:56:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/true-skate-inspired-work-shoes-composite-toe-size-3w-black>: HTTP status code is not handled or not allowed 2026-01-25 06:56:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/polaris-star-twin-snowmobile-replacement-battery-12v-i>: HTTP status code is not handled or not allowed 2026-01-25 06:56:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-crafted-western-work-boots-w-steel-toe-and-internal-met-guard-size-11m-brown>: HTTP status code is not handled or not allowed 2026-01-25 06:56:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stone-skate-inspired-work-shoes-composite-toe-size-7m-black-red>: HTTP status code is not handled or not allowed 2026-01-25 06:56:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stone-skate-inspired-work-shoes-composite-toe-size-5m-blue-navy>: HTTP status code is not handled or not allowed 2026-01-25 06:56:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35bs34hx114-38-pitch-1-14-finished-bore-34-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:56:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/safety-crafted-western-work-boots-w-steel-toe-and-internal-met-guard-size-10-5w-brown>: HTTP status code is not handled or not allowed 2026-01-25 06:56:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kontron-1334336-medical-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 06:56:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dsr-psj2212-jump-starter-replacement-battery-75ah-12v-it>: HTTP status code is not handled or not allowed 2026-01-25 06:56:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stone-skate-inspired-work-shoes-composite-toe-size-14w-black-red>: HTTP status code is not handled or not allowed 2026-01-25 06:56:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-40a27hx58-a-plate-12-pitch-58-bore-27-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:56:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/philips-652007-medical-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 06:56:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/perforated-anti-static-air-bubble-rolls-12-x-250-x-1-2-4-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:56:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clear-pallet-covers-36-x-24-x-43-2-mil-50-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:56:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nanoflex-tr-athletic-work-shoes-composite-toe-size-9m-black-seafoam-green-white>: HTTP status code is not handled or not allowed 2026-01-25 06:56:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premier-work-casual-oxford-shoes-steel-toe-size-10eee-navy>: HTTP status code is not handled or not allowed 2026-01-25 06:56:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50bs19hx118-58-pitch-1-18-finished-bore-19-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:56:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premier-work-casual-slip-on-sneaker-steel-toe-size-11d-cognac>: HTTP status code is not handled or not allowed 2026-01-25 06:56:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60bs15hx138-34-pitch-1-38-finished-bore-15-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:56:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/self-seal-bubble-bags-4-x-4-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:56:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premier-work-casual-oxford-shoes-steel-toe-9-5d-black>: HTTP status code is not handled or not allowed 2026-01-25 06:56:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nanoflex-tr-athletic-work-shoes-composite-toe-size-7w-black-camo-brown>: HTTP status code is not handled or not allowed 2026-01-25 06:56:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reclosable-bags-3-x-5-2-mil-black-1000-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:56:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50a28hx34-a-plate-58-pitch-34-bore-28-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:56:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-100a11hx1-a-plate-1-14-pitch-1-bore-11-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:56:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paper-twist-ties-10-x-5-32-yellow-2000-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:56:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80a13hx34-a-plate-1-pitch-34-bore-13-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:56:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nanoflex-tr-athletic-work-shoes-composite-toe-size-10-5w-black-seafoam-green-white>: HTTP status code is not handled or not allowed 2026-01-25 06:56:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nanoflex-tr-athletic-work-shoes-composite-toe-size-11m-black-seafoam-green-white>: HTTP status code is not handled or not allowed 2026-01-25 06:56:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anti-static-bubble-bags-4-x-5-1-2-1500-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:56:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premier-work-casual-slip-on-sneaker-steel-toe-size-8d-cognac>: HTTP status code is not handled or not allowed 2026-01-25 06:57:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/paper-twist-ties-10-x-5-32-orange-2000-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:57:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/r-d-5648-sealed-lead-acid-replacement-battery-26ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 06:57:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/anti-static-bubble-bags-4-x-7-1-2-1100-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:57:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simplex-4208a-industrial-replacement-battery-26ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 06:57:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60bs16hx78-34-pitch-78-finished-bore-16-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:57:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nano-x1-adventure-athletic-work-shoes-composite-toe-size-8-5m-slate-blue-salmon>: HTTP status code is not handled or not allowed 2026-01-25 06:57:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ge-60-778-alarm-replacement-battery-18ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 06:57:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nano-x1-adventure-athletic-work-shoes-composite-toe-size-5m-slate-blue-salmon>: HTTP status code is not handled or not allowed 2026-01-25 06:57:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-10bs20hx24-metric-58-pitch-24mm-finished-bore-20-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:57:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rb3315-nanoflex-tr-athletic-work-shoes-composite-toe-size-5m-black>: HTTP status code is not handled or not allowed 2026-01-25 06:57:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-50bs21hx112-58-pitch-1-12-finished-bore-21-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:57:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nanoflex-tr-athletic-work-shoes-composite-toe-size-11-5m-black>: HTTP status code is not handled or not allowed 2026-01-25 06:57:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/plastic-eraser-latex-free-w-sleeve-2-1-2x7-8x1-2-white>: HTTP status code is not handled or not allowed 2026-01-25 06:57:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/clic-retractable-eraser-refillable-blue-barrel>: HTTP status code is not handled or not allowed 2026-01-25 06:57:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 78279 pages (at 100 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:57:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/altronix-al1024ulm-alarm-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 06:57:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/altronix-maxim55e-alarm-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 06:57:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-08bs21hx25-metric-12-pitch-25mm-finished-bore-21-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:57:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-40btl36-taper-bushed-12-pitch-36-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:57:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60bs09hx1-34-pitch-1-finished-bore-9-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:57:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-40a14hx12-a-plate-12-pitch-12-bore-14-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:57:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nano-x1-adventure-athletic-work-shoes-composite-toe-size-8w-slate-blue-salmon>: HTTP status code is not handled or not allowed 2026-01-25 06:57:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/altronix-maxim75-alarm-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 06:57:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/altronix-al201ul-alarm-replacement-battery-4-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 06:57:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-35bs15hx78-38-pitch-78-finished-bore-15-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:57:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80bs18hx2316-1-pitch-2-316-finished-bore-18-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:57:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lavante-trail-2-running-work-shoes-composite-toe-size-9-5m-gray-black>: HTTP status code is not handled or not allowed 2026-01-25 06:57:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lavante-trail-2-running-work-shoes-composite-toe-size-7w-army-green-black-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:57:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ag10-series-4-x-48-solid-alum-fixed-bar-supply-return-air-vent-grille-brushed-satin>: HTTP status code is not handled or not allowed 2026-01-25 06:57:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/altronix-smp5pmp4cb-alarm-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 06:57:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-80bs21x158-1-pitch-1-58-finished-bore-21-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:57:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-06bs18hx25-metric-38-pitch-25mm-finished-bore-18-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:57:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ul-rated-hook-n-loop-tape-fasteners-black-2-inch-x-15-feet>: HTTP status code is not handled or not allowed 2026-01-25 06:57:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lavante-trail-2-running-work-shoes-composite-toe-size-16m-gray-red-black>: HTTP status code is not handled or not allowed 2026-01-25 06:57:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quickie-z500-wheelchair-replacement-battery-55ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 06:57:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lavante-trail-2-running-work-shoes-composite-toe-size-10m-gray-black>: HTTP status code is not handled or not allowed 2026-01-25 06:57:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-hook-with-acrylic-adhesive-5-8-inch-x-75-feet>: HTTP status code is not handled or not allowed 2026-01-25 06:57:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/white-hook-sew-on-2-inch-x-75-feet>: HTTP status code is not handled or not allowed 2026-01-25 06:57:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-hook-sew-on-2-inch-x-75-feet>: HTTP status code is not handled or not allowed 2026-01-25 06:57:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nano-x1-adventure-athletic-work-mid-cut-shoes-composite-toe-size-7-5m-black>: HTTP status code is not handled or not allowed 2026-01-25 06:57:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/black-loop-with-acrylic-adhesive-4-inch-x-75-feet>: HTTP status code is not handled or not allowed 2026-01-25 06:57:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sprocket-60bs35x134-34-pitch-1-34-finished-bore-35-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:57:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elan-1660b-emergency-light-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 06:57:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/floatride-energy-3-adventure-men-s-athletic-work-shoes-composite-toe-size-9m-black>: HTTP status code is not handled or not allowed 2026-01-25 06:57:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/column-protector-18x-18-square-opening-38-od-x-42h-yellow>: HTTP status code is not handled or not allowed 2026-01-25 06:57:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/floatride-energy-3-adventure-men-s-athletic-work-shoes-composite-toe-size-6-5m-black>: HTTP status code is not handled or not allowed 2026-01-25 06:57:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/selfpowered-collision-warning-system-6l-x-4w-x-834h>: HTTP status code is not handled or not allowed 2026-01-25 06:57:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shoprider-serie-wheelchair-replacement-battery-7ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 06:57:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/universal-air-flow-sample-probe-tubing-kit>: HTTP status code is not handled or not allowed 2026-01-25 06:57:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elan-st3-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 06:57:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/floatride-energy-3-adventure-men-s-athletic-work-shoes-composite-toe-size-10-5w-black>: HTTP status code is not handled or not allowed 2026-01-25 06:57:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/classic-leather-boots-brown-15f-comfort-rating-size-12>: HTTP status code is not handled or not allowed 2026-01-25 06:57:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rotary-screw-air-compressor-460v-150-psi-25-cfm-75hp-3ph>: HTTP status code is not handled or not allowed 2026-01-25 06:57:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evolve-skate-inspired-high-top-work-shoes-composite-toe-size-7-5m-triple-black>: HTTP status code is not handled or not allowed 2026-01-25 06:57:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mojo-mallard-decoy-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 06:57:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-tapered-roller-bearing-e32009j-45mm-bore-75mm-od>: HTTP status code is not handled or not allowed 2026-01-25 06:57:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/series-e-a-standard-aluminum-beam-fe8066-3-for-96-and-102-trailers>: HTTP status code is not handled or not allowed 2026-01-25 06:57:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/kaufel-2272-emergency-light-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 06:57:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/winch-strap-423010-with-delta-ring-30-x-4-gold>: HTTP status code is not handled or not allowed 2026-01-25 06:57:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gs-gtx14ah-bs-powersports-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 06:57:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metric-tapered-roller-bearing-e32311j-55mm-bore-120mm-od>: HTTP status code is not handled or not allowed 2026-01-25 06:57:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/two-stage-gas-powered-truck-mount-compressor-with-honda-engine-l001154>: HTTP status code is not handled or not allowed 2026-01-25 06:57:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evolve-skate-inspired-work-shoes-composite-toe-size-9w-rust>: HTTP status code is not handled or not allowed 2026-01-25 06:57:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evolve-skate-inspired-work-shoes-composite-toe-size-13w-triple-black>: HTTP status code is not handled or not allowed 2026-01-25 06:57:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/scada-sp12-75-solar-replacement-battery-75ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 06:57:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/evolve-skate-inspired-work-shoes-composite-toe-size-8w-olive-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:57:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pressguard-report-cover-prong-clip-letter-3-capacity-light-green>: HTTP status code is not handled or not allowed 2026-01-25 06:57:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extreme-xt4l-b-powersports-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 06:57:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/elsar-16216-emergency-light-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 06:57:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/side-opening-pressguard-report-cover-prong-fastener-letter-red>: HTTP status code is not handled or not allowed 2026-01-25 06:57:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chrome-4l-bs-powersports-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 06:57:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/side-opening-pressboard-report-cover-prong-fastener-letter-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:57:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/honeywell-diaphragm-gas-valve-v48a2151-normally-closed-34-npt-12-max-psi>: HTTP status code is not handled or not allowed 2026-01-25 06:57:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-frame-motor-wwe75-9-444t-gp-tefc-rigid-3-ph-444t-95-fla-rb>: HTTP status code is not handled or not allowed 2026-01-25 06:57:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/46-ada-round-expanded-metal-picnic-table-black>: HTTP status code is not handled or not allowed 2026-01-25 06:57:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jet-pump-motor-atj12-36-56jb-gp-tefc-rem-c-3-ph-56j-208-230-460v-1-2-hp>: HTTP status code is not handled or not allowed 2026-01-25 06:57:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/comfortie-casual-work-shoes-steel-toe-size-11-5w-black-white>: HTTP status code is not handled or not allowed 2026-01-25 06:57:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shr830-spool-holder-rod-and-brackets>: HTTP status code is not handled or not allowed 2026-01-25 06:57:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pm-dc-motor-wpmdc34-18-180v-56cb-tenv-tefc-56c-180v-3-4-hp-1800-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:57:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cc-pump-motor-wwe3-36-145jm-tefc-rigid-c-3-ph-145jm-3-hp-3600-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:57:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/btlslsph101p-54-commercial-bottleless-hot-cold-water-cooler-w-filtration-silver>: HTTP status code is not handled or not allowed 2026-01-25 06:57:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fh101b-02-b1120-16-commercial-hot-cold-water-cooler-black-w-blue-trim>: HTTP status code is not handled or not allowed 2026-01-25 06:57:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/phano-non-toxic-china-marker-green-lead>: HTTP status code is not handled or not allowed 2026-01-25 06:57:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bb4500-high-top-work-sneaker-composite-toe-size-11-5m-black-white>: HTTP status code is not handled or not allowed 2026-01-25 06:57:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-frame-motor-wwht50-18-326t-f2-gp-tefc-rigid-3-ph-f2-326t-596-fla>: HTTP status code is not handled or not allowed 2026-01-25 06:57:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drr-scooter-and-moped-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 06:57:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-drainage-for-48-in-portable-evaporative-cooler-604152>: HTTP status code is not handled or not allowed 2026-01-25 06:57:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/replacement-water-level-sensor-for-48-in-portable-evaporative-cooler-604152>: HTTP status code is not handled or not allowed 2026-01-25 06:57:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/healthflex-privacy-screen-5-panel-stone>: HTTP status code is not handled or not allowed 2026-01-25 06:57:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-gsx-ltd-800r-800cc-snowmobile-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 06:57:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/400-industrial-carton-sealing-tape-3-x-110-yds-clear-pack-of-24>: HTTP status code is not handled or not allowed 2026-01-25 06:57:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-frame-motor-wwht200-18-445-7tc-f2-gp-tefc-rigid-3-ph-f2-445-7tc-230-fla>: HTTP status code is not handled or not allowed 2026-01-25 06:57:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/jet-pump-motor-tj15-36-56cb-nol-gp-tefc-rem-c-1-ph-56c-15-hp>: HTTP status code is not handled or not allowed 2026-01-25 06:57:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/motor-atj13-36-56cb-gp-tefc-rem-c-3-ph-56c-1-3-hp-3600-rpm-08-fla>: HTTP status code is not handled or not allowed 2026-01-25 06:57:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1590625-elbow-latch-bright-chrome>: HTTP status code is not handled or not allowed 2026-01-25 06:57:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-frame-motor-wwht100-18-405t-gp-tefc-rigid-3-ph-405t-115-fla-rb>: HTTP status code is not handled or not allowed 2026-01-25 06:57:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/t-frame-motor-wwht20-18-256t-gp-tefc-rigid-3-ph-256t-246-fla>: HTTP status code is not handled or not allowed 2026-01-25 06:58:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/arctic-cat-cougar-2up-550cc-snowmobile-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 06:58:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cc-pump-motor-wwe30-36-286jp-tefc-rigid-c-3-ph-286jp-30-hp-3600-rpm>: HTTP status code is not handled or not allowed 2026-01-25 06:58:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/welded-wire-security-partition-swing-door-hlso-4w-x-8h-black>: HTTP status code is not handled or not allowed 2026-01-25 06:58:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/knee-shin-instep-guards-web-straps-aluminum-alloy-14l-x-5w-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 06:58:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-legend-600-ho-sdi-593cc-snowmobile-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 06:58:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mens-metal-foot-guards-steel-toe-clip-rubber-strap-5w-standard-1-pair>: HTTP status code is not handled or not allowed 2026-01-25 06:58:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-expedition-4-tek-700cc-snowmobile-replacement-battery-12v-i>: HTTP status code is not handled or not allowed 2026-01-25 06:58:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ft-one-fit-non-belted-full-body-harness-standard-1-d-ring-tongue-buckle-legs-2x-large>: HTTP status code is not handled or not allowed 2026-01-25 06:58:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/premium-strap-galvanized-steel-hooks-36l-blue>: HTTP status code is not handled or not allowed 2026-01-25 06:58:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ft-one-fit-non-belted-full-body-harness-standard-1-d-ring-quick-connect-legs-large>: HTTP status code is not handled or not allowed 2026-01-25 06:58:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/brp-mx-z-x-800-ho-ptek-800cc-snowmobile-replacement-battery-12v-b>: HTTP status code is not handled or not allowed 2026-01-25 06:58:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/multi-purpose-metal-access-panel-cam-lock-12l-x-16w>: HTTP status code is not handled or not allowed 2026-01-25 06:58:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/u-boat-utility-cart-1750-lb-capacity-63l-x-16w-x-60h>: HTTP status code is not handled or not allowed 2026-01-25 06:58:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/deluxe-black-plastic-tilt-truck-1-cubic-yard-1250-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 06:58:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/apc-smartups-rm-sua3000rm2u-ups-replacement-battery-5ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 06:58:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 78376 pages (at 97 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:58:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/apc-back-ups-back-ups-be550g-ups-replacement-battery-9ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 06:58:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbi-sala174-nano-lok-xl-self-retracting-lifeline-w-aluminum-carabiner-11l>: HTTP status code is not handled or not allowed 2026-01-25 06:58:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/35ft-high-speed-hdmi-cable-digital-video-w-audio-m-m-35>: HTTP status code is not handled or not allowed 2026-01-25 06:58:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6ft-low-profile-vga-coax-monitor-cable-high-resolution-hd15-m-m>: HTTP status code is not handled or not allowed 2026-01-25 06:58:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6ft-straight-through-serial-rs232-extension-cable-db9-m-f-6>: HTTP status code is not handled or not allowed 2026-01-25 06:58:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-extinguisher-cabinet-vertical-acrylic-window-semi-recessed-6d-ss-3-rolled>: HTTP status code is not handled or not allowed 2026-01-25 06:58:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-extinguisher-cabinet-solid-door-fully-recessed-6d-steel>: HTTP status code is not handled or not allowed 2026-01-25 06:58:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/displayport-to-hdmi-converter-video-adapter-1920x1200-1080p-m-f>: HTTP status code is not handled or not allowed 2026-01-25 06:58:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fire-extinguisher-cabinet-vertical-acrylic-window-fully-recessed-6d-steel>: HTTP status code is not handled or not allowed 2026-01-25 06:58:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ft-one-fit-non-belted-full-body-harness-standard-1-d-ring-quick-connect-legs-x-large>: HTTP status code is not handled or not allowed 2026-01-25 06:58:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/automated-time-clock-fingerprint-system-black-fpt-80-a959>: HTTP status code is not handled or not allowed 2026-01-25 06:58:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dbi-sala-nano-lok-xl-twin-leg-self-retracting-lifeline-w-quick-connect-rebar-hook-9l>: HTTP status code is not handled or not allowed 2026-01-25 06:58:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ajc-light-ce1-5bf-6v-5ah-emergency-light-battery>: HTTP status code is not handled or not allowed 2026-01-25 06:58:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rotary-screw-air-compressor-w-dryer-460v-115psi-55cfm-15hp-3ph-120-gal-tank>: HTTP status code is not handled or not allowed 2026-01-25 06:58:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ajc-lithonia-elb-06010-6v-12ah-emergency-light-battery>: HTTP status code is not handled or not allowed 2026-01-25 06:58:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rotary-screw-air-compressor-200v-150psi-50cfm-15hp-3ph-120-gal-tank>: HTTP status code is not handled or not allowed 2026-01-25 06:58:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rotary-screw-air-compressor-230v-125psi-38cfm-10hp-3ph-120-gal-tank>: HTTP status code is not handled or not allowed 2026-01-25 06:58:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-in-heavy-duty-plastic-chain-50-feet-black>: HTTP status code is not handled or not allowed 2026-01-25 06:58:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simplex-112-136-industrial-replacement-battery-10ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 06:58:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/epdm-commercial-grade-60a-rubber-sheet-1-8-thick-x-3-x-8-length-black>: HTTP status code is not handled or not allowed 2026-01-25 06:58:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/epdm-commercial-grade-60a-rubber-sheet-1-2-thick-x-3-x-10-length-black>: HTTP status code is not handled or not allowed 2026-01-25 06:58:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caterpillar-d35-industrial-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 06:58:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/personal-first-aid-kit-1-24-persons-34-pieces-osha-compliant-plastic-case>: HTTP status code is not handled or not allowed 2026-01-25 06:58:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/products-350-unassembled-pedestal-cement-mixer>: HTTP status code is not handled or not allowed 2026-01-25 06:58:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/new-f-180-folding-saw-180mm-large-teeth>: HTTP status code is not handled or not allowed 2026-01-25 06:58:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ready-built153-6-tier-18-doors-locker-36w-x-12d-x-78h-black-assembled>: HTTP status code is not handled or not allowed 2026-01-25 06:58:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pbq-5-2-12-sealed-lead-acid-replacement-battery-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 06:58:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/new-holland-l775-industrial-replacement-battery-5ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 06:58:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/extreme-cold-weather-kit-for-liquid-cooled-standby-generators-with-15l-engine>: HTTP status code is not handled or not allowed 2026-01-25 06:58:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nonin-medical-lark-6500-medical-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 06:58:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ready-built153-1-tier-3-doors-locker-36w-x-12d-x-78h-black-assembled>: HTTP status code is not handled or not allowed 2026-01-25 06:58:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yellow-post-safety-barrier-15-ft-yellow-black-belt-danger-keep-out-w-roller-base>: HTTP status code is not handled or not allowed 2026-01-25 06:58:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/metal-sign-frame-10-stem-shovel-base-11w-x-7h-black>: HTTP status code is not handled or not allowed 2026-01-25 06:58:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4-ply-poster-board-28w-x-22h-dark-blue-25-carton>: HTTP status code is not handled or not allowed 2026-01-25 06:58:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leoch-djw6-4-0-sealed-lead-acid-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 06:58:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vent-cap-male-thread-2-1>: HTTP status code is not handled or not allowed 2026-01-25 06:58:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/high-capacity-waterfall-deck-shelving-5-shelves-72w-x-36d-x-96h-midnight-ebony>: HTTP status code is not handled or not allowed 2026-01-25 06:58:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-tech-open-shelving-8-shelves-starter-unit-800-lb-cap-36w-x-18d-x-87h-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:58:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/digital-wheelchair-ramp-scale-1000lb-capacity-emr-connectivity>: HTTP status code is not handled or not allowed 2026-01-25 06:58:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/powered-drum-truck-f89503p-plastic-chime-hook-10-wheels>: HTTP status code is not handled or not allowed 2026-01-25 06:58:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sbr-red-65a-sheet-1-8-thick-x-3-x-18-length>: HTTP status code is not handled or not allowed 2026-01-25 06:58:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sbr-red-65a-sheet-3-16-thick-x-4-x-4-length-8-pack>: HTTP status code is not handled or not allowed 2026-01-25 06:58:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eye-level-digital-beam-scale-500lb-capacity-emr-connectivity-built-in-height-rod>: HTTP status code is not handled or not allowed 2026-01-25 06:58:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-tech-closed-stand-alone-shelving-6-shelves-500-lb-cap-36w-x-18d-x-87h-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:58:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/12-galvanized-spike-and-washer>: HTTP status code is not handled or not allowed 2026-01-25 06:58:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ready-built153-6-tier-6-doors-locker-12w-x-18d-x-78h-light-gray-assembled>: HTTP status code is not handled or not allowed 2026-01-25 06:58:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aleco-anchorbar-chairmat-rectangle-46w-x-60l-intermediate-thick-beveled-edge>: HTTP status code is not handled or not allowed 2026-01-25 06:58:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sensory-table-with-shelf>: HTTP status code is not handled or not allowed 2026-01-25 06:58:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exploration-lab-6962jc>: HTTP status code is not handled or not allowed 2026-01-25 06:58:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/big-book-pick-a-book-stand-1-sided>: HTTP status code is not handled or not allowed 2026-01-25 06:58:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-tech-open-shelving-8-shelves-starter-unit-400-lb-cap-48w-x-12d-x-87h-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:58:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/closed-style-deep-bin-shelving-7-shelves-add-on-unit-36w-x-72d-x-87h-bluegray>: HTTP status code is not handled or not allowed 2026-01-25 06:58:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/5-step-heavy-duty-double-entry-work-platform-36w-steps-dep5-3648>: HTTP status code is not handled or not allowed 2026-01-25 06:58:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leoch-djw6-1-2-sealed-lead-acid-replacement-battery-1-3ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 06:58:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exell-eb1213-sealed-lead-acid-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 06:58:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/970-06-platform-landing-48-x-48-floor-height-72>: HTTP status code is not handled or not allowed 2026-01-25 06:58:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exell-eb1222-sealed-lead-acid-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 06:58:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exell-eb12220-sealed-lead-acid-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 06:58:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aprilaire-95-pint-per-day-whole-home-hehumidifer-1850>: HTTP status code is not handled or not allowed 2026-01-25 06:58:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eco-flo-sup54-submersible-utility-pump-manual>: HTTP status code is not handled or not allowed 2026-01-25 06:58:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/yuasa-npx50-250-sealed-lead-acid-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 06:58:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/30-in-galvanized-flanged-motorized-damper>: HTTP status code is not handled or not allowed 2026-01-25 06:58:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mighty-products-mt-tt-0812-utility-trailer-mesh-cover-8x12>: HTTP status code is not handled or not allowed 2026-01-25 06:58:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/16-in-stainless-steel-wall-exhaust-shutter>: HTTP status code is not handled or not allowed 2026-01-25 06:58:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-tech-closed-stand-alone-shelving-5-shelves-500-lb-cap-48w-x-24d-x-87h-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:58:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-tech-closed-stand-alone-shelving-5-shelves-375-lb-cap-48w-x-12d-x-87h-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:58:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mighty-products-mn-dn-1630-debris-safety-netting-16-x-30>: HTTP status code is not handled or not allowed 2026-01-25 06:58:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e-track-d-ring-tie-down-strap-4400-lb-capacity-2-x-9>: HTTP status code is not handled or not allowed 2026-01-25 06:58:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mighty-products-mn-dn-2050-debris-safety-netting-20-x-50>: HTTP status code is not handled or not allowed 2026-01-25 06:58:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hi-tech-open-style-shelving-sway-braces-6-shelves-36w-x-24d-x-87h-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:58:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/solo-agm1265t-sealed-lead-acid-replacement-battery-55ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 06:58:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mk-m27-sld-g-sealed-lead-acid-replacement-battery-100ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 06:58:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/caterpillar-d22-d25-industrial-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 06:58:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leoch-lp12-5-4-sealed-lead-acid-replacement-battery-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 06:58:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mighty-products-mn-dn-1220-debris-safety-netting-12-x-20>: HTTP status code is not handled or not allowed 2026-01-25 06:58:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/36-in-aluminum-wall-exhaust-shutter-rear-flange>: HTTP status code is not handled or not allowed 2026-01-25 06:58:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/mighty-products-mn-pf90-b0812-8-x-12-premier-privacy-fence-screen-90-blockage>: HTTP status code is not handled or not allowed 2026-01-25 06:58:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stair-tread-rubber-square-nose-48l-light-brown>: HTTP status code is not handled or not allowed 2026-01-25 06:58:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/topin-tp12-21-sealed-lead-acid-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 06:58:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cove-base-700-series-tpr-4x18x48-dolphin>: HTTP status code is not handled or not allowed 2026-01-25 06:58:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/zeus-pc9-12s-f1-sealed-lead-acid-replacement-battery-9ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 06:58:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stair-tread-rubber-square-nose-60l-black>: HTTP status code is not handled or not allowed 2026-01-25 06:58:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bb-hr9-12-hr9-12t2-12v-9ah-ups-battery>: HTTP status code is not handled or not allowed 2026-01-25 06:58:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/atlite-24-1002-sealed-lead-acid-replacement-battery-4-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 06:59:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/maintenance-free-quiet-2-tier-6-doors-locker-45w-x-24d-x-78h-dark-gray-assembled>: HTTP status code is not handled or not allowed 2026-01-25 06:59:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e-track-weld-on-contoured-single-strap-anchor-zinc>: HTTP status code is not handled or not allowed 2026-01-25 06:59:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/maintenance-free-quiet-1-tier-3-doors-locker-45w-x-21d-x-78h-dark-gray-unassembled>: HTTP status code is not handled or not allowed 2026-01-25 06:59:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/panasonic-up-rw1245p1-uprw1245p1-12v-9ah-ups-battery>: HTTP status code is not handled or not allowed 2026-01-25 06:59:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e-track-multi-purpose-j-hook-2-hanger-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-25 06:59:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cove-base-vinyl-4x080x120-coil-brown>: HTTP status code is not handled or not allowed 2026-01-25 06:59:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/unison-mx-06045-sealed-lead-acid-replacement-battery-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 06:59:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/e-track-tie-down-single-truck-trailer-anchor-kit-w-2-x-192-cam-strap>: HTTP status code is not handled or not allowed 2026-01-25 06:59:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sota-sa1272-sa-1272-12v-8ah-ups-battery>: HTTP status code is not handled or not allowed 2026-01-25 06:59:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/upsonic-pcm-140-sealed-lead-acid-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 06:59:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/fast-track-e-track-horizontal-vertical-anchor-track-32-galvanized-steel>: HTTP status code is not handled or not allowed 2026-01-25 06:59:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/uniwell-sla1050-sealed-lead-acid-replacement-battery-7ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 06:59:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/volcano-kb1240-sealed-lead-acid-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 06:59:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/vision-cp12180-sealed-lead-acid-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 06:59:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/newmox-fnc-1270-sealed-lead-acid-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 06:59:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 78475 pages (at 99 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 06:59:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/leoch-lpx12-5-4-sealed-lead-acid-replacement-battery-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 06:59:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/accuglide-3-upper-taping-head-3>: HTTP status code is not handled or not allowed 2026-01-25 06:59:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/karryall-model-76-adjustable-conductive-small-pcb-rack-15w-1314d-4h>: HTTP status code is not handled or not allowed 2026-01-25 06:59:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-in-close-galvanized-steel-pipe-nipple-150-psi-lead-free>: HTTP status code is not handled or not allowed 2026-01-25 06:59:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/stainless-steel-manual-chain-hoist-1-ton-capacity-15-lift>: HTTP status code is not handled or not allowed 2026-01-25 06:59:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/exair-soft-grip-safety-air-gun-wadjustable-super-air-nozzle-and-chip-shield-zinc>: HTTP status code is not handled or not allowed 2026-01-25 06:59:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/altronix-al600ulxpd16-alarm-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 06:59:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/centrifuge-tube-rack-set-sterile-15ml-capacity-amber-pack-of-500>: HTTP status code is not handled or not allowed 2026-01-25 06:59:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/we3020kd-sp-steam-table-pan-rack-18-pan-capacity-17w-x-71-12h-x-20d>: HTTP status code is not handled or not allowed 2026-01-25 06:59:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/1-2-in-1-1-2-in-black-steel-pipe-nipple-150-psi-lead-free>: HTTP status code is not handled or not allowed 2026-01-25 06:59:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/altronix-al802uladaj-alarm-replacement-battery-12ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 06:59:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/n187248-5-equipment-stands-5-shelf-shelving-unit-18w-x-72h-x-48l-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 06:59:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/volt153-portable-electric-heater-w-thermostat-240v-3-phase-40000w>: HTTP status code is not handled or not allowed 2026-01-25 06:59:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ws3026sc-7st-slicermixerscale-cart-w-pan-slides-28-14w-x-30h-x-20-12d>: HTTP status code is not handled or not allowed 2026-01-25 06:59:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/chill2-chill-tray-dollies-double-22-34w-x-10h-x-39d-aluminum>: HTTP status code is not handled or not allowed 2026-01-25 06:59:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2-in-galvanized-malleable-cap-150-psi-lead-free>: HTTP status code is not handled or not allowed 2026-01-25 06:59:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/altronix-al600ulpd8cb-alarm-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 06:59:19 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/belkin-pro-f6c100-4-3-x-12v7ah-12v-8ah-ups-battery>: HTTP status code is not handled or not allowed 2026-01-25 06:59:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eaton-ex-exb-1000-ex-exb-1000-rt2u-12v-7ah-ups-battery>: HTTP status code is not handled or not allowed 2026-01-25 06:59:20 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/coopower-cpd12-7-sealed-lead-acid-replacement-battery-7ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 06:59:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gauze-sponge-absorbent-2l-x-2w-4-plys-200bx-fabric-wh>: HTTP status code is not handled or not allowed 2026-01-25 06:59:21 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/werker-wka12-7f2-sealed-lead-acid-replacement-battery-7ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 06:59:22 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cryo-cube-box-polypropylene-100-places-blueyellow-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-25 06:59:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/invacare-arrow-3g-wheelchair-replacement-battery-75ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 06:59:23 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/type-i-steel-safety-can-with-funnel-2-5-gal-9-5l-self-close-lid-yellow-7125210>: HTTP status code is not handled or not allowed 2026-01-25 06:59:24 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/laboratory-scoops-pp-white-pack-of-7>: HTTP status code is not handled or not allowed 2026-01-25 06:59:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/drench-shower-floor-mount-galvanized-pipe-nylon-6-showerhead-20-gpm-240v>: HTTP status code is not handled or not allowed 2026-01-25 06:59:25 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/type-i-steel-safety-can-with-funnel-5-gallon-19l-self-close-lid-red-7150110>: HTTP status code is not handled or not allowed 2026-01-25 06:59:26 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6222-deep-groove-ball-bearing-open-110mm-bore-200mm-od>: HTTP status code is not handled or not allowed 2026-01-25 06:59:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/desiccator-non-vacuum-white-base-polypropylenepolycarbonate-12-dia>: HTTP status code is not handled or not allowed 2026-01-25 06:59:27 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/62112rs-deep-groove-ball-bearing-double-sealed-55mm-bore-100mm-od>: HTTP status code is not handled or not allowed 2026-01-25 06:59:28 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/32013-tapered-roller-bearing-metric-65mm-bore-100mm-od>: HTTP status code is not handled or not allowed 2026-01-25 06:59:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/shepard-agm1265t-wheelchair-replacement-battery-75ah-12v-it>: HTTP status code is not handled or not allowed 2026-01-25 06:59:29 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/22500-lbs-capacity-tow-strap-3-inch-x-20-ft-looped-ends>: HTTP status code is not handled or not allowed 2026-01-25 06:59:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merits-p318-2s-c-wheelchair-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 06:59:30 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/c-clamp-2-inch>: HTTP status code is not handled or not allowed 2026-01-25 06:59:31 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/merits-p1201-tbmu-wheelchair-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 06:59:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/rack-for-microcentrifuge-tubes-pc-48-places-white-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-25 06:59:32 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/cryo-rack-for-5ml-capacity-vials-81-places-pack-of-4>: HTTP status code is not handled or not allowed 2026-01-25 06:59:33 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6004zz-p53-deep-groove-ball-bearing-double-shielded-abec-5-20mm-bore-42mm-od>: HTTP status code is not handled or not allowed 2026-01-25 06:59:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-tier-test-tube-rack-for-13mm-tubes-plastic-62-places-white-pack-of-2>: HTTP status code is not handled or not allowed 2026-01-25 06:59:34 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pride-mobility-lx-11-wheelchair-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 06:59:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/reagent-bottle-wide-mouth-pp-1000ml-capacity-clear-pack-of-50>: HTTP status code is not handled or not allowed 2026-01-25 06:59:35 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/quantum-rehab-melody-wheelchair-replacement-battery-5ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 06:59:36 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/teardrop-bolted-pallet-rack-add-on-96w-x-36d-x-96h>: HTTP status code is not handled or not allowed 2026-01-25 06:59:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6202zz-deep-groove-ball-bearing-double-shielded-15mm-bore-35mm-od>: HTTP status code is not handled or not allowed 2026-01-25 06:59:37 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/invacare-power-9000-wheelchair-replacement-battery-35ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 06:59:38 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/aerial-shelf-for-bench-22630lg-dove-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:59:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/teardrop-bolted-pallet-rack-add-on-96w-x-42d-x-144h>: HTTP status code is not handled or not allowed 2026-01-25 06:59:39 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/pacesaver-pediatric-wheelchair-replacement-battery-26ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 06:59:40 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6311zznr-deep-groove-ball-bearing-double-shielded-snap-ring-55mm-bore-120mm-od>: HTTP status code is not handled or not allowed 2026-01-25 06:59:41 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ev-rider-power-bike-wheelchair-replacement-battery-75ah-12v-it>: HTTP status code is not handled or not allowed 2026-01-25 06:59:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/3-tier-3-door-wood-locker-12-w-x-15-d-x-72-h-cherry-assembled>: HTTP status code is not handled or not allowed 2026-01-25 06:59:42 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/gilson-14he-lawn-and-garden-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 06:59:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/4ink-color-retractable-pen-medium-blueblackredgreen-ink>: HTTP status code is not handled or not allowed 2026-01-25 06:59:43 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/makita-um401dw-lawn-and-garden-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 06:59:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/630042rs-deep-groove-ball-bearing-wide-width-double-sealed-20mm-bore-42mm-od>: HTTP status code is not handled or not allowed 2026-01-25 06:59:44 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sears-502-25538-lawn-and-garden-replacement-battery-12ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 06:59:45 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/simplicity-22h-lawn-and-garden-replacement-battery-22ah-12v-nb>: HTTP status code is not handled or not allowed 2026-01-25 06:59:46 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6006zz-deep-groove-ball-bearing-double-shielded-30mm-bore-55mm-od>: HTTP status code is not handled or not allowed 2026-01-25 06:59:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/6307-deep-groove-ball-bearing-open-35mm-bore-80mm-od>: HTTP status code is not handled or not allowed 2026-01-25 06:59:47 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelf-truck-36l-x-14w-x-80h-polyurethane-swivel-casters>: HTTP status code is not handled or not allowed 2026-01-25 06:59:48 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hiliter-pen-style-highlighter-chisel-tip-fluorescent-yellow-ink-dozen>: HTTP status code is not handled or not allowed 2026-01-25 06:59:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60440211401-cloth-belt-340d-3w-x-21l-50-grit>: HTTP status code is not handled or not allowed 2026-01-25 06:59:49 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sears-256321-lawn-and-garden-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 06:59:50 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60440196826-cloth-belt-341d-6w-x-108l-80-grit>: HTTP status code is not handled or not allowed 2026-01-25 06:59:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/twisterase-express-mechanical-pencil-refillable-09mm-smoke>: HTTP status code is not handled or not allowed 2026-01-25 06:59:51 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/nexel-black-epoxy-wire-shelf-truck-24l-x-24w-x-69h-polyurethane-brake-casters>: HTTP status code is not handled or not allowed 2026-01-25 06:59:52 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60440200479-341d-3-8dia-x-1w-60-grit-100-case>: HTTP status code is not handled or not allowed 2026-01-25 06:59:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/60440200800-341d-1-2dia-x-1w-80-grit-100-case>: HTTP status code is not handled or not allowed 2026-01-25 06:59:53 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8020-15-series-1-12-x-34-lite-mono-t-slotted-extrusion-profile-120l-stock-bar>: HTTP status code is not handled or not allowed 2026-01-25 06:59:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/tru-ray-construction-paper-18x12-blue-50-sheets>: HTTP status code is not handled or not allowed 2026-01-25 06:59:54 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sunworks-groundwood-construction-paper-18x12-light-brown-50-sheets>: HTTP status code is not handled or not allowed 2026-01-25 06:59:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8020-25-series-1-x-1-mono-t-slotted-extrusion-profile-118-18l-stock-bar>: HTTP status code is not handled or not allowed 2026-01-25 06:59:55 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/baby-changing-station-w-stainless-steel-veneer-polyethylene-gray>: HTTP status code is not handled or not allowed 2026-01-25 06:59:56 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/twintip-permanent-marker-fineultra-point-black-ink>: HTTP status code is not handled or not allowed 2026-01-25 06:59:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/big-beam-windsor-6-emergency-light-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 06:59:57 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/energel-liquid-gel-ink-pen-refillable-10mm-blue-barrelink>: HTTP status code is not handled or not allowed 2026-01-25 06:59:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8020-25-series-1-x-1-1516-six-t-slotted-extrusion-profile-118-18l-stock-bar>: HTTP status code is not handled or not allowed 2026-01-25 06:59:58 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/petropad153-smart-polymer-spill-pad-53-78l-x-53-78w-x-large-white-pk-of-2>: HTTP status code is not handled or not allowed 2026-01-25 06:59:59 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/medium-executive-quick-cart-1902466-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 07:00:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/large-executive-quick-cart-1902465-dark-gray>: HTTP status code is not handled or not allowed 2026-01-25 07:00:00 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/portalac-lac-pe76r-emergency-light-replacement-battery-7ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 07:00:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/858-1001-external-grinder-kit-series-57-34hp-1ph>: HTTP status code is not handled or not allowed 2026-01-25 07:00:01 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/8020-10-series-1-x-1-opposite-bi-t-slotted-extrusion-profile-120l-stock-bar>: HTTP status code is not handled or not allowed 2026-01-25 07:00:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/toro-titan-mx5400-lawn-and-garden-replacement-battery-9ah-12v-f2>: HTTP status code is not handled or not allowed 2026-01-25 07:00:02 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/46405-332-loop-thandle-cushion-grip-hex-key-6-handle>: HTTP status code is not handled or not allowed 2026-01-25 07:00:03 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precise-p700-gel-rollerball-pen-fine-07mm-black-barrelink-dozen>: HTTP status code is not handled or not allowed 2026-01-25 07:00:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/precise-p500-gel-rollerball-pen-extra-fine-05mm-red-barrelink-dozen>: HTTP status code is not handled or not allowed 2026-01-25 07:00:04 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bta210-digital-balance-210g-x-0-01g-4-11-16-diameter-platform>: HTTP status code is not handled or not allowed 2026-01-25 07:00:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/iq-square-auxiliary-led-flood-beam-light-with-16-led-4-14-clear>: HTTP status code is not handled or not allowed 2026-01-25 07:00:05 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/razor-point-marker-pen-extra-fine-black-ink-dozen>: HTTP status code is not handled or not allowed 2026-01-25 07:00:06 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/technologies-ez-shopper-electric-grocery-cart-ezs-1772-8000-bea-blue-750-lb-cap>: HTTP status code is not handled or not allowed 2026-01-25 07:00:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/lopro-wireless-solar-powered-truckcab-led-light-white-pack-of-5>: HTTP status code is not handled or not allowed 2026-01-25 07:00:07 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/ac-converter-battery-charger-25-amp>: HTTP status code is not handled or not allowed 2026-01-25 07:00:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-hipolymer-lead-refill-b-leads-07mm-medium-black-12tube>: HTTP status code is not handled or not allowed 2026-01-25 07:00:08 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/dual-lite-sdg-emergency-light-replacement-battery-5ah-6v-f1>: HTTP status code is not handled or not allowed 2026-01-25 07:00:09 [scrapy.extensions.closespider] (PID: 96) INFO: Closing spider since no items were produced in the last 1800 seconds. 2026-01-25 07:00:09 [scrapy.core.engine] (PID: 96) INFO: Closing spider (closespider_timeout_no_item) 2026-01-25 07:00:09 [scrapy.extensions.logstats] (PID: 96) INFO: Crawled 78572 pages (at 97 pages/min), scraped 35565 items (at 0 items/min) 2026-01-25 07:00:09 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/light-duty-no-bump-dump-hopper-steel-2-cu-yd-2000-lb-capacity-yellow>: HTTP status code is not handled or not allowed 2026-01-25 07:00:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/heavy-duty-no-bump-dump-hopper-steel-3-cu-yd-6000-lb-capacity-machine-gray>: HTTP status code is not handled or not allowed 2026-01-25 07:00:10 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/eg1f-p-genesis-horn-white-marked-fire-steady-horn>: HTTP status code is not handled or not allowed 2026-01-25 07:00:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/johnlite-2996-emergency-light-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 07:00:11 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/2447th-r-temporal-horn-red-horn-only>: HTTP status code is not handled or not allowed 2026-01-25 07:00:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/sd-st-terminal-sensor-4-wiretb>: HTTP status code is not handled or not allowed 2026-01-25 07:00:12 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/johnlite-2922-emergency-light-replacement-battery-7ah-12v-f1>: HTTP status code is not handled or not allowed 2026-01-25 07:00:13 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/9x3-4-high-density-poly-fabric-roller-cover-24-case-9sc075>: HTTP status code is not handled or not allowed 2026-01-25 07:00:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/super-sticky-pads-3-inchx3-inch-45-sheets-pad-3-pads-pk-assorted>: HTTP status code is not handled or not allowed 2026-01-25 07:00:14 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/defender-articulated-extendable-ladder-aluminum-4-type-ia-300-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 07:00:15 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/velocity-articulated-extendable-ladder-aluminum-6-type-ia-300-lb-capacity>: HTTP status code is not handled or not allowed 2026-01-25 07:00:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bb-ep712-t2-12v-7ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-25 07:00:16 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/bb-bp456-sealed-lead-acid-agm-vrla-battery>: HTTP status code is not handled or not allowed 2026-01-25 07:00:17 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/electrostatic-air-filter-merv-8-10w-x-1d-x-30h>: HTTP status code is not handled or not allowed 2026-01-25 07:00:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/hawker-cyclon-08090012-6v-5ah-sealed-lead-acid-battery>: HTTP status code is not handled or not allowed 2026-01-25 07:00:18 [scrapy.spidermiddlewares.httperror] (PID: 96) INFO: Ignoring response <403 https://www.globalindustrial.com/p/power-patrol-sla1056-sealed-lead-acid-agm-vrla-battery>: HTTP status code is not handled or not allowed 2026-01-25 07:00:18 [global_industrial] (PID: 96) INFO: [Spidermon] ------------------------------ MONITORS ------------------------------ 2026-01-25 07:00:18 [global_industrial] (PID: 96) INFO: [Spidermon] Extracted Items Monitor/test_stat_monitor... FAIL 2026-01-25 07:00:18 [global_industrial] (PID: 96) INFO: [Spidermon] Item Validation Monitor/test_stat_monitor... SKIPPED (Unable to find 'spidermon/validation/fields/errors' in job stats.) 2026-01-25 07:00:18 [global_industrial] (PID: 96) INFO: [Spidermon] Error Count Monitor/test_stat_monitor... OK 2026-01-25 07:00:18 [global_industrial] (PID: 96) INFO: [Spidermon] Warning Count Monitor/test_stat_monitor... FAIL 2026-01-25 07:00:18 [global_industrial] (PID: 96) INFO: [Spidermon] Finish Reason Monitor/Should have the expected finished reason(s)... FAIL 2026-01-25 07:00:18 [global_industrial] (PID: 96) INFO: [Spidermon] Unwanted HTTP codes monitor/Should not hit the limit of unwanted http status... OK 2026-01-25 07:00:18 [global_industrial] (PID: 96) INFO: [Spidermon] Field Coverage Monitor/test_check_if_field_coverage_rules_are_met... OK 2026-01-25 07:00:18 [global_industrial] (PID: 96) INFO: [Spidermon] Retry Count monitor/Should not hit the limit of requests that reached the maximum retry amount... OK 2026-01-25 07:00:18 [global_industrial] (PID: 96) INFO: [Spidermon] Downloader Exceptions monitor/test_stat_monitor... OK 2026-01-25 07:00:18 [global_industrial] (PID: 96) INFO: [Spidermon] Successful Requests monitor/Should have at least the minimum number of successful requests... OK 2026-01-25 07:00:18 [global_industrial] (PID: 96) INFO: [Spidermon] Total Requests monitor/Should not hit the total limit of requests... OK 2026-01-25 07:00:18 [global_industrial] (PID: 96) INFO: [Spidermon] ---------------------------------------------------------------------- 2026-01-25 07:00:18 [global_industrial] (PID: 96) ERROR: [Spidermon] ====================================================================== FAIL: Extracted Items Monitor/test_stat_monitor ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/twisted/internet/task.py", line 526, in _oneWorkUnit result = next(self._iterator) ^^^^^^^^^^^^^^^^^^^^ StopIteration During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/spidermon/contrib/scrapy/monitors/base.py", line 184, in test_stat_monitor assertion_method( AssertionError: Expecting 'item_scraped_count' to be '>=' to '100000.0'. Current value: '35565' 2026-01-25 07:00:18 [global_industrial] (PID: 96) ERROR: [Spidermon] ====================================================================== FAIL: Warning Count Monitor/test_stat_monitor ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/twisted/internet/task.py", line 526, in _oneWorkUnit result = next(self._iterator) ^^^^^^^^^^^^^^^^^^^^ StopIteration During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/spidermon/contrib/scrapy/monitors/base.py", line 184, in test_stat_monitor assertion_method( AssertionError: Expecting 'log_count/WARNING' to be '<=' to '2.0'. Current value: '3994' 2026-01-25 07:00:18 [global_industrial] (PID: 96) ERROR: [Spidermon] ====================================================================== FAIL: Finish Reason Monitor/Should have the expected finished reason(s) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/twisted/internet/task.py", line 526, in _oneWorkUnit result = next(self._iterator) ^^^^^^^^^^^^^^^^^^^^ StopIteration During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/spidermon/contrib/scrapy/monitors/monitors.py", line 110, in test_should_finish_with_expected_reason self.assertTrue(finished_reason in expected_reasons, msg=msg) AssertionError: Finished with "closespider_timeout_no_item" the expected reasons are ['finished'] 2026-01-25 07:00:18 [global_industrial] (PID: 96) INFO: [Spidermon] 11 monitors in 0.006s 2026-01-25 07:00:18 [global_industrial] (PID: 96) INFO: [Spidermon] FAILED (failures=3, skipped=1) 2026-01-25 07:00:18 [global_industrial] (PID: 96) INFO: [Spidermon] -------------------------- FINISHED ACTIONS -------------------------- 2026-01-25 07:00:18 [global_industrial] (PID: 96) INFO: [Spidermon] ---------------------------------------------------------------------- 2026-01-25 07:00:18 [global_industrial] (PID: 96) INFO: [Spidermon] 0 actions in 0.000s 2026-01-25 07:00:18 [global_industrial] (PID: 96) INFO: [Spidermon] OK 2026-01-25 07:00:18 [global_industrial] (PID: 96) INFO: [Spidermon] --------------------------- PASSED ACTIONS --------------------------- 2026-01-25 07:00:18 [global_industrial] (PID: 96) INFO: [Spidermon] ---------------------------------------------------------------------- 2026-01-25 07:00:18 [global_industrial] (PID: 96) INFO: [Spidermon] 0 actions in 0.000s 2026-01-25 07:00:18 [global_industrial] (PID: 96) INFO: [Spidermon] OK 2026-01-25 07:00:18 [global_industrial] (PID: 96) INFO: [Spidermon] --------------------------- FAILED ACTIONS --------------------------- 2026-01-25 07:00:18 [global_industrial] (PID: 96) INFO: [Spidermon] CustomTemplateSendSlackMessageSpiderFinished... ERROR 2026-01-25 07:00:18 [global_industrial] (PID: 96) INFO: [Spidermon] ---------------------------------------------------------------------- 2026-01-25 07:00:18 [global_industrial] (PID: 96) ERROR: [Spidermon] ====================================================================== ERROR: CustomTemplateSendSlackMessageSpiderFinished ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/twisted/internet/task.py", line 526, in _oneWorkUnit result = next(self._iterator) ^^^^^^^^^^^^^^^^^^^^ StopIteration During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/spidermon/core/actions.py", line 39, in run self.run_action() File "/usr/local/lib/python3.11/site-packages/spidermon/contrib/actions/slack/__init__.py", line 252, in run_action self.manager.send_message( File "/usr/local/lib/python3.11/site-packages/spidermon/contrib/actions/slack/__init__.py", line 58, in send_message return [ ^ File "/usr/local/lib/python3.11/site-packages/spidermon/contrib/actions/slack/__init__.py", line 59, in self.send_message( File "/usr/local/lib/python3.11/site-packages/spidermon/contrib/actions/slack/__init__.py", line 85, in send_message return self._send_channel_message( ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/spidermon/contrib/actions/slack/__init__.py", line 129, in _send_channel_message attachments=self._parse_attachments(attachments), ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/spidermon/contrib/actions/slack/__init__.py", line 171, in _parse_attachments python_attachments = ast.literal_eval(attachments) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/ast.py", line 64, in literal_eval node_or_string = parse(node_or_string.lstrip(" \t"), mode='eval') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/ast.py", line 50, in parse return compile(source, filename, mode, flags, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 4 "text": "• _Extracted Items Monitor/test_stat_monitor_: Expecting 'item_scraped_count' to be '>=' to '100000.0'. Current value: '35565'\n• _Warning Count Monitor/test_stat_monitor_: Expecting 'log_count/WARNING' to be '<=' to '2.0'. Current value: '3994'\n• _Finish Reason Monitor/Should have the expected finished reason(s)_: Finished with "closespider_timeout_no_item" the expected reasons are ['finished']\n", ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ SyntaxError: invalid syntax. Perhaps you forgot a comma? 2026-01-25 07:00:18 [global_industrial] (PID: 96) INFO: [Spidermon] 1 action in 0.226s 2026-01-25 07:00:18 [global_industrial] (PID: 96) INFO: [Spidermon] FAILED (errors=1) 2026-01-25 07:00:19 [scrapy.extensions.feedexport] (PID: 96) INFO: Stored bq feed (565 items) in: bq://response-elt.scraper_data.catalog_item_scrape/batch:36 2026-01-25 07:00:19 [scrapy.statscollectors] (PID: 96) INFO: Dumping Scrapy stats: {'HeadersSpooferDownloaderMiddleware/spoofed': 82545, 'NotFoundHandlerSpiderMiddleware/HttpError': 40658, 'NotFoundHandlerSpiderMiddleware/NotProductPage': 9460, 'big_query/url': 78588, 'downloader/exception_count': 2, 'downloader/exception_type_count/twisted.internet.error.TimeoutError': 2, 'downloader/request_bytes': 142828764, 'downloader/request_count': 82545, 'downloader/request_method_count/GET': 82545, 'downloader/response_bytes': 5286642525, 'downloader/response_count': 82543, 'downloader/response_status_count/200': 37930, 'downloader/response_status_count/307': 5, 'downloader/response_status_count/308': 3950, 'downloader/response_status_count/403': 40658, 'elapsed_time_seconds': 50409.541267, 'feedexport/success_count/BigQueryFeedStorage': 36, 'finish_reason': 'closespider_timeout_no_item', 'finish_time': datetime.datetime(2026, 1, 25, 7, 0, 18, 734467, tzinfo=datetime.timezone.utc), 'httpcompression/response_bytes': 25690100716, 'httpcompression/response_count': 37930, 'httperror/response_ignored_count': 40658, 'httperror/response_ignored_status_count/403': 40658, 'item_scraped_count': 35565, 'items_per_minute': None, 'log_count/ERROR': 2369, 'log_count/INFO': 41577, 'log_count/WARNING': 3994, 'memusage/max': 402735104, 'memusage/startup': 125915136, 'poet/injector/catalog_extraction.pages.global_industrial.GlobalIndustrialProductsPageObject': 78588, 'product_status/not_available_online': 6, 'proxy_manager/ignored/proxy_defined': 3957, 'proxy_manager/processed': 78588, 'response_received_count': 78588, 'responses_per_minute': None, 'retry/count': 2, 'retry/reason_count/twisted.internet.error.TimeoutError': 2, 'scheduler/dequeued': 82545, 'scheduler/dequeued/memory': 82545, 'scheduler/enqueued': 82545, 'scheduler/enqueued/memory': 82545, 'spider_exceptions/NotProductPage': 2365, 'spidermon/validation/fields': 746865, 'spidermon/validation/items': 35565, 'spidermon/validation/validators': 1, 'spidermon/validation/validators/item/jsonschema': True, 'spidermon_field_coverage/dict/brand': 1.0, 'spidermon_field_coverage/dict/categories': 1.0, 'spidermon_field_coverage/dict/countryOfOrigin': 1.0, 'spidermon_field_coverage/dict/description': 1.0, 'spidermon_field_coverage/dict/imageUrl': 1.0, 'spidermon_field_coverage/dict/inStock': 1.0, 'spidermon_field_coverage/dict/isFreeShipping': 1.0, 'spidermon_field_coverage/dict/leadTime': 1.0, 'spidermon_field_coverage/dict/manufacturer': 1.0, 'spidermon_field_coverage/dict/manufacturerSku': 1.0, 'spidermon_field_coverage/dict/name': 1.0, 'spidermon_field_coverage/dict/packagingIncrement': 1.0, 'spidermon_field_coverage/dict/prices': 1.0, 'spidermon_field_coverage/dict/productStatus': 1.0, 'spidermon_field_coverage/dict/relatedSkus': 1.0, 'spidermon_field_coverage/dict/specifications': 1.0, 'spidermon_field_coverage/dict/supplier': 1.0, 'spidermon_field_coverage/dict/supplierSku': 1.0, 'spidermon_field_coverage/dict/uom': 1.0, 'spidermon_field_coverage/dict/url': 1.0, 'spidermon_field_coverage/dict/weight': 1.0, 'spidermon_item_scraped_count': 35565, 'spidermon_item_scraped_count/dict': 35565, 'spidermon_item_scraped_count/dict/brand': 35565, 'spidermon_item_scraped_count/dict/categories': 35565, 'spidermon_item_scraped_count/dict/countryOfOrigin': 35565, 'spidermon_item_scraped_count/dict/description': 35565, 'spidermon_item_scraped_count/dict/imageUrl': 35565, 'spidermon_item_scraped_count/dict/inStock': 35565, 'spidermon_item_scraped_count/dict/isFreeShipping': 35565, 'spidermon_item_scraped_count/dict/leadTime': 35565, 'spidermon_item_scraped_count/dict/manufacturer': 35565, 'spidermon_item_scraped_count/dict/manufacturerSku': 35565, 'spidermon_item_scraped_count/dict/name': 35565, 'spidermon_item_scraped_count/dict/packagingIncrement': 35565, 'spidermon_item_scraped_count/dict/prices': 35565, 'spidermon_item_scraped_count/dict/productStatus': 35565, 'spidermon_item_scraped_count/dict/relatedSkus': 35565, 'spidermon_item_scraped_count/dict/specifications': 35565, 'spidermon_item_scraped_count/dict/supplier': 35565, 'spidermon_item_scraped_count/dict/supplierSku': 35565, 'spidermon_item_scraped_count/dict/uom': 35565, 'spidermon_item_scraped_count/dict/url': 35565, 'spidermon_item_scraped_count/dict/weight': 35565, 'start_requests/big_query': 78588, 'start_time': datetime.datetime(2026, 1, 24, 17, 0, 9, 193200, tzinfo=datetime.timezone.utc)} 2026-01-25 07:00:19 [scrapy.core.engine] (PID: 96) INFO: Spider closed (closespider_timeout_no_item)